What is unspoken between Herbert and Pip in their ironic discussion in Chapter 41 of "Great Expectations"?

The thing that remains unspoken between Herbert and Pip in their ironic discussion is that they both have a deep dislike for Pip's benefactor, the convict Abel Magwitch.  Pip says that Herbert

"had no occasion to say...that he had conceived an aversion for my patron, neither had I occasion to confess my own.  We interchanged that confidence without shaping a syllable".

Despite their feelings, Herbert and Pip are both basically good-hearted young men, and they speak of their dilemma in deciding how to deal with the convict delicately and with civility.  They actually do not know a lot about the specifics of Magwitch's past, but because of his reputation and demeanor they feel they may have reason to fear him.  Nevertheless, Pip understands that the convict is risking his life by returning to London on his behalf, and because of this, Pip feels a sense of responsibility towards him despite his revulsion.  Pip and Herbert devise a plan by which they will spirit Magwitch out of London, and once the man is safe, Pip will discreetly leave him and have nothing more to do with him (Chapter 41).