|Fig. 1: Dudley Ryder, 1st Earl of Harrowby|
This story is worth quoting here as the 'infamous inkstand incident' (my turn of phrase). As a confirmed Pittite, this wrangles with me. Let's see what you make of it...
Pitt's friend Dudley Ryder, 1st Earl of Harrowby, lived until 1844. Thus, he knew Philip Henry Stanhope, and they seem to have conferred about Pitt. Below is one of the stories Harrowby related to Stanhope:
“…It appears that Mr. Pitt on his deathbed told the Bishop of Lincoln that he should like to leave him some memorial of his friendship, & bid him take a silver inkstand which was on his table & which he had always used. This the Bishop did accordingly. Some years later, as it chanced the Bishop was giving a dinner to some of Mr. Pitt’s most intimate friends, the conversation happened on him, and the well-known inkstand was mentioned. Some of the party expressed a strong wish to see it once again, & asked that it might be brought in. The Bishop made various excuses, but as there seemed no indiscretion in pressing for merely a sight of this memorial, the Bishop at last was obliged to say that having not long since to purchase some expensive plate, & above all two large wine-coolers, he had allowed Mr. Pitt’s inkstand to be taken as part of the price. Nothing I presume was said to the Bishop at the time; but Lord Harrowby added that the anecdote was never forgotten & caused great indignation among Pitt’s surviving friends.” 
Harrowby clearly never forgot the insult to Pitt's memory.
1. Kent History & Library Centre. 'Story told to Earl Stanhope by Lord Harrowby of Bishop Pretyman, afterwards Tomline' - Pitt MSS: U1590/S5/C60.
Figure 1: Dudley Ryder, 1st Earl of Harrowby by H. Robinson after T. Phillips (1844). Source