Henry Addington, the Speaker of the House of Commons at time of Pitt's one reported foray into courtship, was privy to the secret of Pitt's interest in Eleanor Eden. Judging by the contents of Pitt's letter to Addington dated January 23, 1797 - the day after the relationship with Eden was officially considered as over - Addington may have also known the reasons underpinning why Pitt broke it off. Unfortunately, Pitt does not tell all in his letters. Indeed, he was always very careful with what he wrote, and this was particularly the case after he became First Lord of the Treasury.
At one pm on Monday, January 23, 1797, Pitt wrote to Addington from Hollwood to confirm that the intended marriage to Lord Auckland's eldest daughter Eleanor Eden was not going to happen:
My dear Sir,
Knowing your friendly Anxiety and not liking to trust to the Accidents of the Post, I send this by a Messenger; and for so distressful a Subject, I have at length as good an Account to send you as I could expect. The first Answer indeed which I received on Saturday [January 21], tho' thoroughly kind, was the most embarrassing possible, as it stated the Sentiments entertained to be mutual, and pressed for Explanation and discussion, proposing at the same time any Interval of delay in order to take the Chance of overcoming the difficulties, and desiring me to continue coming [to Eden Farm, the Auckland family residence] in the Interval as if nothing had happened. - I had then nothing left but to convey in my Answer quite explicitly, tho with as much Tenderness as I could, that the decision I had felt myself obliged to take was final, and that farther discussion could only produce encreased Anxiety, and could lead to no Good. - This was understood and received as I meant it should; and the Answer I received last night considers the Thing as over, and proposes to contradict the Reports [of the intended marriage] gradually, and with the delicacy which the Subject requires. I hope I may collect from the Manner in which it is written that the Shock has been as little distressful in its Consequences, to any Part of the Family [a reference to Eleanor as well] as I could flatter myself. - If that Hope should be well founded, I trust I can command my Feelings enough to bear the rest, and not to be wanting either to the Calls of Public Duty, or to what yet remains to me of the Private Relations of Life. Among these, the Recollection of what I owe to your kindness and friendship will I trust always hold a principal place in my Thoughts.
Ever Sincerely Yours,
I mean to return to Town [London], to a Cabinet tomorrow, and have almost resolved to go to Lord Cornwallis's at the End of the Week." 
Although Pitt's reasoning for ending the courtship remains a mystery, from this letter it seems as though he was emotionally affected by it. His resolve was firm, and unwavering; he would not change his mind once his decision had been made. Discussion was futile, but Pitt wasn't unfeeling: he hoped that "the Shock has been as little distressful in its Consequences, to any Part of the Family."  He is thinking of Eleanor's feelings. His emotions are there in the passage, "...I trust I can command my Feelings enough to bear the rest."  He assures Addington that his adherence to "the Calls of Public Duty" will still be paramount, and also hints at not lacking in "...what yet remains to me of the Private Relations of Life."  Pitt is cagey; he isn't explicit on what he means here. Perhaps it was meant to be a general statement, but Pitt's feelings were genuine.
When Addington - then Viscount Sidmouth - was 82 years old, many years after Pitt's death, the Irish statesman and writer John Wilson Croker came to visit him. Amongst other preserved anecdotes, Sidmouth told Croker that "Pitt is said never to have had a female attachment; it is not true. He had, I believe, more than one. One I know of; it was to the present Dowager Lady Buckinghamshire, then Miss [Eleanor] Eden."  This conversation took place in 1839; most of his and Pitt's contemporaries were deceased, and I cannot see what, at that long distance of time, would have been Sidmouth's motivation for lying. That he was clearly privy to Pitt's relationship with Eleanor Eden is beyond a question. He attested that Pitt had 'female attachments,' and believed Pitt had more than one. He knew Pitt from childhood, and he was a close personal friend. I trust Sidmouth's testimony.
1. William Pitt to Henry Addington, January 23, 1797. Devon Record Office, Sidmouth MSS: 152M/C1797/OZ/7.
5. Jennings, L. J. (ed.) (1884) The Croker Papers: The Correspondence and Diaries of the late Right Honourable John Wilson Croker, Secretary to the Admiralty from 1809 to 1830, Volume 2, p. 340.