Matthias d'Amour was a Belgian "groom of the chamber" who served Jane Maxwell, the Duchess of Gordon . Many years later, d'Amour wrote his memoirs recounting the days he worked for such an illustrious mistress. The Duchess of Gordon (d. 1812) was a fashionable Scottish political hostess for Mr. Pitt's administration, and in the late 1780s in particular, she spent countless hours in the cheerful and witty company of Henry Dundas (then Secretary of State), Mr. Pitt, and other members of Pitt's government. As d'Amour was working for the Duchess of Gordon at such an important period, he thus had numerous opportunities of seeing Pitt in some of his most private and companionable hours.
The two early 20th century photographs above depict the interior of the former Buckingham House at 91 Pall Mall. These photographs of the former Buckingham House (once owned by the Pitt and Grenville families and subsequently used by the War Department), shows the rather curious staircase on the first floor of the building. In the 1780s, Buckingham House was tenanted by the Duke and Duchess of Gordon, and it served as a social hub of Pitt's administration. Unfortunately, the mansion has long since been demolished, and The Royal Automobile Club now occupies part of the site. This is the house where the Duchess of Gordon held some of her political dinners in the late 1780s for the Pitt administration.
Below is d'Amour's amusing recollection of a nonsensical conversation the Duchess of Gordon enjoyed with Pitt and Dundas:
"As I had always, on these occasions, to be in immediate attendance on Her Grace [the Duchess of Gordon], I had ample opportunities of witnessing, sometimes the most grave and weighty discourse; and at other times the most light, witty, and amusing repartee. I was often astonished, especially when Mr. Pitt was present, out of what trifles they could a spin a whole web of pleasing conversation. On one occasion, when on a visit at Mr. Harry [Henry] Dundas's [at Wimbledon], then Secretary of State, afterwards Lord Melville, our Duchess, as I remember, and Mr. Dundas with some others were seated in a room into which the moon shone brightly during the dusk of the evening. Her Grace made a passing remark, "How beautifully the moon shone behind the window." "No, your Grace," replied Mr. D[undas], "the moon does not shine behind the window - it shines before the window." Her Grace was as tenacious in defending her assertion as Mr. Harry was in maintaining his amendment; and as neither party were disposed to yield, they actually reserved the point in dispute for the deliberation next day, of the whole congregated Administration of [King] George the Third: and for a full hour the Secretary of State, Mr. Pitt, Lord Thurlow (who was the Lord Chancellor), Mr. Wilberforce (then a young man), the Marquis of Aberdeen, and a number more almost equally distinguished, were employed in the most lively and humorous manner to decide the question." 
Perhaps my favourite passage from d'Amour's Memoirs relates to Mr. Pitt and a certain pony at the Duchess of Gordon's during the time she leased the Marquis of Buckingham's mansion on Pall Mall:
"As I was one day passing through the rooms after my accustomed avocations, I met with Mr. Pitt and Lady Charlotte Gordon conversing together in the drawing room. Lady Charlotte, having some order to give me, commenced as usual, "Mr. D'Amour -" Mr. Pitt purposely interrupting her speech by taking the sentence from her lips, added, "You are desired to bring one of the Shetland ponys upstairs immediately." I smiled and bowed acquiescence, but stood a moment or two to give the lady time to finish what she intended to have said. What Mr. Pitt had proposed, however, in jest, she determined to surprise him with in earnest, and while they were mutually laughing, she stepped towards me, and in a low tone of voice, bade me do as he had said. I hastened down stairs, being always well pleased to fulfil a good-humoured command, sought the groom, got the pony saddled, and had him led up stairs; the easy ascent of which he mounted very gracefully. When I opened the door and announced the arrival, (and surely it was the first announcement of the kind ever made,) Mr. Pitt's powerful voice, exercised in abundant laughter, resounded through a great part of the mansion. After parading the pony round, the Prime Minister, to finish the joke, tied a white handkerchief to the bit of the bridle, and led him down stairs with his own hand: not, however, till the animal had deposited upon the floor of the drawing room an indubitable proof of his having made himself quite at home."
It's lovely to see how Pitt could unbend his mind!
I do wonder what Pitt and Lady Charlotte Gordon were discussing whilst they were alone in the drawing room?
1. Bulloch, J. M. (ed.) (1902) The Gordon Book. p. 17.
2. D'Amour, M. (1836) Memoirs of Mr. Matthias d'Amour. London: Longman, pp. 154-5.
3. Ibid., pp. 158-9.