After being puzzled by the discrepancy in the correct number of Pitt's house on Park Place, I believe I can safely deduce with confidence that John Ehrman was correct. Pitt rented 12 Park Place just after his resignation in early 1801. In the 1860s, Lord Stanhope had written in his notes that the address was 5 Park Place, but he did not provide a source for his information.  Ehrman believed that Pitt resided at 12 Park Place, and although he also did not include a source, this information was cited in one of the footnotes. 
I mentioned in a previous post that numbers 5 and 12 Park Place are on opposite sides and ends of this very small street, so it amazed me that there would be that discrepancy. Fortunately, I believe I've touched upon Ehrman's source - and the answer to the mystery.
|Fig. 1: A map of Park Place, St. James from Horwood's 1799 map|
London Past and Present: Its History, Associations, and Traditions (1891: 33) mentions that "William Pitt retired to No. 12 in 1801."  Although they do not give their source, I can only assume that Ehrman gleaned his information from either a letter in one of the manuscripts, or from this book. As you can see from the screenshot I took of my digital copy of Horwood's 1799 map of London, Park Place is located just off St. James's Street. Brooks's gentleman's club is on the top right-hand corner, and no. 12 is one of the terraced houses directly opposite. Pitt was very familiar with the area of St. James, not only due to its proximity to Westminster, St. James's Palace, and gentleman's clubs he was a member of (including White's and the former Goostree's), but his father also formerly had a house at No. 10 St. James's Square. His brother Chatham also rented a property on St. James's Square at one time, and Pitt frequently stayed at Nerot's Hotel on King Street, St. James in the late 1770s. Renting a property on Park Place would have provided Pitt with somewhere central, convenient, and within easy reach of Westminster.
1. Kent History and Library Centre, Maidstone. Stanhope of Chevening manuscripts, Pitt MSS: U1590/S5/C60-4.
2. Ehrman, J. (1996) The Younger Pitt: The Consuming Struggle. London: Constable, p. 534 (footnote).
3. Wheatley, H. B. & Cunningham, P. (1891) London Past and Present: Its History, Associations, and Traditions. Cambridge: Cambridge University Press, p. 33.
Figure 1: Detail of Park Place, St. James, from my digital copy of Richard Horwood's 1799 map of London.