Jane (neè Pryce) was Elizabeth and Louisa Jane Williams' mother. Today I decided to research her burial record on the Westminster Archives via findmypast.co.uk in order to follow-up a source. Sure enough, here's what I learned:
Jane Williams was buried at St Margaret's church, Westminster on 27 February 1797. Unfortunately, the source does not mention her age when she died.
St Margaret's church cemetery is no longer in use; in fact, all trace of its former external use as a burial ground has been erased, so it is now impossible to ascertain exactly where Jane Williams's final resting place is located.
I do wonder how intimate the Williams family were to William Pitt the younger. This certainly needs to be explored further.