The southern portion used to be the site for Millbank Penitentiary, a notorious six sided structure that housed prisoners of transportation. They were shipped off to Australia in the 19th century direct from a jetty on Millbank itself. It was a desolate, dark, dank prison, where by all accounts, prisoners not only suffered psychological isolation but also physical deprivation and died from dysentery, cholera and scurvy and many other associated diseases. Those who died were interred in an onsite cemetery. There was also enforced silence and a policy to break the morale and spirit of each prisoner, with the aim of rehabilitation. The prison was so vast and claustrophobic, the wardens themselves could only retrace their journey throughout, by marking the walls with chalk. The prison was subject to subsidence, since the land had not been drained adequately prior to construction. It was surrounded by a moat, the remains of which can still be seen today at the western end of John Islip Street. Eventually it was deemed unsanitary, and in any case Australia did not want any more prisoners to colonise it, and so the role and purpose diminished, and it was eventually closed in the late 19th century. The Australian term POM is rumoured to be based on the acronym from Prisoner of Millbank.

 http://sally-gethin-poetry.blogspot.co.uk/p/millbank.html