Manchester House was built in an area of London acquired by the Portman family in 1533.
The family still retains ownership of much of the land in the vicinity, which is called the Portman Estate. In 1757 the New Road, now the Marylebone-Euston Road, was built, marking the northern boundary of the estate.
At this time there were still open fields to the north of Oxford Street. In 1761 the Portman family started to develop Portman Square and a building boom began. In 1771, Samuel Adams, a speculative builder, bought the leaseholds of eight plots of land on the north side of the square, including the plot on which Hertford House now stands.
Adams built the shell of the building but it was only when the 4th Duke of Manchester bought the leasehold in 1776 that work continued on the house again. Completed in 1788 by the architect Joshua Brown, the house consisted of 5 bays on its south front and 3 storeys. The front façade had at its centre a large Venetian window.
By 1807 the 2nd Marquess had added two first floor rooms on each wing and a conservatory above the main entrance. In 1871 Richard Wallace redeveloped the house, completing the three sides of the inner quadrangle with a range of galleries on the first floor, including the top-lit Great Gallery at the rear of the building.
For the Centenary Project in 2000, Rick Mather Architects created educational facilities including a Lecture Theatre on the Lower Ground Floor and an attractive restaurant area with a glass roof in the courtyard.