170's doorway Norman Shaw designed this house for Frederick Anthony White, a wealthy cement manufacturer and amateur of art and architecture. It was completed in 1889. The initials of the first owner and his wife may be seen on the rainwater heads on the south façade, and the White family crest forms part of the decoration of the front door. Mr White seems to have been very proud of his house and issued a portfolio of photographs in the early nineties, with exterior and interior views of his home. "When Mr Fred White lived there, with all his beautiful things round him, [170] was one of the most attractive houses in London" wrote Sir Reginald Blomfield in his biography of Shaw.

Mr White sold his house in 1925 to the 6th Marquess of Anglesey, who lived there until about 1938, when he put it on the market. Imperial College made enquiries about a possible purchase but nothing came of this and the house was leased to the Secretarial Appointments Bureau for 21 years. The College became interested in the house again after the war, and its purchase was stated to be part of the College quinquennial policy for 1947-52. After some negotiation, the University Grants Committee offered to help to buy the house and the College became owner in 1947, with vacant possession in 1959. The lease of the occupiers, the London College of Secretaries, was in fact extended to 1960.

It was agreed that the house should be adapted for use by the Governing Body and the Senior Common Room, with provision for the Rector's Lodging and other staff accommodation. The consulting architect to the College, Sir Hubert Worthington, was at first in charge of these alterations, which were continued by Mr T W Sutcliffe and completed in 1962. Dame Sylvia Crowe designed the landscaping of the garden, the original southern boundary wall was moved, and some minor architectural features were changed.

Norman Shaw (1831-1912) was one of the most important English architects of the 19th century, specialising in town and country houses. 170 Queens Gate was scheduled as a building of special architectural or historic interest in 1958. It is generally agreed to be a very important example of English domestic architecture of its period.