Cadhay was built by John Haydon in 1550 on the site of an earlier house. His nephew Robert built a long gallery, a feature of late 16^th century housebuilding, closing in the south side of the house to form a courtyard. The four statues of Henry VIII, Edward VI, Mary and Elizabeth which stand over the doors in the courtyard were carved in 1617.

By 1737 the house was in a poor state of repair and the new owner, Peere Williams, restored the house in the style of that period. He plastered up most of the Tudor hearths and panelled a number of the rooms. He inserted a lower ceiling in the great hall under the magnificent oak-timbered roof to form the current dining room and roof chamber.

Being a large house, there were times in the 18th and 19th centuries when Cadhay was divided into two. It also fell into a bad state of repair. It was bought by Dampier Whetham in 1910 who uncovered the old Tudor hearths and put the house into sound structural condition. He let the house to the William-Powletts who bought the property in 1935, and have occupied it since.