Guildford Castle started out as a Norman motte and bailey castle, built soon after 1066. The wooden defences were replaced by stone ones during the 12th century, and the motte acquired first a shell keep and then the strong but not very large tower keep that survives today. As the only royal castle in Surrey it became the centre for administration and justice, with the keep converted to use as the county gaol. The royal apartments were moved to a palace in the bailey below, which under Henry III became one of the most luxurious royal residences in England.