A Norman castle of earth and timber was built at Skipton by Robert de Romille, soon after 1090. The defences were later rebuilt in stone. The castle is most closely associated with the Clifford family who were granted the property by Edward II in 1310. Robert Clifford immediately began to strengthen the fortifications but he was killed at the Battle of Bannockburn in 1314 leaving the work to be continued by his son Roger.