Takatori Castle was constructed in 1332 during Nanboku-cho period. It is located between Nara and Yoshino, so this castle was very important for the traffic. This original castle was constructed by Ochi clan. However, that was a very traditional style of mountain castle at this time. There is no stone walls or baileys. In 1580, Tsutsui Jukei extended this castle as the protection of Koriyama Castle. After him, Hashiba Hidenaga, the brother of Hideyoshi, moved to Nara and he ordered his samurai, Honda Toshisada and re-structure this castle. The current structure of this castle was built by Honda. In 1640, Uemura clan was moved to this castle and he kept extending this castle. Usually, during the Edo period, all castle lords must report to Shogun to fix their castle, but this castle could fix and extend without reporting and getting the permission from Shogun.

It is chosen as the 3 major mountain castles in Japan and registered as the national historical site. Unlike, other mountain castles, Takatori Castle has many stone walls and basement of the Tenshu, keeps and gates. The Baileys are well composted and you can see how strong this castle was.