It was first built for a Russian General around 1895. The first structure was actually a wooden cottage romantically named "The Love Castle." Later, ownership passed to a court doctor of the Russian Tsar named A.K. Tobin. In 1911, Baron Von Steinheil acquired the wooden cottage and built on top of it what stands today. The castle itself sits 130 feet above the water below.
In 1927, Swallow's Nest survived a magnitude 6 to 7 earthquake and was closed to the public for nearly 40 years due to a massive crack that developed on the cliff itself. In 1968, renovations began and it was re-opened to the public in 1975 and houses a fine Italian restaurant inside.
The castle itself has been in numerous films including Agatha Christie's And Then There Were None.