When George William died in 1787, the land and remains of Belvoir were willed to his nephew, Ferdinando Fairfax, son of his brother Bevan. Ferdinando and his wife Elizabeth lived on the grounds of Belvoir in a house known as the White House. Records show that Ferdinando lived on the property when he prepared his will in 1799. However, Ferdinando also owned several other properties, and only lived at Belvoir for a short time.
Ferdinando Fairfax's will reveals his personal convictions, including his belief that slavery was an injustice. Ferdinando bequeathed slaves to family members stipulating they were never to be sold. He insisted on their fair and just treatment and settlements for their freedom.
Ferdinando donated a large portion of the Fairfax family fortune to causes and institutions, such as public libraries and free schools. He served as a civilian volunteer during the four-day Battle of the White House in the War of 1812 and was commended for his bravery. Ferdinando Fairfax died in 1820 and with him died the Fairfax family association with the Belvoir property.
In the 1830s, William Herbert bought the land and in 1841 the property was purchased by Philip Otterback, and became known as Otterback's Woods.