Built in 1785 by Daniel Sowers on a lot granted by Lord Fairfax dated 1753. The Sowers' owned many horses, possibly for a livery business. In 1801, the house conveyed to Adam Douglas, an Irish export merchant and author of The Irish Immigrant. Dr. Benjamin Grayson owned the property until 1812, when John Miller purchased it. It passed to his son, Godfrey Sperry Miller, in 1857. Confederate and Union wounded soldiers received care here during the Civil War. The Millers made brick additions and reportedly installed the first elevator in the city. Margaretta, daughter of Godfrey, left the home in 1938 for use as a residence for elderly ladies. It became a senior community center in 1976.