The Friends Meeting House is the oldest religious building in Baltimore. In 1781, the Patapsco Friends Meeting, formerly located on Harford Road two miles north of the Inner Harbor, moved to this site. In 1784 a group of Quakers established a school here, which "provided guarded education for their children." The school eventually became the Friends School of Baltimore.
By the mid eighteenth century the Society of Friends exerted a strong influence socially, politically, and economically in Baltimore. Among the influential Baltimore Quakers who worshipped at this site were Philip E. Thomas (first President of the Baltimore & Ohio Railroad) Johns Hopkins, Moses Sheppard (philanthropist) and the Ellicott family (who founded Ellicott City). Meeting members Elisha Tyson, Robert Townsend, and James Carey organized the Baltimore Abolition Society in 1789. In 1796, they were successful in overturning a 1753 law that forbade the freeing of slaves (manumisson) at the time of their owners' death. On occasion, the Baltimore Abolition Society was represented in court by Francis Scott Key. By the first decade of the nineteenth Century, Baltimore Quakers were deeply involved in the Committee of Indian Affairs, which advocated and at times litigated on behalf of the rights of Native Americans.