Joseph Emerson Brown (1821-1894), born in Pickens District, South Carolina, moved to Union County, Georgia, as a boy. The old Brown home was on the present site of the Woody Gap School, opened in 1941 for mountain students. Brown worked on his father's farm until he was nineteen, when he went to school in South Carolina. Returning to Georgia, he settled in Canton as head of the local academy. Admitted to the bar in 1845, Brown entered Yale Law School, practicing in Canton after graduating.
In 1849, Brown became a State Senator. He was elected Governor, 1857, as the Democratic compromise candidate and reelected 1859, 1861, 1863. During the Civil War, Brown's extreme states' rights views conflicted with President Davis' efforts to centralize the Confederate government. After the war, Brown, unpopular for affiliating with the Republican Party and advocating submission to Reconstruction, was defeated in the U.S. Senate race of 1868. Appointed Chief Justice of the Georgia Supreme Court, 1868, he remained on the bench until in 1870 he became President of the Western & Atlantic Co. After Georgia regained home rule, Brown returned to the Democratic party and was appointed to the U.S. Senate in 1880. Reelected, he served until 1891.