In area first settled by James Manor (1804-1881), who came from Tennessee with Sam Houston in 1832, later returning for his family and a sister and brother. Until 1852, area was subject to Indian raids. Other pioneers included A.F., W.M., and James Boyce, Sterling Chamberlain, Dave Eppright, J.I. Haynes, Ed Harrington, A.C. and W.H. Hill, W.B. Howse, N.A. Rector, W.L. Shipp, E.D. Townes, and Joe, Bill, Sam, and Walter Vaughan. In 1854 a boys' school was opened and a Methodist church organized. Isaac Wilbahn (1857) gave site, and Parsons Female Academy (named for a leading contributor) was established, to become one of the celebrated schools of 19th century Texas. Its teachers included the Rev. and Mrs. D.H. Bittle, and T.C. Bittle; its students, T.B. Wheeler (1840-1913), Mayor of Austin (1872-77), judge of 12th district (1880-86), Lieutenant Governor of Texas (1887-91); and John C. Townes (1852-1912), judge of 33rd District (1882-85), 26th District (1888), and Dean of the University of Texas Law School (1901-02, 1907-23).
In 1871 James Manor donated right of way for the Houston & Texas Central Railroad. The town founded and named for him in 1872 was incorporated in 1913.
Cotton, king here for years, has been supplanted by cattle raising as base for the local economy.