Columbia Primitive Baptist Church was formally constituted on the first Sunday in October, 1833, after serving as an arm of Bethany Church more than a year. Moses Dees was the first delegate from Columbia to the annual meeting of the mother church, Union on the Alapaha River, late in October 1833. On May 3,1833, while Columbia was still an arm of Bethany Church, Thomas Newbern made a deed to the church property to Samuel T. Henderson, Moses Dees, andJarvis T. Frier as Trustees for Columbia Church.
The first edifice was a little log house across the road from the present church. Later a larger building of hewn logs with puncheon floor was constructed. A more commodious edifice was built in 1872, and has since been improved.
The Rev. Matthew Albritton was one of the earliest pastors, serving the church until 1847.
Columbia Church is a member of the Ochlocknee Association, as it was originally.