In 1864, Ellerslie stood in the middle of the Confederate defense line along Swift Creek. On May 9-10, Confederate Gens. Johnson Hagood and Bushrod Johnson, with 4,200 men, contested the advance of a much larger Federal force, composed of elements of Gen. Benjamin F Butler's Army of the James. During the fighting on May 9, a Confederate battery near the house dueled with Federal gunners across the creek at Arrowfield Farm. A cannonball struck the house's wall and remained embedded until 1910.
At noon on May 9, the 63rd Tennessee Infantry occupied hastily dug rifle pits here as part of the Confederate reserve. When Union skirmishers advanced to a fence line about 600 yards from Ellerslie, two Tennessee companies pushed them back. During the night, the Confederates drove a Federal gun from the bank of Swift Creek. The Federals retreated to their camps at Point of Rocks the next morning.
Federal operations then shifted toward Richmond and culminated in the Battle of Drewry's Bluff on May 16. Gen. P.G.T. Beauregard established his headquarters here and kept them intermittently through June. On September 2, Hagood's South Carolina brigade established a rest camp at Ellerslie and remained here until September 28.
On the night of April 2, 1865, fires and explosions illuminated Ellerslie as commissary supplies and munitions were burned at Dunlop's Station near here on the Richmond & Petersburg Railroad. Several civilians were killed.
Pierre Gustave Toutant Beauregard was born on Contreras Plantation near New Orleans to a Creole family on May 28, 1818. He graduated from the U.S. Military Acadamy at West Point in 1838 and was wounded in the Mexican War. He was superintendent at West Point in 1861 and resigned his commission to join the Confederacy. He gave the command to fire on Fort Sumter in Charleston, S.C.—the first shot of the war. He became a Southern hero for the Confederate victory in the First Battle of Manassas. In Charleston in 1864, he authorized CSA H.L. Hunley
to make the first attack by a submarine. He defeated Union Gen. Benjamin F. Butler in the Bermuda Hundred Campaign in Virginia in 1864. Beauregard defended Petersburg with 2,000 men against 16,000 Federals until Gen. Robert E. Lee arrived with the Army of Northern Virginia. After the war, Beauregard returned to New Orleans, assisted freed slaves, and declined offers to command foreign armies. He served as a railroad president and in Louisiana government posts. He died in New Orleans on Feb. 20, 1893, and was buried there.
Robert Young, a Belfast architect, designed Ellerslie in 1856 as a castellated mansion for tobacco magnate David Dunlop. Dunlop's grandson engaged the Richmond architectural firm of Carneal and Johnston in 1910 to remodel it in the Bungaloid mode, retaining the house's mass and tower but replacing the original flat roof with a hipped roof and dormers.