On the night of October 20, 1861, a small Federal scouting party crossed the Potomac River from Maryland to determine whether recent troop movements indicated a Confederate withdrawal from Leesburg. Advancing inland from Ball's Bluff, the Federals moved past this point, crested a low ridge near the Jackson house, and saw in the dim moonlight what appeared to be a Confederate Camp. Upon learning of this, the Federal commander, Brig. Gen. Charles P. Stone, saw a target of opportunity and quickly organized a raiding party.
Five companies of the 15th Massachusetts led by Col. Charles Devens spent several hours quietly crossing the river from Harrison's Island. At dawn they marched up the path on their way to destroy the supposed enemy camp and return to Maryland. The "camp," however, did not exist. In the dark, the scouts had mistaken a row of trees for tents. By the time the error was reported to General Stone, an unintended battle had begun.
At 7:30 a.m. on October 21, Company K of the 17th Mississippi clashed with the Massachusetts men near the Jackson house.
General Stone remained in Maryland at Edward's Ferry. On hearing of the patrol's mistake, but not yet knowing that fighting had occurred, he ordered Col. Edward D. Baker to evaluate the situation. On his way upriver to do this, Baker learned of the fighting from a messenger who then proceeded downriver to alert General Stone. Col. Nathan G. "Shanks" Evans directed his Confederate forces from an earthen fort named for him on nearby Edwards Ferry Road. By midmorning, Evans had committed four more companies of Mississippi infantry and three companies of Virginia cavalry to the Ball's Bluff fight. Devens' men withdrew to a wood line near the Jackson house. About 13:30, the 8th Virginia Infantry arrived. Shortly thereafter, the Confederate line, now nearly 700 strong, attacked Devens' roughly 650 Federal troops. The skirmish lasted perhaps an hour. Afterwards, Devens withdrew to the bluff. The most serious fighting was about to begin.