Companies D and I of the 20th Massachusetts (the "Harvard Regiment") followed the 15th Massachusetts across the Potomac with orders to serve as a rear guard and cover the withdrawal of the 15th Massachusetts following what was hoped would be a successful raid. Those two companies, led by regimental commander Colonel William R. Lee, deployed along the bluff here and waited. They spent much of the day in the area immediately beyond this sign.
While waiting, Colonel Lee sent out scouting parties upriver and downriver to secure his flanks. The upriver party stumbled into a small group of pickets from Co. K, 17th Mississippi, and a few shots were exchanged. The Mississippians withdrew and alerted Colonel Evans to the presence of Union troops at Ball's Bluff. Unfortunately for the Federals, no one from the 20th Massachusetts went forward to inform their comrades in the 15th Massachusetts that contact had been made with the enemy.
Around mid-afternoon, the 20th became involved in the main fighting, an action later described by Lieut. Henry L. Abbott as a fight "made up of charges" as individual companies would advance, fire, and fall back. Later, Lieut. Oliver Wendell Holmes, Jr. received the first of his three Civil War wounds here and was evacuated from the field. During the route of the Federal troops, Colonel Lee and Major Paul J. Revere (grandson of the Revolutionary War hero) were captured along with many other soldiers. Capt. William F. Bartlett led a mixed group of some 80 men upriver where they found a small skiff and managed to cross to safety.