Brown's Island is named for Elijah Brown who acquired it in 1826. Brown came from Rhode Island in 1811 to be a gunsmith at the Virginia Manufactory of Arms. In 1818, he entered the Public Guard, which was stationed at the Manufactory, and served as Lieutenant and Paymaster. For a time the Island was called Neilson's Island, after a subsequent owner, but the name Brown's Island eventually stuck.
Since Elijah Brown's day, the island has had a varied history, sketched in plaques around the island. In 1982, the City of Richmond traded Gambles Hill Park to the Ethyl Corporation for Brown's Island. After improvements sponsored by the James River Discovery Program, the island became part of the City's James River Park in 1987.