Madison was incorporated in 1809 and named for James Madison, who had negotiated a treaty with the Muscogee (Creek) Nation. Until Walton County was created in 1818, Morgan County was Georgia's western frontier, and everything west of it was still Muscogee land. Many of the first settlers held land grants for service in the Revolution, and they laid the town out with a geometric regularity.
Cotton made the town, and enslaved people made the cotton. The 1820 census counted 13,520 people in Morgan County, 44.7 percent of them enslaved, and the share grew as large plantations spread over the surrounding hills. The wealth paid for the columned houses still lining the streets and for two women's colleges founded in 1849 and 1850. In November 1864 Sherman's troops burned the depots, gins and warehouses and wrecked the railroad for miles, but spared the town's center. Local folklore calls Madison the town too pretty to burn and credits Joshua Hill, a former congressman who had opposed secession, with persuading Sherman, though accounts vary and Hill's own plantation house was among the few burned. Fire came anyway on April 9, 1869, when forty-two stores downtown were destroyed.
The historic district went on the National Register in 1974. Start at the welcome center on East Jefferson Street. The Morgan County African-American Museum, founded in 1993, is in the Horace Moore House of about 1895 on Academy Street; the painter Benny Andrews and his brother, the novelist Raymond Andrews, grew up near Madison.
The closest stops worth working into your route
Roadside plaques and monuments within a short detour