Politicians voted for the change, after more than 13 hours of strong debate over the flag, which some see as a symbol of black slavery.
There've been calls for its removal for decades, but recently photos emerged of Dylann Roof, the 21-year-old charged with the killings of the black churchgoers in Charleston, holding the flag.
The flag - used by the South in the American Civil War (1860-65) - is seen by some as an icon of slavery and racism while others say it symbolises US heritage and history.
The banner could now be removed within days.
As a group of Republicans fought to keep the flag, Representative Jenny Horne scolded her colleagues for stalling the debate with dozens of amendments.
"I cannot believe that we do not have the heart in this body to do something meaningful, such as take a symbol of hate off these grounds on Friday," Horne cried.
Opponents of removing the flag talked about grandparents who passed down family treasures and lamented that the flag had been "hijacked" or "abducted" by racists.
Horne reminded her colleagues that she was a descendant of Confederate President Jefferson Davis.
She later said that she hadn't intended to speak but got frustrated with fellow Republicans.