Lundi Tyamara, 38, had been admitted to hospital four weeks ago but passed away this morning. He became ill after returning from a trip to China.
In a statement, his family said: "We got the call just after 1am this morning. We are all shocked. We never expected this to happen. The family is taking it very hard."
Johannesburg mayor Herman Mashaba tweeted: "Deep condolences to the family of gospel singer and star Lundi Tyamara who passed on earlier. May his soul rest in peace."
He sold more than 3 million copies of his gospel albums during his career and received recognition from the South African Music Awards and Kora All Africa Awards for his work.