Reacting to the company's decision, archbishop Justin said: "The big issue is to create a financial system that gives access to the poor and hope for the poorest in our lands, to be able to flourish and develop and have proper access to finance, not just for loans but for savings.
"For lives in which finance is a good servant, not a bad master."
Last year Most Rev Justin Welby was left red-faced after he said he wanted to put the company "out of business" before reports later revealed that the Church of England had invested part of its portfolio in the lender.
He has since helped launch the Church Credit Champions Network (CCCN) as part of a bid to get churches and credit unions working together with the ultimate aim of creating a network of community-based church banks.
The Church has also made changes to its investments policy, which no longer includes payday lenders.
David Barclay, who heads up the CCCN, told Premier's News Hour that the Financial Conduct Authority (FCA) was finally having an impact on the industry.
He said: "It reveals that actually what people have thought all along really is true, which is they have been systemically lending money to people they thought could never pay it back."
In a statement, the FCA said Wonga's changes were as a result of a "voluntary agreement" reached between the lender and regulator.
"This should put the rest of the industry on notice," said Clive Adamson, director of supervision at the FCA. "They need to lend affordably and responsibly."
Wonga has also been told it must appoint a "skilled person" to monitor its lending decisions and report back to the FCA.
Listen to more from David Barclay at the Church Credit Champions Network: