Most Revd Justin Welby is in Africa for the Anglican Consultative Council which has been taking place in Zambia.
According to the Zimbabwean newspaper, Archbishop Justin said: "We talked about the past in Zimbabwe, the mistakes that have been made, the breakdowns of the relationships from time to time and we ended with prayer for the future of this country.
"The relations between the Church and the State greatly improved and President Mugabe also talked about that."
President Mugabe, who identifies as Catholic, has previously faced criticism from church leaders. In 2008 Archbishop of York Dr John Sentamu called for him to be removed from power.
Links between local church leaders and the government are however now said to be improving.
In a statement to Premier, a Lambeth Palace spokesman confirmed the meeting.
He said: "The Archbishop of Canterbury paid a pastoral visit to President Mugabe of Zimbabwe yesterday late afternoon.
"This last minute visit, made at the request of the Bishop of Harare, was in support of the church in Zimbabwe.
"It lasted less than an hour."