Justin Welby confirmed on Thursday that he'd taken action against Christopher Lowson following advice from the police.
"If these matters are found to be proven I consider that the bishop would present a significant risk of harm by not adequately safeguarding children and vulnerable people," he said.
Welby also confirmed there was no allegation that Bishop Christopher has committed abuse of a child or vulnerable adult.
The Bishop of Grimsby, David Court, will take on episcopal leadership of the diocese.
Responding to the news, Bishop Christopher said: "I am bewildered by the suspension and will fully cooperate in this matter.
"For the sake of the diocese and the wider Church I would like this to be investigated as quickly as possible to bring the matter to a swift conclusion."
Lowson has been Bishop of Lincoln since 2011. Two years ago he also took his seat in the House of Lords.
The news of his suspension comes just weeks after a BBC Panorama show investigated abuse in the Diocese of London.
In a statement following the suspension, Lincolnshire Police said: "
We are aware of the decision today of the Archbishop of Canterbury to suspend the Bishop of Lincoln from office and it would not be appropriate for us to comment on that decision.
The first phase of the Lincolnshire Police Operation Redstone investigation into non-recent sexual abuse cases and resulted in three men being convicted.
Phase 2 of the investigation is continuing into wider safeguarding issues and management decisions within the Diocese. Because it is a live investigation and we do not want to jeopardise the outcome, we do not intend to make any further comment."
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.