Rev Gordon Dickenson in the Chester diocese had been involved in a child sex abuse scandal in 1970s and was convicted this month of eight counts of sexual assault against boys.
In 2009, Dickenson wrote to Bishop Peter admitting the abuse, saying he would "never do it again" and the bishop did not pass it on.
Dickenson was allowed to stay on as a retired clergy member and hold services for another five years.
On Friday, the Bishop of Chester issued a statement saying: "I have asked the Bishop of Birkenhead, Keith Sinclair, to lead on all safeguarding arrangements in the Diocese of Chester and have formally delegated this responsibility to him with immediate effect.
"I have taken this decision in response to recent comment into my handling of the Gordon Dickenson case in 2009.
"An independent review will seek to identify where any failures in procedures arose, and what lessons can be learned and I look forward to contributing to the review and to giving a full account of my actions in relation to this matter.
"The Diocese of Chester takes seriously its safeguarding responsibilities at every level. Whilst an independent review into my actions takes place, I recognise that I should not continue to lead the safeguarding arrangements in the Diocese.
"I will continue in all other duties relating to my role of Bishop of Chester.
"I will not be making any further public comments in relation to this matter until the outcome of the independent review."
Peter Forster has been the Bishop of Chester since January 1997, making him the longest serving bishop.
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.
Tell us what you think about our news service, what you want and how Premier fits into your news landscape. Take our quick survey!