Bishop Wallace Parke Benn told the Inquiry into Child Sexual Abuse the Rev Roy Cotton was restricted from public ministry to avoid contact with children but it was "difficult" to enforce.
Fiona Scolding QC, the lead lawyer for the Anglican strand of the inquiry, is examining how the Church of England handled allegations of sexual misconduct stretching back to the 1950s, first focusing on the Diocese of Chichester.
She questioned the former Bishop of Lewes at the public hearing in London on Monday over paedophile priests Roy Cotton, his friend Colin Pritchard, Gordon Rideout, Jonathan Graves and Robert Coles - who all operated in the Diocese.
Ms Golding described the sanctions imposed on them as a "stern telling off" rather than anything more substantial.
Bishop Benn, who served from 1997 to 2012, said he had "inherited" a paedophile ring when he took his role in East Sussex and that "hindsight was a blessed thing" when questioned before chairman Alexis Jay and her panel.
The inquiry heard Cotton was convicted in 1954 but did not admit this to the Diocese until 2001. He died in 2006 before facing justice for more crimes.
Bishop Benn knew of Cotton's arrest in November 1997 but not the specifics of the allegations, he said.
He told the inquiry he took steps to restrict Cotton's contact with children while he was working in the Eastbourne parish and insisted he never learned of his conviction until 2001.
He said a ban placed on Cotton practising publicly was "very difficult to monitor", adding: "I can't honestly remember what was done to actually reinforce that."
In December 2008 police dropped the investigation but Bishop Benn had a "hunch" about Cotton - who he distrusted and described as a "villain", the inquiry heard.
But he granted him permission to officiate (PTO) on his retirement in 1998 after it was cleared by Bishop Eric Waldram Kemp who said he had been "treated badly by the police", Bishop Benn said.
Yet in a letter read to the inquiry he told Cotton he would be "happy" to grant the PTO. He defended this as "standard" wording.
He told how there was fear of legal repercussions if priests were suspended and later cleared of wrongdoing, adding: "Any ultimate disciplinary action was never mine to take, it was not my responsibility, it was the diocese's responsibility."
He told how the diocese tried to "push" Cotton into retirement and said they were "eager" to get him out of a parish where there was "more room to groom".
Despite the restriction, the inquiry heard Cotton took five public services between 2001 and 2003, four of which were in one week for Pritchard - who was handed his second jail sentence last month.
Ms Scolding said this showed the "toothlessness" for monitoring PTO, to which he replied: "Yes. Well it shows how easily it's overcome.
"I shouldn't have agreed at all. I should have just said no, and no is no."
Years later, in 2009, he contacted Cotton offering help "from a friend" after an anonymous article emerged about him in a free publication in Eastbourne.
But he said the Church was "really disappointed" when the police had not acted on the allegations. He claimed they did not "join the dots" but said it was not his responsibility to contact the police directly.
Facing similar questions over the other convicted priests, the inquiry heard Bishop Benn's assistant granted Pritchard a PTO in 2007 after he was arrested in November the previous year.
He told the inquiry: "It was a very regrettable mistake but as soon as that came to light, we put it right.
"It was not my intention to issue it."
The licence was revoked on the advice of the Diocese safeguarding adviser from 2007 to 2010, Shirley Hosgood.
During the hearing Bishop Benn described her as "aggressive" and "nit picking" rather than someone who was trying to help, although Ms Scolding said this is not how she came across when giving evidence.
He also criticised the damning independent Meekings report published in 2012 which he claimed made him a scapegoat and blamed press coverage at the time.
But he denied ever threatening to sue the Church over the publication, adding that the only legal action considered was "after some very, very bad reporting from the BBC".
He described "panic" in the Diocese after the report was published and "systemic problems", adding: "There was dysfunctionality. It was a rabbit in the headlights moment for about two years."
He said he had considered "stepping aside" but the "truth" was more important, adding: "I would walk over hot coals to do it better.
"My concern in safeguarding has always been the well-being of the victims who have to cope for the rest of their lives with the shocking experience that they have been through."
The inquiry continues on Tuesday.
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.
