Rt Rev Dr Steven Croft became Bishop of Oxford in September.
At his inauguration, a demonstration took place outside Christ Church Cathedral led by an abuse survivor - known only as Michael - wanting the Bishop to do more to help victims of sexual abuse.
Michael, a retired priest himself, claimed that he told Dr Croft about the abuse in 2012.
Michael said he did not feel supported by the then Bishop of Sheffield and that Dr Croft "could not get off the phone quick enough."
Defending himself in an interview with the BBC, Bishop Stephen said: "Clearly this person feels he didn't receive the support he needed and I regret that very much."
He added that he had arranged for someone to provide pastoral care for Michael and kept a written record of the actions he took.
Dr Croft elaborated further on his blog on the Diocese of Oxford's website. He wrote: "I want to make it quite clear that the care of vulnerable people and those who are survivors of sexual abuse and exploitation is a high priority for me, and for the Church of England as a whole.
"As a society we have all learned a great deal in recent years about the appalling prevalence of sexual abuse.
"I would encourage any survivors who are yet to speak up to do so. Please be assured that we work in close co-operation with the police and other agencies to ensure that all allegations of abuse."