The Rev Matthew Ineson criticised the archbishops as he gave evidence to the Independent Inquiry into Child Sexual Abuse (IICSA) investigation into the Anglican Church.
The vicar called for changes to the way the church investigates safeguarding issues and complaints about clerics, describing the current system as "totally unsuitable".
The witness told the inquiry how he suffered abuse at the hands of priest Trevor Devamanikkam, who took his own life on the day he was due to appear in court accused of sexual offences against Mr Ineson.
He said he later made disclosures to a number of bishops and archbishops but they were ignored and no further action was taken by the Archbishop of Canterbury Justin Welby when he made complaints against the clerics.
Mr Ineson said bishops abuse their power to protect themselves and force victims of clerical sexual abuse to face further suffering by not acting upon their disclosures.
He told the inquiry: "Bishops sit on thrones. They live in fine palaces and houses, they wear the finest robes and garments, they bully people. People literally kneel down and kiss the ring on their finger.
"It really does drive people to distraction and I say no more. Enough is enough. I think the victims are far stronger people than the bishops."
He continued: "I cannot see the face of Jesus in the Archbishop of Canterbury or York. I see hypocrites and I see Pharisees. I see the people that Jesus stood up against.
"I don't think those people are fit for office."
Mr Ineson said he met the Archbishop of York Dr John Sentamu at a meeting for survivors of clerical abuse at a General Synod in York.
The witness said: "I'd never seen John Sentamu before and if I ever see him again it will be too soon.
"He came over to me, he came really in my face, too close, he grabbed me by the shoulder and he held me by the shoulder and said, 'One day me and you will talk'."
Mr Ineson said he asked the archbishop for an apology for his failure to act on his disclosures but Dr Sentamu replied: "Apologies mean different things to different people."
He told the inquiry: "He's arrogant, he's rude and he's a bully."
Mr Ineson, who was ordained in 2000 and practised as a vicar in Rotherham, South Yorkshire, for more than 10 years, said he made his first disclosures between 2012 and 2013 to the Bishop of Doncaster Peter Burrows, the then bishop of Sheffield Steven Croft, and the then archdeacon of Rotherham Martyn Snow, but that nothing came of his reports.
He said: "I could not believe it. I could not believe they were doing nothing."
Mr Ineson went on to make two written disclosures to Mr Croft, now the Bishop of Oxford, and sent copies to the Bishop of Beverley and Dr Sentamu.
He said he wrote in his second letter: "You will never know what it took to tell you but you will also never know of the hurt that you and your suffragan have caused me by doing absolutely nothing about it."
Mr Ineson told the inquiry that the only person to reply was Dr Sentamu, who wrote: "Please be assured of my prayers and best wishes at this difficult time."
He said he submitted complaints about the clerics under the clergy discipline measure but was told that the complaints fell outside of the church's one-year rule.
Mr Ineson appealed but told the inquiry that part of the process was to contact those being complained about to ask for their opinion about whether the one-year rule should be extended in their cases.
He said: "Who in their right mind thinks it's acceptable to write to a priest who is under investigation by the police for child sexual abuse and give him the opportunity to object to being investigated?"
Mr Ineson described the complaints system as "totally unsuitable".
He said: "Bishop investigates bishop and they're all conflicted."
Mr Ineson said he believed that safeguarding should be taken out of the hands of the church so that complaints are investigated by an independent body and that clerical sexual abuse should be the subject of mandatory reporting.
He said: "I have no desire to damage the church at all or bring the church down. The overriding motive for me is to help prevent abuse from happening again."
Mr Ineson continued: "The church is not going to change unless they are made to. They can't be trusted."
Giving evidence on Wednesday afternoon, Dr Sentamu said he did not believe he had made personal mistakes in the course of responding to disclosures of clerical abuse.
He said: "Hand on heart, I don't think so. Where there have been disclosures, I have been willing to apologise to the person, trying to do the best I can to support them."
When asked about the case of Mr Ineson, Dr Sentamu said he received a copy of the cleric's letter but assumed the Bishop of Sheffield would deal with it because it was his responsibility.
He said that, if he had behaved at the General Synod in the way described by Mr Ineson, that behaviour would have been "totally inappropriate".
Dr Sentamu agreed Mr Ineson's treatment by the Church had been "shabby and shambolic" and said he should have received more support.
He added the police investigation and the complaints issued under the clergy discipline measure meant it took too long to set up an inquiry into his abuse.
Richard Scorer, a specialist abuse lawyer from Slater and Gordon, who acts for many survivors of abuse in the Anglican Church, said: "I applaud the bravery of the Rev Matthew Ineson in giving evidence today.
"Never has the case for independent safeguarding been so powerfully made.
""It is absolutely appalling that so many senior church figures chose to ignore such serious allegations.
"This surely demonstrates that a mandatory reporting law is the only possible way forward to make sure the mistakes of the past are not repeated."
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.