Simon Hodgkinson's wife Olivia has been linked to the former Bishop of Arundel and Brighton, Kieran Conry, by several newspapers.
Hodgkinson says Bishop Kieran should no longer be a priest and is compiling a formal complaint.
He alleges the bishop had a relationship with his wife that was not appropriate and will submit his claims to the Pope's representative in the UK and Cardinal Vincent Nichols, the head of the Catholic Church in England and Wales.
Mr Hodgkinson will claim the 63-year-old bishop broke his vow of celibacy and the Church's teaching on adultery and the sanctity of marriage.
Last month, parishioners in the bishop's diocese were told he had resigned after he admitted being 'unfaithful' to his vows, but he says he did not have an affair with Mrs Hodgkinson.
The bishop denies the claims that he had an inappropriate relationship with the woman.
At the time of his resignation he apologised to all who had been "hurt by my actions and then to all of those inside and outside the diocese who will be shocked, hurt and saddened".
He added: "I am sorry for the shame that I have brought on the diocese and the Church and I ask for your prayers and forgiveness."