The MP said he started to fall away from the Church following the death of Pope John Paul II.
He told the Huffington Post: "I'm Catholic by upbringing, but I'm not particularly religious now.
"But because of my background, when I voted it always caused a kind of tension in terms of people I know, friends, family, so I do find it fairly hurtful when people accuse me of lacking commitment [to gay rights - he once voted for an amendment seeking to put a named father on IVF registrations]."
He said the Catholic Church's approach to issues like gay rights and contraception left him disillusioned with his faith.
"I find that quite difficult because if I think of the church of my youth, and the priests that I knew, the feeling and overriding mood was quite forgiving really, quite humane, humorous, irreverent, even the priests.
"That's my memory of the church that I grew up with. And it seemed at some point with the change of Popes to click into a more judgemental mode and became much more obsessed with sexuality and issues related to sexual behaviour. And in that period, I drifted more and more away and Ratzinger [Pope Benedict XVI] said he wanted a 'smaller, purer' church, which I found quite terrifying actually.
"I did feel that those years were difficult ones but I have high hopes for the new Pope. A humble man with great warmth and a fantastic character. When the vote happened in Ireland recently [the gay marriage referendum] I did hope that it might be a moment for him to move the church on. I still live in hope."
He adds: "Catholic social teaching underpins my politics, we did have to read the catechism at school but it is powerful and strong and right".
He said his children attend a Catholic school because he still believes "in the values and the grounding it gives you".
Andy Burnham faces challenges from Yvette Cooper, Liz Kendall and Jeremy Corbyn for leadership of the Labour Party.
In a YouGov poll for The Times newspaper on Monday, Jeremy Corbyn was found to have the support of 53 percent of those currently eligible to vote in the leadership contest.
Andy Burnham was second, with the support of 21 percent of those eligible to vote.
