He explained that's why he joined the new Catholics for Labour group that was launched at the Labour Party Conference this week.
He said the group aims to apply Catholic social teaching to public policy and meets for mass on Wednesdays.
"We have joint meetings and discussion on some of the most unbelievably contentious issues, like pro-life issues, to see if we can somehow arrive at some sort of way of living the Gospel in parliament," he said.
The MP for Ealing North said the Church's teaching on the poor, refugees and war needs to be brought to the forefront in the Labour party.
He said: "Christ's message has never ever been more relevant than it is today.
"There's never been a time where this country and world has been crying out for that message more than it is at the moment.
"For too long we've just said, 'oh this is a Christian country'.
"I'm perfectly fine with having a multi-faith [parliament] but there has to be a strong voice for Christ within parliament otherwise frankly we're just wasting our time."
Pound also addressed concerns of the group being too sectarian by using "Catholics" and not "Christians" for the group's name.
He told Premier that wasn't the intention.
"Catholics are one of the few groups that have been totally discriminated within parliament," he said.
"It was only in the 1830s that Catholics were even allowed into parliament. That hasn't happened to Jews, it's not happened to non-conformists, it's not happened to Anglicans.
"It's not just that sort of historical significance, but also because Catholic social teaching is absolutely crucial in the way we need to go forward."
When asked what the feedback on the group is like, he said it's been mixed.
He told Premier 150 people attended the group's mass in Brighton on the first day of the Labour Conference, but it has been the subject of some snide jokes within the party.
However, Pound sees it as an opportunity to stand up for his faith.
He said: "In some ways it's sort of a good thing.
"If people are sort of subliminally saying 'those Christians are all barking mad', now we're challenging them saying 'here we are, this is what I do, this is what I believe in, let's talk about it, let's see where you differ' and more often than not they'll see the message applies to so many of us and people resonate with that."
Founders of the group include Mike Kane, Jon Cruddas, Keith Vaz, David Crausby, Emma Lewell-Buck and Conor McGinn.
