The Rt Rev Peter Forster, argued the £55.7 billion scheme was "unnecessarily expensive" during a debate in the House of Lords on the Chancellor's recent Spring Statement.
Bishop Peter told peers: "If you want to find good things to tax, I always say start with sin. Find a new sin and tax it.
"I rather agree that HS2 is a sin.
"Not for having capacity, which I am all in favour of, but in doing it in such an unnecessarily expensive way.
"Trains go quite fast enough for me as they are.
"It could have been done far more cheaply."
Former Treasury chief Lord Macpherson of Earl's Court backed his comments.
The independent crossbencher, who was permanent secretary at the Treasury for eleven years, said: "Public investment needs to be focused on projects which yield the highest return.
"That probably means more expenditure on roads and I know I am in a minority but it also suggests that we should cancel HS2."
Meanwhile, a new report from the New Economics Foundation has found that 40 per cent of the benefits of the HS2 project would go to London and that the £56bn budget would be better spent on upgrading the existing network and smaller-scale local projects.
Bishop Peter also called for the government to do more to help tackle the "chronic lack" of low-cost and social housing.
He said: "As I look at the housing issues, it just seems to me there is something missing in the analysis in making it all joined up to put the market-based solutions together with appropriate government initiatives.
"If the market delivers 300,000 units by the mid-2020s, I shall eat my cassock."
The prime minister has said she remains "fully committed" to HS2, after a Channel 4 documentary claimed that ministers are seeking to cancel the project, or at least restrict it to Phase 1, which is the London to Birmingham section.
Estimates for building the line have increased by two-thirds since 2011 after taking account of inflation, from £33.3bn to £55.8bn.
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.
Tell us what you think about our news service, what you want and how Premier fits into your news landscape. Take our quick survey!
