A letter from the department cited passages from Leviticus, Matthew and Exodus, and said the Book of Revelation is "filled with imagery of revenge, destruction, death and violence".
The refusal notice, sent to an Iranian national, added: "These examples are inconsistent with your claim that you converted to Christianity after discovering it is a 'peaceful' religion."
The Bishop of Durham, Rt Rev Paul Butler, said the case shows the Home Office must radically change its understanding of all religious beliefs.
He said: "I am extremely concerned that a Government department could determine the future of another human being based on such a profound misunderstanding of the texts and practices of faith communities.
"To use extracts from the Book of Revelation to argue that Christianity is a violent religion is like arguing that a Government report on the impact of climate change is advocating drought and flooding."
Excerpt from a home office reasons for refusal letter for a convert to Christianity. I've seen a lot over the years, but even I was genuinely shocked to read this unbelievably offensive diatribe being used to justify a refusal of asylum. pic.twitter.com/R1wA1HMNwH
— Nathan Stevens (@nathestevens) March 19, 2019
Mr Butler, who leads for the bishops in the House of Lords on matters relating to immigration, asylum and refugees, welcomed the Home Office's acknowledgement that the letter was not in accordance with its policy.
But he added: "The fact that these comments were made at all suggests that the problem goes deeper than a lack of religious literacy among individual civil servants and indicates that the management structures and ethos of the Home Office, when dealing with cases with a religious dimension, need serious overhaul."
The letter was flagged up on Twitter by Nathan Stevens, an immigration caseworker representing the asylum seeker.
Mr Stevens, who said his client is appealing against the decision, wrote: "I've seen a lot over the years, but even I was genuinely shocked to read this unbelievably offensive diatribe being used to justify a refusal of asylum."
Stephen Evans, chief executive of the National Secular Society, said it was "wholly inappropriate for the Home Office to use theological justifications for refusing asylum applications".
I'd better not give that as it easily links to my client, but here's a redacted version of the front page, with our firm's name and the usual headers pic.twitter.com/LZH0zxTN34
— Nathan Stevens (@nathestevens) March 20, 2019
He added: "Decisions on the merits of an asylum appeal should be based on an assessment of the facts at hand - and not on the state's interpretation of any given religion. It's not the role of the Home Office to play theologian."
A Home Office spokeswoman said: "This letter is not in accordance with our policy approach to claims based on religious persecution, including conversions to a particular faith."
She added that the department works closely with key partners to improve policy guidance and training provided to asylum decision-makers, "so that we approach claims involving religious conversion in the appropriate way".
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.
Tell us what you think about our news service, what you want and how Premier fits into your news landscape. Take our quick survey!