Speaking in the House of Lords, Lord Pearson of Rannoch asked the government to respond to the potential risk of the changes which come into effect in April.
He'd suggested some might be offended if they claim that Jesus is God and denies the "divinity of Allah".
Home Office minister Lord Ahmed replied stating that reading the Bible in public would never become a criminal offence. He said: "Reading texts from the Holy Bible in public or preaching the divinity of Christ is not a crime and never will be in this country.
"The government's counter extremism strategy makes it clear that the government will protect free speech and the right to profess, practice and propagate any religion or indeed none."
Lord Pearson went on to praise Prime Minister David Cameron for stating in his Christmas message that Jesus was the son of God, suggesting "that won't have gone down all that well with the Islamists".
Last year a Christian street preacher was been found guilty of a public order offence while preaching on the streets of Taunton.
Michael Overd from Creech St Michael, in Taunton, claimed he was simply reading the Bible but was found guilty of making offensive remarks about homosexuality through a loudspeaker.
He later successfully appealed the decision.