Rt Rev Nick Baines warned that some secularists "have a problem" with religion being talked about and it was making many believers too scared to talk about Christ.
Some Christians feel "picked on" by "intolerant" liberalism in the UK, he warned.
His comments come just a day after Prime Minister Theresa May said it is important people can talk freely about their faith following claims some Christians dare not speak about Christmas.
Bishop Nick told the Press Association: "Clearly there are some Christians who are concerned about whether they are free to talk about their faith in a respectful and appropriate way in the workplace.
"Equally, there are plenty of people who are not Christians who think that Christians shouldn't, or think there is an issue around it. I call it religious illiteracy.
"There are people who have been disciplined or threatened with discipline for talking about their faith even though they have been asked about it. Someone makes a complaint and says they have been inappropriate. This is absurd."
He added: "There is, amongst some Christians - this isn't universal - a sense of being a little picked on or beleaguered.
"I think if you claim to be open-minded and liberal, why are you so frightened of religious expression?"
The bishop said that relations between different religious communities are good and he suggested it was some secularists that were creating issues.
This can end up in some people trying to strip the religion out of Christmas, he said.
"The joke with all of this is that most people of other faiths have absolutely no problem with anyone talking about their faith. It's the secularists that have a problem," he said.
"It is a Christian festival. Are they going to tell Muslims they have got to strip Islam out of Eid? It's just ludicrous."
And he warned that Britain has become a liberal society where some people believe you are only acceptable if you accept liberalism.
"And liberalism can become very intolerant of anything that doesn't fit its own parameters," he added.
