Sir Bob Russell, the town's High Steward, was speaking after the council decided to take down a wooden sculpture depicting Jesus on the cross and a soldier stabbing a child with a spear - a reference to King Herod's order for all infants in Bethlehem to be killed when Jesus was born - because it could upset children.
The sculpture, called Crucifixion of Mankind, was originally created in 1961 by Connor Barrett and given to the library in Colchester.
According to the Mail on Sunday, Mr Russell said: "The excuse that some people do not like it is not a reason for its removal.
"This is censorship, which is bad enough anywhere but not in a public library."
Andrea Williams from Christian Concern told the paper: "The decision to remove this piece of art which depicts the Christian story demonstrates ignorance of history and our nation's Christian cultural heritage."
Crucifixion of Mankind is currently in storage and Essex County Council has pledged to re-hang the sculpture in another place where the public can see it.
