According to the Diocese of East Anglia, a man walked into St Mary's Catholic Church on 27thMarch, lit a couple of small fires and attempted to burn a sacred icon taken off the church's altar.
There was also a picture of a satanic figure left in the icon's place on the holy altar.
The attack has been condemned by Parish Priest Fr Anthony Nwankwo, who pledged that the incident would not stop the Diocese leaving its churches across the region open as places of prayer and sanctuary.
Fr Anthony said: "I was shocked and saddened when I discovered what had happened. When I looked at the altar I saw this picture of something satanic. My immediate thought was to get some holy water and use it with prayer to nullify what the person had done.
"It is a clear attack on the church and our religion which is not acceptable. If people do not like what we do here they can just stay away. Such an attack on our church can never succeed.
"Our parishioners are shocked and sad about what has happened but I was very encouraged by the number of people who came to Mass this morning in support."
He said a similar incident happened four months ago when someone left a satanic picture at the foot of a at the back of the church. He also said two weeks ago someone came in and stole a microphone we use on the altar.
"We will not close the church but to use the altar and sanctuary we need to go through a service to reconsecrate the area. As part of that we will also ask for forgiveness for the person who did this," Fr Anthony said.
Brian Lafferty, who has been a parishioner for 33 years and is on the parish council, said he was appalled: "I have never seen anything like it before and it is quite disturbing really, especially in this day and age. I was totally shocked and disgusted when I saw it and there is no room for it in today's society.
"If you want to celebrate like that and worship other gods there are other places you can go. Here, we worship the one God."
Police are investigating the incident as a hate crime and have CCTV footage a suspect.
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.