They have since been charged following the incident at Boghall Parish Church on Elizabeth Drive in Bathgate last Thursday.
Sergeant John Fleming from Bathgate Police Station said: "Hate crime in any form will not be tolerated within West Lothian and whenever such incidents occur we will investigate them thoroughly to identify those responsible.
"If you have been a victim of a hate crime then please contact us on 101, make an anonymous report to the charity Crimestoppers on 0800 555 111 or visit the Police Scotland website to learn more about our third-party reporting mechanism."
The boys - one of whom is 13-years-old, two are 14 and a fourth is 15 - will all now be reported to the Scottish Children's Reporter Administration (SCRA).
Often, youngsters are brought to the attention of a Children's Reporter at the SCRA because of concern surrounding an aspect of their lives.
A Children's Reporter can decide whether a child should be the subject of a compulsory supervision order.