Firefighters tackled a blaze at Park Congregational Church on Thursday night, which a spokesman from the local council called "a scene of utter devastation".
The fire completely destroyed Park Congregational Church's roof, with the rest of the building suffering "substantial" damage according to police.
Some locals had to be evacuated for the whole night, sleeping in a nearby care home.
The arrested boys are 15 or 16 years of age.
Eifion Bowen, from Carmarthenshire council, told the BBC: "It's a scene of devastation really. The roof is completely gone.
"The spire stands alone but clearly the loss of the roof will affect the structure itself and that's what we're looking at at the moment. There was pockets of hot spots there which were being monitored by the fire service and dealt with appropriately."
