An 18-year-old holding a piece of paper with a hand drawn Islamic State symbol ran towards a priest at St Joseph's in Medan on Sunday before he was restrained by members of the congregation.
Reports suggest he was carrying an axe and a bomb but failed to detonate it properly.
Local police said the explosives in his backpack only burned and didn't fully explode.
Local chief detective Nur Fallah said the bomb was very basic.
"Somebody tried to kill the priest by pretending to attend the church service and at that time tried to explode something, like a firecracker, but the firecracker didn't explode, it only fumed," he said.
Sixty-year-old priest Albert Pandiangan received a slight injury to his left hand as a result of the attack.
Despite the suggestion the attacker claimed allegiance to Islamic State, police have not confirmed a motive. They're also investigating as to whether he was working alone.