Organisations who have been campaigning against the change are urging caution after a Westminster source confirmed to a Sunday newspaper the plans were to be altered.
Initially ministers wanted every institution caring for children for more than six hours per week to register with the education inspectorate Ofsted.
That would mean Sunday schools, Bible clubs and even church days out could be forced to declare their activities to the state.
After a meeting with the Archbishop of Canterbury Justin Welby, government officials have backed down and sources say registration will not be necessary, but it has not appeased concerned Christians.
James Mildred from the charity Care urged "a lot of caution" and warned there was "absolutely zero guarantee that they will not be inspected".
The government has always insisted churches will not be targeted under the new rules, to be included in the counter extremism bill, which targets Islamic hate preaching.
"This is state interference on a grotesque level," warned Mildred.
He said the U-turn changed nothing and Ofsted would still be able to show up at Sunday school demanding access.
"It's fine if you're going to say let's not have a register of Sunday schools but they have already said that Ofsted will still be able to go an inspect Sunday schools if they receive a tip off that undesirable teaching, that's the government's own term, is being promoted.
"The government's problem is that they've got dodgy definitions both on British values and undesirable teaching, they're hugely subjective contexts.
"How are you going to make sure that's legally watertight? The government still has not answered that."
The Home Office said it would comment in due course.
