Revd Keith Hebden said the safety of his two children is at risk by the needles in his garden at St Mark's Church in Mansfield.
He's proposed building a mesh fence to stop them being thrown in but the council says it would not be "in keeping with the neighbourhood".
Mansfield District Council had instead recommended Victorian railings but Revd Hebden says that wouldn't solve the problem.
It's claimed several needles a week are found outside the church.
Revd Hebden spoke to Premier. He said: "We'd just like some common sense really.
"I want someone from the council to come out and have a look because I think if they saw they'd make sense of it and wouldn't just tick a box and say no."
A council spokesman said: "It was considered the proposed fence would be visually obtrusive and harmful to the character and appearance of this conservation area."