Commenting on plans outlined by Rochdale Borough Council leader Richard Farnell, Hebron Pentecostal Church said it's "time to raise the bar", regarding standards of behaviour.
Talking on Premier's News Hour, senior pastor David Corke said: "Morals and standards have become very low over the last 30 years in this nation but I believe it's time to raise the bar, so I'm glad the subject's being addressed."
Begging, loitering, nuisance car parking and loud music would also be outlawed, while children could be barred from going into the town centre after 11pm.
Human rights organisation Liberty said the plans would be difficult to put into practise and society's most vulnerable people could be unfairly affected.
Laraten Caten, Liberty's legal officer, told the Manchester Evening News: "These proposals would unjustifiably curb the rights and freedoms of Rochdale residents."
Mr Farnell was quoted by the Guardian as responding: "With all the horrific human rights abuses happening around the world right now, I would have thought Liberty had bigger things to worry about.
"We are clamping down on a small minority of antisocial ne'er-do-wells who drunkenly shout and swear and harangue shoppers in our town centre."
David Corke from Hebron Pentecostal went on to say: "I question whether foul of abusive language is real speech. My English teacher ...taught me that swearing was not good English.
"Therefore, I would like to support the council leader ...in his endeavour to stamp out this bad behaviour in our brilliant town."
Click here to listen to Premier's Alex Williams speaking with David Corke: