With concerns that some would miss out on funding or face punishment, many charities have been wary about how explicit they can be about the faith element of their work.
Speaking at the Raising Funds for Christian Charities and Churches conference in London, Crouch acknowledged there had been problems in the past but she wanted that changed.
"I know that some of you here feel that successive governments and local authorities have wanted to applaud all the good things that you do as Christians, but discourage you from speaking openly about the faith that motivates you," she said. "I assure you that that is not the intention and I encourage you to challenge anyone in government who makes you feel you need to suppress your faith.
"I very much appreciate that, for so many of you, the inspiration to do such amazing acts of service in your communities comes directly from your Christian faith."
She also gave an assurance that no Christian organisation would face discrimination when it came to government contracts and grants.
She said: "I know that some faith organisations feel they miss out on commissioning opportunities.
"I want to make it clear today that any organisation, religious or otherwise, is eligible to apply and be considered for government funding.
"We want to make sure that funding is awarded to those who are best placed to deliver positive outcomes for our citizens. Churches have exceptional local networks that help them understand people's needs and come up with effective solutions.
"I would urge you to emphasise this unique ability when engaging with commissioners."
Welcoming her comments, Stewardship's chair of conferences, Rev David Senior, said: "The Minister has also invited responses from churches and charities and is keen to hear from those who might feel that are facing difficulties in their relationships with national or local Government, or government agencies on the ground of their faith.
"We are working with her officials to facilitate this."