A new book from Christian charity Jubilee Plus says negative stereotypes put out by the media and politicians are shaping our attitudes.
The Myth of the Underserving Poor challenges believers to lead the way in changing the way society views those who are homeless or on benefits.
Christian peer Lord Alton has given his support to the book.
He said: "You should never stigmatise people because of their misfortune.
"There is a danger that because of the headlines of 'scroungers' and all these things that instead of looking at people as they are, each of them in the image of God, we instead see them as a liability."
As part of the research, the authors looked at a large number of news stories to analyse how people living in poverty were portrayed via websites and newspapers.
They found that too often they were negative stories and that the voices of those living below the breadline were not being heard.
Co-author Natalie Williams wants Christians to realise they can be influenced by the media but that it is possible to stand against it.
She said: "As Christians, it's something we can do in our churches, workplaces, our families.
"To really just tell a different story, which is one that says we don't look to someone in poverty and judge them; we look to God and ask what he's like and how I can best represent him."
More details can be found at www.jubilee-plus.org.