On the UN Day for Older Persons, their letter says many old people suffer from other's outdated attitudes that prevent them living life to the full.
18 councillors have also signed the letter, as well as council leaders and people who work in health and social care.
They state: "Even seemingly casual ageism is deeply damaging both to individuals and to our communities, with recent research suggesting that a quarter of people over 50 have felt discriminated against whilst doing everyday tasks or accessing services. Even seemingly positive phrases like 'you look young for your age' can reinforce damaging stereotypes.
"If we are to make real progress towards age equality, we must all work to root out ageism in our own communities and organisations. We will challenge ageism wherever we see it, whether it's in public or in private, and ensure that the voices of older people are at the heart of our local decision-making.
"As well as making this commitment, we're asking everyone to join us: we can all re-think our own prejudices, think more carefully about the impact our words can have on others, and be part of a movement to end ageism once and for all."
Louise Morse from Pilgrim's Friends Society, and author of What's Age Got To Do With It? told Premier this is a problem within churches as well.
"In churches older people are not often engaged in church meetings or in decisions that are made about the church and many churches will have a pastor for younger people, a youth worker, but they'll have nothing to actualise the ideas of older people themselves."
"If you think about it, God takes the whole of our lives to hone in us the attributes that older people have."
She added that the church needs to recognise that older people have often have a whole lifetime growing a relationship with God, with God growing them to influence others.
Morse cited phrases to avoid, such as "'she's very good for her age', or 'she's evergreen' or 'Isn't it wonderful he can do that at that age?'"
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.