She was shot and stabbed as she left her constituency surgery in West Yorkshire last week. Thomas Mair, 52, has been remanded in custody as a terrorism suspect.
MPs queued outside St Margaret's Church in the grounds of Westminster Abbey as hundreds of Mrs Cox's colleagues filled the pews.
Inside, they heard readings from Speaker John Bercow and Baroness Kinnock, a close friend of Mrs Cox.
It was led by the rector of St Margaret's Church, Canon Andrew Tremlett, and Commons chaplain Rose Hudson-Wilkin.
Opening the service, Canon Tremlett said: "We wish to God we were not here to mourn Jo Cox today; that Thursday's terrible events had never happened.
"But as people of good faith, of unshakeable determination, and unswerving commitment to serve the people of this United Kingdom, we come here to offer our prayers and petitions, the pain of our hearts, and the cries of our souls, as we remember and pray for Jo Cox, Member of Parliament for Batley and Spen, for Brendan, and for their children."
Mr Bercow read a passage from Deuteronomy 15: 7-11 which urges people to "open your hand to the poor and needy neighbour in our land", reflecting Mrs Cox's passionate humanitarian campaign work.
The choir sang Psalm 130 before Baroness Kinnock of Holyhead read Philippians 4: 4-9.
"Do not be anxious about anything, but in every situation, by prayer and petition, with thanksgiving, present your requests to God," it says.
Before the church, service MPs gathered in the House of Commons to pay tribute to Mrs Cox.
They met in "heartbreaking sadness" and "heartfelt solidarity" to pay tribute to Jo Cox, Speaker John Bercow said as several MPs broke down in tears.
Several of her Labour colleagues, including Heidi Alexander, Stephen Kinnock and Carolyn Harris, were crying before Mr Bercow began the tributes.
Jeremy Corbyn told the House Mrs Cox "believed in loving her neighbour's neighbour" and thought that "every life counted equally".
Mrs Cox's husband, two young children and family were sat in the public gallery watching as the Prime Minister joined those paying tribute to Mrs Cox.
As MPs, who all wore the white rose of Mrs Cox's native Yorkshire, wiped away tears, David Cameron spoke of the sadness the country felt at the killing of the "loving, determined, passionate and progressive politician".