Speaking at Prime Minister's Questions, David Cameron said there was a "strong case for change" as he maintained plans to devolve the power to councils.
If approved the change to the law would mean each area could decide how long large retailers can open for on a Sunday.
Currently shops in England and Wales with over 280 sq m (3,000 sq ft) floorspace are limited to opening for six hours on Sundays.
Ministers want to give councils the choice on their own local laws which they say would be worth up to £1.4 billion a year to the economy while ensuring high streets remain the "heartbeat of our communities".
Premier's Antony Bushfield was in the House of Commons:
"The first question of PMQs went to Susan Elan Jones who, to some cheers from both sides of the chamber, questioned proposals to extend Sunday Trading.
"As the PM replied an MP jeered at him asking 'what about families?'
"David Cameron replied that families are annoyed at not being able to go shopping when they want."
Susan Elan Jones, the Labour MP for Clwyd South, asked the Prime Minister if he agreed that current laws were a "workable compromise that most people seemed satisfied with most of the time".
David Cameron replied that he did not agree: "I think there's a strong case for change but I think it's a change that we should allow local authorities to decide.
"Let me just give... two examples of where I don't think the current situation works.
"You've got these restrictions on opening hours for many stores that families would like to shop in and you have to go to these stores and you have to walk around for hours before you're actually allowed to buy anything.
"The second point I'd make is that you can shop on Sunday, you can shop anywhere on Sunday, you can do it on the internet.
"I think it's time to modernise our approach to give families more choice and to help create jobs at the same time."
The PM's already faced harsh criticism of the proposal from the Church of England which said it would have a "dramatic effect on family life".
It has also been reported a number of Mr Cameron's own MPs plan to vote against the party and reject a change in the law.
Communities minister Brandon Lewis said: "This Government is determined to devolve powers previously held in Whitehall to local people.
"We have already taken a range of measures to boost the Great British high street and now we are giving local areas another tool to encourage shoppers to the town centre and get shops to grow and thrive."