Sir Gerald Howarth told a Commons debate on immigration that Britain was "a Christian country".
The Conservative member said: "Our services simply cannot continue to accommodate a quarter of a million new arrivals a year, quite apart from the serious cultural issues arising from people taking advantage of our liberal society while seeking to impose their medieval ways on us."
Sir Gerald was the Parliamentary Under-Secretary of State at the Ministry of Defence as Minister for International Security Strategy from May 2010 to September 2012.
His statement has prompted an angry response from the National Secular Society.
Campaigns manager Stephen Evans said: "His remarks are completely at odds with reality in Britain today. By some measures, Sir Gerald is excluding around two-thirds of the population from his definition of who belongs in Britain.
"What are 'Christian values'? The phrase is as ambiguous as 'British values'. We're going to need to find more inclusive language if we want people to buy into notions of shared citizenship and universal values.
"While there are significant problems around integrating religious minority groups into wider society - appealing to this outdated notion of Christian nationalism is completely wrong, inaccurate and counter-productive."
At Easter the Prime Minister David Cameron reiterated that Britain was a Christian country.
