Allieu Bangura told Premier people in West Africa are being discriminated against by the international community because of fears the illness will spread.
The World Health Organization (WHO) says 4,447 people have died from the virus which is mainly affecting Sierra Leone, Liberia and Guinea.
Heathrow and Gatwick airports and Eurostar terminals are now screening arrivals from the region to check if they could have Ebola.
Mr Bangura said: "People are being stigmatised, people are being discriminated against because they are coming from an Ebola affected country.
"For Heaven sake, not everyone in the country has been affected."
Asked how he would feel if he flew from Sierra Leone to London and was stopped for questioning he replied: "I certainly would not feel comfortable.
"Unfortunately though the decision is not within my control to determine what the UK government does or not."
He said some people were unable to leave the country because some flights had been cancelled.
"We don't have anywhere to go and we feel, ourselves, as a country, as being isolated, as being stigmatised," he said.
Meanwhile the Prime Minister has pledged that the Government "will do everything we can to keep this country safe" from the outbreak of Ebola.
Speaking at PMQs David Cameron told MPs: "There'll be another Cobra meeting chaired today by the Foreign Secretary, I'll be chairing one tomorrow. We're looking at all these issues about where people are arriving to and co-operating properly with all the devolved authorities.
"It is worth stressing there are no direct flights from Liberia, Sierra Leone or Guinea into the United Kingdom, so we're talking about people who come here indirectly, and that is why it's so important we put in place the screening processes starting at Heathrow but to be rolled out more as the days go by.
"I'm absolutely convinced that we will do everything we can to keep this country safe."
Listen to Premier's Antony Bushfield's full interview with Allieu Bangura here: