Workplaces that prohibit the wearing of visible political, philosophical or religious signs - such as wearing headscarves or cross necklaces - do not break discrimination laws, the judges concluded.
The judgment was made as part of a ruling on a case where two Belgian women were fired from their jobs after insisting on wearing headscarves to work.
The first case was referred to the ECJ by the Belgian courts. Samira Achbita had been a receptionist for the Belgian branch of G4S.
After three years at the company she decided she wanted to start wearing a headscarf at work for religious reasons.
Achbita was fired in June 2006 for refusing to remove the scarf.
In the second case, Asma Bougnaoui, a design engineer, lost her job at an IT consultancy firm, Micropole, after a customer complained about the head covering.
After losing their cases in Belgium, they went before the ECJ but the judges upheld the ruling made by the lower courts.
The ruling said "an employer's desire to project an image of neutrality towards both its public and private sector customers is legitimate".
The ruling may also impact people of other beliefs such as Christians who wear crosses and Jewish men who wear skull caps.
But a Christian charity has criticised the decision.
Speaking on Premier News Hour, Sophia Kuby from ADF International - which defends religious freedom around the world - said the decision contradicts European Court of Human Rights rulings on freedom of religion in the workplace.
"The situation is now that we have very contradictory rulings from the ECJ on the one hand and the European Court of Human Rights on the other hand.
"The European Court of Human Rights protecting freedom of religion and manifestation of belief in the workplace. We have a really conflicting interpretation of what freedom of religion means so we will probably need another judgement to make it clearer," Kuby concluded.