The European Court of Human Rights (ECHR) in Strasbourg rejected an appeal by Richard Williamson, saying his case was "manifestly ill-founded".
According to the German international broadcaster, Deutsche Welle, the 78-year-old was also told his punishment - a fine of 1,800 Euros - was "very lenient".
Williamson made the comments during an interview with Swedish broadcasters in Germany in 2008.
He had argued the conviction violated his right to freedom of expression, and that Swedish law should have been applied to his case. Holocaust denial is not a crime in Sweden but it is illegal in Germany.
Williamson was originally fined 12,000 Euros but the figure was reduced following a legal challenge.
The interview took place while he was a bishop in the ultra-traditionalist Society of Saint Pius X, a group which has been estranged from the Vatican for a number of years.
Williamson has since been expelled by the Society. He is also currently ex-communicated from the Vatican.
Six million Jews were systematically murdered by the Nazis in the 20th century.
Pictures: Auschwitz concentration camps
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.