The British actor, who has been a long-standing supporter of gay marriage, addressed the topic on BBC's Newsnight.
When asked who has the greater right - the bakers or the customer, he said: "It was not because this was a gay couple that they objected; it was not because they were going to be celebrating some kind of marriage or agreement between them. It was the actual words on the cake that they objected to, because they found them offensive.
"And I would support their right to say 'no this is personally offensive to my beliefs, I will not do it'. But I feel bad for them, that it cost them £600 or whatever."
His comments come just weeks after a judge ruled the Christian run bakery had discriminated against the customer on grounds of sexuality.
Back in May 2014, Ashers declined an order placed at its Belfast store. A gay rights activist had requested a cake with the campaign slogan, 'Support Gay Marriage'.
Owners Colin and Karen McArthur refused to make the cake claiming it went against their firmly-held Christian beliefs.
Action was subsequently taken by the Northern Ireland Equality Commission.
The McArthurs are now appealing the verdict. In a statement released last week they said: "We continue to insist that we have done nothing wrong as we have discriminated against no individual but rather acted according to what the Bible teaches regarding marriage.
"As many other people have already noted, Christian beliefs seem to have been trampled over in this judgment and we believe this only has negative effects for our society."
A date is yet to be set for their appeal.