In an interview with HuffPostUK, the singer said meetings with Cowell: "would be like speaking to a devout Catholic or something, someone anti-gay marriage and anti-contraception."
Young, 36, who is releasing his first album under a new record label, also criticised the X Factor judge's style of management: "I don't like bullies, and I don't know if there's a nurturing side to his shows. It always comes from the top.
"There definitely wasn't enough pastoral care, and I can't bear seeing people that aren't looked after.
"I didn't like it at school, and with my friends. I've never liked it, and that's my question for him."
Over the past ten years Young has appeared on the London stage in 'Cabaret', published a best-selling memoir, and most recently has been teaching performance and song writing in schools.
Will Young's album '85% Proof' is out on Monday 25 May
