The politician, who lost his seat in the House of Commons in May's election, said he associated good memories with that type of music.
"My parents met in a church choir in Norwich when they were 15, and singing in choirs was something they've done throughout their adult lives," he told the Gramophone website.
"So from the earliest I can remember there was music in the house, particularly choral music; I probably knew the key bits of Messiah and Zadok the Priest before I could really remember anything else."
In the wide ranging interview Balls said he had learned to play the violin from age seven.
He said: "I went to Keble College, Oxford, which then, as now, had a strong Anglican music tradition.
"During my time there, I heard the music of Howells for the first time, which has been a big thing for me ever since.
"It's not very often in music that you can say the 20th century was the century - but in Anglican cathedral music it was brilliant.
"When I was Secretary of State for Children, Schools and Families, my office was opposite Westminster Abbey, which held a service for the department. I was allowed to choose the music, so I chose Dyson in D."
He added that he started piano lessons whilst Shadow Chancellor, in January 2011.
"I've since played the Schumann live on Jeremy Vine's programme, with seven million people listening to me.
"These days when I get into a black cab, the driver won't say anything about politics - they'll turn round and say, 'How's the piano going?'"
