Most Revd Justin Welby said he was determined to listen to both sides of the debate but admitted to "moments" of frustration about the issue.
"In human terms it looks impossible and many people say that we shouldn't be bothering to try," he said.
"The reality is that when I listen to people I know that I'm listening to people for whom not just the issue of sexuality but the whole way in which the Church lives and exists and reaches out to people, what it looks like to be a holy church, is something on which they feel passionately and are deeply, deeply, deeply disagreeing.
"And if you love them you listen carefully, and in obedience to the work of the spirit of Christ, we have to seek to love one another."
Appearing on BBC Radio 4's Desert Island Discs Archbishop Justin refused to discuss his previous stance on gay marriage.
He said: "I'm really not going to answer the question very well because we're now into conversations within the Church, both globally and locally, and I think if I weigh in at this stage it's inappropriate.
"It's something as you go around the communion, and having visited all the provinces I am very aware of this, that is seen by many as an absolutely central understanding of obedience to Christ, in both directions, either in favour or against."
Gay marriage was introduced to England and Wales in October. The Church of England was against the move.
He also spoke on the programme about his mother, Jane Gillian, taking him for tea with Winston Churchill when he was very young. She was the former prime minister's private secretary.
"I remember a very, very old man. And he cried, I don't know why. And because he cried, I cried. And then we sat and had tea," he said.
"I've talked to her about it since, and she said 'Well, he cried quite a lot'."
The archbishop also described growing up with an alcoholic father and said one Christmas he was left alone all day while his father lay in bed.
He said that was 'very painful'.
"I did think this was a pretty bad day. I think I went out once or twice but everything was closed. I didn't know really what to do with myself all day. I suspect I watched telly a bit.
"It was a grim day. That was a grim, grim day."
The Archbishop chose Jerry And The Pacemakers' You'll Never Walk Alone as a tribute to Liverpool, where he was Dean for nearly four years.
"I remember being at Anfield on the anniversary of the Hillsborough disaster and this being sung, and then going back to the cathedral and finding Everton and Liverpool scarves all over the cathedral, and this amazing, wonderful, poor, battered, thrusting, lively, humorous city coming together."
Alongside the Bible, he choose Gibbon's weighty historical tome The Decline And Fall Of The Roman Empire as his book, and the complete series of the West Wing as a luxury.