Fr Edward Simpson told the jury at Birmingham Crown Court he couldn't understand why the woman would make up the claim.
The 85-year-old is alleged to have groped the child under her top and then put part of his hand into the waistband of her skirt at a house in Solihull, West Midlands.
The court heard how the priest had no clear recollection of the details of the day when the assault is alleged to have taken place.
When asked why the girl would make up the claim, Fr Simpson told jurors: "I can't understand it. It never happened."
He told the court he thought the girl and her family were friends, and said: "I would not do anything like that. I was quite surprised by the allegation."
Yesterday the court heard from the alleged victim, who said she had told a doctor in 1988 and her mother in the mid-1990s about her claims.
Fr Edward Simpson denies the charge and the trial continues.