Vickery House, 69, faces a total of eight charges relating to alleged incidents in the 1970s and 80s, with one alleged victim as young as 14.
Giving evidence in his defence, House accepted some of the incidents described had happened, but insisted that he had mistakenly thought that his advances would be welcomed.
Cross examining, prosecutor Jake Hallam said in each case he was motivated by his "own sexual gratification" and "without having any interest" in gaining the consent of his victims. House denied it.
The married father of two had twice touched a young monk friend, once at his home in Devon in 1977 and again after he moved to Berwick in East Sussex, the Old Bailey heard.
They did not discuss what happened until 1995, when the alleged victim confronted him about it in a telephone call.
House told him he was "very sorry you were so hurt" and the alleged victim agreed to keep it between them, jurors were told.
But when he was asked if it had happened to anyone else, House said: "No, this has not happened, I have never done this to anyone else."
Mr Hallam said: "You told him something that was not true", and House replied: "Yes."
Asked why he lied to his close friend, House said: "I found the phone call extremely uncomfortable. I was not happy with either myself or...I just wanted the phone call to finish. I wanted it to be over. It was too painful."
The prosecutor went on: "Is the reason you did not tell him the truth that you did not want the truth to come out?"
House replied: "Yes. That's the whole business of pressing it down and the whole consequences. It was better for me, my family and my career and the work that I was doing.
"I love my wife and children dearly and I felt this was a course that I would take that would protect them. "
He denies the charges against him.