Officers armed with sub-machine guns, were sent to Saint Alfege Church, in Greenwich, after a member of the public heard gunshots.
According to the Evening Standard newspaper, Revd Chris Moody was hosting a fund-raising party for Christian Aid following his Sunday service.
He said: "The police just dashed through our courtyard with their guns and ran into the park. It was extraordinary.
"We didn't have any communication with the police or prior warning at all.
"There was a crowd of people outside the church because we had just finished the service."
Robert Gray, 58, the owner of the Number Sixteen bed and breakfast, said: "We were having cream tea in the churchyard and I saw a huge flashing light of the police car drive down the road and park by my B&B.
"I was next to the vicar's wife Jill when two big men in black clothing came bursting in.
"I said to her: 'look at that' but she didn't see it and was almost knocked over by the police man with the huge machine gun."
Mr Gray added that: "Greenwich is a bit like London in the country.
"So there was a country fair going on with all these armed police bursting in like the film Hot Fuzz or Midsomer Murders.
"And nobody batted an eyelid. It was just a great example of a British stiff upper lip.
"There was no hysteria. People just carried on drinking their tea."
A Scotland Yard spokesman said that "Police were called by a member of the public at approximately 11.30am on Sunday who has reported hearing a gun shot in Trafalgar Road.
"Officers, including armed officers, attended the scene and a man was identified and arrested in Saint Alfege Park.
"The man, aged 49, has been arrested on suspicion of possession of a firearm with intent to cause fear or alarm. He remains in custody. A firearm was recovered."
