Joining the city's mayor Sadiq Khan and Home Secretary Amber Rudd, Most Rev Justin Welby laid flowers during the vigil in Potters Fields Park near London's City Hall on the southern bank of the River Thames.
Writing on Twitter afterwards, he said: "Moving vigil led superbly by [the] Mayor of London, London united in calm dignified memory. Great civic leadership, let's pray on & persist in hope."
Archbishop Justin's support came after Donald Trump appeared to criticise Mr Khan after the mayor sought to reassure Londoners about a greater police presence in the capital.
The US President wrote: "At least 7 dead and 48 wounded in terror attack and Mayor of London says there is "no reason to be alarmed!"
A minute's silence was also held, a short distance away from Saturday night's attack in which seven people were killed and 48 others injured.
Addressing those gathered, Mr Khan said: "As a proud and patriotic British Muslim I say this you do not commit these disgusting acts in my name.
"Your perverse ideology has nothing to do with the true values of Islam. You will never succeed in dividing our city."
Metropolitan Police Commissioner Cressida Dick and shadow home secretary Diane Abbott also joined the vigil, supported by representatives from various faith groups.
John Loughrey, a 63 year old bell ringer from Streatham who held a Union flag during the service, said: "I am here to show respect to the people who were murdered and the injured.
"The whole world shows them respect. We will win and continue normal life."