Stephen Harper was speaking after a soldier was shot dead at the National War Memorial in Ottawa.
The gunman was killed in a shootout with police, although officers still don't know if there were accomplicies.
Mr Harper named the dead solider in a live TV address to his nation: "Our thoughts and prayers are with the family and friends of Cpl Nathan Cirillo of the Argyll and Sutherland Highlanders."
He added: "We also pray for the speedy recovery of the others injured in these despicable attacks.
"Fellow Canadians, we've also been reminded today of the compassionate and courageous nature of so many Canadians, like those private citizens and first responders who came to provide aid to Cpl Cirillo as he fought for his life."
Canadian officials identified the gunman who killed the soldier as Michael Zehaf-Bibeau, who government sources said was a recent convert to Islam.
Cpl Nathan Cirillo was shot at a war memorial
Thousands of people have also been offering prayers on a dedicated Facebook page for Cpl Nathan Cirillo.
One person wrote on the site: "RIP Nathan. Thank you for your service and the ultimate sacrifice. So tragic, so unnecessary. My prayers go out for you and to your family, friends and colleagues."
Prime Minister Harper added: "Well, today has been, without question, a difficult day.
"I have every confidence that Canadians will pull together with the kind of firm solidarity that has seen our country through many challenges.
"Together, we will remain vigilant against those at home or abroad who wish to harm us.
"For now, Laureen and Ben and Rachel and I join all Canadians in praying for those touched by today's attack.
"May God bless them, and keep our land glorious and free."
Prayer for Canada
Lord, keep this nation under your care,
and guide us in the way of justice and truth.
Let your way be known upon earth,
your saving health among all nations. Amen.
The Anglican Bishop of Ottawa, Rt Revd John H Chapman, has released this statement: "Like all Canadians, we are following today's news from Parliament Hill-just blocks from our Synod office with shock and trepidation.
"In this moment of huge uncertainty, building lockdowns, evacuated streets, barricaded shopping malls, and minute-by-minute updates, we draw strength and courage from our faith and pray that this event will soon be over.
"We trust in the professionalism of our police and armed forces. We pray for the soul of the soldier who lost his life at the outset of today's incident. We have faith in the resilience of those charged with countering acts of violence in our land.
"Please pray for all those at the centre of this situation and for a return to calm in our homes, hearts and streets."
Fr Dan Hawkins from St Basil's Catholic Church in the Canadian capital told Premier's News Hour there has to be more of a focus on stopping young people becoming radicalised.
He said: "Maybe we have to return to more prayer in our lives, community prayer, not individual prayer.
"Coming together as communities and looking at the problems of the world and seeing what we can do to help solve them."
He added it was important Christians played a role in stopping extremism: "The United Kingdom has been a victim of a lot of violence as well regarding these things and I think we have to understand we are a global community.
"We have a responsibility to see what Christ would like us to do in the world to bring peace."
Listen to Antony Bushfield's full interview with Fr Dan Hawkins: