Most Rev Justin Welby and the senior Jewish leader later visited Yad Vashem on Wednesday Israel's Holocaust memorial where they also laid a wreath.
The two men toured the museum and discussed efforts since the Holocaust - in which six million Jews died - to improve relations between Jews and Christians.
Marking the latest stop on his twelve-day tour of the Middle East, Archbishop Justin acknowledged the history of anti-Semitism within the Church of England and restated his commitment to stopping it.
He said: "Within our Christian tradition there have been century upon century of these terrible, terrible hatreds.
"In the last year, we see uncovered even in England afresh that sense of anti-Semitism. Until that is expelled from our culture there will be a root, a tap root, for all racism, all discrimination, all cruelty, because of the nature of the human being in our culture.
Chief Rabbi Mirvis highlight the importance of Holocaust education, a strengthening of the Jewish identity and the building of "bridges of understanding, of tolerance, of hope, of unity and of peace".
He said: "Being here right now in Yad Vashem must prompt us to ask the question, how can we best pay tribute to the memory of the six million victims of the Shoah.
"Or better put, what would those Jewish victims be wanting us to be doing?"