Justin Humphreys, from the Churches Child Protection Advisory Service (CCPAS), was speaking after Ben Emmerson QC (above) and Elizabeth Prochaska both announced they were quitting.
Ben Emmerson QC was suspended on Wednesday over concerns about his leadership. He has denied that his resignation is about disagreements with the fourth and current chairperson Professor Alexis Jay (pictured below).
Prof Jay said she had accepted Ben Emmerson QC's decision to step down from the post of senior counsel after two years.
There have been calls for the inquiry to be broken up into smaller strands after the departure of the two lawyers, as well as the departure of three previous chairperson.
It was planned to take five years, however is likely to take more and will look at how the Church and other public figures and bodies dealt with child abuse. It's costed £100m so far and hasn't heard from a single witness yet.
Prime Minister Theresa May has backed the inquiry, saying it retained both her and Home Secretary Amber Rudd's confidence.
Speaking to Premier Christian Radio Justin Humphreys, from CCPAS, said: "I think this is a really sad set of events.
"The biggest thing we have to face here is not whether Ben Emmerson or Elizabeth Prochaska can be replaced, but the impact on public confidence - I would imagine that's got to be at an all-time low right now.
"They are in a state of flux and chaos but in some ways it's not surprising - we have a new chair, a fourth chair... that was going to cause issues from the very beginning.
"It's very, very disappointing... those institutions who're under the spotlight have spent an inordinate amount of time, thousands and thousands of hours, millions and millions of documents, in trying to move this thing forward."
Listen to Premier's Aaron James speaking to Justin Humphreys:
