The Washington Post has reported that protesters have been gathering wherever the bishop attends to express their anger, and that the priest has been forced to sneak out of back exits and use riot police to escort him to engagements.
However the priest has said that relations are improving in the region.
His appointment in March by Pope Francis was met with a backlash - 1,300 church members, 30 diocesan priests and almost half of Chile's Parliament sent letters urging the Holy Father to rethink his decision.
The Pope has not yet spoken publicly about the case, but the Vatican released a statement defending Barros in March, that said the Congregation of Bishops "did not find any objective reasons to prelude the appointment."
On April 10th 600 people protested outside Osorno Cathedral holding black umbrellas, a colour they said represented sex abuse in the church.
Mario Vargas, 52, said: "We are beginning to energize our movement and make it more mainstream."
The bishop has celebrated Mass since his appointment but attendance is reported to have been low and the bishop used police escorts.
43-year-old parishioner Carlos Meza said: "You can feel something sour that transcends all kinds of church activities, it's just not during Masses."
Canon law experts have been reported saying Barros is likely to retain his position until his resignation.
