Bishop Rogelio Ricardo Livieres Plano has been criticised for appointing a priest who was eventually sacked following sexual abuse allegations.
However, Bishop Rogelio's diocese strongly supports him. In a statement, it said that under his leadership the number of priests rose from 79 to 140 and annual baptisms had risen from less than 10,000 per year in 2004, to more than 20,000 per year in 2013.
The pontiff released a statement through Vatican Radio, saying: "The grave decision taken by the Holy See, under the weight of serious pastoral concerns, is for the greater good and unity of the Church of Ciudad del Este and episcopal communion in Paraguay.
"The Holy Father...calls for the clergy and the entire people of God of Ciudad del Este to accept the Holy See's decision with the spirit of obedience, docility and an open heart, guided by faith."
The Vatican has appointed Msgr Ricardo Jorge Valenzuela RÃos to lead Bishop Rogelio's diocese until a permanent replacement is found.