The 25-year-old man (given the protective alias Adane) was reported to the human rights office in the overwhelmingly Muslim-majority eastern region of Somali.
He was encouraged to return to Islam but was arrested when he refused, despite citing his constitutional right to freedom of religion.
Intervention by the chairman of the human rights office led to his release.
On his release, Adane, who became a Christian two years ago, discovered he had been dismissed from paramilitary Liyou police force.
According to a local source who spoke with World Watch Monitor, he was then urged by the chairman to relocate because he had upset too many people.
Addis Ababa, Ethiopia's capital city.
The case is only the latest drawing attention to religious liberties in Ethiopia, a country included in Open Doors' World Watch List of countries where persecution against believers is worst.
World Watch Monitor, which highlights incidences of persecution, said: "Christians in the Muslim-majority eastern Somali region can experience hostility from family and community because of their faith, and also can be vulnerable when ethno-political tension erupts."
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.