Franco Mulakkal was formally arrested after police questioned him for three days in the southern Indian city of Kochi.
The nun with the Missionaries of Jesus order filed a police complaint in June accusing the bishop of raping her from 2014 to 2016.
Mulakkal is the bishop of the northern Indian city of Jalandhar.
He denied the accusations, saying the nun was angry because he had ordered her to be disciplined.
Police officer Hari Sankar told The Press Trust of India news agency that Mulakkal will appear in court.
Dozens of nuns, activists and advocates have protested outside a court demanding his prosecution.
In her police complaint, the nun alleged that Mulakkal raped her at a guest house in May 2014 and later sexually exploited her on several occasions. She said she had to approach police because church authorities did not act on her complaints.
Earlier this month, she wrote to the Vatican's representative in New Delhi, alleging that the bishop was using "political and money power" to "bury" the case and seeking his removal.
On Thursday, the Vatican relieved Mulakkal "temporarily" of all pastoral responsibilities as the bishop of the Jalandhar diocese, the Press Trust reported.
Mr Sankar told reporters on Friday that the police would now investigate charges of rape, illegal confinement, unnatural sex and intimidation. A judge would decide whether Mulakkal would be formally charged.
India has been shaken by a series of sexual assaults since 2012, when a young woman was fatally gang-raped on a New Delhi bus.
While the government has passed a series of laws increasing punishment for rape of an adult to 20 years in prison, it is rare for more than a few weeks to pass without another brutal sexual assault being reported.
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.
