The Gideons donated 344 New Testament Bibles for the new Dumfries and Galloway Royal Infirmary.
The hospital said the Bibles were placed in the sanctuary within the hospital, but "repeated requests from patients for Bibles saw medical staff take responsibility for storing supplies on wards - with a small number placed in individual rooms".
However, according to The Express, the hospital decided to remove Bibles from rooms after someone complained that having them available would be giving Christianity "preferential treatment".
Rev David Robertson of the Free Church of Scotland told Premier that Christianity was actually treated poorly in the situation.
"[It's been treated as] very much a minority... as a hobby and with a great deal of disrespect," he said.
"There is a privilege here, but it's not for Christianity. It's privilege for militant secularism which wants to exclude all religion form the public sphere.
"Having a Bible in a country where the majority of people still profess to be Christian, in an area where there are many Christian churches, is not rocket science.
"It can't do any harm and it could actually do some people a great deal of good. So why would you prevent it just for ideological reasons?"
Rev Robertson said, in his experience of being in a long-term hospital patient, having a Bible readily available provides comfort for many people.
He believed the hospital shouldn't have given into the complainant.
He told Premier: "The lesson is - if you are one of these boards who are deciding these things - don't give into the bullies. Don't allow one person; don't allow a tiny minority to dictate for everyone else.
"Encourage people to show a great deal more tolerance because this is fundamentally intolerant in the name of tolerance."
NHS Dumfries and Galloway told Premier in a statement: "All 344 bibles remain available to patients, but NHS Dumfries and Galloway took the decision to halt the direct provision of bibles into social areas and single-bed rooms in order to consider the most easily accessible yet appropriate locations, consulting with the Person Centred Health and Care Committee."
Listen to Rev Robertson speaking with Premier's Tola Mbakwe here:
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.