Some were accused of breaking building regulations, while others were said to be causing noise pollution.
According to the BBC, churches have been accused of having thousands of worshippers but not having adequate space to fit them all in and using loud public speakers to attract crowds.
Government official Justus Kangwagye told the BBC's Focus on Africa programme that churches need to meet "modest standards".
He also told Rwandan New Times newspaper that some churches left worshippers open to avoidable risks and could "cause danger to those worshipping".
The BBC said some of the churches have already been reopened after being approved by inspectors.
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.
