Workers at St Aloysius' Catholic College in Garnethill discovered the damage late on Sunday night.
Spray paint had been used on the side of the wall at the school. It said "the famine is over it's time to go home".
It's believed this refers to the immigration of Irish Catholics to the west of Scotland during a famine in the mid-1800s.
The famine left many thousands starving and has become a source of taunting against Catholics.
A police spokeswoman said: "The incident is being treated as a hate crime and enquiries are ongoing.
"It is thought to be of a sectarian nature."
A school spokeswoman said the vandalism has now been removed.
St Aloysius is the only Jesuit private school in Scotland and fees at the school can range between £2,000 and almost £11,000 per year.
The spokeswoman said: "Graffiti was discovered on the Dalhousie Street side of the main building on the morning of Monday, August 22, following the weekend.
"The matter was reported to police and enquiries are ongoing. The graffiti has now been removed."