Christian, Bill Haslam, said that the Bible was a 'sacred text' and should therefore not be the state's official book.
Legislators said they will try to overturn the veto.
Mr Haslam said it 'trivialised the Bible' and breaches the constitutional guarantee of freedom of religion.
He wrote to the state's top lawmakers saying that he planned to veto the legislation, according to The Guardian.
The Republican said: "My personal feeling is that this bill trivializes the Bible, which I believe is sacred text."
This veto comes one week after the state senate voted to make it Tennessee's official state book.
Bill Haslam was asked by a number of civil liberties groups not to allow the measure to become law.
If Mr Haslam had signed the bill, Tennessee would have become the first US state to declare the Bible as its official state book.
A majority in each legislative chamber would make his veto null and void.
