Nicholas Chamberlain said he is in a long-term relationship with his male partner but he is following Church rules and they do not sleep together.
He made the announcement to the Guardian newspaper on Friday after an unnamed Sunday paper threatened to publish a story about his sexuality.
"It was not my decision to make a big thing about coming out," he said.
"People know I'm gay, but it's not the first thing I'd say to anyone. Sexuality is part of who I am, but it's my ministry that I want to focus on."
He said the church was aware of his sexual orientation when he was appointed in November last year.
"I was myself. Those making the appointment knew about my sexual identity," he added.
Bishop Nicholas said he adheres to church guidelines.
Most Rev Justin Welby, the Archbishop of Canterbury, said his sexuality is "completely irrelevant".
"I am and have been fully aware of Bishop Nick's long-term, committed relationship," he added.
"His appointment as Bishop of Grantham was made on the basis of his skills and calling to serve the church in the diocese of Lincoln.
"He lives within the bishops' guidelines and his sexuality is completely irrelevant to his office."
A Church of England spokesman said: "The Church has said for some time that it would be unjust to exclude from consideration for the episcopate anyone seeking to live fully in conformity with the Church's teaching on sexual ethics or other areas of personal life and discipline.
"Whilst Bishop Nick's appointment is notable in the gifts and talents that he brings to the episcopate, it is wholly consistent and unexceptional in other regards given the testing of that call by those responsible for the selection process in each case."
Rev Peter Ould comments on issues of sexuality in the Church and told Premier it was a "non story".
"Yes, the bishop has come out and said that he's gay but he's also said that he's celibate and is abiding by the Church's rules.
"There's no indication that he ever hasn't been celibate so really this isn't actually a big deal."
He added that the Sunday newspaper's threats to "out" Bishop Nicholas could be part of a wider agenda.
"It's part of politics," he said, "there are agendas that different wings in the Church have and it could be part of that".
