The Labour Party has pointed to a series of controversial past tweets from Rev Richard Cameron, who posted that "homosexual behaviour is a sin" and "allowing children to change their gender is wicked".
In September he tweeted: "Christ has the power to help and change anyone. Obviously many gays hate this because want to carry on in their perversion."
He also shared a series of controversial views on Islam, describing terrorism as "a problem Islam needs to deal with", a full face veil as "oppressive and unBritish" and the prophet Muhammad as "a violent man".
Labour revealed the minister's tweets after Rev Cameron, the minister at Scotstoun Parish Church, branded Jeremy Corbyn a "terrorist sympathiser" on Wednesday.
As the Labour leader was entering the Heart of Scotstoun community centre in Glasgow, he stopped to tell reporters about a scarf he was wearing, which had been given to him by representatives of the Who Cares Scotland charity.
But he was interrupted by Rev Cameron who shouted: "I thought you'd be wearing your Islamic jihad scarf."
He added: "Who's going to be the first terrorist invited to the House of Commons when you're prime minister?"
Footage of the encounter attracted much attention online.
Green MSP Ross Greer, a member of the Church of Scotland, tweeted: "Bigoted nonsense directed at Corbyn today doesn't represent the @churchscotland. The bigoted tweets surfacing certainly don't.
"Do I wish the Kirk were robust in rejecting fringe figures like this guy? Yes. We must do so much better to say this isn't who we are & isn't acceptable."
A Church of Scotland spokeswoman said: "There has been significant concern raised today about the comments made by Rev Richard Cameron and his social media use.
"At this stage all we can say is that there is a formal complaints process and that any complaints we receive in relation to this matter will be taken seriously and addressed.
"We do deplore any comments which are Islamophobic or homophobic. The Church of Scotland works closely with our Islamic neighbours and the General Assembly has taken a strong position and said formally that we decry homophobia in any form."
Rev Cameron had also said to Mr Corbyn: "Do you think that the man who is going to be prime minister of this country should be a terrorist sympathiser, Mr Corbyn?"
The Labour leader did not react to Mr Cameron and was ushered into the community centre by Scottish Labour leader Richard Leonard.
Rev Cameron then accused Mr Corbyn of "running away".
The Church of Scotland rebuked Mr Cameron for the way he approached the situation, saying: "Whilst we may occasionally robustly challenge policy issues with which we disagree, we always intend to do that in a way that is polite and measured and allows for reasoned debate."
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.