Shneur Odze says he found the New Testament Bible masked as a Jewish Bible in his Manchester synagogue. He believes the book was left on purpose by proselytising religious sect with the purpose of converting Jews.
His tweet, which has since been deleted, read: "Grateful to whoever put a missionary bible amongst our synagogue's books. Was wondering what I'd burn my Chametz with #LikeWeWouldn'tNotice"
Odze told The Daily Mail what annoyed him the most was that the Bible was placed in the synagogue during Passover, a holy period on the Jewish calendar.
He explained what he did was in line with Orthodox tradition and his only option. He said giving the Bible away 'would be to compound the fraud', and to toss it out would 'disrespect what is still a religious tract'.
He later said his decision to post the photo was 'unwise.'
A UKIP spokesman told the Mail: "That while we understand the act, it was ill-advised to put a picture of it on social media which was also provocative and likely to be misunderstood."