Caviezel recounted the phone conversation during an interview in front of students at Christian school Liberty University on Wednesday.
He said he received a call from his agent telling him that Mel Gibson wanted to speak with him.
"[Mel told me], 'If you take this part [of Jesus], you may not work in this town again,'" Caviezel said.
"I felt fear come into me. That's one thing about our faith. When that [fear] happens, that's not Jesus."
Caviezel said although he was nervous about what taking the role would mean for the rest of his career, he ignored his fears and accepted the offer to play Jesus.
He added: "I said, 'My initials are J.C., and I'm 33 years old,' [Mel said], 'You're freaking me out,' and he hung up the phone."
Caviezel also shared about the moment that he felt God told him to be an actor. He said it happened at the age of 19.
"I felt this love that I had never felt before, this peace I never felt before," Caviezel said.
"I heard God say, 'I'd like you be an actor' ... I did not know where to go. I had a call to be an actor, but it was going to take Him to teach me and bring me to the right teachers, the right films to do."
Caviezel has been promoting his new film Paul, Apostle of Christ, in which he plays Luke. The film, which premiers on 23rd March, takes viewers on the journey of Paul's ministry.
Caviezel said one of the main reasons he's a part of the movie is because of the similarities between the violent persecution churches faced in the days of Paul and the persecution Christians around the world today .
He encouraged Christians to stand up for their faith.
"You're going to be given a power and an opportunity. You're here for a reason," Caviezel said. "Jesus said, 'If they persecute you, know they persecuted me first.'"
Stay up to date with the latest news stories from a Christian perspective. Sign up to our daily newsletter and receive more stories like this straight to your inbox every morning.