Family, friends and fans of the three-time World Heavyweight Champion sung hymns and prayed at King Solomon Missionary Baptist Church in Louisville, Kentucky - his hometown.
Muhammad Ali's brother Rahman Ali was present with his wife Caroline as various tributes were paid and a collection was taken.
Their father, Cassius Clay Sr, was a painter and a member of King Solomon for several years before his death and painted a mural of Jesus' baptism which still hangs behind the preacher's pulpit.
Muhammad Ali still occasionally accompanied his father Cassius to church after his conversion to Islam.
Revd Wanda McIntyre, who presided over the service, said that fact reminded her that he believed above all in living life with tolerance and an open heart.
Mr Ali once said: "Rivers, lakes, ponds, streams, oceans all have different names, but they all contain water.
"So do religions have different names, and they all contain truth, expressed in different forms and times. It doesn't matter if you're a Muslim, a Christian, or a Jew.
"When you believe in God, you should believe that all people are part of one family."
Muhammad Ali's body has now arrived in Louisville, Kentucky, before a public memorial is held on Thursday.
