The American Atheists had argued it violated religious freedom protections.
The 17 foot tall structure was found in the rubble after the World Trade Centre collapsed during the terrorist attacks on September 11 2001.
Emergency services prayed and left messages at the cross in the following days and now it's held at the National September 11 Memorial & Museum.
A judge panel rejected the claims by the Atheists and decided the cross was not just a religious symbol and a "genuine historical artefact".
Judge Reena Raggi said: "The stated purpose of displaying The Cross at Ground Zero to tell the story of how some people used faith to cope with the tragedy is genuine and an objective observer would understand the purpose of the display to be secular.
"Taking hope from what he perceived to be a religious symbol, Silecchia brought the column and the cross-piece to the attention of other rescue workers, many of whom shared his reaction.
"The Cross at Ground Zero thus came to be viewed not simply as a Christian symbol, but also as a symbol of hope and healing for all persons.
" [The] stated purpose of displaying The Cross at Ground Zero to tell the story of how some people used faith to cope with the tragedy is genuine, and an objective observer would understand the purpose of the display to be secular."
The American Atheists hasn't said whether it plans to appeal or not.