Lord Michael Bates has been a minister of state since May 2015 and served in a junior role in the Home Office during the coalition government.
He has quit his job to walk across countries in South American to raise money for Unicef, the children's charity, as well as to raise awareness of the UN Olympic Truce.
Lord Bates said: "The Olympic Truce states that countries should take initiatives for peace and reconciliation during the period of the Olympic and Paralympic Games."
After he leaves at the end of the month, Lord Keen of Elie will act as a Lords spokesperson for the Home Office in addition to his other responsibilities as Advocate General for Scotland, according to Downing Street.
Prime Minister David Cameron has paid tribute to Lord Bates' time in his role, saying he made an "outstanding contribution".
He went on: "I am immensely grateful for his efforts in steering important pieces of legislation through the Lords, including the Modern Slavery Bill, the Psychoactive Substances Bill and, most recently, the Immigration Bill. He has been very effective in representing the Home Office in the Lords.
"I wish him well in the commendable undertaking for which he vacates his post, a solo walk across South America in support of the UN Olympic Truce and to raise funds for Unicef, a cause close to his heart."
Home Secretary Theresa May also commented, saying: "Lord Bates has been a highly valued member of the ministerial team at the Home Office and played a key role in steering hugely important legislation through the House of Lords. I would like to thank him for his contribution to the Home Office and wish him the very best on his fundraising walk across South America."
Lord Bates will leave Buenos Aires, where the 2018 Youth Olympics will be held, and aims to cover around 2,000 miles to the Olympic and Paralympic host city Rio de Janeiro in four-and-a-half months.
He aims to arrive in the Brazilian city in late August, as the Olympic closing ceremony takes place.
The 54-year-old has already walked 3,000 miles from Olympia in Greece to London for the Truce in 2012, and hopes to raise £250,000 for Unicef this year.
He told the Press Association: "I'm personally very loyal to the Prime Minister and really appreciate the opportunity to serve in the Home Office.
"These are obviously difficult times - my final act as a minister was to make a statement in the House of Lords about the Brussels attacks - but I feel very passionately about the Olympic Truce and rather than talking about peace, I thought I would walk for peace."
