Spokesman Rob Carr's been speaking as reports yesterday suggested up to 20 Labour MPs wanted Ed Miliband to step down, for fears the party couldn't win next year's general election under his leadership.
A spokesman for the Labour leader called these reports "ludicrous".
Last week, two members of Miliband's front bench - Shadow Home Secretary Yvette Cooper and Shadow Health Secretary Andy Burnham - were forced to deny they were making plans in case he stepped down.
Rob Carr, from Christians on the Left, told Premier's News Hour: "He's fine. He's not going anywhere. There's a lot of speculation and a lot rumour, and I think that's all it is - it is literally a few malcontents. Ed's not going anywhere.
"The fact is, the Tories have lost two MPs to UKIP, the deficit's up, downward mobility's on the rise, 22 Tory MPs are sending letters against David Cameron - and we're all taking about Ed Miliband in the media.
"In five years time people aren't going to be asking themselves 'how was Ed's oratory?' They're going to be asking: 'am I better off?'
"We tend to be very superficial and wanting slick PR people.
"What we should be talking about is politics and policies, rather than PR and soap opera."
According to The Independent newspaper, a recent poll found only a third of Labour voters thought he was fit to be Prime Minister.
The latest YouGov poll placed Labour in the lead by just 1%, at 33% versus the Conservatives' 32%.
