Mr Anderson, the former deputy party chair of the Conservative Party, was suspended at the weekend after he refused to apologise for claiming "Islamists" had achieved "control" over London and that Mr Khan had "given our capital city away to his mates".

On Monday, the now independent MP for Ashfield stood by his position and refused to apologise - while also describing his words as "clumsy".

Politics latest: Tories label MP's suspension 'final nail in the party's coffin'

Speaking on BBC Radio York, the prime minister denied the Tory party has "Islamophobic tendencies" and said: "Lee's comments weren't acceptable, they were wrong. That's why he's had the whip suspended."

He added: "Words matter, especially in the current environment where tensions are running high. I think it's incumbent on all of us to choose them carefully."

But speaking to reporters later on Monday, the prime minister repeatedly refused to call Mr Anderson's remarks Islamophobic.

'Clearly racist'

Mr Khan said he was "bewildered why Rishi Sunak and the cabinet are refusing to call this out".

The London mayor added: "They should say what the problem is. The problem is you have a senior Conservative saying things that are clearly racist, anti-Muslim and Islamophobic.

"That is leading to an environment where anti-Muslim crime is spiralling and what they are doing is pouring petrol on the flames of Islamophobia."

Labour leader Sir Keir Starmer also attacked Mr Sunak for being "too weak" on the issue.

He told broadcasters: "This is really basic. Islamophobia is something which should be called out by every political leader and the prime minister isn't calling it out because he's too weak.

"I don't think anybody can make excuses for this prime minister and say it's all too difficult. It's very straightforward.

"He lacks the backbone to call this out for what it is, because he's leading a divided party, a chaotic party, and it's no wonder people have just had enough of this after 14 years and desperately want change."

Mr Sunak's statement to local radio was the first time he has directly addressed the comments made by Mr Anderson.

The MP told GB News last week: "I don't actually believe that the Islamists have got control of our country, but what I do believe is they've got control of Khan and they've got control of London...

"He's actually given our capital city away to his mates."

Mr Anderson's comments sparked condemnation from across the political divide, including from Tory peer Baroness Warsi, who said she was "really disturbed by where the Conservative Party has gone" and that "anti-Muslim racism is being used as an electoral campaign tool".

'My words were clumsy'

In a fresh statement published by GB News on Monday, Mr Anderson said his words were "clumsy" and "borne out of sheer frustration at what is happening to our beautiful capital city".

However, he doubled down on his refusal to apologise, saying: "If you are wrong, apologising is not a sign of weakness but a sign of strength.

"But when you think you are right you should never apologise because to do so would be a sign of weakness."

He said: "Seeing the words 'From the river to the sea' on [the] Elizabeth Tower made me feel sick to the pit of my stomach."

Read more from Sky News:
US airman dies after setting himself ablaze in 'genocide' protest
Everton Football Club's 10-point deduction reduced
Cruise ship blocked from docking over 'health risks'

Repeating his opposition to the mayor, Mr Anderson said: "Khan has stood by and allowed our police to turn a blind eye to the disgusting scenes around parliament.

"It is not my intention to upset anyone, I believe in free speech and have 100% respect for people of all backgrounds.

"The vast majority of Muslims are not Islamists in the same way the vast majority of Christians are not conservatives or socialists.

"My words may have been clumsy but my words were borne out of sheer frustration at what is happening to our beautiful capital city.

"We've got to get Khan out at the elections in May."

He added: "Sadiq Khan is failing London across the board. On transport with ULEZ and the strikes which he pledged to end. On knife crime and violence against women and girls. The list is endless.

"He's more interested in virtue signalling to his trendy lefty mates by renaming train lines and street signs at a cost of millions of pounds rather than in delivering for our capital city which should be the greatest city on Earth."

Mr Anderson's statement, published by GB News, was drawn up on Saturday, shortly after he made his provocative comments to the channel.

? Click to subscribe to the Sky News Daily wherever you get your podcasts?

However, it was not released because of Mr Anderson's refusal to apologise to the London mayor as requested by Tory chief whip Simon Hart.

Mr Sunak said it was "incumbent on all of us, especially those elected to parliament, not to inflame our debates in a way that's harmful to others".

"Words matter, especially in the current environment where tensions are running high and I think it's incumbent on all of us to choose them carefully, he said.

Elsewhere in his interview with local radio, Mr Sunak rejected suggestions his party had a problem with Islamophobia.

The prime minister said it was "not a fair characterisation at all" to argue he had taken a tough stance on antisemitism but had overlooked Islamophobia in his party - saying racism or prejudice "any kind" was "completely unacceptable" and "not British".

Earlier today, a cabinet minister appeared to leave the door open to Mr Anderson regaining the Tory whip, which allows him to sit as a Conservative MP in the Commons.

'Reflection needed'

Asked what Mr Anderson needed to say to be welcomed back, Transport Secretary Mark Harper told Kay Burley on Sky News he should "reflect on what he said" and "retract those comments and apologise".

"He's contributed a lot in the past. I'd like to see him be able to contribute to the Conservative Party in the future," he said.

Read more:
More than 40% of public 'think Labour still has antisemitism problem'
Anderson row blunts Sunak's moral authority on racism
Speaker retracts offer for emergency ceasefire debate

While a number of senior Tories have criticised Mr Anderson for his comments, Mr Harper was the second cabinet minister to refuse to say whether the remarks were "racist".

Yesterday, Mr Dowden backed the decision to suspend the Ashfield MP but refused to say whether he was racist.

WhatsApp messages leaked to Sky News reveal some Tory MPs regard Mr Anderson's suspension as a mistake.

The WhatsApp forum is called the "109 group" of Tory MPs elected in 2019.

Jill Mortimer, the Conservative MP for Hartlepool, shared messages from constituents saying they would not vote Tory again and that "Lee Anderson's suspension is the final nail in the party's coffin".