Labour branded the decision not to kick him out of the party “shocking” but a spokesman for Government chief whip Mark Spencer said he wanted to await the conclusion of the police inquiry.
According to reports, a Conservative MP was arrested after a former parliamentary aide accused him of rape, sexual assault and coercive control.
The ex-minister, who has not been named, is alleged to have assaulted a woman and forced her to have sex, the Sunday Times and Mail on Sunday said.
While pending a police investigation for a sexual crime, I think it is only right that the whip is withdrawn
The abuse is alleged to have taken place during a relationship.
Shadow safeguarding minister Jess Phillips criticised the Tories for failing to suspend the whip from the MP, which would effectively eject him from the party.
The Labour MP told Times Radio it sent a “terrible message” that senior figures were still able to secure “protection” with their Westminster status.
She added: “While pending a police investigation for a sexual crime, I think it is only right that the whip is withdrawn.”
But a spokesman for Mr Spencer said: “These are serious allegations and it is right that they are investigated fully.
“The whip has not been suspended. This decision will be reviewed once the police investigation has been concluded.”
Housing Secretary Robert Jenrick said he did not know the name of the MP involved, but added that he was “confident” the party was taking the matter seriously.
“I’ve read the accusations in the newspapers this morning but I don’t know more than you do in that respect,” the Cabinet minister told Times Radio.
“These are very serious allegations and they should be investigated. They’re in the hands of the police, so I don’t think I can comment any more than that.”
Asked whether he wanted to see the Tory whip removed from the MP, Mr Jenrick said: “We need to take allegations of this nature very seriously and I’m confident that the party is, and will do.”
The Metropolitan Police said they received allegations on Friday of sexual offences and assault relating to four incidents at addresses in London, including in Westminster, between July last year and January this year.
A spokesman for the force said: “The Met has launched an investigation into the allegations.”
A man was arrested on Saturday on suspicion of rape and taken into custody at an east London police station, the force added. He has been released on bail to a date in mid-August.
The papers said the allegations were raised with Mr Spencer, and the complainant also spoke to him, but no action was taken.
The PA news agency understands Mr Spencer advised her to make a formal complaint to authorities, who would be able to investigate the claims.
A Conservative Party spokeswoman said: “We take all allegations of this nature extremely seriously.
“As this matter is now in the hands of the police it would be inappropriate to comment further.”
The investigation comes days after former Tory MP Charlie Elphicke was convicted at Southwark Crown Court in a separate case of sexually assaulting two women, one of whom was a parliamentary worker.
Elphicke had the whip removed in November 2017 when the allegations first surfaced, but it was controversially restored before a confidence vote in Theresa May’s leadership the following year.
However, when Elphicke was formally charged in July 2019, the party suspended him again on the same day.
