Police on duty in the area next to the Houses of Parliament spotted the man "acting suspiciously" in an "area known for anti-social behaviour", Scotland Yard said in a statement.
Officers stopped the man who "was found in possession of a large knife" in a black case near the cathedral in Francis Street.
Scotland Yard said the incident was not terror-related.
Op Bobcat officers stopped & searched a male acting suspiciously near Westminster Cathedral. Found in possession of large knife #knifecrime pic.twitter.com/FBhppJipmU
— Westminster Police (@MPSWestminster) August 31, 2017
A Metropolitan Police spokesman said: "At approximately 11:45hrs on Thursday, 31 August, officers on routine patrol in the Victoria Street area of Westminster attempted to speak with a man who was seen acting suspiciously in an area known for anti-social behaviour."
"He was arrested and taken to a central London police station where he remains in custody.
"Enquiries continue."
The officers were working as part of Operation Bobcat, which sees plain-clothes police tackle knife crime and robbery in Westminster.
