The dead included the assailant and the policeman he stabbed, while the other two victims were among the pedestrians hit by the car as it tore along Westminster Bridge, which is right next to parliament.
"We've declared this as a terrorist incident and the counter-terrorism command are carrying out a full-scale investigation into the events today," Mark Rowley, Britain's most senior counter-terrorism officer, told reporters.
"The attack started when a car was driven over Westminster Bridge, hitting and injuring a number of members of the public, also including three police officers on their way back from a commendation ceremony.
"A car then crashed near to parliament and at least one man, armed with a knife continued the attack and tried to enter parliament."
Reuters reporters who were inside parliament at the time heard loud bangs and shortly afterwards saw the knifeman and the stabbed policeman lying on the ground in a courtyard just outside, within the gated perimeter of the parliamentary estate.
A Reuters photographer said he saw at least a dozen people injured on the bridge. His photographs showed people lying on the ground, some of them bleeding heavily and one under a bus. A woman was pulled alive, but with serious injuries, from the Thames, the Port of London Authority said.
The circumstances of her fall into the river were unclear. Three French schoolchildren aged 15 or 16 were among those injured in the attack, French officials said. The attack took place on the first anniversary of attacks by Islamist militants that killed 32 people in Brussels.
"I just saw a car go out of control and just go into pedestrians on the bridge," eyewitness Bernadette Kerrigan told Sky News. She was on a tour bus on the bridge at the time.
"As we were going across the bridge, we saw people lying on the floor, they were obviously injured. I saw about 10 people maybe. And then the emergency services started to arrive. Everyone was just running everywhere."
The House of Commons, which was in session at the time, was immediately suspended and lawmakers were asked to stay inside. Prime Minister Theresa May was safe after the incident, a spokesman for her office said.
He declined to say where May was when the attack took place. Journalist Quentin Letts of the Daily Mail newspaper told LBC radio that he had witnessed the stabbing of the policeman and the shooting of the assailant from his office in the parliament building.
"He (the assailant) ran in through the open gates ... He set about one of the policemen with what looked like a stick," Letts said.
"The policeman fell over on the ground and it was quite horrible to watch and then having done that, he disengaged and ran towards the House of Commons entrance used by MPs (members of parliament) and got about 20 yards or so when two plain-clothed guys with guns shot him."
Britain is on its second-highest alert level of "severe" meaning an attack by militants is considered highly likely. In May 2013, two British Islamists stabbed to death soldier Lee Rigby on a street in southeast London.
In July 2005, four British Islamists killed 52 commuters and themselves in suicide bombings on the British capital's transport system in what was London's worst peacetime attack.