File photo: Relatives weep after identifying the body of Johnny Mutinda Musango, 48, one of the victims of al-Shabaab's attack on the Westgate mall.

For two years, Kenyans have been bracing themselves for the barbarism we have just witnessed here, says Ian Birrell.

Ever since their army invaded neighbouring Somalia following the killing and kidnapping of aid workers and tourists on their terrain, people have worried about the near-inevitable carnage to come their way.

There were attacks on churches, small-scale bombings. But many suspected the ultimate target was Westgate, with its wealthy shoppers from the newly rich elites, and armies of expats. Now their worst fears have been realised.

The details are chilling: people out shopping on a Saturday morning lined up, given a religious test and shot dead if they failed. Most were Kenyan, but families from nations from China to Peru are united in grief with Kenyans. Among the British victims was an architect who had designed a clinic for HIV patients in Kenya free of charge, and his heavily pregnant wife, a malaria specialist working for the Gates Foundation. Good people, slaughtered by gun-toting gangsters.

Now the heat will be turned on al-Shabaab, the Somali-based group responsible for the savagery. Already you can hear talk on the radio blurring them with al-Qaeda. There will be calls for reprisals, demands to step up the Western-backed war on terror already being ramped up in the Horn of Africa. We must hope cool heads prevail.

Somalia is the ultimate failed state, an ungovernable hellhole plagued with poverty and riven with historic rivalries between hundreds of nomadic clans and sub-clans. The last person to rule with any authority was a military dictator, Siad Barre, whose overthrow in 1991 sparked the chaos that still engulfs the country.

A million Somalis have been killed since then, twice as many displaced and much of Somalia reduced to rubble. Half a million Somali exiles live in Kenya. But while many of the country’s problems were self-induced, with warlords growing rich as they ripped apart their own country, Somalia’s problems were worsened by bungled interventions from outside.

Go back eight years, and a semblance of normality had returned to the country. Western-backed warlords had been defeated and the Union of Islamic Courts, a coalition of Islamic conservatives, enforced the rule of law. There was security in Mogadishu. The possibility of genuine peace hovered on the horizon.

But a united, stable and Islamic country was the last thing Ethiopia wanted on its doorstep. So it invaded, persuading Britain and the US to back their key ally in the region. The incursion was disastrous, with Somalia spiralling back out of control, while grotesque human rights abuses boosted the militant cause. The biggest beneficiary was the security wing of the Islamic Courts movement, al-Shabaab, which soon had control of much of Mogadishu and swathes of the country.

The increasingly hardline group murdered Westerners, stoned adulterers, banned bras and yanked gold teeth out of mouths. While condemning criminality, they raked off millions from maritime kidnapping and the illegal charcoal trade. But when cross-border raids by criminal gangs into Kenya became too much, culminating in the murder of British publishing executive David Tebbutt at a beach cottage, Kenya sent 2 000 troops over the border. The recent narrative has been of normalisation. There have been stories about the stuttering rebirth of a shattered state, of al-Shabaab riven by divisions and on the run. Unfortunately, the optimism and talk of a “new era” has been overdone.

The mall slayings show al-Shabaab to be a lethal force that in classic insurgency style melted out of sight in cities while regrouping under a new hardline leader. It was significant the hardy souls at Médecins Sans Frontières quit Somalia last month after 22 years due to “extreme attacks”.

We should not be deluded by cheap comparisons with al-Qaeda. Yes, the terrorists are Islamic militants, as shown by their sparing of Muslims in Nairobi, and last year they pledged allegiance to al-Qaeda. But the group is primarily a nationalist response to interference in Somalia, muddled up in the complexity of clan rivalry.

Their emergence was to a point down to outside interference. The latest invading army is accused of human rights abuses while the UN says Kenyan officers aided the export of illegal charcoal that earns millions for militias. And the West has backed some clans and warlords over others.

Somali fears that Kenya and Ethiopia want to carve up their country were strengthened last month with the signing of a deal recognising Jubaland as an independent entity. Its leader has been denounced by the UN and linked to al-Shabaab, yet the deal was welcomed.

Yet it is Somaliland that shows the clearest path to peace here. Destroyed in the 1991 civil war, it has emerged as one of the most inspirational stories of rising Africa, with a functioning democracy, self-determination and comparatively free expression. The reason: it was not recognised by the outside world when it declared independence. This meant no outside intervention, little aid and clan elders were left to negotiate their way forward. This former British protectorate offers cause for hope. The world must do all it can to ensure the bloodshed in the mall is never repeated, by aiding the re-creation of a functioning Somali state. The lesson from history is for outsiders to tread warily if they are not to make matters worse and, however unpalatable, embrace all parties in any solution.