The historic port, some 100 kilometres south of the capital Mogadishu, was captured in August 2012 by African Union troops after four years under Islamist control.
Somali government troops backed by African Union peacekeepers on Saturday recaptured the key port of Merka from Shebab insurgents, just one day after the Islamists swept into the city, the army and residents said.
"The Somali forces and AMISOM peacekeepers secured control of Merka again and now the situation has returned to normal," a Somali military official, Abdirisak Mohamed, told AFP by telephone from Mogadishu.
"There was brief exchange of gunfire, but the (Shebab) militants have fled."
He said "several" Shebab fighters and one Somali soldier were killed in the clashes, and "the security forces are now conducting clearing operations inside town."
A resident reached by telephone said the AU force, known by its acronym AMISOM, used tanks to recapture the port, and four civilians were killed after they were caught in the fighting.
"Four people died in our neighbourhood and two others were wounded, I can see AMISOM and Somali troops returned to the city now and they are conducting security operations", Muhidin Osman said.
Another resident, Shamso Moalim, said "Shebab fighters pulled out of the town after putting up slight resistance, the Somali forces and AMISOM are back".
"There are civilian casualties but I don't know the number", she added.
AU troops had fled the city, the state capital of Lower Shabelle, as heavily armed Shehab fighters swept in with black Islamist flags on Friday, residents and local authorities said, in one of the most dramatic reverses for the multi-national force in its nearly decade-long battle against the Shebab.
The Al Qaeda-linked Shebab said on its website that one of its leaders, Sheikh Mohamed Abu-Abdallah, addressed hundreds of people gathered at the regional government headquarters in Merka after the seizure on Friday.
"The enemy has lost and running away, they are fleeing from the Islamic regions," it quoted him as saying.
Residents said earlier Saturday that the Shebab had begun imposing Sharia law on the city.
The historic port, some 100 kilometres (60 miles) south of the capital Mogadishu, was captured in August 2012 by African Union troops after four years under Islamist control.
The Shebab are fighting to overthrow the internationally-backed government in Mogadishu, which is protected by 22,000 AU troops.
The group has lost ground since being routed from Mogadishu in 2011, but continues to stage regular shooting and suicide attacks, and in recent months has staged a series of brazen raids on AU bases.
Last month they stormed a Kenyan army base at El-Adde in southwest Somalia, in the latest incident of an AMISOM base being overrun, before retreating.
In September, Shebab fighters stormed a Ugandan AMISOM base in Janale district, southwest of Mogadishu in the Lower Shabelle region.
Kenyan President Uhuru Kenyatta, speaking at an AU summit last month, called for more "resources for the forces in Somalia so that AMISOM can have robust power on land, air and the sea."
AU troops have been hampered by a lack of air power -- including attack helicopters -- leaving their bases often isolated and supply lines vulnerable to attack by Shehab gunmen controlling surrounding rural areas.