The battles erupted a day before a deadline set by Libya's interim National Transitional Council (NTC) for pro-Gaddafi outposts to surrender or face onslaughts.
NTC officials said the outbreaks of fighting meant the ceasefire had effectively been scrapped. That could pave the way for some of the final battles of a six-month civil war.
Fighters besieging Bani Walid went in a day ahead of the deadline to protect civilians and were locked in street-to-street fighting with Gaddafi forces, an NTC official near the town said.
Battles also began outside Sirte in response to barrages of rockets fired by pro-Gaddafi forces.
The two towns, along with the remote desert outpost of Sabha, are the main areas still in control of Gaddafi loyalists since NTC fighters overran his Tripoli headquarters last month. The NTC said it had also dispatched hundreds of fighters deep into the desert, headed towards Sabha where it was advancing.
NTC official Abdallah Kanshil said fighters had entered Bani Walid from the north, east and south and had advanced as close as 2 km (1.5 miles) from a market in the centre.
Anti-Gaddafi forces believe some of Gaddafi's sons, and possibly even the fallen leader himself, are holed up in the town, 150 km (95 miles) southeast of the capital Tripoli.
NTC spokesman Ahmed Bani said battles also had erupted close to a river near Sirte after pro-Gaddafi forces fired rockets at NTC forces laying siege to it. The coastal city was just a village when Gaddafi was born there 69 years ago, but has grown into a power base, lavished with cash under his rule.
"They attacked our forces so we had to answer them. There was shelling and there was no way to avoid that," Bani said. Asked if the fighting meant Saturday's ceasefire deadline no longer applied, he said: "Yes, it was breakable."
AVOIDING "FULL SCALE WAR"
Kanshil estimated loyalist forces in Bani Walid at around 600 men, more than previous estimates.
"People are very afraid, that is why we have to go in," he said. Troops were still hoping to avert "full-scale war" and had entered Bani Walid without orders from central NTC command.
One NTC fighter had been killed so far and four wounded in Bani Walid while three Gaddafi fighters were killed and three wounded, Kanshil said. Seven Gaddafi loyalists were captured.
Ambulances streamed back and forth with casualties from near Bani Walid, and NTC fighters grabbed crates of rocket-propelled grenades and mortars and raced to the front. Smoke rose from the frontline and NATO warplanes roared overhead.
Officials at the NTC's Tripoli military headquarters told Reuters that between 100 and 150 vehicles had set off into the desert for Sabha on Friday. More would leave in convoy later in the day and into Saturday.
"There are people going from Jadu, from Zintan, from Tripoli, there is a mixture," NTC fighter Abu Aziz Selem told Reuters as he rushed out of the capital with a Kalashnikov assault rifle slung over each of his shoulders.
Families trickled out of Bani Walid before the fighting intensified, belongings crammed into their cars.
"I'm taking my family away from war," said Khalid Ahmouda, stopping his car briefly to speak to Reuters.
His veiled wife, Oum Abdurahman, leaned from a window, holding her baby son. "There's no power, no food, no water. Many people want to leave but have no fuel for their cars and Gaddafi forces are preventing people from leaving," she said.
"They fire in the air to terrorise people. Today we managed to leave," she said, adding her brother-in-law was among 11 people killed in a May 25 crackdown on anti-Gaddafi protests.
NTC officials at a checkpoint 30 km from Bani Walid said Gaddafi fighters had been captured. Reuters witnesses saw some men driven away with their hands tied behind them, as well as two bodies, said to be Gaddafi fighters, in a pick-up truck.
Muammar Gaddafi's location is still a mystery. In a defiant message broadcast on Thursday, though, he said he was still in Libya to lead the fight against what he called "rats" and "stray dogs" who had taken over the capital.
Interpol issued arrest warrants for him, his son Saif al-Islam and intelligence chief Abdullah al-Senussi, all wanted by the International Criminal Court (ICC) for war crimes.
Four top officials, including Gaddafi's air force commander and a general in charge of his forces in the south, were among a new group of Libyans who have fled to neighbouring Niger.
Niger said it would respect its commitments to the Hague-based international court if Gaddafi or his sons arrived, but had accepted the lower-ranked officials as a humanitarian case.
Hisham Buhagiar, military official coordinating the NTC's hunt for Gaddafi, said he had indications the former strongman was in or near the town of Birak, some 700 km south of Tripoli. NATO forces had bombed the area late on Thursday, he said.
"We thought he was in Birak. I saw NATO heavily bombed Birak. They're following the same trail," he said.
"We always talk," he said of his relations with NATO, which since March has been conducting air strikes on Gaddafi forces and military assets to enforce a United Nations resolution protecting Libyan civilians.
Buhagiar said he would move to the city of Sabha, close to Birak, within two days to continue the hunt. Gaddafi is said to be relying on loyal tribes for protection in Libya's south, where the NTC has little influence.
Sabha, deep in the desert, has been cut off from the rest of the country since shortly after the fall of Tripo
Libya rebels move on last Gaddafi bastions