Iraqi forces on Tuesday recaptured from the Islamic State group the first three districts of militant bastion Tal Afar as the Pentagon chief visited Baghdad in a show of support.

The United Nations said thousands of civilians had fled Tal Afar in the two days since the start of the broad offensive backed by the US-led coalition fighting IS.

In remarks in Baghdad after meeting Prime Minister Haider al-Abadi, US Defence Secretary Jim Mattis said the militants were “on the run”. “Cities have been liberated, people freed from ISIS, from Daesh,” Mattis said, using alternative names for IS.

The militants had not been able “to stand up to our team in combat, and they have not retaken one inch of ground that they lost”, he said.

Iraqi troops backed by a US-led international coalition routed IS in Mosul in July after a gruelling nine-month fight for Iraq’s second city. They launched an offensive Sunday to recapture Tal Afar, once a key IS supply hub between Mosul – around 70 kilometres (45 miles) further east – and the Syrian border to the west.

IS fighters inside Tal Afar, estimated to number around 1,000, responded with artillery fire Tuesday as Iraqi forces massed outside the city. Army, police and units of the Hashed al-Shaabi paramilitary coalition later took “full control” of the Al-Kifah, Al-Nur and al-Askari districts of Tal Afar, the Hashed said.

Iraqi forces had encircled the city despite what Hashed spokesman Ahmed al-Assadi described as “intense” fighting. He said the battle for the city would probably last weeks, in contrast to the months-long battle for Mosul.

The International Organization for Migration said “thousands of civilians” had fled Tal Afar since the offensive began.

Since Friday, more than 3,000 people had arrived at two IOM emergency sites, many with nothing but the clothes they were wearing, the UN agency said.

The UN’s refugee agency UNHCR said it had received some 9,000 people at the Hamman Al Alil transit centre over the past week, and it was preparing to accommodate nearly 30,000 more.

Mattis declined to make any predictions about the battle.

“ISIS’s days are certainly numbered, but it’s not over yet and it’s not going to be over anytime soon,” he said.

Iraqi forces had “fought like the dickens in Mosul, (it) cost them over 6,000 wounded, somewhere over 1,200 killed,” he said.

Yet that comeback restored the confidence of the Iraqi forces after their shock loss of Mosul and swathes of northern Iraq to IS in 2014.

Mattis stressed that retaking Mosul would not have happened “without… Abadi’s steady hand”, but was also due in part to extensive US support.

Mattis said his discussions in Iraq would focus on the way ahead, including how to keep the country from again politically fragmenting or falling further under Iran’s influence.

“Secretary Mattis is going to be very much focused on a pathway for the United States to continue to have a residual force in Iraq to continue to train Iraqi security forces” and avoid a successor from IS emerging, said Nicholas Heras, Middle East Security Fellow at the Center for a New American Security in Washington.

A key issue is a plan for an independence referendum on September 25 in Iraqi Kurdistan, where Mattis met on Tuesday afternoon with the autonomous region’s president Massud Barzani.

The poll is strongly opposed by the US, which believes it could undermine Abadi and distract from the fight against IS.

White House envoy to the anti-IS coalition, Brett McGurk, said the battle for Tal Afar was “going well” but that a referendum on Kurdish independence would be “potentially catastrophic to the counter-ISIS campaign”.

“It’s not just the United States; every member of our coalition believes that now is not the time to hold this referendum,” he said.

Before leaving Baghdad, Mattis stressed the need to “stay focused like a laser beam on the defeat of ISIS and to let nothing distract us”.

Mattis, who is on a five-day trip to Jordan, Iraq, Turkey and Ukraine, said he would also talk about reconstruction and resettlement of hundreds of thousands of Iraqis driven from their homes and towns by the fighting, especially in Mosul.

“It’s not going to happen overnight. It’s going to be a heavy lift for them going forward.”

But Heras said Mattis, whom he said has earned firm trust among Iraqis, needs to help Abadi further build his power as a moderate for the post-war period, with elections looming for next year.

“That will be a political pickle that Mattis will have to work Abadi through,” he said.