Defense chiefs are “embarrassed” over a sex scandal involving Secret Service agents and military service members as it distracted from President Barack Obama’s visit to Colombia, the US military’s top officer said.

His comments came as the Pentagon revealed the extent of the case was wider than initially reported, with more than five military service members implicated.

The incident served as a distraction that overshadowed “a very important” presidential visit, General Martin Dempsey, chairman of the US Joint Chiefs of Staff, told a news conference.

“We are embarrassed,” said Dempsey, referring to fellow chiefs of the armed forces.

“So we let the boss down, because nobody’s talking about when went on in Colombia other than this incident.”

A military investigation would hold accountable any personnel that “violated orders or policies or laws,” Dempsey said.

The incident saw 11 Secret Service and at least five military personnel pulled from their security duties in Cartagena, Colombia at the Summit of the Americas.

The US Secret Service, which has sent the men back to the United States, is investigating claims they brought prostitutes to their hotel rooms in Cartagena late Wednesday and had a dispute over payment with one of the women.

“Whether our forces are in Colombia or any other country, or here in this country, we expect them to abide by the highest standard of behavior. That’s a requirement,” Defense Secretary Leon Panetta told the same press conference. “And for that reason, we are conducting a full investigation into this matter.”

A Pentagon spokesman said the service members who are under investigation were not handling security for the US president but were performing important work in support of the Secret Service.

“We believe that there maybe more than five involved in this incident. I don’t have specific numbers. We’re going to look at the facts wherever they may go,” spokesman George Little told reporters.

The personnel were from more than one branch of the military, and some of them were at the same hotel as other Secret Service agents alleged to have brought prostitutes to their rooms, he said.