The boss of ChatGPT-maker OpenAI says people should have more faith in AI firms to keep the tech safe, amid rising public concerns about the risks.

“The world should trust that we are going to do the right thing because it’s the right thing and we feel the magnitude of this,” Sam Altman said on Tuesday during a conference in San Francisco.

He accepted people were justified in being concerned about AI, as the capability of the tools has progressed rapidly.

But critics say firms cannot be trusted to self-regulate, with left-wing Senator Bernie Sanders and Steve Bannon, a former adviser of US President Donald Trump, among those calling for greater governance of AI.

Trump himself has criticised calls for more guardrails on the fast-moving technology, calling fears about AI’s safety a “hoax”.

He said the only “guardrails” needed for AI was a “strong and smart” president.

Altman’s comments on Tuesday came during an appearance at an annual conference hosted by software firm Salesforce.

“It doesn’t take as much imagination as it used to for [us] to imagine how this could go wrong,” he said. “I think the world is right to be afraid of this.”

But AI bosses themselves have also admitted it may be difficult to regulate a technology which is changing so quickly.

In an interview with Axios published on 3 September, external, Altman said he had expected Congress to create a “basic framework” for advanced AI after his 2023 testimony, but that has yet to happen.

He suggested lawmakers have struggled to keep pace with the technology and to regulate it without “slow[ing] down innovation”.

New safety legislation on AI is unlikely to emerge from the US Congress anytime soon, despite the fact lawmakers returned from their summer break on Monday.

There is a lack of consensus within both US parties on how any regulation would happen – and the House is set to enter recess again at the end of the week ahead of the midterm elections in early November.

Altman’s comments on Tuesday were the first time he had spoken publicly since a post went viral last week by a researcher who quit the AI firm Anthropic. The researcher claimed AI could “kill all humans” by the end of the decade if left unchecked.

A handful of other AI executives and experts responded by saying they agreed with the assessment, though they didn’t explain how they came to their conclusions or how exactly AI could accomplish such a thing.

After Altman’s comments, Mark Zuckerberg wrote on X, external that every AI firm had the ability and the incentive “to take its own actions” towards safety.

“Any lab that doesn’t focus on alignment will fall behind,” the Meta chief executive wrote.

“Labs face significant liability if their models cause harm, so they have a strong incentive to prevent this as well.”

Alignment is a field which aims to build human ethical ideas and principles into AI. In other words, it aims to keep it on track with what humans value.

Some political figures have pushed back against the view that AI firms should be able to self-regulate.

Bernie Sanders said decisions about AI had been left to “a handful of the richest people in the world”, while former White House strategist Steve Bannon said the public could not trust tech oligarchs to regulate themselves.

Nvidia boss Jensen Huang said at the same conference on Tuesday that AI firms should decide if new versions of the technology should be released – and not be managed by outside forces.

“We don’t need new laws or regulations,” he said, adding there should not be a “false choice” between the speed of innovation and the safety of AI products.

Nvidia is the biggest company in the world by valuation, with its profits having boomed as a result of rampant demand for its hardware.

Huang also argued safety is an “engineering problem” and companies should “take a pause” if they lose confidence in a product.

The recent claims from inside the industry have intensified scrutiny of AI development, prompting Anthropic chief executive Dario Amodei to call for the pace of AI development to slow, external and urge governments to regulate the industry.

The post was applauded by Altman, Google’s DeepMind co-founder Demis Hassabis and Elon Musk.

By Tuesday, however, more AI leaders were voicing support for self-regulation rather than government involvement.

Altman said AI companies were capable of regulating themselves.

“We will get it right,” Altman said, adding that if firms could not keep “alignment and safety way ahead of capabilities” they would “slow down or stop”.

But Yoshua Bengio, one of the pioneers of modern AI, posted on X, external “ambitious efforts outside the for-profit sector” were needed to “avert the worst risks from AI”.

Some industry figures have argued resurgent AI fears were overblown and were being jumped on to generate hype for the industry.

During the conference, Altman addressed a crowd of hundreds of business people, telling them they should use AI tools to aid in their work, but also needed them in order to protect their businesses from potentially AI-enabled cyber attacks.

The idea of AI executives being left to entirely regulate themselves has struck some in the industry as a bad idea.

Jack Clark, an Anthropic executive and co-founder, told the BBC on Monday that leaving AI to be a “totally unregulated industry” was “rolling dice with immense risks”.

Patrick Hillman, the chief operating officer of Logical Intelligence, which is chaired by Yann LeCun, a sage of the AI industry, noted Tuesday how little faith people have in tech companies to do anything truly in the public interest, external.

“The only institution that Americans might trust less than Washington these days is Silicon Valley. I have worked and lived in both and I assure you both have earned this scepticism,” Hillman said.

“If you believe what you are building is dangerous, show us what you are prepared to stop doing,” he added.

Leaders of OpenAI and Anthropic have said they have recently started working towards some sort of industry-wide agreement on safety.

Amodei, during a cameo at the conference on Tuesday, said Anthropic is now in “a dialogue with the rest of the industry” about committing to better safety standards and checks on AI tools and development.

He also said that one of the biggest surprises of the AI boom was not how the technology itself has advanced, but its broader impact.

“We didn’t appreciate that it would lead to companies growing so fast, and how quickly they would become central to things,” Amodei said.

OpenAI executive Chris Lehane said last week that the company was also working with other AI labs, external “to advance frontier AI standards, building a voluntary effort now, with or without government support.” Those labs include Anthropic and Google Deepmind.

“With stakes this high, we cannot let the perfect become the enemy of the good,” Lehane wrote.