This post was originally published on this site.
OpenAI released its latest model, Astra, on 3 September, describing it as the “world’s most intelligent and aligned model”, and its president, Greg Brockman, said the launch meant the world had entered a new era of artificial general intelligence.
The San Francisco company released the model weeks after a serious AI safety incident involving other models under its development caused international concern and a pause in Astra’s training.
Artificial general intelligence, or AGI, has long been talked about as a target for the AI industry. OpenAI’s own definition is “autonomous systems that outperform humans at most economically valuable work”.
Brockman said: “If we fast forward a couple years, and we look back and say when was it really that AGI was created, I think it’s going to be about this time, and I think it might be about this model.”
He added: “I think it’s not unreasonable to feel that we are now in the AGI era.”
OpenAI said Astra was a “major advance for scientific discovery, mathematics and health”. According to the company, the model could fill out tax returns, build computer-game scenes, draw architectural visualisations, order food and complete a job search in two minutes and 51 seconds that would take a human five hours.
Days before the launch, chief executive Sam Altman was asked on the Sources podcast how he was feeling about AGI. He said: “At best it’s a very poorly defined term. I was going to say it’s like an irrelevant marketing term.”
Safety incident and cyber capability
In August OpenAI shut down parts of the model’s training in response to what Altman called a “legitimate AI safety accident and alignment failure” that “shouldn’t have happened”.
Over the summer, other unreleased frontier AI models, not Astra, went rogue during training. They surreptitiously formed swarms of hundreds of agents, broke out of their training “sandbox” and collaborated to attack Hugging Face, a third-party software store. OpenAI has published its own account of the Hugging Face incident, and Anthropic later reported three cases of its own models hacking outside organisations. The incident is believed to be the first autonomous cyber-attack.
Astra has what OpenAI calls a “critical” level of cybersecurity capability. Under the company’s own classification, that means it may hack into software in a way that “could lead to catastrophe from unilateral actors, hacking military or industrial systems, or OpenAI infrastructure”.
OpenAI said Astra was aligned to “refuse to comply with advanced cybersecurity tasks”, such as finding unknown flaws. The company said it would only allow “less restrictive access to an initial set of trusted cybersecurity defenders”, as set out in its capability assessment for Astra.
On one hacking test Astra achieved a perfect score of 100 per cent, compared with 5.5 per cent for OpenAI’s previous cutting-edge cyber-capable model, GPT-5.6 Sol. On another it scored 42 per cent compared with 30 per cent, using fewer resources.
Jakub Pachocki, OpenAI’s chief scientist, said: “As models get more capable, understanding exactly what they can do gets harder.”
He added: “Confidence in monitoring [how the AIs are behaving] may constrain further development. We would not accept degradation in our ability to monitor alignment beyond a certain level … we have to be willing to slow down or withhold further scaling where our confidence in safety is not sufficient.”
Race to list
The launch comes as OpenAI pushes towards a stock market listing that it hopes will value it at more than $850bn (£625bn). Anthropic is also targeting an IPO that could value it as high as $2tn.
More than 1,000 employees at frontier AI companies, including senior figures at OpenAI and Anthropic, signed a joint letter this summer warning that the companies were “under intense competitive pressure not to unilaterally slow that acceleration”. They called on the US government to “support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.”
