This post was originally published on this site.
OpenAI is rolling out its latest advanced LLM, Sol, for wide public access. Sol is considered to be at least on par with Anthropic’s Fable, a model whose capabilities (or ownership) stressed out the White House enough to that it was briefly banned from public access.
So how did these models get the ok for release? Short answer: Nobody’s quite sure.
“Frankly, I don’t have visibility into those exact processes, so yes, I don’t feel like I have enough information to say whether they’re adequate or not,” Mina Narayanan, a senior research analyst at Georgetown’s Center for Security and Emerging Technology, told TechCrunch. “Anthropic did say that they were in conversations with the government, and that they developed a classifier to detect jailbreak attempts, and they’ve implemented defensive gap strategies to prevent future jailbreaks, but exactly what that dialog looked like between the government and Anthropic and OpenAI is unclear.”
Dean W. Ball, a former Trump policy advisor who now works for OpenAI, wrote that “nobody knows what the requirements are to get licensed” in his newsletter last month.
Andy Konwinski, a computer scientist who co-founded Databricks, Perplexity, and the Laude Institute, said he’s never spoken to anyone who understands the process, even employees at frontier labs. “It’s existentially a problem,” he tells TechCrunch. “Safety or not, it’s about who has the power to make decisions—who gatekeeps and decides on permissions?”
Eighteen months into the Trump administration, there is still little clarity about how to move forward, despite—or, some critics allege, because—of the industry figures setting policy. Last month, after weeks of infighting, an executive order was published laying out a roadmap for evaluating frontier models, but the specifics have yet to be filled in, other than what won’t exist. “There will not be an FDA for AI,” Sriram Krishnan, a former Andreesen Horowitz partner who served as a senior advisor for AI in the White House until last month, told the Financial Times.
Notably, there’s still no agreement on what kinds of models require government scrutiny, or what agency or agencies should perform those evaluations. For now, the Department of Commerce’s Center for AI Standards and Innovation seems to be taking the lead, but the executive order instructs six cabinet agencies to determine a final process by early August. What has emerged in the meantime is, at best, ad hoc.
OpenAI CEO Sam Altman said on CNBC that the process involved conversations with the officials like Secretary of Commerce Howard Lutnick, Secretary of the Treasury Scott Bessent, and US national cyber director Sean Cairncross, but it’s not clear who the experts that tested the models were or how they did that. OpenAI declined to share details on the government’s process with TechCrunch, but pointed to the results of several external evaluations by organizations like UK AISI, SecureBio and Irregular in the latest model’s safety card.
As with Anthropic’s Fable roll-out, OpenAI previewed the model for the government and select users ahead of wider release, but we don’t know who who all of those users were or how they were chosen. In a late June blog post, the company said “we don’t believe this kind of government access process should become the long-term default,” saying it would work with the government to develop a different path forward.
The backdrop to those conversations, however, includes Altman reportedly offering as much as 5% to OpenAI’s equity for the administration’s so-called “Trump Accounts,” and OpenAI president Greg Brockman’s role as the largest publicly-known donor to Trump’s mid-term political operation. It’s hard for outside observers to separate those activities from the government’s apparently lighter-touch approach to regulating Sol.
Amthropic’s Fable, on the other hand, was briefly pulled from wider access when the US government forbade its use by foreign nationals, partly because of real concerns about users jail-breaking the model to access hacking capabilities and partly due to personality clashes between Anthropic and the Trump administration. The threat of an export ban may have also led OpenAI to be more cooperative with the government’s (unknown) requests.
From an industry perspective, a hands-off approach to regulation might be nice, but one that depends on personal connections to administration officials creates uncertainty and bad incentives.
Konwinski told TechCrunch that he worries true experts in this technology—”safety researchers, alignment researchers, interpretability researchers, but also data people, and people from all over the stack”—aren’t playing enough of a role in the model release process.
Konwinski argues that an “open commons” is the best way to actually balance safety and innovation. He points to models like the FDA, the NIH, or the national labs, which convene researchers, government officials, and private companies to reach a consensus on safety issues.
Some of that comes down to the incentives of capitalism that have motivated AI researchers for more than a decade, and played out in the court room during Elon Musk’s lawsuit challenging OpenAI’s corporate structure. Ball points out that the nature of the AI business requires companies to recoup much of their training costs shortly after their models are released and are further ahead of the competition.
“Even if their intentions are good, there’s very clear legal obligations and fiduciary responsibility that are built right into the operating procedures,” Konwinski points out.
Ball, in his post, argued that the way forward will depend on third-party auditing organizations, licensed by the government, that will evaluate frontier labs’ approach to safety. Konwinski, too, is bullish about new institutional formats like focused research organizations that could help more disinterested experts from academia and the non-profit world access and evaluate frontier models.
For now, the secrecy around the development of AI isn’t going away, but it also will seed political challenges for an industry that Americans increasingly view with skepticism. “There’s not a sense that responsible people are driving forward these changes,” University of Wisconsin-Madison computer science professor Remzi Arpaci-Dusseau said last week at the Open Frontier conference.
At the same event, David Siegel, the computer scientist who founded Two Sigma, one of the most successful quantitative hedge funds, asked attendees to “imagine a situation, which I think would be very bad, [where] a small number of firms control the technology; the government, in their secretive laboratories, is evaluating whether or not the technology is suitable for use; and the general public and scientific community doesn’t really have any access to any of that stuff.”
It seems like we don’t need to imagine it.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
