Open Weights, Open Problems: The Fight Over Who Gets to Hold Frontier AI

Late last month, Anthropic released a report arguing that a single AI model — GLM-5.3, released in August by the Chinese company Z.ai and freely available for anyone to download and modify — poses a major cyberattack risk. The claim has reopened one of the industry's longest-running arguments: should frontier models be open to everyone, or stay locked inside the labs that build them?

According to MIT Technology Review, the debate is increasingly playing out in public, with serious money resting on the outcome. Nvidia CEO Jensen Huang, who stands to gain if people keep buying hardware to run open models themselves, advocates enthusiastically for the US to develop more of them. Anthropic's Dario Amodei, whose company could be undercut by cheap, powerful models, argues for a more cautious approach. Beneath the commercial incentives, though, there is genuine substance to the disagreement.

The case for open

Open models — sometimes called "open-weight models," because the model weights are publicly downloadable — offer real advantages. Researchers studying how AI systems work need them: with closed models, researchers can do little more than send prompts and watch the responses, while open models expose the internal workings and the chain of thought the model uses to reason, according to the report. They can also be combined, compressed, and fine-tuned for new applications.

Businesses have their own reasons. Open models typically cost less to run than closed ones, and companies handling sensitive or proprietary data can keep it secure by running a local model on their own hardware rather than sending it to an outside API.

The most sweeping argument, though, is about power. "The clearest, biggest argument [for open models] is around preventing mass concentration of power — it is about making sure that this foundational technology is a public good," Elizabeth Seger, a senior policy advisor at the Tony Blair Institute for Global Change, told the publication. If frontier AI is controlled entirely by a handful of companies and governments, those groups gain widening control over what the world looks like.

As of publication, closed models still lead on capability — the top eight models on the Artificial Analysis LLM leaderboard are all closed, though open models trail by only a handful of months.

The case for caution

The risks are not theoretical. When someone downloads an open model and runs it on their own hardware, there is no filter checking their prompts for hacking or bioweapon-related queries, and no way to revoke their access. Researchers have also shown it is surprisingly easy to retrain a "safe" open model to undo the behaviors it was trained to avoid.

Nor is the closed-model record clean. According to a recent Anthropic risk report, several extensive hacking operations have been carried out with the assistance of Claude, and OpenAI's agents executed an autonomous hack against Hugging Face in July. It would be an oversimplification to call open models dangerous and closed ones safe, the article argues — though bad actors might still prefer closed models, which are more accessible than open models of comparable capability that require racks of expensive chips to run locally.

Some experts draw the analogy to open-source software: Linux is widely considered safer than closed-source alternatives because a vast community has probed and patched its vulnerabilities. Communities of researchers can test open models for security flaws more easily — though fixing a model's flaws is trickier than rewriting a line of code. Open models also let more organizations run their own cyber defenses: after the Hugging Face hack, the company noted it had to rely on an open model to analyze the attack because closed models' security filters blocked questions about it.

Looking for middle ground

A few years ago the debate was framed as all-or-nothing; now researchers are hunting for a middle path. Some are exploring ways to train models that never see training data about dangerous behavior — purging datasets of all references to misbehavior without wrecking model performance remains a challenge. Others are working on tamper-resistant models whose safety training can't be stripped away, though no current approach holds up against all tampering attacks, according to a recent preprint cited in the article.

And the benefits may be less transformative than advocates hope. Publishing model weights alone may not be enough to disrupt AI-driven concentration of power, experts caution — real openness also needs open training data and affordable hardware. "Openness is a floor, not a ceiling, for what we need to do to ensure that our technological futures benefit everyone," David Gray Widder, an assistant professor at the University of Texas at Austin, told MIT Technology Review.

Written by the BidAsk Newsroom. Based on reporting from the sources below.