The debate over AI safety is shifting from whether to regulate to who gets to write the rules. In a September 13, 2026 post on the Cohere blog, CEO Aidan Gomez argues that a proposal from Anthropic CEO Dario Amodei—asking governments for antitrust exemptions so leading labs can coordinate on safety standards—would effectively let a handful of Silicon Valley companies define the guardrails for everyone else.
The core tension: safety vs. competition
Gomez is careful to separate the goal from the mechanism. He supports independent review of highly capable AI systems. But he questions who writes the standards, who conducts the review, and who gets a seat at the table. The Anthropic roadmap, he writes, would have a small group of powerful labs agree on shared standards and limits on how fast the technology should advance—and then ask governments to make that coordination lawful and require every other developer to follow.
This isn’t a new pattern. Gomez points to two historical examples: the SEC’s 1975 designation of three bond-rating agencies, which later rated subprime securities triple-A; and Europe’s 1985 Motor Vehicle Block Exemption, which let car manufacturers set standards for who could sell and service vehicles. Both started with safety as the stated goal. Both ended up protecting incumbents and limiting competition.
Why the current proposal worries builders
For product teams, the practical concern is that a safety regime designed by a few labs will be rigorous only about the risks those labs have already built systems to assess. Gomez notes that risk in existing frameworks gets defined as a function of scale—compute thresholds, monitoring infrastructure, dedicated security orgs. That makes the companies with enormous systems the only ones qualified to judge.
But smaller models orchestrated well, using tools and verification steps, can do things large models can’t. A cyber swarm is a different risk surface than a single model. None of that shows up in a regime built around massive compute. The entry requirements in the proposal—vast computing power, resident evaluator teams, shutdown architecture—read like a moat, not a safety checklist.
What an alternative looks like
Gomez proposes four pillars for better rules:
- An evidence-based risk framework built across countries, in the open, with technologists, policy experts, and sector specialists—and published disagreements.
- Mandatory transparency around model cards, incident reporting, and accountability for real harm.
- Testing scoped by the evidence, targeting specific dangerous capabilities (cyberattacks, synthetic fraud, bioweapons) rather than every system for every risk.
- Rules that bind based on what a system can do, not who built it.
This last point matters for builders. If a dangerous capability is treated the same whether it comes from a trillion-dollar lab or a university department, then the playing field stays open. That’s the same principle behind choosing a model by its actual performance on your task rather than its brand—a theme we’ve explored before in picking the checkpoint, not the brand.
The takeaway for product teams
If you’re building AI products, the regulatory debate isn’t abstract. The rules that get written will shape what you can ship, how you test, and who you can compete with. A framework that equates safety with scale will favor incumbents. A framework that ties rules to demonstrated capabilities—regardless of who built the system—leaves room for smaller teams and new approaches.
Gomez’s argument is not that AI needs no guardrails. It’s that the guardrails should be written by a broad, evidence-driven process, not by the companies that stand to gain from them.
Sources
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.