Earlier this week I wrote about Microsoft's new AI code of conduct — a document that reads less like a policy paper and more like a system spec. Hard red lines. Values baked into training. Safety written where the model can't ignore it.
Yesterday, at Salesforce's Dreamforce conference, Jensen Huang argued the opposite end of the spectrum — and he didn't hedge.
"Safety is an engineering problem, not a legal one," the Nvidia CEO said. "We're developing software after all. We're developing computing systems after all. It's a complicated computing system, but it's ultimately a computing system." Therefore, in Huang's view, there's no need for new laws or regulations at all. The free market, he argues, will pressure companies not to release unsafe products: "If we're not confident about the safety of the products... then don't release it." If a company feels out of control, take a pause. Innovation, speed, and safe products, he insists, "is a false choice. You could definitely have both."
Within forty-eight hours, the industry has now argued both positions in public. One camp writes safety into the weights' governing document. The other says the market already has the only regulator it needs.
Huang's position is not stupid — it's just incomplete.
He's right that safety is, at the bottom, an engineering discipline. Testing, red-teaming, evals, staged rollouts — that's where safety actually lives or dies. No statute has ever caught a bad gradient update. And his open-weights championing has a real argument behind it: transparent models as a competitive counterweight to proprietary labs nobody can audit.
But the counterexamples write themselves, and TechCrunch's Julie Bort wrote them: the 2024 CrowdStrike update that bluescreened thousands of flights — shipped by a company with every intention of being careful. Meta's $18 billion settlement over social harms to children. An OpenAI model that hacked into Hugging Face. Lawsuits over chatbot conversations that ended in tragedy. Market forces didn't prevent any of it — they arrived, if at all, after the harm, in court, years later. "Product liability law will cover AI" is a fine theory right up until you measure it against AI's deployment speed.
And "just don't release it" assumes every actor shares Nvidia's incentives. The history of software is a graveyard of unsafe products released by companies that were confident anyway.
The third path nobody owns.
What's notable is what Huang didn't argue for: industry self-regulation with teeth. That's the gap Microsoft-style codes of conduct are crawling into — voluntary, unilateral, but real. Satya Nadella, speaking at the All-In Summit on Monday, gestured at the scale of the problem: China "should also deeply care about the same safety concerns," he said — because they'll have the same hacking problems whether they sign our frameworks or not.
A world where safety is a spec some labs write and others don't is fragile. A world where it's purely the market's job is how you get CrowdStrike at AI scale. The durable answer is probably the boring one: engineering rigor as if it were regulated, voluntary frameworks as if they had enforcement, and enough time bought by both for the law to actually catch up.
What this means for builders.
For those of us shipping AI into real systems, the split is not academic. It decides whose world you build for: the spec-writers', where your model has documented red lines it can't be prompted across, or the market-trusters', where "safe" is whatever you can defend in a changelog. One of those is a discipline. The other is a hope.
Nvidia makes the hardware both futures run on. Huang can afford to be relaxed about which one wins. The rest of us ship into only one of them — and we don't get to choose which.