Who decides when AI is too dangerous?

The question of who decides when artificial intelligence becomes too dangerous is one of the most urgent policy debates of our time, yet it remains frustratingly unresolved. The rapid advancement of AI systems—from capable chatbots to autonomous agents—has outpaced the development of governance frameworks that can meaningfully assess risk. This isn’t just a technical oversight; it’s a systemic failure of accountability. Without clear thresholds for danger, we risk normalizing harm under the guise of innovation, where companies proceed with development while regulators scramble to catch up long after the damage is done. The lack of consensus is rooted in competing priorities. Tech companies argue for self-regulation, citing the need for flexibility in a fast-evolving field, while critics demand preemptive bans on high-risk applications like autonomous weapons or deepfake propaganda tools. Governments, meanwhile, are caught between geopolitical competition and public safety—some nations see AI leadership as a matter of national security, which can discourage strict oversight. This tension was starkly visible in recent debates over generative AI, where even well-intentioned voluntary guidelines failed to prevent misuse in areas like disinformation or fraud. What makes this issue particularly thorny is that danger is often subjective. A system that seems benign in one context—say, a language model used for creative writing—could become hazardous when repurposed for manipulation or surveillance. The absence of standardized risk assessments means that decisions about safety are frequently left to the developers themselves, who may prioritize speed over scrutiny. This raises a critical question: Can we trust the same institutions that profit from AI’s proliferation to police its dangers objectively? Moving forward, the most plausible path may lie in international coordination—think of it as an AI equivalent of the Non-Proliferation Treaty, where nations agree to common rules on high-risk applications. But even that would require unprecedented cooperation in an era of technological nationalism. Until then, the power to define danger remains dangerously decentralized, leaving society to navigate a landscape where the line between progress and peril is drawn not by careful governance, but by the next headline-grabbing failure.