AI Red Lines: Insights from the New U.N. Campaign Launch

Highlights

• Initiative: The AI Red Lines initiative debuted at the United Nations General Assembly to address AI risks.
• Signatories: Over 200 Nobel laureates and AI experts, including Wojciech Zaremba, endorsed a letter for global accountability.
• Call for Action: The letter stresses the need for clear and verifiable red lines by 2026 to mitigate future AI dangers.
• Concerns: Experts like Stuart Russell emphasize the need for built-in safety measures for AI systems from the start.

The AI Red Lines initiative launched at the United Nations General Assembly Tuesday – the perfect place for a very nonspecific declaration.

More than 200 Nobel laureates and other artificial intelligence experts (including OpenAI co-founder Wojciech Zaremba), plus 70 organizations that deal with AI (including Google DeepMind and Anthropic), signed a letter calling for global “red lines to prevent unacceptable AI risks.” However, it was marked as much by what it didn’t say as what it did.

“AI systems have already exhibited deceptive and harmful behavior, and yet these systems are being given more autonomy,” the letter said, laying out a deadline of 2026 for its recommendation to be implemented: “An international agreement on clear and verifiable red lines is necessary for preventing universally unacceptable risks.”

Fair enough, but what red lines, exactly? The letter says only that these parameters “should build upon and enforce existing global frameworks and voluntary corporate commitments, ensuring that all advanced AI providers are accountable to shared thresholds.”

The lack of specifics may be necessary to keep a very loose coalition of signatories together. They include AI alarmists like 77-year-old Geoffrey Hinton, the so-called “AI godfather” who has spent the last three years predicting various forms of doom from the impending arrival of AGI (artificial general intelligence); the list also includes AI skeptics like cognitive scientist Gary Marcus, who has spent the last three years telling us that AGI isn’t coming any time soon.

What could they all agree on? For that matter, what could governments already at loggerheads over AI, mainly the U.S. and China, agree on, and trust each other to implement? Good question.

This Tweet is currently unavailable. It might be loading or has been removed.

Probably the most concrete answer by a signatory came from Stuart Russell, veteran computer science professor at UC Berkeley, in the wake of a previous attempt to talk red lines at the 2023 Global AI Safety Summit. In a paper titled “Make AI safe or make safe AI?” Russell wrote that AI companies offer “after-the-fact attempts to reduce unacceptable behavior once an AI system has been built.” He contrasted that with the red lines approach: ensure built-in safety in the design from the very start, and “unacceptable behavior” won’t be possible in the first place.

“It should be possible for developers to say, with high confidence, that their systems will not exhibit harmful behaviors,” Russell wrote. “An important side effect of red line regulation will be to substantially increase developers’ safety engineering capabilities.”

In his paper, Russell got as far as four red line examples: AI systems should not attempt to replicate themselves; they should never attempt to break into other computer systems; they should not be allowed to give instructions on manufacturing bioweapons. And their output should not allow any “false and harmful statements about real people.”

From the standpoint of 2025, we might add red lines that deal with the current ongoing threats of AI psychosis, and AI chatbots that can allegedly be manipulated to give advice on suicide.

We can all agree on that, right?

Trouble is, Russell also believes that no Large Language Model (LLM) is “capable of demonstrating compliance”, even with his four minimal red-line requirements. Why? Because they are predictive word engines that fundamentally don’t understand what they’re saying. They are not capable of reasoning, even on basic logic puzzles, and increasingly “hallucinate” answers to satisfy their users.

So true AI red line safety, arguably, would mean none of the current AI models would be allowed on the market. That doesn’t bother Russell; as he points out, we don’t care that compliance is difficult when it comes to medicine or nuclear power. We regulate regardless of outcome.

But the notion that AI companies will just voluntarily shut down their models until they can prove to regulators that no harm will come to users? This is a greater hallucination than anything ChatGPT can come up with.