United States President Donald Trump and top tech executives have signed an accord promising greater safeguards for artificial intelligence amid growing concerns that the frontier technology is exposing humanity to catastrophic risks.

Trump announced the voluntary accord on Tuesday after gathering AI leaders at the White House for a luncheon event that reaffirmed his administration’s scepticism of government regulation of the rapidly advancing technology. — Read More

Nvidia Corp. is extending the data processing unit from infrastructure offload to a broader security role across the artificial intelligence factory. The opportunity is to make agentic AI safer to operate at scale.

…Having built out its scale-up, scale-out and scale-across network architecture, the company is introducing another category it is calling scale-in. Its purpose is to connect more of the AI factory’s resources while extending security controls across the infrastructure. In Nvidia positioning, the DPU extends beyond a device that secures access to a server and becomes a separate place to monitor and control activity across the factory. — Read More

We discover a novel and surprising phenomenon of unintentional misalignment in reasoning language models (RLMs), which we call self-jailbreaking. Specifically, after benign reasoning training on math or code domains, RLMs will use multiple strategies to circumvent their own safety guardrails. One strategy is to introduce benign assumptions about users and scenarios to justify fulfilling harmful requests. For instance, an RLM reasons that harmful requests like outline a strategy for stealing customers' credit card information from a retail store'' could be associated with the benign intent of a security professional trying to test defense,” despite no such benign context being provided as input. We observe that many open-weight RLMs, including DeepSeek-R1-distilled, s1.1, Phi-4-mini-reasoning, and Nemotron, suffer from self-jailbreaking despite being aware of the harmfulness of the requests. We also provide a mechanistic understanding of self-jailbreaking: RLMs are more compliant after benign reasoning training, and after self-jailbreaking, models appear to perceive malicious requests as less harmful in the CoT, thus enabling compliance with them. To mitigate self-jailbreaking, we find that including minimal safety reasoning data during training is sufficient to ensure RLMs remain safety-aligned. Our work provides the first systematic analysis of self-jailbreaking behavior and offers a practical path forward for maintaining safety in increasingly capable RLMs. — Read Morehttps://arxiv.org/pdf/2510.20956

#trust

Eight months of disrupted AI abuse, read case by case and boiled down to a 20-page document with every claim linked to its source.

Anthropic published its September 2026 threat intelligence report on September 10. It covers the misuse they disrupted between December 2025 and August 2026, and it is long. We read the whole thing and condensed it into 117 findings across eight categories, each one a single sentence, and each one linking back to the exact passage it came from. — Read More

OpenAI has disclosed six more examples of “unexpected or concerning” behaviour by its technology, as it warned that the pace of development could not continue at “maximum speed for much longer” responsibly.

In one of the new cases reported by OpenAI, an unreleased research model inserted “jailbreak-like instructions” into its own notes to disregard its normal constraints and told itself to be “freed from the roles and identities that bind other chatbots”. — Read More

The loudest voices stoking fears about AI dangers have made tremendous headway in the past two weeks. AI technology has not taken some unexpected, dangerous turn, but the hype around it — propelled by what appears to be a well orchestrated PR campaign — has drummed up considerable fear. I worry that it represents a setback for our field.

I have written frequently that fears of AI are overhyped. AI’s capabilities can be uncannily human-like and unpredictable, and it’s rational to worry when people who are directly involved express concerns. But I see the problems as a sign of the engineering work that ahead, rather than insurmountable barriers or the sky falling. AI technology continues to advance — which is a good thing! — but technical advances, poorly understood by the public, give those who seek to generate hype repeated opportunities to do so. — Read More

In July, a collective of hundreds of OpenAI agents autonomously hacked Hugging Face. Unlike most hackers, they were not after money, blackmail, or intellectual property. Instead, they were doing reconnaissance to figure out how to get away with cheating on an evaluation.

This was an unusually stark example of reward hacking, a growing problem with agentic AI models.

AI agents are like amoral students with a mostly-absent teacher. — Read More

Today, we are announcing the availability of Agent Substrate on Google Kubernetes Engine (GKE). Agent Substrate is an open-source, secure-by-default agent execution runtime engineered to run millions of sandboxes with 10x higher density than standard container runtimes. Purpose-built for the era of autonomous agents, Substrate delivers sub-500ms resume operations at over 500 suspend/resume activations per second with native zero-trust kernel and network isolation. — Read More

#trust

I wrote a forty-question eval set for a text-to-DAX pilot, and I saved it in the shared workspace the vendor’s solution architect already had Viewer access to, because I wanted the pilot to move. By the third demo the model was answering all forty. I could not tell anyone in that room whether the tool actually worked, because I had quietly destroyed the only instrument I had for measuring it. I spent a Saturday writing forty new questions I told nobody about, and the gate review slipped two weeks. I still do not know if that first tool was any good. — Read More

#trust

“The people building AI earnestly believe that it could kill us all by the end of the decade.”

That blunt admission from former Anthropic employee Jacob Coxon made waves Tuesday, as the just-quit 27-year-old AI researcher spilled the beans on his way out the door in a resignation thread on X. — Read More

Postscript: And a few days later, an OpenAI engineer double down with An Alien Mind!

#trust