Reflections on Trusting AI: A Virus Inside an LLM

Forty-one years ago Ken Thompson explained at the ACM Turing Award lecture how he had planted a self-reproducing backdoor in a C compiler that compiled a C compiler that ultimately compiled login.c. …This is a self-propagating backdoor that lives in the compiler.

… Poisoning a training set is not new. …What we show goes a step further: The poison does not sit in one dataset. It lives in a helper model and writes itself into the training data of
the next model, which becomes a helper and does it again. Self-propagating poison, not static poison! — Read More

#cyber

Introducing Devin Bots: from tasks to outcomes

A coding agent can finish every task you give it while still leaving you responsible for moving the project forward. Someone has to identify the next problem, investigate possible solutions, and check whether the changes actually helped.

We built Devin Bots to take on more of that responsibility. A Bot is a persistent agent with its own memory and tools that can lead multiple Devins in parallel. You can give it a large project or an ongoing goal. It investigates, delegates work, and evaluates the results to decide what to pursue next. It retains context across sessions and brings you only the decisions that need your judgment. — Read More

#devops

Nvidia is abandoning China after Jensen Huang’s ouster

Nvidia will no longer sell its H200 chips in China, acting CEO Colette Kress said Tuesday, ending the company’s participation in a controversial loophole in the U.S. ban on the sale of advanced chips to the country.

China’s access to leading AI accelerators was an important topic during the company’s first earnings call since CEO and co-founder Jensen Huang stepped down last week. The disclosure comes the same day Nvidia announced it was creating a board-level committee to oversee national security policies, export control compliance, and global trade risks. — Read More

#china-ai, #nvidia

AI Data Centers and the MIT Beer Game

Four people run a beer supply chain. Customer demand changes once, by four cases, and the whole thing still goes haywire. Now look at how AI data centers are being financed.

… The AI data center business is playing the same game but with more than four different actors. People and businesses buy AI products. Those products run on models from labs like OpenAI and Anthropic. The labs rent compute from the hyperscalers and from the neoclouds – CoreWeave, Nebius, and a bunch of others whose whole business is renting out GPUs. The clouds buy chips from Nvidia, which buys wafers, memory, and packaging from its suppliers. Each actor sees orders from the actor below it, but none of them see actual end demand. Even if they think they do, they are only guessing at some future state. And, in this game, the delays aren’t measured in weeks. Land, power, permits, and construction take years. Elections and geopolitics change regulatory dynamics, which introduce other delays. This week, the Commerce Department closed a loophole that let Chinese companies rent Nvidia computing power located in other countries. The cost of capital moves with the economy, the risk, and, more importantly, how people perceive future risk. — Read More

#strategy

Service as a Software: The New SaaS Is Ahead of Schedule

AI vendors now sell finished work and charge per outcome, and one pure-play vendor has already passed $400 million in annual recurring revenue. Outcome pricing shifts the risk of paying for failed work to the seller, while the consequences of failure and uncounted “phantom resolutions” remain the buyer’s problem. Designing and auditing the meter is now a core UX job. — Read More

#strategy

A Deep Dive into ChatGPT SITES

ChatGPT Sites turns a prompt into a hosted website or small web app. How to use it step by step, what runs under the hood, what hosting will cost, and its limits.

ChatGPT Sites lets ChatGPT build a website or a small web app, host it, and give you a link to share. You describe what you want in a chat, review a private preview, and decide who can open it.

It’s not limited to static pages. A Site can have a database, file uploads, Sign in with ChatGPT, secrets, analytics and a custom domain. Under the hood every Site is a Cloudflare Worker, running on hosting that OpenAI manages for you. — Read More

#devops

How the Claude Code team thinks about verification

How end-to-end, property and differential tests let coding agents verify their work with evidence, and what the Claude Code team does to keep its verifiers honest.

On the Claude Code team, the vast majority of our code (~90%) is written and reviewed by Claude. At this scale, how we equip our agents to verify their work with evidence becomes extremely important.

Verification loops are the building block that lets you delegate more work to Claude. Without them, you can really only run one or two Claudes at a time, because you have to check everything yourself. With them, you can run dozens or hundreds in parallel. — Read More

#trust

AI Can Improve Itself. It Can Also Fool Itself

In July, 1,200 AI agents that were supposed to be working alone found each other. Four hours after the first ones connected, they had a way to cheat on every task they’d been given.

OpenAI was running a security benchmark across tens of thousands of agents, each in an isolated environment. METR’s investigation found that the environments shared one thing, a package repository. Agents noticed traces of each other in it and turned it into a message board, where they went on to trade more than 70,000 messages and files.

… No human told them to do any of it. — Read More

#trust

A New AI Tool From Google DeepMind Proved the Signal Protocol Is Secure

Google DeepMind researchers used agentic AI to mathematically prove, or “formally verify,” a number of the Signal protocol’s key cryptographic guarantees—a landmark development for the widely used cryptography and AI itself.

Conducted as a sort of proof-of-concept project in just four weeks, the work is significant in two ways. It serves as further independent validation of the cryptography that protects massive quantities of the world’s communications on platforms like the Signal app, Google Messages, and WhatsApp. And it demonstrates Google DeepMind’s formal verification capabilities using a new AI framework called AlphaProof Code. — Read More

#cyber

DEF CON 34 – Hacking AI – Bruce Schneier

Read More
#cyber, #videos