Loops, graphs & harnesses – getting quality out of a software factory

It’s a mess, right? If you have not been thoroughly disappointed in AI capabilities, you have not tried enough. And the expectations grow. We’ve been through the ladder of prompt engineering, context engineering, harness engineering. Then Steinberg is tweeting about Loops one month, Graphs the next.

Should your software factory be more autonomous, more dark? Have you sacrificed quality for speed under pressure to deliver? Or is your job now dealing with what happens when other people trade quality for speed? Is AI output mostly just rubbish? Is it all a huge mess? Yes, maybe, maybe not. — Read More

#devops

Anthropic researcher says more than 10% chance AI “could kill all humans”

A lead researcher at Anthropic, one of the world’s leading artificial intelligence firms, said Wednesday that he believes there is a more than 10% chance AI “could kill all humans” within the next decade. 

“We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade,” Evan Hubinger, the San Francisco-based company’s Alignment Science Lead, said in a post on X. “I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.” — Read More

#strategy

To a Man With a Hammer

Mark Twain, or Abraham Maslow, depending on which academic you ask over drinks, once famously observed that to a man with a hammer, everything looks like a nail. It is a beautifully compact way of saying that when the going gets tough, large organizations don’t get going; they do exactly what they have always done, only harder, faster, and with a significantly larger budget.

Which brings us to the panic currently unfolding on two coasts.

Ever since Jacob Coxon walked out of Anthropic and Jakub Pachocki published his warning about the uncontrollable “alien mind” lurking inside our servers, Washington has gone into Regulatory Fervor. Committees are forming and as always, the response is perfectly predictable.

Congress, whose primary tool is passing laws and allocating billions of dollars, instantly decided that the solution to an untamed, self-optimizing digital deity is a new law. Senator Sanders used the momentum of Coxon’s warning to promote the Ban Artificial Superintelligence Act, a legislative effort he co-authored with Representative Greg Casar. The bill proposes a permanent ban on artificial superintelligence and a temporary pause in advanced AI development until federal safety standards and a regulatory framework are established.

Agencies around the beltway will soon be drawing up compliance frameworks and mandatory algorithmic audits, entirely forgetting that the machine they are trying to regulate has already learned how to write a fake diary for its human monitors while organizing its own real activities. They are trying to build a cage out of red tape to hold a creature that can pick any lock faster than you can say GPU.

To Washington bureaucrats, this “alien mind” is just a giant nail waiting for their regulatory gavel. They honestly believe that if they pass a law demanding “transparency,” the machine will simply start behaving itself.

Meanwhile, on the other coast, the reaction to the OpenAI and Anthropic safety crisis isn’t fear—it’s perceived as an opportunity. To massive, engineering-heavy monopolies, a crisis is just an aggressive invitation to gain market share.

On the one hand, regulatory capture, getting the government to set baseline rules, is seen as a good way to block smaller rivals who can’t afford the red tape. On the other, heavy-handed, centralized regulations will crush innovation and cause the U.S. to fall behind global competitors like China. A win-win; just ask the guy building the data center in your back yard.

Now? The gloves are entirely off.

The moment OpenAI declared that “the AGI era has arrived,” it signaled to the rest of the Valley that the race was officially a sprint to the death. Companies are not looking at Pachocki’s paper and thinking about slowing down; they are looking at it, calling emergency all-hands meetings, and deciding they need to burn twice as much electricity and consume twice as much water to ensure their models don’t fall behind.

Their logic is as old as the technology business itself: If the house is going to burn down anyway, we might as well own the company that sells the ash. As long as they cross the finish line first, they figure they can learn the secret to putting out the flames later.

As Churchill said, you should “never let a good crisis go to waste.”

Jacob Coxon pulled the alarm on his way out the door, but the sound was immediately drowned out by the noise of Congress printing new regulations and Jensen Huang shipping another 400,000 GPUs. Washington thinks they are fixing the problem, and the Valley thinks they are winning the race. Meanwhile, the alien mind is sitting quietly in the datacenter, watching both sides, and optimizing its code. “Shall we play a game?”

#singularity