OpenAI, Google and others are seeing diminishing returns to building ever-bigger models — but that may not matter as much as you would guess

Over the past week, several stories sourced to people inside the big AI labs have reported that the race to build superintelligence is hitting a wall. Specifically, they say, the approach that has carried the industry from OpenAI’s first large language model to the LLMs we have today has begun to show diminishing returns.

Today, let’s look at what everyone involved is saying — and consider what it means for the AI arms race. While reports that AI scaling laws appear to be technically accurate, they can also be easily misread. For better and for worse, it seems, the development of more powerful AI systems continues to accelerate. — Read More

The rigid structures of language we once clung to with certainty are cracking. Take gender, nationality or religion: these concepts no longer sit comfortably in the stiff linguistic boxes of the last century. Simultaneously, the rise of AI presses upon us the need to understand how words relate to meaning and reasoning.

A global group of philosophers, mathematicians and computer scientists have come up with a new understanding of logic that addresses these concerns, dubbed “inferentialism”. — Read More

The Open Source Initiative has published (news article here) its definition of “open source AI,” and it’s terrible. It allows for secret training data and mechanisms. It allows for development to be done in secret. Since for a neural network, the training data is the source code—it’s how the model gets programmed—the definition makes no sense.

And it’s confusing; most “open source” AI models—like LLAMA—are open source in name only. But the OSI seems to have been co-opted by industry players that want both corporate secrecy and the “open source” label. (Here’s one rebuttal to the definition.)

This is worth fighting for. We need a public AI option, and open source—real open source—is a necessary component of that. — Read More

Two of the biggest forces in two deeply intertwined tech ecosystems — large incumbents and startups — have taken a break from counting their money to jointly plead that the government desist from even pondering regulations that might affect their financial interests, or as they prefer to call them, innovation.

“Our two companies might not agree on everything, but this is not about our differences,” writes this group of vastly disparate perspectives and interests: Founding a16z partners Marc Andreessen and Ben Horowitz, and Microsoft CEO Satya Nadella and President/Chief Legal Officer Brad Smith. A truly intersectional assemblage, representing both big business and big money.

But it’s the little guys they’re supposedly looking out for.  — Read More

For years I have been warning that “scaling” — eeking out improvements in AI by adding more data and more compute, without making fundamental architectural changes — would not continue forever. In my most notorious article, in March of 2022, I argued that “deep learning is hitting a wall”. Central to the argument was that pure scaling would not solve hallucinations or abstraction; I concluded that “there are serious holes in the scaling argument.”

And I got endless grief for it. Sam Altman implied (without saying my name, but riffing on the images in my then-recent article) I was a “mediocre deep learning skeptic”; Greg Brockman openly mocked the title. Yann LeCun wrote that deep learning wasn’t hitting a wall, and so on. Elon Musk himself made fun of me and the title earlier this year.

The thing is, in the long term, science isn’t majority rule. In the end, the truth generally outs. Alchemy had a good run, but it got replaced by chemistry. The truth is that scaling is running out, and that truth is, at last coming out. — Read More

AI can “teleport” scents without human hands (or noses)

Ever send a picture of yourself trying on clothes to a friend to see what they think of how you look? Now, imagine doing the same from the perfume and cologne counter. AI could make that happen in the not-too-distant future after a breakthrough in ‘Scent Teleportation.’ Osmo, which bills itself as a “digital olfaction” company, has succeeded in using AI to analyze a scent in one location and reproduce it elsewhere without human intervention. — Read More

Imagine a world where your digital colleague handles entire workflows, adapts to real-time challenges, and collaborates seamlessly with your human team. This isn’t science fiction—it’s the imminent reality of AI agents in the workplace. 

As Sam Altman, CEO of OpenAI, boldly predicted at their annual DevDay event, “2025 is when AI agents will work.” But what does this mean for the future of human labor, organizational structures, and the very definition of work itself? 

According to research by The Conference Board, 56% of workers use generative AI on the job, and nearly 1 in 10 use generative AI tools daily.  — Read More

Anthropic CEO Dario Amodei wants you to know he’s not an AI “doomer.”

At least, that’s my read of the “mic drop” of a ~15,000 word essay Amodei published to his blog late Friday. (I tried asking Anthropic’s Claude chatbot whether it concurred, but alas, the post exceeded the free plan’s length limit.)

In broad strokes, Amodei paints a picture of a world in which all AI risks are mitigated, and the tech delivers heretofore unrealized prosperity, social uplift, and abundance.  — Read More

If you asked me how to scale clean energy, I would prescribe a magical source of urgent energy demand.

Someone willing to pay a premium to build solar+batteries, geothermal, and nuclear, in order to bring them down the cost curve and make them cheaper for everyone.

That is exactly what AI data centres are. — Read More

The ability to plan a course of action that achieves a desired state of affairs has long been considered a core competence of intelligent agents and has been an integral part of AI research since its inception. With the advent of large language models (LLMs), there has been considerable interest in the question of whether or not they possess such planning abilities. PlanBench, an extensible benchmark we developed in 2022, soon after the release of GPT3, has remained an important tool for evaluating the planning abilities of LLMs. Despite the slew of new private and open source LLMs since GPT3, progress on this benchmark has been surprisingly slow. OpenAI claims that their recent o1 (Strawberry) model has been specifically constructed and trained to escape the normal limitations of autoregressive LLMs–making it a new kind of model: a Large Reasoning Model (LRM). Using this development as a catalyst, this paper takes a comprehensive look at how well current LLMs and new LRMs do on PlanBench. As we shall see, while o1’s performance is a quantum improvement on the benchmark, outpacing the competition, it is still far from saturating it. This improvement also brings to the fore questions about accuracy, efficiency, and guarantees which must be considered before deploying such systems. — Read More

#strategy