The United States and China are often said to be in a “race” with one another with respect to artificial intelligence. In a sense this is true, but the metaphor manages to miss almost all that is interesting about US-China dynamics in emerging technology. Today I’d like to offer some brief thoughts about how I see this “race” and where it might be headed.
All metaphors are lossy approximations of reality. But “race” is an especially inapt metaphor for this context. A race is a competition with clear boundaries and a clearly defined finish line. There are no such luxuries to be found here. Beyond the rhyme, “the Space Race” made intuitive sense because the objective was clear: landing humans on the Moon.
Stating that there is an “AI race” underway invites the obvious follow-up question: the AI race to where? And no one—not you, not me, not OpenAI, not the U.S. government, and not the Chinese government—knows where we are headed. — Read More
Monthly Archives: November 2025
Andrew Ng: LLMs as the Next Geopolitical Weapon & Do Margins Still Matter in AI?
Continuous Autoregressive Language Models
The efficiency of large language models (LLMs) is fundamentally limited by their sequential, token-by-token generation process. We argue that overcoming this bottleneck requires a new design axis for LLM scaling: increasing the semantic bandwidth of each generative step. To this end, we introduce Continuous Autoregressive Language Models (CALM), a paradigm shift from discrete next-token prediction to continuous next-vector prediction. CALM uses a high-fidelity autoencoder to compress a chunk of K tokens into a single continuous vector, from which the original tokens can be reconstructed with over 99.9\% accuracy. This allows us to model language as a sequence of continuous vectors instead of discrete tokens, which reduces the number of generative steps by a factor of K. The paradigm shift necessitates a new modeling toolkit; therefore, we develop a comprehensive likelihood-free framework that enables robust training, evaluation, and controllable sampling in the continuous domain. Experiments show that CALM significantly improves the performance-compute trade-off, achieving the performance of strong discrete baselines at a significantly lower computational cost. More importantly, these findings establish next-vector prediction as a powerful and scalable pathway towards ultra-efficient language models. Code: this https URL. Project: this https URL. — Read More
Introducing Nested Learning: A new ML paradigm for continual learning
We introduce Nested Learning, a new approach to machine learning that views models as a set of smaller, nested optimization problems, each with its own internal workflow, in order to mitigate or even completely avoid the issue of “catastrophic forgetting”, where learning new tasks sacrifices proficiency on old tasks.
The last decade has seen incredible progress in machine learning (ML), primarily driven by powerful neural network architectures and the algorithms used to train them. However, despite the success of large language models (LLMs), a few fundamental challenges persist, especially around continual learning, the ability for a model to actively acquire new knowledge and skills over time without forgetting old ones.
When it comes to continual learning and self-improvement, the human brain is the gold standard. It adapts through neuroplasticity — the remarkable capacity to change its structure in response to new experiences, memories, and learning. Without this ability, a person is limited to immediate context (like anterograde amnesia). We see a similar limitation in current LLMs: their knowledge is confined to either the immediate context of their input window or the static information that they learn during pre-training.
The simple approach, continually updating a model’s parameters with new data, often leads to “catastrophic forgetting” (CF), where learning new tasks sacrifices proficiency on old tasks. Researchers traditionally combat CF through architectural tweaks or better optimization rules. However, for too long, we have treated the model’s architecture (the network structure) and the optimization algorithm (the training rule) as two separate things, which prevents us from achieving a truly unified, efficient learning system.
In our paper, “Nested Learning: The Illusion of Deep Learning Architectures”, published at NeurIPS 2025, we introduce Nested Learning, which bridges this gap. Nested Learning treats a single ML model not as one continuous process, but as a system of interconnected, multi-level learning problems that are optimized simultaneously. We argue that the model’s architecture and the rules used to train it (i.e., the optimization algorithm) are fundamentally the same concepts; they are just different “levels” of optimization, each with its own internal flow of information (“context flow”) and update rate. By recognizing this inherent structure, Nested Learning provides a new, previously invisible dimension for designing more capable AI, allowing us to build learning components with deeper computational depth, which ultimately helps solve issues like catastrophic forgetting. — Read More
Introducing SIMA 2, the next milestone in our research creating general and helpful AI agents.
Computational models for brain science
The No. 1 Country Song in America Is AI-Generated
According to Billboard’s “Country Digital Song Sales” chart, the No. 1 song in the U.S. is “Walk My Walk” by Breaking Rust—an artist that was created by artificial intelligence (AI).
This is a new development in the music industry as it is the first time an AI-created song has reached the top of the charts.
There have long been concerns about the use of generative AI in creative sectors. Discourse about this came into the fold a few years ago following protests in Hollywood from the writer and actor guilds, which took place shortly after the public release of ChatGPT, and concerns that came in tandem with the new technology and its implications.
… As the AI revolution continues to impact creative industries, it could be that more AI-generated artists continue to pop up in the charts, with pushback likely to be inevitable. — Read More
AI Red-Teaming Design: Threat Models and Tools
Red-teaming is a popular evaluation methodology for AI systems, but it is still severely lacking in theoretical grounding and technical best practices. This blog introduces the concept of threat modeling for AI red-teaming and explores the ways that software tools can support or hinder red teams. To do effective evaluations, red-team designers should ensure their tools fit with their threat model and their testers.
AI red-teaming is an evaluation methodology to discover flaws and vulnerabilities in AI systems. Although this type of evaluation has been adopted across the AI industry (as seen in Anthropic’s Responsible Scaling Policy, Google Deepmind’s Frontier Safety Framework, and OpenAI’s Safety & Responsibility documents), red-teaming practices vary widely, and there are few established standards or best practices. This is due in part to the versatility and flexibility of the methodology, such that red-team designers and testers have to make many decisions in the red-teaming process. While this blog post is primarily aimed at AI red-teamers, it may also be useful for policymakers and other readers interested in the design of AI evaluation.
This post will discuss two key factors in designing an AI red-teaming exercise: the red team’s threat model, and the selection of the software tools that testers use to engage with the target system. The threat model is the key concept around which the red-teaming exercise is constructed, while the design features of various tools shape which testers can use them and which threat models they can address. Appropriate tools can empower testers, but inappropriate ones can obscure evaluation results and lead to false conclusions. — Read More
Spiky Superintelligence vs. Generality
The Piss Average Problem
The Age of AI is a Crisis of Faith
The fundamental question facing online spaces in 2025 is no longer can AI pass as human? but rather can humans prove they’re not AI?
This represents a profound shift from technical doubt to existential uncertainty. It’s a crisis of faith where the bedrock assumption that we interact with other humans online has collapsed. And I’m not being hyperbolic. In 2024, bot traffic exceeded human traffic for the first time in a decade, hitting 51%. We’ve crossed the threshold. The internet is now majority non-human.
When I personally veer onto the Internet, particularly places like LinkedIn or Substack or any social media’s comment section, Dead Internet Theory truly shines as a valid hypothesis. This once-fringe conspiracy theory which speculates that the Internet is now mostly bots talking to bots is now many people’s lived experience — Read More
Rick's Cafe AI 