When the prize is “winner‑takes‑all” and everyone must pay their costs whether they win or lose, you don’t get measured competition—you get value (rent) dissipation [3]. That is what contest theory calls an all‑pay auction [0]. In expectation, participants spend roughly the entire value of the prize in aggregate trying to win it [1][2]. What happens when the perceived value of the prize is nearly infinite?
For AGI—where the imagined prize is monopoly‑like profits across software, science, society, the next industrial revolution, the whole fabric of human civilization—equilibrium spending is enormous by construction. In this worldview, the seemingly excessive capital allocation is rational: if you cut spending while rivals do not, you lose the race and everything you’ve already invested. Google co‑founder Larry Page has allegedly asserted (as relayed by investor Gavin Baker): “I am willing to go bankrupt rather than lose this race” [4]. — Read More
Daily Archives: October 31, 2025
The real problem with AI coding
The problem with AI coding isn’t technical debt. It’s comprehension debt.
And most teams don’t realize it until it’s too late.
…When you write code manually, you build up a clear mental model of the logic and trade-offs as you go. Every line you write, you understand why it exists. You see the edge cases. You know what alternatives you considered and rejected.
When AI writes code for you, that process inverts. You’re reverse-engineering someone else’s thinking after the fact. It’s like trying to learn calculus by reading a textbook instead of solving problems yourself.
… [V]olume amplifies the comprehension problem. You’re not just reverse-engineering one function. You’re reverse-engineering entire systems. — Read More
Signs of introspection in large language models
Have you ever asked an AI model what’s on its mind? Or to explain how it came up with its responses? Models will sometimes answer questions like these, but it’s hard to know what to make of their answers. Can AI systems really introspect—that is, can they consider their own thoughts? Or do they just make up plausible-sounding answers when they’re asked to do so?
Understanding whether AI systems can truly introspect has important implications for their transparency and reliability. If models can accurately report on their own internal mechanisms, this could help us understand their reasoning and debug behavioral issues. Beyond these immediate practical considerations, probing for high-level cognitive capabilities like introspection can shape our understanding of what these systems are and how they work. Using interpretability techniques, we’ve started to investigate this question scientifically, and found some surprising results.
Our new research provides evidence for some degree of introspective awareness in our current Claude models, as well as a degree of control over their own internal states. We stress that this introspective capability is still highly unreliable and limited in scope: we do not have evidence that current models can introspect in the same way, or to the same extent, that humans do. Nevertheless, these findings challenge some common intuitions about what language models are capable of—and since we found that the most capable models we tested (Claude Opus 4 and 4.1) performed the best on our tests of introspection, we think it’s likely that AI models’ introspective capabilities will continue to grow more sophisticated in the future. — Read More
On-Policy Distillation
LLMs are capable of expert performance in focused domains, a result of several capabilities stacked together: perception of input, knowledge retrieval, plan selection, and reliable execution. This requires a stack of training approaches[.]
… Smaller models with stronger training often outperform larger, generalist models in their trained domains of expertise. There are many benefits to using smaller models: they can be deployed locally for privacy or security considerations, can continuously train and get updated more easily, and save on inference costs. Taking advantage of these requires picking the right approach for the later stages of training.
Approaches to post-training a “student” model can be divided into two kinds:
Off-policy training relies on target outputs from some external source that the student learns to imitate.
On-policy training samples rollouts from the student model itself, and assigns them some reward.
We can do on-policy training via reinforcement learning, by grading each student rollout on whether it solves the question. This grading can be done by a human, or by a “teacher” model that reliably gets the correct answer. — Read More
LEVERAGING MACHINE LEARNING TO ENHANCE ACOUSTIC EAVESDROPPING ATTACKS
This multi-part series explores how machine learning can enhance eavesdropping on cellular audio using gyroscopes and accelerometers — inertial sensors commonly built into mobile devices to measure motion through Micro-Electro-Mechanical Systems (MEMS) technology. The research was conducted over the summer by one of our interns, Alec K., and a newly hired full-time engineer, August H.
Introduction
Acoustic eavesdropping attacks are a potentially devastating threat to the confidentiality of user information, especially if these attacks are implemented on smartphones, which are now ubiquitous. However, conventional microphone-based attacks are limited on smartphone devices by the fact that the user must consent to the collection of microphone information by applications. Recently, researchers on eavesdropping have taken to performing side-channel attacks that leverage information leaks from a piece of hardware to reconstruct some kind of secret (i.e. the audio we want to listen in on).
Unlike the microphone, which requires explicit user permission to access, sensors like the gyroscope and accelerometer do not require explicit user consent for an application to access their readings on Android. These devices are sensitive to the vibrations caused by sound, and since some Android devices allow sampling these sensors at frequencies up to 500 Hz, it is possible to reconstruct sound using these devices. — Read More
Rick's Cafe AI 