… [T]he models we now write code with learned from a web that is already old. I made the model gap to show this concretely: measured in Chrome releases (I know, I know, the web is far broader than just Chrome, but also Chrome has easy data to access on chromestatus.com), even the freshest model is several versions behind, and most are ten to twenty behind. The “knowledge” cutoff is a serious issue for the web platform, and the ecosystem of libraries and tools that are being launched but are not easily available to these models is massively gaining traction (Claude Code).

That connects to model half-life, where I looked at how quickly models are superseded, and to dead framework theory: if a framework stops appearing in fresh training data, the models stop reaching for it, and the framework quietly dies regardless of its merits. I wrote this thesis at least 6 months ago, and I think I’ve been proven correct (which is why we built Modern Web Guidance). The flip side, though, is that I’ve found guided output getting better than what people create (I think auto-research loops to optimize web performance, as an example, will massively raise the bar for the quality of the web people experience). — Read More