When you live in tech bubble central, signs of a tech bubble become easier to spot every time. Drive to Silicon Valley on any of the Bay Area’s main arteries right now, and you’ll notice nearly every billboard pumping a product “driven by AI.”
On the same drive five years ago, you’d see the same scene with the word “blockchain.” Ten years ago: “big data.” Twenty-five years ago: literally any word followed by “.com.” Each one in turn, for all its promise, became a punchline.
It’s not a question of whether the Silicon Valley machine was wrong on any of these technologies. Especially not the dotcom thing. Heck, the entire internet had just dropped into our laps in the 1990s; you can’t blame anyone for dreaming about creating all the stuff we now take for granted. It’s a question of impatience: all the investors, startup shysters and panicked CEOs that rush in when a promising new technology emerges are eager for immediate results. — Read More
Daily Archives: August 7, 2024
Why AI’s Tom Cruise problem means it is ‘doomed to fail’
LLMs’ ‘reversal curse’ leads it to fail at drawing relationships between simple facts. It’s a problem that could prove fatal
In 2021, linguist Emily Bender and computer scientist Timnit Gebru published a paper that described the then-nascent field of language models as one of “stochastic parrots”. A language model, they wrote, “is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning.”
… If a human learns the fact, “Valentina Tereshkova was the first woman to travel to space”, they can also correctly answer, “Who was the first woman to travel to space?” This is such a basic form of generalization that it seems trivial. Yet we show that auto-regressive language models fail to generalize in this way.
This is an instance of an ordering effect we call the Reversal Curse.
[R]esearchers “taught” a bunch of fake facts to large language models, and found time and again that they simply couldn’t do the base work of inferring the reverse. — Read More
Rick's Cafe AI 