Jeff Ding is the leading US scholar on China and AI and author of one of the earliest China-focused Substacks, ChinAI.
He recently published a fire paper called, “The diffusion deficit in scientific and technological power: re-assessing China’s rise.” It makes the argument that diffusion capacity (not just innovation capacity) is critical to economic growth — and China actually fares much worse in diffusion capacity than mainstream narratives imply. — Read More
Monthly Archives: May 2023
In Battle Over A.I., Meta Decides to Give Away Its Crown Jewels
The tech giant has publicly released its latest A.I. technology so people can build their own chatbots. Rivals like Google say that approach can be dangerous.
In February, Meta made an unusual move in the rapidly evolving world of artificial intelligence: It decided to give away its A.I. crown jewels.
The Silicon Valley giant, which owns Facebook, Instagram and WhatsApp, had created an A.I. technology, called LLaMA, that can power online chatbots. But instead of keeping the technology to itself, Meta released the system’s underlying computer code into the wild. Academics, government researchers and others who gave their email address to Meta could download the code once the company had vetted the individual.
Essentially, Meta was giving its A.I. technology away as open-source software — computer code that can be freely copied, modified and reused — providing outsiders with everything they needed to quickly build chatbots of their own.
… Its actions contrast with those of Google and OpenAI, the two companies leading the new A.I. arms race. — Read More
AI and the future of humanity
Meta Wars, Microsoft vs Meta: Who Will Come Out on Top?
The Meta vs Microsoft debate is that of both principle and the future of Metaverse technology. No doubt the Metaverse, as vague as the word is at the moment, can be seen by tech giants as a new world ripe for the taking. Two of the largest companies building the Metaverse are Meta, formerly Facebook, and Microsoft, are racing to stake their claim.
Now, of course, as mentioned, no one knows what the Metaverse will end up looking like. There are a few things we do know, however.
Firstly, the Metaverse is a virtual environment – or many of them – in which individuals can share their experiences and engage in real-time activities with other users in simulated settings.
Secondly, we do know what technologies will constitute the infrastructure on which it is built. — Read More
OpenAI bets $30M on this GPT-powered education appㅣSpeak
This AI-Powered, Point-Based Photo Manipulation System is Wild
Researchers have developed a point-based image manipulation system that uses generative artificial intelligence (AI) technology to allow users to precisely control the pose, shape, expression, and layout of objects.
The research outlines how users can control generative adversarial networks (GANs) with intuitive graphical control. The technology is called DragGAN. — Read More
Spider-Man by Wes Anderson Trailer (100% AI)
StableStudio is Stability AI’s latest commitment to open-source AI
Stability AI has announced StableStudio, a new open-source variant of its DreamStudio AI text-to-image web app.
Stability AI is releasing an open-source version of DreamStudio, a commercial interface for the company’s AI image generator model, Stable Diffusion. In a press statement on Wednesday, Stability AI said the new release — dubbed StableStudio — “marks a fresh chapter” for the platform and will serve as a showcase for the company’s “dedication to advancing open-source development.” — Read More
‘Low Background’ Content
and the possibility of a self-referential AI death spiral …
One of the unexpected side-effects of humanity’s entry into the nuclear age was a scramble for so-called ‘low-background’ steel. After the bombings of Hiroshima and Nagasaki and prolific atmospheric nuclear testing, new radioactive elements filled our atmosphere. As a result, due to the air injection process in steelmaking, any steel made after the summer of 1945 had an increased radioactive signature. For most uses, like cars or buildings, this didn’t matter. But for certain sensitive scientific and medical equipment, steel’s radioactivity became a real issue. Thus was created a market need for steel that was created in the less-radioactive atmosphere before 1945—low-background steel. Interestingly, a big source of this important resource came from the enthusiastic scrapping of sunken battleships, including the scuttled WWI German fleet.
I mention this because we are now entering another new age—the age of AI. The story of low-background steel came to mind recently as I started working with AI/ML companies in my consulting business. Like anybody with a healthy sense of self preservation, I’ve been immersing myself in this extraordinary, fast-moving revolution. I lived through several previous ones (personal computer, internet, smartphone), but I don’t remember any of them moving quite this fast. It’s obviously going to change just about every aspect of our lives. And the more I delve into the mechanics of Large Language Models and generative AI—and the more I watch AI’s light-speed propagation into daily life—the more I wonder if we’d crossed a line (in roughly the spring of 2022) where any content that existed before that moment should be considered “low-background content.” That is to say, content that was certifiably created by actual human beings rather than AI. Everything after should be considered suspect. — Read More
TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
Language models (LMs) are powerful tools for natural language processing, but they often struggle to produce coherent and fluent text when they are small. Models with around 125M parameters such as GPT-Neo (small) or GPT-2 (small) can rarely generate coherent and consistent English text beyond a few words even after extensive training. This raises the question of whether the emergence of the ability to produce coherent English text only occurs at larger scales (with hundreds of millions of parameters or more) and complex architectures (with many layers of global attention).
In this work, we introduce TinyStories, a synthetic dataset of short stories that only contain words that a typical 3 to 4-year-olds usually understand, generated by GPT-3.5 and GPT-4. We show that TinyStories can be used to train and evaluate LMs that are much smaller than the state-of-the-art models (below 10 million total parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate reasoning capabilities.
We also introduce a new paradigm for the evaluation of language models: We suggest a framework which uses GPT-4 to grade the content generated by these models as if those were stories written by students and graded by a (human) teacher. This new paradigm overcomes the flaws of standard benchmarks which often requires the model’s output to be very structures, and moreover provides a multidimensional score for the model, providing scores for different capabilities such as grammar, creativity and consistency.
We hope that TinyStories can facilitate the development, analysis and research of LMs, especially for low-resource or specialized domains, and shed light on the emergence of language capabilities in LMs. — Read More
Rick's Cafe AI 