he Biden administration is proposing requiring U.S. cloud companies to determine whether foreign entities are accessing U.S. data centers to train AI models, U.S. Commerce Secretary Gina Raimondo said on Friday.
“We can’t have non-state actors or China or folks who we don’t want accessing our cloud to train their models,” Raimondo said in an interview with Reuters. “We use export controls on chips,” she noted. “Those chips are in American cloud data centers so we also have to think about closing down that avenue for potential malicious activity.” – Read More
Monthly Archives: January 2024
Meta releases ‘Code Llama 70B’, an open-source behemoth to rival private AI development
Meta AI, the company that brought you Llama 2, the gargantuan language model that can generate anything from tweets to essays, has just released a new and improved version of its code generation model, Code Llama 70B. This updated model can write code in various programming languages, such as Python, C++, Java and PHP, from natural language prompts or existing code snippets. And it can do it faster, better and more accurately than ever before. – Read More
The Cult of AI
I WAS WATCHING a video of a keynote speech at the Consumer Electronics Show for the Rabbit R1, an AI gadget that promises to act as a sort of personal assistant, when a feeling of doom took hold of me.
It wasn’t just that Rabbit’s CEO Jesse Lyu radiates the energy of a Kirkland-brand Steve Jobs. And it wasn’t even Lyu’s awkward demonstration of how the Rabbit’s camera can recognize a photo of Rick Astley and Rickroll the owner — even though that segment was so cringe it caused me chest pains.
No, the real foreboding came during a segment when Lyu breathlessly explained how the Rabbit could order pizza for you, telling it “the most-ordered option is fine,” leaving his choice of dinner up to the Pizza Hut website. After that, he proceeded to have the Rabbit plan an entire trip to London for him. The device very clearly just pulled a bunch of sights to see from some top-10 list on the internet, one that was very likely AI-generated itself.
Most of the Rabbit’s capabilities were well in line with existing voice-activated products, like Amazon Alexa. Its claim to being something special is its ability to create a “digital twin” of the user, which can directly utilize all of your apps so that you, the person, don’t have to. It can even use Midjourney to generate AI images for you, removing yet another level of human involvement and driving us all deeper into the uncanny valley. – Read More
New Theory Suggests Chatbots Can Understand Text
Artificial intelligence seems more powerful than ever, with chatbots like Bard and ChatGPT capable of producing uncannily humanlike text. But for all their talents, these bots still leave researchers wondering: Do such models actually understand what they are saying? “Clearly, some people believe they do,” said the AI pioneer Geoff Hinton in a recent conversation with Andrew Ng, “and some people believe they are just stochastic parrots.”
This evocative phrase comes from a 2021 paper co-authored by Emily Bender, a computational linguist at the University of Washington. It suggests that large language models (LLMs) — which form the basis of modern chatbots — generate text only by combining information they have already seen “without any reference to meaning,” the authors wrote, which makes an LLM “a stochastic parrot.”
These models power many of today’s biggest and best chatbots, so Hinton argued that it’s time to determine the extent of what they understand. The question, to him, is more than academic. “So long as we have those differences” of opinion, he said to Ng, “we are not going to be able to come to a consensus about dangers.”
New research may have intimations of an answer. A theory developed by Sanjeev Arora of Princeton University and Anirudh Goyal, a research scientist at Google DeepMind, suggests that the largest of today’s LLMs are not stochastic parrots. The authors argue that as these models get bigger and are trained on more data, they improve on individual language-related abilities and also develop new ones by combining skills in a manner that hints at understanding — combinations that were unlikely to exist in the training data. – Read More
A Theory for Emergence of Complex Skills in Language Models
Skill-Mix: a Flexible and Expandable Family of Evaluations for AI models
New Texas Center Will Create Generative AI Computing Cluster Among Largest of Its Kind
The University of Texas at Austin is creating one of the most powerful artificial intelligence hubs in the academic world to lead in research and offer world-class AI infrastructure to a wide range of partners.
UT is launching the Center for Generative AI, powered by a new GPU computing cluster, among the largest in academia. The cluster will comprise 600 NVIDIA H100s GPUs — short for graphics processing units, specialized devices to enable rapid mathematical computations, making them ideal for training AI models. The Texas Advanced Computing Center (TACC) will host and support the cluster, called Vista. – Read More
Self-Rewarding Language Models
We posit that to achieve superhuman agents, future models require superhuman feedback in order to provide an adequate training signal. Current approaches commonly train reward models from human preferences, which may then be bottlenecked by human performance level, and secondly these separate frozen reward models cannot then learn to improve during LLM training. In this work, we study Self-Rewarding Language Models, where the language model itself is used via LLM-as-a-Judge prompting to provide its own rewards during training. We show that during Iterative DPO training that not only does instruction following ability improve, but also the ability to provide high-quality rewards to itself. Fine-tuning Llama 2 70B on three iterations of our approach yields a model that outperforms many existing systems on the AlpacaEval 2.0 leaderboard, including Claude 2, Gemini Pro, and GPT-4 0613. While only a preliminary study, this work opens the door to the possibility of models that can continually improve in both axes. – Read More
PHOENIX: Open-Source Language Adaption for Direct Preference Optimization
Large language models have gained immense importance in recent years and have demonstrated outstanding results in solving various tasks. However, despite these achievements, many questions remain unanswered in the context of large language models. Besides the optimal use of the models for inference and the alignment of the results to the desired specifications, the transfer of models to other languages is still an underdeveloped area of research. The recent publication of models such as Llama-2 and Zephyr has provided new insights into architectural improvements and the use of human feedback. However, insights into adapting these techniques to other languages remain scarce. In this paper, we build on latest improvements and apply the Direct Preference Optimization(DPO) approach to the German language. The model is available at this https URL. – Read More
A new AI model called Morpheus-1 claims to induce lucid dreaming
Artificial intelligence has entered every aspect of our technological lives in the past few years, from chatbots to catflaps — but one company wants AI to enter your dreams.
Neurotechnology startup Prophetic has a new AI model called Morpheus-1 that it claims can help people both enter a lucid dream state and stabilize that dream.
Lucid dreaming is a state of dreaming where the dreamer is aware that they are dreaming and often has some control over the dream characters, narrative, and environment. It was the main plot device in Christopher Nolan’s confusing 2010 modern classic Inception. – Read More
OpenAI launches new generation of embedding models and other API updates
OpenAI, the artificial intelligence research company, announced on Thursday a new generation of embedding models, which can convert text into a numerical form that can be used for various machine learning tasks. The company also introduced new versions of its GPT-4 Turbo and moderation models, new API usage management tools, and lower pricing on its GPT-3.5 Turbo model. – Read More
Google’s latest AI video generator can render cute animals in implausible situations
On Tuesday, Google announced Lumiere, an AI video generator that it calls “a space-time diffusion model for realistic video generation” in the accompanying preprint paper. But let’s not kid ourselves: It does a great job of creating videos of cute animals in ridiculous scenarios, such as using roller skates, driving a car, or playing a piano. Sure, it can do more, but it is perhaps the most advanced text-to-animal AI video generator yet demonstrated. – Read More
Rick's Cafe AI 