Two great resources for those wanting to track the when and the how of AI progress.
AI Timelines is the discussion of how long until various major milestones in AI progress are achieved, whether it’s the timeline until a human-level AI is developed, the timeline until certain benchmarks are defeated, the timeline until we can simulate a mouse-level intelligence, or something else.
AI Takeoff refers to the process of an Artificial General Intelligence going from a certain threshold of capability (often discussed as “human-level”) to being super-intelligent and capable enough to control the fate of civilization. There has been much debate about whether AI takeoff is more likely to be slow vs fast, i.e., “soft” vs “hard”.
Monthly Archives: March 2023
Sparks of Artificial General Intelligence:Early experiments with GPT-4
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition. The latest model developed by OpenAI, GPT-4 [Ope23], was trained using an unprecedented scale of compute and data. In this paper, we report on our investigation of an early version of GPT-4, when it was still in active development by OpenAI. We contend that (this early version of) GPT4 is part of a new cohort of LLMs (along with ChatGPT and Google’s PaLM for example) that exhibit more general intelligence than previous AI models. We discuss the rising capabilities and implications of these models. We demonstrate that, beyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more, without needing any special prompting. Moreover, in all of these tasks, GPT-4’s performance is strikingly close to human-level performance, and often vastly surpasses prior models such as ChatGPT. Given the breadth and depth of GPT-4’s capabilities, we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system. In our exploration of GPT-4, we put special emphasis on discovering its limitations, and we discuss the challenges ahead for advancing towards deeper and more comprehensive versions of AGI, including the possible need for pursuing a new paradigm that moves beyond next-word prediction. We conclude with reflections on societal influences of the recent technological leap and future research directions. Read More
Songwriting at the Dawn of AI: When Machines Can Write, Who Is the Artist?
The United States Copyright Office recently issued new guidelines regarding copyright applications for works created with artificial intelligence tools. The new rules recognize that work made with both AI input and human creation can be eligible for copyright protection, but any part of it that is entirely made by AI is not eligible. Which is to say, copyright protections can extend only to work that is attributable to human authorship.
“If a work’s traditional elements of authorship were produced by a machine, the work lacks human authorship and the Office will not register it,” the report states. “For example, when an AI technology receives solely a prompt from a human and produces complex written, visual, or musical works in response, the ‘traditional elements of authorship’ are determined and executed by the technology—not the human user.” The report hypothesizes that a work that creatively combines AI-generated elements into something new or AI-generated work that an artist then heavily modifies after the fact would indeed be eligible. Read More
How a tiny company with few rules is making fake images go mainstream
Midjourney, the year-old firm behind recent fake visuals of Trump and the pope, illustrates the lack of oversight accompanying spectacular strides in AI
The AI image generator Midjourney has quickly become one of the internet’s most eye-catching tools, creating realistic-looking fake visuals of former president Donald Trump being arrested and Pope Francis wearing a stylish coat with the aim of “expanding the imaginative powers of the human species.”
But the year-old company, run out of San Francisco with only a small collection of advisers and engineers, also has unchecked authority to determine how those powers are used. Read More
Deep Learning Is Hitting a Wall
What would it take for artificial intelligence to make real progress?
Let me start by saying a few things that seem obvious,” Geoffrey Hinton, “Godfather” of deep learning, and one of the most celebrated scientists of our time, told a leading AI conference in Toronto in 2016. “If you work as a radiologist you’re like the coyote that’s already over the edge of the cliff but hasn’t looked down.” Deep learning is so well-suited to reading images from MRIs and CT scans, he reasoned, that people should “stop training radiologists now” and that it’s “just completely obvious within five years deep learning is going to do better.”
Fast forward to 2022, and not a single radiologist has been replaced. Rather, the consensus view nowadays is that machine learning for radiology is harder than it looks; at least for now, humans and machines complement each other’s strengths. Read More
How to Create an AI Generated Video with ChatGPT, Synthesia, and Descript
Learn how we created an AI generated video with a ChatGPT script, a Synthesia avatar and voice, and stock footage from Descript.
There is a lot of buzz around new and exciting artificial intelligence (AI) and machine learning (ML) tools for video production and video creation. So, I wanted to see first-hand how some of these tools perform! As an experiment, I set out to create a high quality video using generative AI in less than 15 minutes. Read More
Andrew Ng Weighs In on Call for Pause
1/The call for a 6 month moratorium on making AI progress beyond GPT-4 is a terrible idea. I’m seeing many new applications in education, healthcare, food, … that’ll help many people. Improving GPT-4 will help. Lets balance the huge value AI is creating vs. realistic risks.
2/There is no realistic way to implement a moratorium and stop all teams from scaling up LLMs, unless governments step in. Having governments pause emerging technologies they don’t understand is anti-competitive, sets a terrible precedent, and is awful innovation policy.
Read More
The Path From APIs to Containers
Explore how microservices fueled the journey from APIs to containers and paved the way for enhanced API development and software integration.
In recent years, the rise of microservices has drastically changed the way we build and deploy software. The most important aspect of this shift has been the move from traditional API architectures driven by monolithic applications to containerized microservices. This shift not only improved the scalability and flexibility of our systems, but it has also given rise to new ways of software development and deployment approaches.
In this article, we will explore the path from APIs to containers and examine how microservices have paved the way for enhanced API development and software integration. Read More
Cerebras releases seven large language models for generative AI, trained on its specialized hardware
Artificial intelligence chipmaker Cerebras Systems Inc. today announced it has trained and now released seven GPT-based large language models for generative AI, making them available to the wider research community.
The new LLMs are notable as they are the first to be trained using CS-2 systems in the Cerebras Andromeda AI supercluster, which are powered by the Cerebras WSE-2 chip that is specifically designed to run AI software. In other words, they’re among the first LLMs to be trained without relying on graphics processing unit-based systems. Cerebras said it’s sharing not only the models, but also the weights and training recipe that was used, via a standard Apache 2.0 license. Read More
Transformers are Sample-Efficient World Models
Deep reinforcement learning agents are notoriously sample inefficient, which considerably limits their application to real-world problems. Recently, many model-based methods have been designed to address this issue, with learning in the imagination of a world model being one of the most prominent approaches. However, while virtually unlimited interaction with a simulated environment sounds appealing, the world model has to be accurate over extended periods of time. Motivated by the success of Transformers in sequence modeling tasks, we introduce IRIS, a data-efficient agent that learns in a world model composed of a discrete autoencoder and an autoregressive Transformer. With the equivalent of only two hours of gameplay in the Atari 100k benchmark, IRIS achieves a mean human normalized score of 1.046, and outperforms humans on 10 out of 26 games, setting a new state of the art for methods without lookahead search. To foster future research on Transformers and world models for sample-efficient reinforcement learning, we release our code and models at https://github.com/eloialonso/iris.
Read More
Rick's Cafe AI 