Tag Archives: Robotics
Covariant Introduces RFM-1 to Give Robots the Human-like Ability to Reason
The key challenge with traditional robotic automation and automation based on manual programming or specialized learned models is the lack of reliability and flexibility in real-world scenarios. To create value at scale, robots must understand how to manipulate an unlimited array of items and scenarios autonomously.
By starting with warehouse pick and place operations, Covariant’s RFM-1 showcases the power of Robotics Foundation Models. In warehouse environments, the technology company’s approach of combining the largest real-world robot production dataset with a massive collection of Internet data is unlocking new levels of robotic productivity and shows a path to broader industry applications ranging from hospitals and homes to factories, stores, restaurants, and more. — Read More
RT-2: New model translates vision and language into action
Robotic Transformer 2 (RT-2) is a novel vision-language-action (VLA) model that learns from both web and robotics data, and translates this knowledge into generalised instructions for robotic control.
High-capacity vision-language models (VLMs) are trained on web-scale datasets, making these systems remarkably good at recognising visual or language patterns and operating across different languages. But for robots to achieve a similar level of competency, they would need to collect robot data, first-hand, across every object, environment, task, and situation.
In our paper, we introduce Robotic Transformer 2 (RT-2), a novel vision-language-action (VLA) model that learns from both web and robotics data, and translates this knowledge into generalised instructions for robotic control, while retaining web-scale capabilities. — Read More
The AI Behind the Curtain: AI Start-Up Figure Shows Off Conversational Robot Infused With OpenAI Technology
Robotics developer Figure made waves on Wednesday when it shared a video demonstration of its first humanoid robot engaged in a real-time conversation, thanks to generative AI from OpenAI.
… The company explained that its recent alliance with OpenAI brings high-level visual and language intelligence to its robots, allowing for “fast, low-level, dexterous robot actions.”
… [Figure 01] can: – describe its visual experience – plan future actions – reflect on its memory – explain its reasoning verbally. — Read More
Figure 01 Status Update – OpenAI Speech-to-Speech Reasoning
An OpenAI spinoff has built an AI model that helps robots learn tasks like humans
In the summer of 2021, OpenAI quietly shuttered its robotics team, announcing that progress was being stifled by a lack of data necessary to train robots in how to move and reason using artificial intelligence.
Now three of OpenAI’s early research scientists say the startup they spun off in 2017, called Covariant, has solved that problem and unveiled a system that combines the reasoning skills of large language models with the physical dexterity of an advanced robot.
The new model, called RFM-1, was trained on years of data collected from Covariant’s small fleet of item-picking robots that customers like Crate & Barrel and Bonprix use in warehouses around the world, as well as words and videos from the internet. In the coming months, the model will be released to Covariant customers. The company hopes the system will become more capable and efficient as it’s deployed in the real world. — Read More
The Cult of AI
I WAS WATCHING a video of a keynote speech at the Consumer Electronics Show for the Rabbit R1, an AI gadget that promises to act as a sort of personal assistant, when a feeling of doom took hold of me.
It wasn’t just that Rabbit’s CEO Jesse Lyu radiates the energy of a Kirkland-brand Steve Jobs. And it wasn’t even Lyu’s awkward demonstration of how the Rabbit’s camera can recognize a photo of Rick Astley and Rickroll the owner — even though that segment was so cringe it caused me chest pains.
No, the real foreboding came during a segment when Lyu breathlessly explained how the Rabbit could order pizza for you, telling it “the most-ordered option is fine,” leaving his choice of dinner up to the Pizza Hut website. After that, he proceeded to have the Rabbit plan an entire trip to London for him. The device very clearly just pulled a bunch of sights to see from some top-10 list on the internet, one that was very likely AI-generated itself.
Most of the Rabbit’s capabilities were well in line with existing voice-activated products, like Amazon Alexa. Its claim to being something special is its ability to create a “digital twin” of the user, which can directly utilize all of your apps so that you, the person, don’t have to. It can even use Midjourney to generate AI images for you, removing yet another level of human involvement and driving us all deeper into the uncanny valley. – Read More
A Robot the Size of the World
…The classical definition of a robot is something that senses, thinks, and acts—that’s today’s Internet. We’ve been building a world-sized robot without even realizing it.
In 2023, we upgraded the “thinking” part with large-language models (LLMs) like GPT. ChatGPT both surprised and amazed the world with its ability to understand human language and generate credible, on-topic, humanlike responses. But what these are really good at is interacting with systems formerly designed for humans. Their accuracy will get better, and they will be used to replace actual humans.
In 2024, we’re going to start connecting those LLMs and other AI systems to both sensors and actuators. In other words, they will be connected to the larger world, through APIs. They will receive direct inputs from our environment, in all the forms I thought about in 2016. And they will increasingly control our environment, through IoT devices and beyond. – Read More
ChatGPT for chemistry: AI and robots join forces to build new materials
An autonomous system that combines robotics with artificial intelligence (AI) to create entirely new materials has released its first trove of discoveries. The system, known as the A-Lab, devises recipes for materials, including some that might find uses in batteries or solar cells. Then, it carries out the synthesis and analyses the products — all without human intervention. Meanwhile, another AI system has predicted the existence of hundreds of thousands of stable materials, giving the A-Lab plenty of candidates to strive for in future. — Read More
Read the Paper
NOIR: Neural Signal Operated Intelligent Robots for Everyday Activities
We present Neural Signal Operated Intelligent Robots (NOIR), a general-purpose, intelligent brain-robot interface system that enables humans to command robots to perform everyday activities through brain signals. Through this interface, humans communicate their intended objects of interest and actions to the robots using electroencephalography (EEG). Our novel system demonstrates success in an expansive array of 20 challenging, everyday household activities, including cooking, cleaning, personal care, and entertainment. The effectiveness of the system is improved by its synergistic integration of robot learning algorithms, allowing for NOIR to adapt to individual users and predict their intentions. Our work enhances the way humans interact with robots, replacing traditional channels of interaction with direct, neural communication. Project website: this https URL. — Read More
Rick's Cafe AI 