In this paper, we give an overview of commonsense reasoning in natural language processing, which requires a deeper understanding of the contexts and usually involves inference over implicit external knowledge. We first review some popular commonsense knowledge bases and commonsense reasoning benchmarks, but give more emphasis on the methodologies, including recent approaches that aim at solving some general natural language problems that take advantage of external knowledge bases. Finally, we discuss some future directions in pushing the boundary of commonsense reasoning in natural language processing. Read More
Monthly Archives: August 2021
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities, and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature. Read More
BeingAI unveils human-like AI character named Zbee
BeingAI is creating virtual beings with artificial intelligence. And its first AI being is a virtual character named Zbee.
Zbee can exist on different platforms and so can interact with people anytime, anywhere, to bring humanness and gamification into digital experiences. Zbee will come with an engaging personality and personal stories. They will offer friendship, entertainment, and mentorship, just like in the movie Her. Read More
Announcing AI21 Studio and Jurassic-1 Language Models
AI21 Labs’ new developer platform offers instant access to our 178B-parameter language model, to help you build sophisticated text-based AI applications at scale
We are thrilled to announce the launch of AI21 Studio, our new developer platform where you can use our state-of-the-art Jurassic-1 language models to build your own applications and services. Jurassic-1 models come in two sizes, where the Jumbo version, at 178B parameters, is the largest and most sophisticated language model ever released for general use by developers. AI21 Studio is currently in open beta, allowing anyone to sign up and immediately start querying Jurassic-1 using our API and interactive web environment. Read More
Elon Musk unveils Tesla Bot, a humanoid robot that uses vehicle AI
“It’s intended to be friendly,” the carmaker’s CEO joked.
Tesla CEO Elon Musk on Thursday unveiled a humanoid robot called the Tesla Bot that runs on the same AI used by Tesla’s fleet of autonomous vehicles. A functioning version of the robot didn’t make an appearance during Musk’s reveal, though a slightly bizarre dance by a performer dressed like a Tesla Bot did.
The unexpected reveal came at the end of Tesla’s AI Day presentation, with Musk providing few details about the slightly creepy, Slenderman-like robot beyond a few PowerPoint slides. The 5-foot-8-inch robot is expected to weigh in at 125 pounds and be built from “lightweight materials,” he said. Read More
Zion Wants to Free Social Media from Big Tech Through the Lightning Network
“
Social network Zion onboards users to the Lightning Network to free them from big tech censorship, revenue restrictions and data collection.
Announced earlier today, Bitcoin-based social network Zion has launched with the mission of eliminating reliance on large tech companies for content creation and sharing, providing an alternative that does not collect user data, censor speech or withhold portions of payments meant for content creators.
“With Zion, technology companies are completely eliminated and users interact in a secure, censorship-resistant, private utility that facilitates free and open flow of content and payments between users and their audiences,” according to a press release from the platform. “There is no way to manipulate how users experience the network and absolutely no data is collected, ever.” Read More
Warner Bros. ‘Reminiscence’ promo uses deepfake tech to put you in the trailer
If you want to see yourself on screen with Hugh Jackman, this is your chance. The promo for Warner Bros. upcoming Reminiscence movie uses deepfake technology to turn a photo of your face — or anybody’s face, really — into a short video sequence with the star. According to Protocol, a media startup called D-ID created the promo for the film. D-ID reportedly started out wanting to develop technology that can protect consumers against facial recognition, but then it realized that its tech could also be used to optimize deepfakes.
For this particular project, the firm created a website for the experience, where you’ll be asked for your name and for a photo. You can upload the photo of anybody you want, and the experience will then conjure up an animation for the face in it. The animation isn’t perfect by any means, and the face could look distorted at times, but it’s still not bad, considering the technology created it from a single picture. Read More
Amazon taps its SocialBot challenge to boost conversational AI
Earlier this week, Amazon announced the winners of its annual Alexa Prize SocialBot Grand Challenge, which promotes research into coherence, context awareness, fluency of response, and other areas fundamental to the future of conversational AI. Participating university teams design social bots for Alexa-enabled devices and can validate their ideas by directly engaging with Amazon’s millions of Alexa customers.
But the competition isn’t just a way for participants to experiment and earn research grants. Each research team maintains ownership of the intellectual property in its systems, and a win might mean an opportunity to integrate their research into Amazon’s future plans. Some fairly significant advancements in conversational AI and subsequent scientific papers typically come out of the event. Read More
Practical Deep Learning for Coders 10
Program Synthesis with Large Language Models
This paper explores the limits of the current generation of large language models for program synthesis in general purpose programming languages. We evaluate a collection of such models (with between 244M and 137B parameters) on two new benchmarks, MBPP and MathQA-Python, in both the few-shot and fine-tuning regimes. Our benchmarks are designed to measure the ability of these models to synthesize short Python programs from natural language descriptions. The Mostly Basic Programming Problems (MBPP) dataset contains 974 programming tasks, designed to be solvable by entry-level programmers. The MathQA-Python dataset, a Python version of the MathQA benchmark, contains 23914 problems that evaluate the ability of the models to synthesize code from more complex text. On both datasets, we find that synthesis performance scales log-linearly with model size. Our largest models, even without finetuning on a code dataset, can synthesize solutions to 59.6% of the problems from MBPP using few-shot learning with a well-designed prompt. Fine-tuning on a held-out portion of the dataset improves performance by about 10 percentage points across most model sizes. On the MathQA-Python dataset, the largest fine-tuned model achieves 83.8% accuracy. Going further, we study the model’s ability to engage in dialog about code, incorporating human feedback to improve its solutions. We find that natural language feedback from a human halves the error rate compared to the model’s initial prediction. Additionally, we conduct an error analysis to shed light on where these models fall short and what types of programs are most difficult to generate. Finally, we explore the semantic grounding of these models by fine-tuning them to predict the results of program execution. We find that even our best models are generally unable to predict the output of a program given a specific input. Read More
#devops, #nlp
Rick's Cafe AI 