We introduce HD (or “Hierarchical-Deep”) models, a new compositional learning architecture that integrates deep learning models with structured hierarchical Bayesian (HB) models. Specifically, we show how we can learn a hierarchical Dirichlet process (HDP) prior over the activities of the top-level features in a deep Boltzmann machine (DBM). This compound HDP-DBM model learns to learn novel concepts from very few training example by learning low-level generic features, high-level features that capture correlations among low-level features, and a category hierarchy for sharing priors over the high-level features that are typical of different kinds of concepts. We present efficient learning and inference algorithms for the HDP-DBM model and show that it is able to learn new concepts from very few examples on CIFAR-100 object recognition, handwritten character recognition, and human motion capture datasets. Read More
Daily Archives: July 18, 2019
Hierarchical Compositional Feature Learning
We introduce the hierarchical compositional network (HCN), a directed generative model able to discover and disentangle, without supervision, the building blocks of a set of binary images. The building blocks are binary features defined hierarchically as a composition of some of the features in the layer immediately below, arranged in a particular manner. At a high level, HCN is similar to a sigmoid belief network with pooling. Inference and learning in HCN are very challenging and existing variational approximations do not work satisfactorily. A main contribution of this work is to show that both can be addressed using max-product message passing (MPMP) with a particular schedule (no EM required). Also, using MPMP as an inference engine for HCN makes new tasks simple: adding supervision information, classifying images, or performing inpainting all correspond to clamping some variables of the model to their known values and running MPMP on the rest. When used for classification, fast inference with HCN has exactly the same functional form as a convolutional neural network (CNN) with linear activations and binary weights. However, HCN’s features are qualitatively very different . Read More
AI pioneer: ‘The dangers of abuse are very real’
Yoshua Bengio is one of three computer scientists who last week shared the US$1-million A. M. Turing award — one of the field’s top prizes.
But alongside his research, Bengio, who is also scientific director of the Montreal Institute for Learning Algorithms (MILA), has raised concerns about the possible risks from misuse of technology. In December, he presented a set of ethical guidelines for AI called the Montreal declaration at the Neural Information Processing Systems (NeurIPS) meeting in the city. Read More
What Every NLP Engineer Needs to Know About Pre-Trained Language Models
Practical applications of Natural Language Processing (NLP) have gotten significantly cheaper, faster, and easier due to the transfer learning capabilities enabled by pre-trained language models. Transfer learning enables engineers to pre-train an NLP model on one large dataset and then quickly fine-tune the model to adapt to other NLP tasks.
This new approach enables NLP models to learn both lower-level and higher-level features of language, leading to much better model performance for virtually all standard NLP tasks and a new standard for industry best practices.
To help you quickly understand the significance of this technical achievement and how it accelerates your own work in NLP, we’ve summarized the key lessons you should know in easy-to-read bullet-point format. We’ve also included summaries of the 3 most important research papers in the space that you need to be aware of. Read More
The Journalism AI global survey: what we’ve learned so far
Over the last few weeks, newsrooms from all over the world have been completing our Journalism AI survey. Contributions came in from Europe, South and North America, Africa, and Asia. We’re immensely grateful to each and every one of the respondents. When we launched Journalism AI earlier this year, we had just a vague picture of how artificial intelligence technologies were being applied to journalism. And that is why we started this ambitious investigation, with the support of the Google News Initiative.
Now, thanks to the invaluable expertise and insights that news organisations are sharing with us through meetings, one-on-one in-depth interviews, and answers to the survey, we can start painting a more detailed picture of what AI actually means for journalism — and what it is likely to mean in the immediate future. Read More
How AI companies can avoid ethics washing
One of the essential phrases necessary to understand AI in 2019 has to be “ethics washing.” Put simply, ethics washing — also called “ethics theater” — is the practice of fabricating or exaggerating a company’s interest in equitable AI systems that work for everyone. A textbook example for tech giants is when a company promotes “AI for good” initiatives with one hand while selling surveillance capitalism tech to governments and corporate customers with the other. Read More
This AI magically removes moving objects from videos
We’ve previously seen developers harness the power of artificial intelligence (AI) to turn pitch black pics into bright colorful photos, flat images into complex 3D scenes, and selfies into moving avatars. Now, there’s an AI-powered software that effortlessly removes moving objects from videos.
All you need to do to wipe an object from footage is draw a box around it, and the software takes care of the rest for you. Read More
Rick's Cafe AI 