Rhodes Scholar Jeff Ding, author of the ChinAI newsletter, breaks down how China stacks up to the rest of the world in the race to develop AI, first, in this October 2018 interview with Jordan Schneider and then in this May 2019 interview with James Wang. His ChinAI newsletter archives are available here — Read More
Monthly Archives: July 2019
Natural Adversarial Examples
We introduce natural adversarial examples – real-world, unmodified, and naturally occurring examples that cause classifier accuracy to significantly degrade. We curate 7,500 natural adversarial examples and release them in an ImageNet classifier test set that we call IMAGENET-A. This dataset serves as a new way to measure classifier robustness. Like `p adversarial examples, IMAGENET-A examples successfully transfer to unseen or black-box classifiers. For example, on IMAGENET-A a DenseNet-121 obtains around 2% accuracy, an accuracy drop of approximately 90%. Recovering this accuracy is not simple because IMAGENET-A examples exploit deep flaws in current classifiers including their over-reliance on color, texture, and background cues. We observe that popular training techniques for improving robustness have little effect, but we show that some architectural changes can enhance robustness to natural adversarial examples. Future research is required to enable robust generalization to this hard ImageNet test set. Read More
Hype And Reality In Chinese Artificial Intelligence
In MIT Technology Review, Jeff Ding shares five takeaways from his experience writing about and translating Chinese-language writing about artificial intelligence (AI) research in China. Ding is a researcher at the University of Oxford who has now published 48 issues of his insightful ChinAI newsletter.
For more discussion of U.S.-China technology connections, listen to this recent Sinica Podcast with Samm Sacks. You can also listen to a ChinaEconTalk interview with Jeff Ding here on SupChina. Read More
Much Ado About Data: How America and China Stack Up
Analysts often cite the amount of data in China as a core advantage of its artificial intelligence (AI) ecosystem compared to the United States. That’s true to a certain extent: 1.4 billion people + deep smartphone penetration + 24/7 online and offline data collection = staggering amount of data.
But the reality is far more complex, because data is not a single-dimensional input into AI, something that China simply has “more” of. The relationship between data and AI prowess is analogous to the relationship between labor and the economy. China may have an abundance of workers, but the quality, structure, and mobility of that labor force is just as important to economic development. Read More
A Deep Generative Model for Graph Layout
Different layouts can characterize different aspects of the same graph. Finding a “good” layout of a graph is thus animportant task for graph visualization. In practice, users often visualize a graph in multiple layouts by using different methods andvarying parameter settings until they find a layout that best suits the purpose of the visualization. However, this trial-and-error processis often haphazard and time-consuming. To provide users with an intuitive way to navigate the layout design space, we presenta technique to systematically visualize a graph in diverse layouts using deep generative models. We design an encoder-decoderarchitecture to learn a model from a collection of example layouts, where the encoder represents training examples in a latent spaceand the decoder produces layouts from the latent space. In particular, we train the model to construct a two-dimensional latent spacefor users to easily explore and generate various layouts. We demonstrate our approach through quantitative and qualitative evaluationsof the generated layouts. The results of our evaluations show that our model is capable of learning and generalizing abstract conceptsof graph layouts, not just memorizing the training examples. In summary, this paper presents a fundamentally new approach to graphvisualization where a machine learning model learns to visualize a graph from examples without manually-defined heuristics. Read More
Parrotron: An End-to-End Speech-to-Speech Conversion Model and its Applications to Hearing-Impaired Speech and Speech Separation
We describe Parrotron, an end-to-end-trained speech-to-speechconversion model that maps an input spectrogram directly toanother spectrogram, without utilizing any intermediate discreterepresentation. The network is composed of an encoder, spectro-gram and phoneme decoders, followed by a vocoder to synthe-size a time-domain waveform. We demonstrate that this modelcan be trained to normalize speech from any speaker regardlessof accent, prosody, and background noise, into the voice of asinglecanonical target speaker with a fixed accent and consistentarticulation and prosody. We further show that this normalizationmodel can be adapted to normalize highly atypical speech froma deaf speaker, resulting in significant improvements in intelli-gibility and naturalness, measured via a speech recognizer andlistening tests. Finally, demonstrating the utility of this modelon other speech tasks, we show that the same model architecturecan be trained to perform a speech separation task.Index Terms: speech normalization, voice conversion, atypicalspeech, speech synthesis, sequence-to-sequence mode. Read More
Learning with Hierarchical-Deep Models
We introduce HD (or “Hierarchical-Deep”) models, a new compositional learning architecture that integrates deep learning models with structured hierarchical Bayesian (HB) models. Specifically, we show how we can learn a hierarchical Dirichlet process (HDP) prior over the activities of the top-level features in a deep Boltzmann machine (DBM). This compound HDP-DBM model learns to learn novel concepts from very few training example by learning low-level generic features, high-level features that capture correlations among low-level features, and a category hierarchy for sharing priors over the high-level features that are typical of different kinds of concepts. We present efficient learning and inference algorithms for the HDP-DBM model and show that it is able to learn new concepts from very few examples on CIFAR-100 object recognition, handwritten character recognition, and human motion capture datasets. Read More
Hierarchical Compositional Feature Learning
We introduce the hierarchical compositional network (HCN), a directed generative model able to discover and disentangle, without supervision, the building blocks of a set of binary images. The building blocks are binary features defined hierarchically as a composition of some of the features in the layer immediately below, arranged in a particular manner. At a high level, HCN is similar to a sigmoid belief network with pooling. Inference and learning in HCN are very challenging and existing variational approximations do not work satisfactorily. A main contribution of this work is to show that both can be addressed using max-product message passing (MPMP) with a particular schedule (no EM required). Also, using MPMP as an inference engine for HCN makes new tasks simple: adding supervision information, classifying images, or performing inpainting all correspond to clamping some variables of the model to their known values and running MPMP on the rest. When used for classification, fast inference with HCN has exactly the same functional form as a convolutional neural network (CNN) with linear activations and binary weights. However, HCN’s features are qualitatively very different . Read More
AI pioneer: ‘The dangers of abuse are very real’
Yoshua Bengio is one of three computer scientists who last week shared the US$1-million A. M. Turing award — one of the field’s top prizes.
But alongside his research, Bengio, who is also scientific director of the Montreal Institute for Learning Algorithms (MILA), has raised concerns about the possible risks from misuse of technology. In December, he presented a set of ethical guidelines for AI called the Montreal declaration at the Neural Information Processing Systems (NeurIPS) meeting in the city. Read More
What Every NLP Engineer Needs to Know About Pre-Trained Language Models
Practical applications of Natural Language Processing (NLP) have gotten significantly cheaper, faster, and easier due to the transfer learning capabilities enabled by pre-trained language models. Transfer learning enables engineers to pre-train an NLP model on one large dataset and then quickly fine-tune the model to adapt to other NLP tasks.
This new approach enables NLP models to learn both lower-level and higher-level features of language, leading to much better model performance for virtually all standard NLP tasks and a new standard for industry best practices.
To help you quickly understand the significance of this technical achievement and how it accelerates your own work in NLP, we’ve summarized the key lessons you should know in easy-to-read bullet-point format. We’ve also included summaries of the 3 most important research papers in the space that you need to be aware of. Read More
Rick's Cafe AI 