The roles played by learning and memorization represent an important topic in deep learning research. Recent work on this subject has shown that the optimization behavior of DNNs trained on shuffled labels is qualitatively different from DNNs trained with real labels. Here, we propose a novel permutation approach that can differentiate memorization from learning in deep neural networks (DNNs) trained as usual (i.e., us-ing the real labels to guide the learning, rather than shuffled labels). The evaluation of weather the DNN has learned and/or memorized, happens in a separate step where we compare the predictive performance of a shallow classifier trained with the features learned by the DNN, against multiple instances of the same classifier, trained on the same input, but using shuffled labels as outputs. By evaluating these shallow classifiers in validation sets that share structure with the training set, we are able to tell apart learning from memorization. Application of our permutation approach to multilayer perceptrons and convolutional neural networks trained on image data corroborated many findings from other groups. Most importantly,our illustrations also uncovered interesting dynamic patterns about how DNNs memorize over increasing numbers of training epochs, and support the surprising result that DNNs are still able to learn, rather than only memorize, when trained with pure Gaussian noise as input. Read More
Monthly Archives: March 2019
Understanding deep learning requires rethinking generalization
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance. Conventional wisdom attributes small generalization error either to properties of the model family, or to the regularization techniques used during training.Through extensive systematic experiments, we show how these traditional approaches fail to explain why large neural networks generalize well in practice.Specifically, our experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a ran-dom labeling of the training data. This phenomenon is qualitatively unaffected by explicit regularization, and occurs even if we replace the true images by completely unstructured random noise. We corroborate these experimental findings with a theoretical construction showing that simple depth two neural networks already have perfect finite sample expressivity as soon as the number of parameters exceeds the number of data points as it usually does in practice.We interpret our experimental findings by comparison with traditional models. Read More
Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation
Deep Neural Networks (DNNs) have been widely applied invarious recognition tasks. However, recently DNNs have been shown tobe vulnerable against adversarial examples, which can mislead DNNs tomake arbitrary incorrect predictions. While adversarial examples are wellstudied in classification tasks, other learning problems may have differ-ent properties. For instance, semantic segmentation requires additionalcomponents such as dilated convolutions and multiscale processing. Inthis paper, we aim to characterize adversarial examples based on spatialcontext information in semantic segmentation. We observe that spatialconsistency information can be potentially leveraged to detect adversar-ial examples robustly even when a strong adaptive attacker has accessto the model and detection strategies. We also show that adversarialexamples based on attacks considered within the paper barely transferamong models, even though transferability is common in classification.Our observations shed new light on developing adversarial attacks anddefenses to better understand the vulnerabilities of DNNs. Read More
Characterizing audio adversarial examples using temporal dependency
Recent studies have highlighted adversarial examples as a ubiquitous threat to dif-ferent neural network models and many downstream applications. Nonetheless,as unique data properties have inspired distinct and powerful learning principles,this paper aims to explore their potentials towards mitigating adversarial inputs.In particular, our results reveal the importance of using the temporal dependencyin audio data to gain discriminate power against adversarial examples. Tested onthe automatic speech recognition (ASR) tasks and three recent audio adversarialattacks, we find that (i) input transformation developed from image adversarial de-fense provides limited robustness improvement and is subtle to advanced attacks;(ii) temporal dependency can be exploited to gain discriminative power againstaudio adversarial examples and is resistant to adaptive attacks considered in ourexperiments. Our results not only show promising means of improving the robust-ness of ASR systems, but also offer novel insights in exploiting domain-specificdata properties to mitigate negative effects of adversarial examples. Read More
Generating Adversarial Examples with Adversarial Networks
Deep neural networks (DNNs) have been foundto be vulnerable to adversarial examples resultingfrom adding small-magnitude perturbations to in-puts. Such adversarial examples can mislead DNNsto produce adversary-selected results. Different at-tack strategies have been proposed to generate ad-versarial examples, but how to produce them withhigh perceptual quality and more efficiently re-quires more research efforts. In this paper, wepropose AdvGAN to generate adversarial exam-ples with generative adversarial networks (GANs),which can learn and approximate the distributionof original instances. For AdvGAN, once the gen-erator is trained, it can generate perturbations effi-ciently for any instance, so as to potentially acceler-ate adversarial training as defenses. We apply Adv-GAN in both semi-whitebox and black-box attacksettings. In semi-whitebox attacks, there is no needto access the original target model after the gener-ator is trained, in contrast to traditional white-boxattacks. In black-box attacks, we dynamically traina distilled model for the black-box model and op-timize the generator accordingly. Adversarial ex-amples generated by AdvGAN on different targetmodels have high attack success rate under state-of-the-art defenses compared to other attacks. Ourattack has placed the first with 92.76% accuracy ona public MNIST black-box attack challenge. Read More
How malevolent machine learning could derail AI
AI security expert Dawn Song warns that “adversarial machine learning” could be used to reverse-engineer systems—including those used in defense.
Artificial intelligence won’t revolutionize anything if hackers can mess with it. That’s the warning from Dawn Song, a professor at UC Berkeley who specializes in studying the security risks involved with AI and machine learning. Read More
Driving the Success of Data Science Solutions: Skills, Roles and Responsibilities
Privacy and machine learning: two unexpected allies?
In many applications of machine learning, such as machine learning for medical diagnosis, we would like to have machine learning algorithms that do not memorize sensitive information about the training set, such as the specific medical histories of individual patients. Differential privacy is a framework for measuring the privacy guarantees provided by an algorithm. Through the lens of differential privacy, we can design machine learning algorithms that responsibly train models on private data. Our works (with Martín Abadi, Úlfar Erlingsson, Ilya Mironov, Ananth Raghunathan, Shuang Song and Kunal Talwar) on differential privacy for machine learning have made it very easy for machine learning researchers to contribute to privacy research—even without being an expert on the mathematics of differential privacy. In this blog post, we’ll show you how to do it. Read More
Semi-supervised knowledge transfer for deep learning from private training data
Some machine learning applications involve training data that is sensitive, such as the medical histories of patients in a clinical trial. A model may inadvertently and implicitly store some of its training data; careful analysis of the model may therefore reveal sensitive information.To address this problem, we demonstrate a generally applicable approach to providing strong privacy guarantees for training data:Private Aggregation of Teacher Ensembles(PATE). The approach combines, in a black-box fashion, multiple models trained with disjoint datasets, such as records from different subsets of users. Because they rely directly on sensitive data, these models are not published, but instead used as “teachers” for a “student” model. The student learns to predict an output chosen by noisy voting among all of the teachers, and cannot directly access an individual teacher or the underlying data or parameters. The student’s privacy properties can be understood both intuitively (since no single teacher and thus no single dataset dictates the student’s training) and formally, in terms of differential privacy. These properties hold even if an adversary can not only query the student but also inspect its internal workings.Compared with previous work, the approach imposes only weak assumptions on how teachers are trained: it applies to any model, including non-convex models like DNNs. We achieve state-of-the-art privacy/utility trade-offs on MNIST and SVHN thanks to an improved privacy analysis and semi-supervised learning. Read More
Alexa, Will I Be Able to Patent My Artificial Intelligence Technology This Year?
The patentability of artificial intelligence (AI) has been increasingly scrutinized in light of the surge in AI technology development and the ambiguity regarding the interpretation of software-related patents. The Federal Circuit has gradually refined the criteria for determining subject matter eligibility for software-related patents, and based in part on such jurisprudence, earlier this year the U.S. Patent and Trademark Office (USPTO) released revised guidance on examining patent subject matter eligibility under 35 U.S.C. §101. See 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50 (Jan. 7, 2019). Considering the advances in AI technology and intellectual property law, how do these recent developments shape the outlook of AI patentability? Read More
Rick's Cafe AI 