Tag Archives: Machine Learning
a16z Podcast: The History and Future of Machine Learning
How have we gotten to where were are with machine learning? Where are we going?
a16z Operating Partner Frank Chen and Carnegie Mellon professor Tom Mitchell first stroll down memory lane, visiting the major landmarks: the symbolic approach of the 1970s, the “principled probabalistic methods” of the 1980s, and today’s deep learning phase. Then they go on to explore the frontiers of research. Along the way, they cover:
– How planning systems from the 1970s and early 1980s were stymied by the “banana in the tailpipe” problem
– How the relatively slow neurons in our visual cortex work together to deliver very speedy and accurate recognition
– How fMRI scans of the brain reveal common neural patterns across people when they are exposed to common nouns like chair, car, knife, and so on
– How the computer science community is working with social scientists (psychologists, economists, and philosophers) on building measures for fairness and transparency for machine learning models
– How we want our self-driving cars to have reasonable answers to the Trolley Problem, but no one sitting for their DMV exam is ever asked how they would respond
– How there were inflated expectations (and great social fears) for AI in the 1980s, and how the US concerns about Japan compare to our concerns about China today
– Whether this is the best time ever for AI and ML research and what continues to fascinate and motivate Tom after decades in the field
Read More
Software Engineering for Machine Learning: A Case Study
Recent advances in machine learning have stimulated widespread interest within the Information Technology sector on integrating AI capabilities into software and services.This goal has forced organizations to evolve their development processes. We report on a study that we conducted on observing software teams at Microsoft as they develop AI-based applications. We consider a nine-stage workflow process informed by prior experiences developing AI applications (e.g., search and NLP) and data science tools (e.g. application diagnostics and bug reporting). We found that various Microsoft teams have united this workflow into preexisting, well-evolved, Agile-like software engineering processes, providing insights about several essential engineering challenges that organizations may face in creating large-scale AI solutions for the marketplace. We collected some best practices from Microsoft teams to address these challenges.In addition, we have identified three aspects of the AI domain that make it fundamentally different from prior software application domains: 1) discovering, managing, and versioning the data needed for machine learning applications is much more complex and difficult than other types of software engineering, 2) model customization and model reuse require very different skills than are typically found in software teams, and 3) AI components are more difficult to handle as distinct modules than traditional software components — models may be “entangled” in complex ways and experience non-monotonic error behavior. We believe that the lessons learned by Microsoft teams will be valuable too their organizations. Read More
More efficient security for cloud-based machine learning
A novel encryption method devised by MIT researchers secures data used in online neural networks, without dramatically slowing their runtimes. This approach holds promise for using cloud-based neural networks for medical-image analysis and other applications that use sensitive data.
Outsourcing machine learning is a rising trend in industry. Major tech firms have launched cloud platforms that conduct computation-heavy tasks, such as, say, running data through a convolutional neural network (CNN) for image classification. Resource-strapped small businesses and other users can upload data to those services for a fee and get back results in several hours.
But what if there are leaks of private data? In recent years, researchers have explored various secure-computation techniques to protect such sensitive data. But those methods have performance drawbacks that make neural network evaluation (testing and validating) sluggish — sometimes as much as million times slower — limiting their wider adoption. Read More
ML Confidential: Machine Learning on Encrypted Data
We demonstrate that, by using a recently proposed leveled homomorphic encryption scheme, it is possible to delegate the execution of a machine learning algorithm to a computing service while retaining confidentiality of the training and test data. Since the computational complexity of the homomorphic encryption scheme depends primarily on the number of levels of multiplications to be carried out on the encrypted data, we define a new class of machine learning algorithms in which the algorithm’s predictions, viewed as functions of the input data, can be expressed as polynomials of bounded degree. We pro-pose confidential algorithms for binary classification based on polynomial approximations to least-squares solutions obtained by a small number of gradient descent steps. We present experimental validation of the confidential machine learning pipeline and discuss the trade-offs regarding computational complexity, prediction accuracy and cryptographic security. Read More
Unsupervised Machine Learning on Encrypted Data
In the context of Fully Homomorphic Encryption, which allows computations on encrypted data, Machine Learning has been one of the most popular applications in the recent past. All of these works,however, have focused on supervised learning, where there is a labeled training set that is used to configure the model. In this work, we take thefirst step into the realm of unsupervised learning, which is an important area in Machine Learning and has many real-world applications, by ad-dressing the clustering problem. To this end, we show how to implement the K-Means-Algorithm. This algorithm poses several challenges in the FHE context, including a division, which we tackle by using a natural encoding that allows division and may be of independent interest. While this theoretically solves the problem, performance in practice is not optimal, so we then propose some changes to the clustering algorithm to make it executable under more conventional encodings. We show that our new algorithm achieves a clustering accuracy comparable to the original K-Means-Algorithm, but has less than 5% of its runtime. Read More
Few-Shot Adversarial Learning of Realistic Neural Talking Head Models
Several recent works have shown how highly realistic human head images can be obtained by training convolutional neural networks to generate them. In order to create a personalized talking head model, these works require training on a large dataset of images of a single person. However, in many practical scenarios, such personalized talking head models need to be learned from a few image views of a person, potentially even a single image. Here, we present a system with such few-shot capability. It performs lengthy meta-learning on a large dataset of videos, and after that is able to frame few- and one-shot learning of neural talking head models of previously unseen people as adversarial training problems with high capacity generators and discriminators. Crucially, the system is able to initialize the parameters of both the generator and the discriminator in a person-specific way, so that training can be based on just a few images and done quickly, despite the need to tune tens of millions of parameters. We show that such an approach is able to learn highly realistic and personalized talking head models of new people and even portrait paintings. Read More
Mona Lisa frown: Machine learning brings old paintings and photos to life
Machine learning researchers have produced a system that can recreate lifelike motion from just a single frame of a person’s face, opening up the possibility of animating not just photos but also paintings. It’s not perfect, but when it works, it is — like much AI work these days — eerie and fascinating.
The model is documented in a paper published by Samsung AI Center, which you can read here on Arxiv. It’s a new method of applying facial landmarks on a source face — any talking head will do — to the facial data of a target face, making the target face do what the source face does. Read More
How to Apply Machine Learning to Business Problems
It’s easy to see the massive rise in popularity for venture investment, conferences, and business-related queries for “machine learning” since 2012 – but most technology executives often have trouble identifying where their business might actually apply machine learning (ML) to business problems.
With new AI buzzwords being created weekly, it can seem difficult to get ahold of what applications are viable, and which are hype, hyperbole or hoax
In this article, we’ll break down categories of business problems that are commonly handled by ML, and we’ll also provide actionable advice to begin a ML initiative with the right approach and perspective (even it’s the first such project you’ve undertaken at your company). Read More
AI System Sorts News Articles By Whether or Not They Contain Actual Information
There’s a thing in journalism now where news is very often reframed in terms of personal anecdote and-or hot take. In an effort to have something new and clickable to say, we reach for the easiest, closest thing at hand, which is, well, ourselves—our opinions and experiences.
I worry about this a lot! I do it (and am doing it right now), and I think it’s not always for ill. But in a larger sense it’s worth wondering to what degree the larger news feed is being diluted by news stories that are not “content dense.” That is, what’s the real ratio between signal and noise, objectively speaking? To start, we’d need a reasonably objective metric of content density and a reasonably objective mechanism for evaluating news stories in terms of that metric. Read More
Rick's Cafe AI 