The last time a rival power tried to out-innovate the U.S. and marshaled a whole-of-government approach to doing it, the Soviet Union startled Americans by deploying the first man-made satellite into orbit. The Sputnik surprise in 1957 shook American confidence, galvanized its government and set off a space race culminating with the creation of NASA and the moon landing 50 years ago this month.
Two years since announcing a national plan to become the world leader in artificial intelligence by 2030, China is making progress toward its goal on an unprecedented scale, raising the question of whether America’s laissez-faire approach to technology is enough and whether another Sputnik moment is around the corner, according to interviews for the latest episode of POLITICO’s Global Translations podcast. Read More
Monthly Archives: July 2019
Why an “AI Race” Between the U.S. and China Is a Terrible, Terrible Idea
Perhaps because it lies at the perfect nexus of genuinely-very-complicated and impossibly-confounded-by-marketing-buzzword-speak, the term “AI” has become a catchall for anything algorithmic and sufficiently technologically impressive. AI, which is supposed to stand for “artificial intelligence,” now spans applications from cameras to the military to medicine.
One thing we can be sure about AI — because we are told it so often and at so increasingly high a pitch — is that whatever it actually is, the national interest demands more of it. And we need it now, or else China will beat us there, and we certainly wouldn’t want that, would we? What is “there,” exactly? What does it look like, how would it work, and how would it change our society? Irrelevant! The race is on, and if America doesn’t start taking AI seriously, we’re going to find ourselves the losers in an ever-widening Dystopia Gap. Read More
Building tools to amplify human abilities
Google’s Selfish Ledger is an unsettling vision of Silicon Valley social engineering
Hierarchical Imitation and Reinforcement Learning
We study how to effectively leverage expert feedback to learn sequential decision-making policies. We focus on problems with sparse rewards and long time horizons, which typically pose significant challenges in reinforcement learning. We propose an algorithmic framework, called hierarchical guidance, that leverages the hierarchical structure of the underlying problem to integrate different modes of expert interaction. Our framework can incorporate different combinations of imitation learning (IL) and reinforcement learning (RL) at different levels, leading to dramatic reductions in both expert effort and cost of exploration. Using long-horizon benchmarks, including Montezuma’s Revenge, we demonstrate that our approach can learn significantly faster than hierarchical RL, and be significantly more label-efficient than standard IL. We also theoretically analyze labeling cost for certain instantiations of our framework. Read More
RL — Imitation Learning
Imitation is a key part in the human learning. In the high-tech world, if you are not an innovator, you want to be a quick follower. In reinforcement learning, we maximize the rewards for our actions. Model-based RL focuses on the model (the system dynamics) to optimize our decisions while Policy Gradient methods improve the policy for better rewards.
On the other hand, Imitation learning focuses on imitating expert demonstrations. Read More
5 Reasons You Need a Better Data Management Solution
Congratulations.It took time, but you’re data-driven. You’ve got data scientists to crunch numbers, a fully stocked data lake, and analysts to make it all make sense. But even with the infrastructure in place, benefiting from data isn’t as easy as flipping a switch. These days, businesses have yet to properly manage their data.
And so, they ask themselves a new question:How do you turn a lake of data into streams of insight?
Read More
Researchers Easily Trick Cylance’s AI-Based Antivirus Into Thinking Malware Is ‘Goodware’
By taking strings from an online gaming program and appending them to malicious files, researchers were able to trick Cylance’s AI-based antivirus engine into thinking programs like WannaCry and other malware are benign. Read More
Observational Learning by Reinforcement Learning
Observational learning is a type of learning that occurs as a function of observing, retaining and possibly replicating or imitating the behaviour of another agent. It is a core mechanism appearing in various instances of social learning and has been found to be employed in several intelligent species, including humans. In this paper, we investigate to what extent the explicit modelling of other agents is necessary to achieve observational learning through machine learning. Especially, we argue that observational learning can emerge from pure Reinforcement Learning (RL), potentially coupled with memory. Through simple scenarios, we demonstrate that an RL agent can leverage the information provided by the observations of an other agent performing a task in a shared environment. The other agent is only observed through the effect of its actions on the environment and never explicitly modeled. Two key aspects are borrowed from observational learning: i) the observer behaviour needs to change as a result of viewing a ’teacher’ (another agent) and ii) the observer needs to be motivated somehow to engage in making use of the other agent’s behaviour. The later is naturally modeled by RL, by correlating the learning agent’s reward with the teacher agent’s behaviour. Read More
Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation
Imitation learning is an effective approach for autonomous systems to acquire control policies when an explicit reward function is unavailable, using supervision provided as demonstrations from an expert, typically a human operator. However, standard imitation learning methods assume that the agent receives examples of observation-action tuples that could be provided, for instance, to a supervised learning algorithm. This stands in contrast to how humans and animals imitate: we observe another person performing some behavior and then figure out which actions will realize that behavior, compensating for changes in viewpoint, surroundings, object positions and types, and other factors. We term this kind of imitation learning “imitation-from-observation,” and propose an imitation learning method based on video prediction with context translation and deep reinforcement learning. This lifts the assumption in imitation learning that the demonstration should consist of observations in the same environment configuration, and enables a variety of interesting applications, including learning robotic skills that involve tool use simply by observing videos of human tool use. Our experimental results show the effectiveness of our approach in learning a wide range of real-world robotic tasks modeled after common household chores from videos of a human demonstrator, including sweeping, ladling almonds, pushing objects as well as a number of tasks in simulation. Read More
Rick's Cafe AI 