This article is part of Demystifying AI, a series of posts that (try to) disambiguate the jargon and myths surrounding AI.
To human observers, the following two images are identical. But researchers at Google showed in 2015 that a popular object detection algorithm classified the left image as “panda” and the right one as “gibbon.” And oddly enough, it had more confidence in the gibbon image. Read More
Tag Archives: Adversarial
Improving Robustness of Deep-Learning-Based Image Reconstruction
Deep-learning-based methods for different applications have been shown vulnerable to adversarial examples. These examples make deployment of such models in safety-critical tasks questionable.Use of deep neural networks as inverse problem solvers has generated much excitement for medical imaging including CT and MRI, but recently a similar vulnerability has also been demonstrated for these tasks. We show that for such inverse problem solvers, one should analyze and study the effect of adversaries in the measurement-space,instead of the signal-space as in previous work. Read More
Does Adversarial Transferability Indicate Knowledge Transferability?
Despite the immense success that deep neural networks (DNNs) have achieved,adversarial examples, which are perturbed inputs that aim to mislead DNNs to make mistakes, have recently led to great concern. On the other hand, adversarial examples exhibit interesting phenomena, such as adversarial transferability. DNNs also exhibit knowledge transfer, which is critical to improving learning efficiency and learning in domains that lack high-quality training data. In this paper, we aim to turn the existence and pervasiveness of adversarial examples into an advantage.Given that adversarial transferability is easy to measure while it can be challenging to estimate the effectiveness of knowledge transfer,does adversarial transferability indicate knowledge transferability? We first theoretically analyze the relationship between adversarial transferability and knowledge transferability and outline easily checkable sufficient conditions that identify when adversarial transferability indicates knowledge transferability. In particular, we show that composition with an affine function is sufficient to reduce the difference between two models when adversarial transferability between them is high. We provide empirical evaluation for different transfer learning scenarios on diverse datasets, including CIFAR-10,STL-10, CelebA, and Taskonomy-data – showing a strong positive correlation be-tween the adversarial transferability and knowledge transferability, thus illustrating that our theoretical insights are predictive of practice. Read More
#adversarialArtificial Intelligence Systems Will Need to Have Certification, CISA Official Says
A process for vetting algorithms and their input data is needed to build confidence in the tech but is still very far off.
Vendors of artificial intelligence technology should not be shielded by intellectual property claims and will have to disclose elements of their designs and be able to explain how their offering works in order to establish accountability, according to a leading official from the Cybersecurity and Infrastructure Security Agency.
“I don’t know how you can have a black-box algorithm that’s proprietary and then be able to deploy it and be able to go off and explain what’s going on,” said Martin Stanley, a senior technical advisor who leads the development of CISA’s artificial intelligence strategy. Read More
How to jam neural networks
Deep neural networks (DNNs) have been a very active field of research for eight years now, and for the last five we’ve seen a steady stream of adversarial examples – inputs that will bamboozle a DNN so that it thinks a 30mph speed limit sign is a 60 instead, and even magic spectacles to make a DNN get the wearer’s gender wrong.
So far, these attacks have targeted the integrity or confidentiality of machine-learning systems. Can we do anything about availability? Read More
Brookings Institute Report: How to improve cybersecurity for artificial intelligence
This report from The Brookings Institution’s Artificial Intelligence and Emerging Technology (AIET) Initiative is part of “AI Governance,” a series that identifies key governance and norm issues related to AI and proposes policy remedies to address the complex challenges associated with emerging technologies. Read More
#adversarial, #cyberThis Bot Hunts Software Bugs for the Pentagon
Late last year, David Haynes, a security engineer at internet infrastructure company Cloudflare, found himself gazing at a strange image. “It was pure gibberish,” he says. “A whole bunch of gray and black pixels, made by a machine.” He declined to share the image, saying it would be a security risk.
Haynes’ caution was understandable. The image was created by a tool called Mayhem that probes software to find unknown security flaws, made by a startup spun out of Carnegie Mellon University called ForAllSecure. Haynes had been testing it on Cloudware software that resizes images to speed up websites, and fed it several sample photos. Mayhem mutated them into glitchy, cursed images that crashed the photo processing software by triggering an unnoticed bug, a weakness that could have caused headaches for customers paying Cloudflare to keep their websites running smoothly. Read More
Learning to Protect Communications with Adversarial Neural Cryptography
We ask whether neural networks can learn to use secret keys to protect information from other neural networks. Specifically, we focus on ensuring confidentiality properties in a multiagent system, and we specify those properties in terms of an adversary. Thus, a system may consist of neural networks named Alice and Bob,and we aim to limit what a third neural network named Eve learns from eavesdrop-ping on the communication between Alice and Bob. We do not prescribe specific cryptographic algorithms to these neural networks; instead, we train end-to-end, adversarially. We demonstrate that the neural networks can learn how to perform forms of encryption and decryption, and also how to apply these operations selectively in order to meet confidentiality goals. Read More
#adversarial, #homomorphic-encryptionFreeLB: Enhanced Adversarial Training for Natural Language Understanding
Adversarial training, which minimizes the maximal risk for label-preserving input perturbations, has proved to be effective for improving the generalization of language models. In this work, we propose a novel adversarial training algorithm, FreeLB, that promotes higher invariance in the embedding space, by adding adversarial perturbations to word embeddings and minimizing the resultant adversarial risk inside different regions around input samples. To validate the effectiveness of the proposed approach, we apply it to Transformer-based models for natural language understanding and commonsense reasoning tasks. Experiments on the GLUE benchmark show that when applied only to the finetuning stage, it is able to improve the overall test scores of BERT-base model from 78.3 to 79.4, and RoBERTa-large model from 88.5 to 88.8. In addition, the proposed approach achieves state-of-the-art single-model test accuracies of 85.44% and 67.75% on ARC-Easy and ARC-Challenge. Experiments on CommonsenseQA benchmark further demonstrate that FreeLB can be generalized and boost the performance of RoBERTa-large model on other tasks as well Read More.
#adversarial, #nlpAttacking Artificial Intelligence: AI’s Security Vulnerability and What Policymakers Can Do About It
The methods underpinning the state-of-the-art artificial intelligence systems are systematically vulnerable to a new type of cybersecurity attack called an “artificial intelligence attack.” Using this attack, adversaries can manipulate these systems in order to alter their behavior to serve a malicious end goal. As artificial intelligence systems are further integrated into critical components of society, these artificial intelligence attacks represent an emerging and systematic vulnerability with the potential to have significant effects on the security of the country.
Unlike traditional cyberattacks that are caused by “bugs” or human mistakes in code, AI attacks are enabled by inherent limitations in the underlying AI algorithms that currently cannot be fixed. Further, AI attacks fundamentally expand the set of entities that can be used to execute cyberattacks. For the first time, physical objects can be now used for cyberattacks (e.g., an AI attack can transform a stop sign into a green light in the eyes of a self-driving car by simply placing a few pieces of tape on the stop sign itself). Data can also be weaponized in new ways using these attacks, requiring changes in the way data is collected, stored, and used. Read More
Rick's Cafe AI 