Deep convolutional neural network (DCNN) based supervised learning is a widely practiced approach for large-scale image classification. However, retraining these large networks to accommodate new, previously unseen data demands high computational time and energy requirements. Also, previously seen training samples may not be available at the time of retraining. We propose an efficient training methodology and incrementally growing DCNN to allow new classes to be learned while sharing part of the base network. Our proposed methodology is inspired by transfer learning techniques, although it does not forget previously learned classes. An updated network for learning new set of classes is formed using previously learned convolutional layers (shared from initial part of base network) with addition of few newly added convolutional kernels included in the later layers of the network. We evaluated the proposed scheme on several recognition applications. The classification accuracy achieved by our approach is comparable to the regular incremental learning approach (where networks are updated with new training samples only, without any network sharing), while achieving energy efficiency, reduction in storage requirements, memory access and training time. Read More
Monthly Archives: March 2019
Transfer Incremental Learning Using Data Augmentation
Due to catastrophic forgetting, deep learning remains highly inappropriate when facing incremental learning of new classes and examples over time. In this contribution, we introduce Transfer Incremental Learning using Data Augmentation (TILDA). TILDA combines transfer learning from a pre-trained Deep Neural Network (DNN) as feature extractor, a Nearest Class Mean (NCM) inspired classifier and majority vote using data augmentation on both training and test vectors. The obtained methodology allows learning new examples or classes on the fly with very limited computational and memory footprints. We perform experiments on challenging vision datasets and obtain performance significantly better than existing incremental counterparts. Read More
Using Transfer Learning to Introduce Generalization in Models
Researchers often try to capture as much information as they can, either by using existing architectures, creating new ones, going deeper, or employing different training methods. This paper compares different ideas and methods that are used heavily in Machine Learning to determine what works best. These methods are prevalent in various domains of Machine Learning, such as Computer Vision and Natural Language Processing (NLP). Read More
The five steps for true digital transformation using artificial intelligence
Artificial intelligence is all the rage. Startups of all ilks are using AI in their services, from generating music playlists to matching job-seekers with employment opportunities, from making purchase recommendations to feeding news aggregators—or so they claim. A report published in January 2018 by McKinsey suggests that early adopters of AI are already reaping benefits, so there is no doubt that its implementation in new arenas will disrupt how we work, consume, and live. But how will this digital transformation come about, and how will startups play a role in a world where AI is no longer a niche but a necessity?
At its core, AI is meant to make work easier for humans. It can handle tasks where our input is often repetitive. Speaking at the Ecosystm Leaders BreakFirst event in Singapore last month, Manoj Menon, a principal advisor at technology research and advisory firm Ecosystm, indicated that there are three prerequisites to a digital transformation for businesses—the technology for machine intelligence, economic backing to roll it out for broad usage, and the human resources to make it happen. Read More
AI and Neuroscience: A virtuous circle
Recent progress in AI has been remarkable. Artificial systems now outperform expert humans at Atari video games, the ancient board game Go, and high-stakes matches of heads-up poker. They can also produce handwriting and speech indistinguishable from those of humans, translate between multiple languages and even reformat your holiday snaps in the style of Van Goghmasterpieces.
These advances are attributed to several factors, including the application of new statistical approaches and the increased processing power of computers. But in a recent Perspective in the journal Neuron, we argue that one often overlooked contribution is the use of ideas from experimental and theoretical neuroscience. Read More
Neuroscience-Inspired Artificial Intelligence
The fields of neuroscience and artificial intelligence (AI) have a long and intertwined history. In more recent times, however, communication and collaboration between the two fields has become less commonplace. In this article, we argue that better understanding biological brains could play a vital role in building intelligent machines. We survey historical interactions between the AI and neuroscience fields and emphasize current advances in AI that have been inspired by the study of neural computation in humans and other animals. We conclude by highlighting shared themes that may be key for advancing future research in both fields. Read More
A Comprehensive Hands-on Guide to Transfer Learning with Real-World Applications in Deep Learning
Humans have an inherent ability to transfer knowledge across tasks. What we acquire as knowledge while learning about one task, we utilize in the same way to solve related tasks. The more related the tasks, the easier it is for us to transfer, or cross-utilize our knowledge.
Conventional machine learning and deep learning algorithms, so far, have been traditionally designed to work in isolation. These algorithms are trained to solve specific tasks. The models have to be rebuilt from scratch once the feature-space distribution changes. Transfer learning is the idea of overcoming the isolated learning paradigm and utilizing knowledge acquired for one task to solve related ones. Read More
Freeze Out: Accelerate training by progressively freezing layers
The early layers of a deep neural net have the fewest parameters, but take up the most computation. In this extended abstract, we propose to only train the hidden layers for a set portion of the training run, freezing them out one-by-one and excluding them from the backward pass. Through experiments on CIFAR, we empirically demonstrate that FreezeOut yields savings of up to 20% wall-clocktime during training with 3% loss in accuracy for DenseNets, a 20% speed up without loss of accuracy for ResNets, and no improvement for VGG networks. Read More
5 Awesome AI Experiences You Can Test Out in Your Browser Right Now
Artificial intelligence is already everywhere, and its influence is growing. It can be hard to get your head around exactly what AI does and how it can be deployed though, which is why we present to you these five fun online experiments—all you need is a web browser and a few minutes to see some of the party tricks AI is already capable of. Read More
Rick's Cafe AI 