With machine learning, the world relies on technology for recommendations recognition systems. But a lot of these systems are corrupted because they have a certain bias associated with them and are hence not accurate with their functioning. Human Biases That Can Result Into ML Biases include: Reporting Bias/Sample Bias, Prejudice Bias, Measurement Bias, Automation Bias, Group Attribution Bias, and Algorithm Bias, among others. What Can Be Done To Prevent Biases? Read More
Tag Archives: Machine Learning
Incremental Convolutional Neural Network Training
Experimenting novel ideas on deep convolutional neural networks (DCNNs) with big datasets is hampered by the fact that network training requires huge computational resources in the terms of CPU and GPU power and hours. One option is to downscale the problem, e.g., less classes and less samples, but this is undesirable with DCNNs whose performance is largely data-dependent. In this work, we take an alternative route and downscale the networks and input images. For example, the ImageNet problem of 1,000 classes and 1,2M training images can be solved in hours on a commodity laptop without GPU by downscaling images and the network to the resolution of 8 8. We attempt to provide the solution to transfer the knowledge (parameters) of a trained DCNN with lower resolution to improve the efficiency of training a DCNN with higher resolution, and continue training incrementally until the full resolution is achieved. In our experiments, this approach achieves clear boost in computing time without the loss of performance. Read More
Fully Homomorphic Encryption for Classification in Machine Learning
Using fully homomorphic encryption scheme, we construct fully homomorphic encryption scheme FHE4GT that can homomorphically compute an encryption of the greaterthan bit that indicates 𝑥 > 𝑥 ′ or not, given two ciphertexts 𝑐 and 𝑐 ′ of 𝑥 and 𝑥 ′, respectively, without knowing the secret key. Then, we construct homomorphic classifier homClassify that can homomorphically classify a given encrypted data without decrypting it, using machine learned parameters. Read More
Conditionals in Homomorphic Encryption and Machine Learning Applications
Homomorphic encryption has the purpose to allow computations on encrypted data, without the need for decryption other than that of the final result. This could provide an elegant solution to the problem of privacy preservation in data-based applications, such as those provided and/or facilitated by machine learning techniques, but several limitations and open issues hamper the fulfillment of this plan. In this work we assess the possibility for homomorphic encryption to fully implement its program without the need to rely on other techniques, such as multiparty computation, which may be impossible in many actual use cases (for instance due to the high level of communication required). We proceed in two steps: i) on the basis of the well-known structured program theorem [28] we identify the relevant minimal set of operations homomorphic encryption must be able to perform to implement any algorithm; and ii) we analyse the possibility to solve -and propose an implementation for- the most fundamentally relevant issue as it emerges from our analysis, that is, the implementation of conditionals (which in turn require comparison and selection/jump operations) in full homomorphic encryption. We show how this issue has a serious impact and clashes with the fundamental requirements of homomorphic encryption. This could represent a drawback for its use as a complete solution in data analysis applications, in particular machine learning. It will thus possibly require a deep re-thinking of the homomorphic encryption program for privacy preservation. Read More
A review of homomorphic encryption and software tools for encrypted statistical machine learning
Recent advances in cryptography promise to enable secure statistical computation on encrypted data, whereby a limited set of operations can be carried out with out the need to first decrypt. We review these homomorphic encryption schemes in a manner accessible to statisticians and machine learners, focusing on pertinent limitations inherent in the current state of the art. These limitations restrict the kind of statistics and machine learning algorithms which can be implemented and we review those which have been successfully applied in the literature. Finally, we document a high performance R package implementing a recent homomorphic scheme in a general framework. Read More
An introduction to Reinforcement Learning
A Beginner’s Guide to Deep Reinforcement Learning
While neural networks are responsible for recent breakthroughs in problems like computer vision, machine translation and time series prediction – they can also combine with reinforcement learning algorithms to create something astounding like AlphaGo.
Reinforcement learning refers to goal-oriented algorithms, which learn how to attain a complex objective (goal) or maximize along a particular dimension over many steps; for example, maximize the points won in a game over many moves. They can start from a blank slate, and under the right conditions they achieve superhuman performance. Like a child incentivized by spankings and candy, these algorithms are penalized when they make the wrong decisions and rewarded when they make the right ones – this is reinforcement. Read More
Parallel and Distributed Deep Learning
This report explores ways to parallelize/distribute deep learning in multi-core and distributed setting. We have analyzed (empirically) the speedup in training a CNN using conventional single core CPU and GPU and provide practical suggestions to improve training times. In the distributed setting, we study and analyze synchronous and asynchronous weight update algorithms (like Parallel SGD, ADMM and Downpour SGD) and come up with worst case asymptotic communication cost and computation time for each of the these algorithms. Read More
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
Deep Neural Networks (DNNs) are becoming an important tool in modern computing applications. Accelerating their training is a major challenge and techniques range from distributed algorithms to low-level circuit design. In this survey, we describe the problem from a theoretical perspective, followed by approaches for its parallelization. We present trends in DNN architectures and the resulting implications on parallelization strategies. We then review and model the different types of concurrency in DNNs: from the single operator, through parallelism in network inference and training, to distributed deep learning. We discuss asynchronous stochastic optimization, distributed system architectures, communication schemes, and neural architecture search. Based on those approaches, we extrapolate potential directions for parallelism in deep learning. Read More
Distributed Deep Neural Networks over the Cloud, the Edge and End Devices
We propose distributed deep neural networks (DDNNs) over distributed computing hierarchies, consisting of the cloud, the edge (fog) and end devices. While being able to accommodate inference of a deep neural network (DNN) in the cloud, a DDNN also allows fast and localized inference using shallow portions of the neural network at the edge and end devices. When supported by a scalable distributed computing hierarchy, a DDNN can scale up in neural network size and scale out in geographical span. Due to its distributed nature, DDNNs enhance sensor fusion, system fault tolerance and data privacy for DNN applications. Read More
Rick's Cafe AI 