In artificial neural networks (ANNs), the activation function is a mathematical “gate” in between the input feeding the current neuron and its output going to the next layer [1].

The activation functions are at the very core of Deep Learning. They determine the output of a model, its accuracy, and computational efficiency. In some cases, activation functions have a major effect on the model’s ability to converge and the convergence speed.

In this article, you’ll learn seven of themost popular activation functions in Deep Learning — Sigmoid, Tanh, ReLU, Leaky ReLU, PReLU, ELU, and SELU — and how to use them with Keras and TensorFlow 2. Read More