the domain of the sigmoid function is unbounded from $-\infty$ to $\infty$.
the range is bounded between $0$ and $1$; the output can become extremely close to $0$ or $1$, but it will never reach the boundaries.

Theorem.

Derivative of Sigmoid Function

The beauty of the sigmoid function is that its derivative is extremely clean:

$$\sigma'(x)=\sigma(x)\cdot\left[1-\sigma(x)\right]$$

This is a very useful property because we often encounter situations in machine learning where we need to compute the derivative of the sigmoid function. For instance, the update rule for gradient descent in logistic regression requires this derivative.

The proof does not require any advanced calculus:

$$\begin{align*} \frac{d\sigma(x)}{dx}&=\frac{d}{dx}\left(\frac{1}{1+e^{-x}}\right) \\ &=\frac{d}{dx}(1+e^{-x})^{-1} \\ &=-(1+e^{-x})^{-2}\cdot \frac{d}{dx}(1+e^{-x}) \\ &=-(1+e^{-x})^{-2}\cdot (-e^{-x}) \\ &=\frac{e^{-x}}{(1+e^{-x})^2} \\ &=\frac{1}{1+e^{-x}}\left(\frac{e^{-x}}{1+e^{-x}}\right) \\ &=\frac{1}{1+e^{-x}}\left(\frac{1+e^{-x}-1}{1+e^{-x}}\right) \\ &=\frac{1}{1+e^{-x}}\left(\frac{1+e^{-x}}{1+e^{-x}}-\frac{1}{1+e^{-x}}\right) \\ &=\frac{1}{1+e^{-x}}\left(1-\frac{1}{1+e^{-x}}\right) \\ &=\sigma(x)\cdot \left(1-\sigma (x)\right) \\ \end{align*}$$

■

Graph of Sigmoid function and its derivative

Here's a graph of the sigmoid function and its derivative:

Implementation using Python

Here's the sigmoid function as well as its derivative implemented in Python using NumPy:


        
        
            
                
                
                    import numpy as np

def sigmoid(x):
    return np.exp(x) / (1 + np.exp(x))

def sigmoid_derivative(x):
    return sigmoid(x) * (1 - sigmoid(x))

Published by Isshin Inada

Edited by 0 others

Did you find this page useful?

thumb_up

thumb_down

Comment

Citation

Ask a question or leave a feedback...

thumb_up

thumb_down

chat_bubble_outline

settings

Enjoy our search

Hit / to insta-search docs and recipes!