Layers of neural networks

No account yet ?

Layer activations

Activation functions

relu function

Tested in Anaconda and Python 3.7

def ReLu(val):
    return max(0.0,val)
 
val = 1.0
print(ReLu(val))
 
val1 = -1.0
print(ReLu(val1))

1.0
0.0

sigmoid function

Tested in Anaconda and Python 3.7

import numpy as np
 
def sigmoid(x):
    return 1.0 / (1.0 + np.exp(-x))
 
print(sigmoid(0.5))

0.6224593312018546

softmax function

Tested in Anaconda and Python 3.7

import numpy as np
 
def softmax(vec):
  exponential = np.exp(vec)
  probabilities = exponential / np.sum(exponential)
  return probabilities
 
vector = np.array([1.0, 3.0, 2.0])
probabilities = softmax(vector)
print("Probability Distribution is:")
print(probabilities)

Probability Distribution is :
[0.09003057 0.66524096 0.24472847]

softplus function

Tested in Anaconda and Python 3.7

# Importing the Tensorflow library
import tensorflow as tf
 
# Importing the NumPy library
import numpy as np
 
# Importing the matplotlib.pyplot function
import matplotlib.pyplot as plt
 
# A vector of size 15 with values from -5 to 5
a = np.linspace(-5, 5, 15)
 
# Applying the softplus function and
# storing the result in 'b'
b = tf.nn.softplus(a, name ='softplus')
 
# Initiating a Tensorflow session
with tf.Session() as sess:
    print('Input:', a)
    print('Output:', sess.run(b))
    plt.plot(a, sess.run(b), color = 'red', marker = "o") 
    plt.title("tensorflow.nn.softplus") 
    plt.xlabel("X") 
    plt.ylabel("Y") 
 
    plt.show()

Input: [-5. -4.28571429 -3.57142857 -2.85714286 -2.14285714 -1.42857143
-0.71428571 0. 0.71428571 1.42857143 2.14285714 2.85714286
3.57142857 4.28571429 5. ]

Output: [0.00671535 0.01366993 0.02772767 0.05584391 0.11093221 0.21482992
0.39846846 0.69314718 1.11275418 1.64340135 2.25378936 2.91298677
3.59915624 4.29938421 5.00671535]

softsign function

Tested in Anaconda and Python 3.7

import numpy as np
import matplotlib.pyplot as plt
 
def SoftSign(x):
    s = x/(1+abs(x))
    plt.plot(x,s)
    plt.xlabel('valus of x')
    plt.ylabel('SoftSign Values')
    plt.title('SoftSign Function')
    plt.legend(["Softsign"])
    plt.show()
    return s
 
x = np.array(np.arange(-10,11,1))
s = print(SoftSign(x))

[-0.90909091 -0.9 -0.88888889 -0.875 -0.85714286 -0.83333333
-0.8 -0.75 -0.66666667 -0.5 0. 0.5
0.66666667 0.75 0.8 0.83333333 0.85714286 0.875
0.88888889 0.9 0.90909091]

tanh function

f(x) = tanh(x)

Tested in Anaconda and Python 3.7

import numpy as np
import matplotlib.pyplot as plt
 
in_array = np.linspace(-np.pi, np.pi, 12)
out_array = np.tanh(in_array)
 
print("in_array : ", in_array)
print("\nout_array : ", out_array)
 
# red for numpy.tanh()
plt.plot(in_array, out_array, color = 'red', marker = "o")
plt.title("numpy.tanh()")
plt.xlabel("X")
plt.ylabel("Y")
plt.show()

in_array : [-3.14159265 -2.57039399 -1.99919533 -1.42799666 -0.856798 -0.28559933
0.28559933 0.856798 1.42799666 1.99919533 2.57039399 3.14159265]

out_array : [-0.99627208 -0.98836197 -0.96397069 -0.89125532 -0.69460424 -0.27807943
0.27807943 0.69460424 0.89125532 0.96397069 0.98836197 0.99627208]

selu function

elu function

exponential function

f(x) = exp(x)

Densing layer

DenseBlock

DenseNet

FC-DenseNet - GitHub

Transposed convolutional layer

Pooling layer

Image augmentation layer

Attention layer

Flattening layer

Concatenate layer

Neural network

Data engineering

Deep learning

Machine learning

Welcome, my name is Eric Soupet and I am the administrator of the site elodees.com. elodees.com is a state of the art of Artificial Intelligence and aims to be collaborative, you can now offer content such as articles, events, tutorials, ... so don't hesitate !