Architecture of a Multi-Layer Perceptron

Multi-layer Perceptron consists of three layers, namely, the Input layer, the Hidden layer and the Output layer. Input is fed at the input layer and output is fetched at the output layer. We can have as many no. of hidden layers as we want. In an MLP every node/neuron of current layer is connected to every other neuron in the next layer. Each of the neurons have weights and biases. These parameters are all trainable i.e. in MLP all the parameters are trainable. We pass the input to model and multiply with weights and add bias at every layer and find the calculated output of the model. In MLP we use the Back-Propagation Algorithm. The loss is calculated and we back propagate the loss. According to which the weights are updated/altered.

The Architecture of a Multi-layer Perceptron is shown. The three layers are of MLP are shown here. P1,P2, …… PR are the input values given to the network. Each of the P values are corresponding to a neuron, and there are R number of neurons. The summation indicates the sum of all the input values, and f indicates activation function. The weights w and biases b are added to each of the neurons. S1 ­indicates the number of inputs at 1st layer, similarly S2 and S3 indicates the number of inputs at layer 2 and layer 3, respectively. W1, W2 and W3 represent the vector of weights at first, second and third layer respectively. The output equations of layer1, layer2 and layer3 are given by a^1, a^2, a^3.

The Back Propagation Algorithm suggests that the error is propagated from the output towards the input, and the weights are tweaked accordingly for the next epoch so as to minimize the loss. In MLP, each neuron of one layer is connected to every other neuron from the previous layer. This creates a complex interconnection of neurons with hundreds or even millions of trainable-parameters.