Weights? Biases? Epoch? Glad you asked.

Some of the important Parameters that we use in Neural Networks are defined below. These parameters are also called Hyperparameters as they determine how the network is trained.

1. Backpropagation Algorithm: BP algorithm is used for supervised learning using gradient descent. After one forward pass, the BP performs a back pass by adjusting the hyperparameters, weights and biases.

2. Neuron: A neuron (perceptron) is defined by the equation:

y=summation((weights*inputs)+bias)

3. Weights: Weights can be considered as the slope of linear regression. It indicates the strength of the connection between neurons and decides the amount of influence a node has on the output. When the weights are negative, it means that on increasing it we will reduce the output value. Weights are actually going to affect the Input values.

4. Biases: The bias can be considered as the intercept of the Linear regression. It will help the model such that it can properly fit for the given data.

5. Epoch: One epoch can be defined as one forward pass and one backward pass. This number indicates the number of times the training has to take place. If there is a high epoch, it can cause overfitting which is seen by increase in training accuracy but decrease in validation accuracy.

6. Batch Size: Also known as mini-batch, indicates the number of samples to be fed to the network for the parameters to update once. There can be many batches in a single Epoch.

7. Training set: The dataset is divided into training (70%), testing (15%) and validation (15%) sets. The training set is given during the training. The model evaluates the classification and verifies by itself using the labels.

8. Testing set: This set is used for finding the accuracy. It consists of samples unseen by the model during training.

9. Validation set: The validation set is used to validate the model during the training process. It is used to calculate the loss and error and the values are updated during training. Based on this the model continues to update the parameters and train the model.

10. Model: In a NN the model gives us the prediction and consists of layers based on the architecture of the neural network.

11. Layers: When multiple neurons are arranged in a certain way, by stacking them together and providing the input to all of them together, they form a layer of a neural network.

12. Training: Each neuron is connected to another neuron via connections. These connections are associated with a certain weight and bias. The aim of training is to tweak the value of weights with respect to the error by trying to make the loss as low as possible and increasing the accuracy.

13. Overfitting: When a model is excessively trained it gives the best accuracy, but when new samples are given, the model cannot classify or predict the input. This is called overfitting.

14. Dropout: It is used for regularization to avoid overfitting of the model. The dropout will randomly drop or ignore neurons. It is denoted by a dropout rate. If the dropout is very high then it means the model is under-learnt and if it is too low then it means that it has caused no significant effect to reduce overfitting.

15. Learning Rate: This hyper parameter indicates how fast a network is learning. If the learning rate is less, then the learning is slow but it has a smooth convergence. And if it is more, there may be no convergence.

16. Optimize: It is the most important ‘Ingredient’ when it comes to ML and its applications. Optimization can be defined as minimization of the cost or the objective function.

17. Activation Function: These functions are what makes the neural network, non-linear. The activation function triggers the neurons and they try to bring the mean to zero. Few activation functions are ReLu, Sigmoid which is used for binary prediction, Softmax is used at the final layer to make predictions of multiple classes.