Batch normalisation part 2 – day 26

Introduction to Batch Normalization Batch normalization is a widely used technique in deep learning that significantly improves the performance and stability of neural networks. Introduced by Sergey Ioffe and Christian Szegedy in 2015, this technique addresses the issues of vanishing and exploding gradients that can occur during training, particularly in deep networks. Why Batch Normalization? In deep learning, as data propagates through the layers of a neural network, it can lead to shifts in the distribution of inputs to layers deeper in the network—a phenomenon known as internal covariate shift. This shift can cause issues such as vanishing gradients, where...

Membership Required

You must be a member to access this content.

View Membership Levels

Already a member? Log in here
Read More
46

3 Types of Gradient Decent Types : Batch, Stochastic & Mini-Batch _ Day 8

Understanding Gradient Descent: Batch, Stochastic, and Mini-Batch Understanding Gradient Descent: Batch, Stochastic, and Mini-Batch Learn the key differences between Batch Gradient Descent, Stochastic Gradient Descent, and Mini-Batch Gradient Descent, and how to apply them in your machine learning models. Batch Gradient Descent Batch Gradient Descent uses the entire dataset to calculate the gradient of the cost function, leading to stable, consistent steps toward an optimal solution. It is computationally expensive, making it suitable for smaller datasets where high precision is crucial. Formula: \[\theta := \theta – \eta \cdot \frac{1}{m} \sum_{i=1}^{m} \nabla_{\theta} J(\theta; x^{(i)}, y^{(i)})\] \(\theta\) = parameters \(\eta\) = learning...

Membership Required

You must be a member to access this content.

View Membership Levels

Already a member? Log in here
Read More
157