Theory Behind 1Cycle Learning Rate Scheduling & Learning Rate Schedules – Day 43
The 1Cycle Learning Rate Policy: Accelerating Model Training In our pervious article (day 42) , we have explained The Power of Learning Rates in Deep Learning and Why Schedules Matter, lets now focus on 1Cycle Learning Rate to explain it in more detail : The 1Cycle Learning Rate Policy, first introduced by Leslie Smith in 2018, remains one of the most effective techniques for optimizing model training. By 2025, it continues to prove its efficiency, accelerating convergence by up to 10x compared to traditional learning rate schedules, such as constant or exponentially decaying rates. Today, both researchers and practitioners...
Transfer learning – day 29
Understanding Transfer Learning in Deep Neural Networks Understanding Transfer Learning in Deep Neural Networks: A Step-by-Step Guide In the realm of deep learning, transfer learning has become a powerful technique for leveraging pre-trained models to tackle new but related tasks. This approach not only reduces the time and computational resources required to train models from scratch but also often leads to better performance due to the reuse of already-learned features. What is Transfer Learning? Transfer learning is a machine learning technique where a model developed for one task is reused as the starting point for a model on a second,...
Batch Normalization – day 25
Understanding Batch Normalization in Deep Learning Understanding Batch Normalization in Deep Learning Deep learning has revolutionized numerous fields, from computer vision to natural language processing. However, training deep neural networks can be challenging due to issues like unstable gradients. In particular, gradients can either explode (grow too large) or vanish (shrink too small) as they propagate through the network. This instability can slow down or completely halt the learning process. To address this, a powerful technique called Batch Normalization was introduced. The Problem: Unstable Gradients In deep networks, the issue of unstable gradients becomes more pronounced as the network depth...
Hyperparameter Tuning with Keras Tuner _ Day 17
A Comprehensive Guide to Hyperparameter Tuning with Keras Tuner Introduction In the world of machine learning, the performance of your model can heavily depend on the choice of hyperparameters. Hyperparameter tuning, the process of finding the optimal settings for these parameters, can be time-consuming and complex. This guide will walk you through the essentials of hyperparameter tuning using Keras Tuner, helping you build more efficient and effective models. Why Hyperparameter Tuning Matters Hyperparameters are critical settings that can influence the performance of your machine learning models. These include the learning rate, the number of layers in a neural network, the...
Activation Function, Hidden Layer and non linearity. _ day 12
Understanding Non-Linearity in Neural Networks Understanding Non-Linearity in Neural Networks Non-linearity in neural networks is essential for solving complex tasks where the data is not linearly separable. This blog post explains why hidden layers and non-linear activation functions are necessary, using the XOR problem as an example. What is Non-Linearity? Non-linearity in neural networks allows the model to learn and represent more complex patterns. In the context of decision boundaries, a non-linear decision boundary can bend and curve, enabling the separation of classes that are not linearly separable. Role of Activation Functions The primary role of an activation function is...
Activation Function _ day 11
Activation Functions in Neural Networks Activation Functions in Neural Networks: Why They Matter ? Activation functions are pivotal in neural networks, transforming the input of each neuron to its output signal, thus determining the neuron’s activation level. This process allows neural networks to handle tasks such as image recognition and language processing effectively. The Role of Different Activation Functions Neural networks employ distinct activation functions in their inner and outer layers, customized to the specific requirements of the network: Inner Layers: Functions like ReLU (Rectified Linear Unit) introduce necessary non-linearity, allowing the network to learn complex patterns in the data....
3 Types of Gradient Decent Types : Batch, Stochastic & Mini-Batch _ Day 8
Understanding Gradient Descent: Batch, Stochastic, and Mini-Batch Understanding Gradient Descent: Batch, Stochastic, and Mini-Batch Learn the key differences between Batch Gradient Descent, Stochastic Gradient Descent, and Mini-Batch Gradient Descent, and how to apply them in your machine learning models. Batch Gradient Descent Batch Gradient Descent uses the entire dataset to calculate the gradient of the cost function, leading to stable, consistent steps toward an optimal solution. It is computationally expensive, making it suitable for smaller datasets where high precision is crucial. Formula: \[\theta := \theta – \eta \cdot \frac{1}{m} \sum_{i=1}^{m} \nabla_{\theta} J(\theta; x^{(i)}, y^{(i)})\] \(\theta\) = parameters \(\eta\) = learning...
Can we make prediction without need of going through iteration ? yes with the Normal Equation _ Day 6
Understanding Linear Regression: The Normal Equation and Matrix Multiplications Explained Understanding Linear Regression: The Normal Equation and Matrix Multiplications Explained Linear regression is a fundamental concept in machine learning and statistics, used to predict a target variable based on one or more input features. While gradient descent is a popular method for finding the best-fitting line, the normal equation offers a direct, analytical approach that doesn’t require iterations. This blog post will walk you through the normal equation step-by-step, explaining why and how it works, and why using matrices simplifies the process. Table of Contents Introduction to Linear Regression Gradient...
Regression & Classification with MNIST. _ day 4
A Comprehensive Guide to Machine Learning: Regression and Classification with the MNIST Dataset Introduction to Supervised Learning: Regression and Classification In the realm of machine learning, supervised learning involves training a model on a labeled dataset, which means the dataset includes both input data and the corresponding output labels. Supervised learning tasks can be broadly categorized into two types: regression and classification. Regression tasks aim to predict continuous numerical values. For example, predicting house prices based on various features such as location, size, and number of bedrooms. The output is a continuous value that can range over...
