• Home
  • Deep Learning 2024 – 2025
    • Announcement : chat ai by api
    • ONNX vs. Core ML: Choosing the Best Approach for Model Conversion in 2024
    • Core ML vs (PyTorch – TensorFlow)for deep learning integration on iOS app on Xcode using SWIFT
    • Announcement : New iOS app coming for Christmas
    • How to Monetize Your Deep Learning Skills and Models as a Solo Developer in 2024 – 2025
    • Understanding Tokens in Deep Learning: Types, Examples, and Use Cases -LLM
    • Understanding Computation Graphs in Pytorch vs Tensorflow
    • Compare iPad M4 chip, Mac M4 Chip , Cloud & GoogleColab for building deep learning model – day 10
    • Solo Developer’s Guide to Building Competitive Language Model Application – day 9
    • Where to Get Data for Machine Learning and Deep Learning Model Creation – day 8
    • GPU and Computing Technology Comparison 2024 – day 7
    • Fine-Tuning vs. Transfer Learning in Voice Synthesis
    • Fine-Tuning in Deep Learning with a practical example – day 6
    • DeepNet – Scaling Transformers to 1,000 Layers new for 2024 – day 5
    • Transformers seems most famous for deep learning on 2024 so lets learn it more – day 4
    • Effortlessly Integrate ChatGPT-4 API on Mac with Xcode: Your Ultimate 2024 Guide to Token-Based Access Without a Monthly Subscription – day 3
    • New on 2024 – 2025 are MLX and Transofermers so lets compare Custom Deep Learning Models for iOS with MLX on Apple Silicon vs. PyTorch – day 2
    • Deep Learning in 2024: Continued Insights and Strategies – day 1
  • Application
    • Flash Card
    • Video Voice Change
    • Todo
    • background img remove INGOAMPT
    • Video Voice Edit INGOAMPT
  • Machine learning _ Deep learning ( day by day )
    • Reinforcement Learning: An Evolution from Games to Real-World Impact – day 77
    • How Dalle Image Generator works ? – day 76
    • Breaking Down Diffusion Models in Deep Learning – Day 75
    • Understanding Unsupervised Pretraining Using Stacked Autoencoders – day 74
    • Unlock the Secrets of Autoencoders, GANs, and Diffusion Models – Why You Must Know Them? -Day 73
    • The Rise of Transformers in Vision and Multimodal Models – Hugging Face – day 72
    • Mastering NLP: Unlocking the Math Behind It for Breakthrough Insights with a scientific paper study – day 71
    • How ChatGPT Work Step by Step – day 70
    • Can ChatGPT Truly Understand What We’re Saying? A Powerful Comparison with BERT” – day 69
    • Leveraging Scientific Research to Uncover How ChatGPT Supports Clinical and Medical Applications – day 68
    • Do you want to read a summery of what is BERT in 2 min read? (Bidirectional Encoder Representations from Transformers) – day 67
    • Transformers Deep Learning – day 66
    • The Revolution of Transformer Models – day 65
    • why transformers are better for NLP ? Let’s see the math behind it – Day 64
    • Natural Language Processing (NLP) and RNN – day 63
    • RNN, Layer Normalization, and LSTMs – Part 8 of RNN Deep Learning- day 62
    • Iterative Forecasting which is Predicting One Step at a Time 2- Direct Multi-Step Forecasting with RNN 3- Seq2Seq Models for Time Series Forecasting – day 61
    • Step-by-Step Explanation of RNN for Time Series Forecasting – part 6 – day 60
    • To learn what is RNN (Recurrent Neural Networks ) why not understand ARIMA, SARIMA first ? – RNN Learning – Part 5 – day 59
    • Understanding RNNs: why compare it with Feedforward Neural Networks with simple Example to show the Math Behind it ? – DAY 58 INGOAMPT
    • Time Series Forecasting with Recurrent Neural Networks (RNNs) – part 3 – day 57
    • Understanding Recurrent Neural Networks (RNNs) – part 2 – day 56
    • RNN Deep Learning – Part 1 – Day 55
    • Mathematics Behind CNN or Convolutional Neural Network in Deep Learning – Day 54
    • CNN- Convolutional Neural Networks – DAY 53
    • Deep Learning types and Best to Learn in 2024 as an iOS Developer – day 52
    • Comprehensive guide to deep learning in 2024 and 2025 covering models, trends, and beginner tips. Explore Feedforward Neural Networks (FNNs), CNNs, RNNs, GANs, and emerging trends like federated learning, explainable AI, and quantum AI – day 51
    • Comprehensive guide to deep learning in 2024 and 2025 covering models, trends, and beginner tips. Explore Feedforward Neural Networks (FNNs), CNNs, RNNs, GANs, and emerging trends like federated learning, explainable AI, and quantum AI – day 51
    • How to Configure and Optimize Deep Neural Networks for Performance and Efficiency – Day 50
    • Learn Max-Norm Regularization to avoid overfitting : Theory and Importance in Deep Learning and proof – day 49
    • DropOut and Monte Carlo Dropout (MC Dropout)- day 48
    • Understanding Regularization in Deep Learning – day 47
    • Comparing TensorFlow (Keras), PyTorch, and MLX -day 46
    • Learning rate – part 4- 1-Cycle Scheduling, exponential decay and Cyclic Exponential Decay (CED) – day 45
    • Exploring Gradient Clipping and Weight Initialization in Deep Learning – day 44
    • Theory Behind 1Cycle Learning Rate Scheduling and Learning Rate Schedules – Day 43
    • Thes Power of Learning Rates in Deep Learning and Why Schedules Matter- day 42
    • Deep Learning Optimizers: NAdam, AdaMax, AdamW, and NAG Comparison – day 41
    • Adam Optimizer deeply explained – day 40
    • Adam vs. SGD: Selecting the Right Optimizer for Your Deep Learning Model
    • AdaGrad, RMSProp, and Adam: A Comparative Guide to Optimization Algorithms – day 38
    • A Comprehensive Guide to AdaGrad: Origins, Mechanism, and Mathematical Proof – day 37
    • Nag as optimiser in deep learning – day 36
    • Momentum – part 3 – day 35
    • Momentum vs Normalisation in Deep learning -part 2 – day 34
    • Momentum Optimization in Machine Learning: A Detailed Mathematical Analysis and Practical Application – day 33
    • Mastering Deep Neural Network Optimization: Techniques and Algorithms for Faster Training – day 32
    • fundamentals of labeled and unlabeled data in machine learning – day 31
    • How do Transfer Learning in Deep Learning Model – with a example – day 30
    • Transfer learning – day 29
    • Understanding Gradient Clipping in Deep Learning – day 28
    • Batch normalisation – trainable and non trainable – day 27
    • Batch normalisation part 2 – day 26
    • Batch Normalization – day 25
    • Activation function progress in deep learning, Relu, Elu, Selu, Geli , mish, etc – include table and graphs – day 24
    • Weight initialazation part 2 – day 23
    • How Create API by Deep Learning to Earn Money and what is the Best Way for Mac Users – Breaking studies on day 22
    • Weight initialisation in Deep Learning well explained: day 21
    • Vanishing gradient explained in detail _ day 20
    • Hyperparameter Tuning and Neural Network Architectures, eg : Bayesian Optimization _ day 19
    • Day 18 _ Automatic vs Manual optimisation in Keras
    • Day 17 _ Hyperparameter Tuning with Keras Tuner
    • Day 16 – TensorFlow: Using TensorBoard, Callbacks, and Model Saving in Keras
    • Day 15 _ Sequential vs Functional Keras API Part 2 explanation
    • Day 14 _ sequential , functional and model subclassing API in Keras
    • Day 13 _ What is Keras
    • Day 12 _ Activation Function, Hidden Layer and non linearity
    • Day 11 _ Activation Function
    • Day 10 _ Regression vs Classification Multi Layer Perceptrons (MLPs)
    • Day 9 _ Deep Learning _ Perceptrons
    • Day 8 _ Gradient Decent Types : Batch, Stochastic and Mini-Batch
    • Day 7 _ Gradient Decent in Machine Learning
    • Day 6 _ Why the Normal Equation Works Without Gradient Descent
    • Day 5 _ Mathematical Explanation behind SGD Algorithm in Machine Learning
    • Day 4 __ Regression & Classification with MNIST
    • Day 3 _ Model vs Instance Models _Train-Test Split _ ML
    • Day 2 _ integrate ML into iOS Apps
    • Day 1 _ Machine learning (ML) Overview
  • About us
    • Contact us
+ 4366565788987
Menu

Tag Archives: relu

Home » Posts Tagged "relu"
19 Aug
Machine Learning Overview

Activation function progress in deep learning, Relu, Elu, Selu, Geli , mish, etc – include table and graphs – day 24

  • August 20, 2024
  • By ingoampt
Activation Function Formula Comparison Why (Problem and Solution) Mathe...

Continue reading

© 2024 INGOAMPT. All rights reserved

  • Home
  • Deep Learning 2024 – 2025
    • Announcement : chat ai by api
    • ONNX vs. Core ML: Choosing the Best Approach for Model Conversion in 2024
    • Core ML vs (PyTorch – TensorFlow)for deep learning integration on iOS app on Xcode using SWIFT
    • Announcement : New iOS app coming for Christmas
    • How to Monetize Your Deep Learning Skills and Models as a Solo Developer in 2024 – 2025
    • Understanding Tokens in Deep Learning: Types, Examples, and Use Cases -LLM
    • Understanding Computation Graphs in Pytorch vs Tensorflow
    • Compare iPad M4 chip, Mac M4 Chip , Cloud & GoogleColab for building deep learning model – day 10
    • Solo Developer’s Guide to Building Competitive Language Model Application – day 9
    • Where to Get Data for Machine Learning and Deep Learning Model Creation – day 8
    • GPU and Computing Technology Comparison 2024 – day 7
    • Fine-Tuning vs. Transfer Learning in Voice Synthesis
    • Fine-Tuning in Deep Learning with a practical example – day 6
    • DeepNet – Scaling Transformers to 1,000 Layers new for 2024 – day 5
    • Transformers seems most famous for deep learning on 2024 so lets learn it more – day 4
    • Effortlessly Integrate ChatGPT-4 API on Mac with Xcode: Your Ultimate 2024 Guide to Token-Based Access Without a Monthly Subscription – day 3
    • New on 2024 – 2025 are MLX and Transofermers so lets compare Custom Deep Learning Models for iOS with MLX on Apple Silicon vs. PyTorch – day 2
    • Deep Learning in 2024: Continued Insights and Strategies – day 1
  • Application
    • Flash Card
    • Video Voice Change
    • Todo
    • background img remove INGOAMPT
    • Video Voice Edit INGOAMPT
  • Machine learning _ Deep learning ( day by day )
    • Reinforcement Learning: An Evolution from Games to Real-World Impact – day 77
    • How Dalle Image Generator works ? – day 76
    • Breaking Down Diffusion Models in Deep Learning – Day 75
    • Understanding Unsupervised Pretraining Using Stacked Autoencoders – day 74
    • Unlock the Secrets of Autoencoders, GANs, and Diffusion Models – Why You Must Know Them? -Day 73
    • The Rise of Transformers in Vision and Multimodal Models – Hugging Face – day 72
    • Mastering NLP: Unlocking the Math Behind It for Breakthrough Insights with a scientific paper study – day 71
    • How ChatGPT Work Step by Step – day 70
    • Can ChatGPT Truly Understand What We’re Saying? A Powerful Comparison with BERT” – day 69
    • Leveraging Scientific Research to Uncover How ChatGPT Supports Clinical and Medical Applications – day 68
    • Do you want to read a summery of what is BERT in 2 min read? (Bidirectional Encoder Representations from Transformers) – day 67
    • Transformers Deep Learning – day 66
    • The Revolution of Transformer Models – day 65
    • why transformers are better for NLP ? Let’s see the math behind it – Day 64
    • Natural Language Processing (NLP) and RNN – day 63
    • RNN, Layer Normalization, and LSTMs – Part 8 of RNN Deep Learning- day 62
    • Iterative Forecasting which is Predicting One Step at a Time 2- Direct Multi-Step Forecasting with RNN 3- Seq2Seq Models for Time Series Forecasting – day 61
    • Step-by-Step Explanation of RNN for Time Series Forecasting – part 6 – day 60
    • To learn what is RNN (Recurrent Neural Networks ) why not understand ARIMA, SARIMA first ? – RNN Learning – Part 5 – day 59
    • Understanding RNNs: why compare it with Feedforward Neural Networks with simple Example to show the Math Behind it ? – DAY 58 INGOAMPT
    • Time Series Forecasting with Recurrent Neural Networks (RNNs) – part 3 – day 57
    • Understanding Recurrent Neural Networks (RNNs) – part 2 – day 56
    • RNN Deep Learning – Part 1 – Day 55
    • Mathematics Behind CNN or Convolutional Neural Network in Deep Learning – Day 54
    • CNN- Convolutional Neural Networks – DAY 53
    • Deep Learning types and Best to Learn in 2024 as an iOS Developer – day 52
    • Comprehensive guide to deep learning in 2024 and 2025 covering models, trends, and beginner tips. Explore Feedforward Neural Networks (FNNs), CNNs, RNNs, GANs, and emerging trends like federated learning, explainable AI, and quantum AI – day 51
    • Comprehensive guide to deep learning in 2024 and 2025 covering models, trends, and beginner tips. Explore Feedforward Neural Networks (FNNs), CNNs, RNNs, GANs, and emerging trends like federated learning, explainable AI, and quantum AI – day 51
    • How to Configure and Optimize Deep Neural Networks for Performance and Efficiency – Day 50
    • Learn Max-Norm Regularization to avoid overfitting : Theory and Importance in Deep Learning and proof – day 49
    • DropOut and Monte Carlo Dropout (MC Dropout)- day 48
    • Understanding Regularization in Deep Learning – day 47
    • Comparing TensorFlow (Keras), PyTorch, and MLX -day 46
    • Learning rate – part 4- 1-Cycle Scheduling, exponential decay and Cyclic Exponential Decay (CED) – day 45
    • Exploring Gradient Clipping and Weight Initialization in Deep Learning – day 44
    • Theory Behind 1Cycle Learning Rate Scheduling and Learning Rate Schedules – Day 43
    • Thes Power of Learning Rates in Deep Learning and Why Schedules Matter- day 42
    • Deep Learning Optimizers: NAdam, AdaMax, AdamW, and NAG Comparison – day 41
    • Adam Optimizer deeply explained – day 40
    • Adam vs. SGD: Selecting the Right Optimizer for Your Deep Learning Model
    • AdaGrad, RMSProp, and Adam: A Comparative Guide to Optimization Algorithms – day 38
    • A Comprehensive Guide to AdaGrad: Origins, Mechanism, and Mathematical Proof – day 37
    • Nag as optimiser in deep learning – day 36
    • Momentum – part 3 – day 35
    • Momentum vs Normalisation in Deep learning -part 2 – day 34
    • Momentum Optimization in Machine Learning: A Detailed Mathematical Analysis and Practical Application – day 33
    • Mastering Deep Neural Network Optimization: Techniques and Algorithms for Faster Training – day 32
    • fundamentals of labeled and unlabeled data in machine learning – day 31
    • How do Transfer Learning in Deep Learning Model – with a example – day 30
    • Transfer learning – day 29
    • Understanding Gradient Clipping in Deep Learning – day 28
    • Batch normalisation – trainable and non trainable – day 27
    • Batch normalisation part 2 – day 26
    • Batch Normalization – day 25
    • Activation function progress in deep learning, Relu, Elu, Selu, Geli , mish, etc – include table and graphs – day 24
    • Weight initialazation part 2 – day 23
    • How Create API by Deep Learning to Earn Money and what is the Best Way for Mac Users – Breaking studies on day 22
    • Weight initialisation in Deep Learning well explained: day 21
    • Vanishing gradient explained in detail _ day 20
    • Hyperparameter Tuning and Neural Network Architectures, eg : Bayesian Optimization _ day 19
    • Day 18 _ Automatic vs Manual optimisation in Keras
    • Day 17 _ Hyperparameter Tuning with Keras Tuner
    • Day 16 – TensorFlow: Using TensorBoard, Callbacks, and Model Saving in Keras
    • Day 15 _ Sequential vs Functional Keras API Part 2 explanation
    • Day 14 _ sequential , functional and model subclassing API in Keras
    • Day 13 _ What is Keras
    • Day 12 _ Activation Function, Hidden Layer and non linearity
    • Day 11 _ Activation Function
    • Day 10 _ Regression vs Classification Multi Layer Perceptrons (MLPs)
    • Day 9 _ Deep Learning _ Perceptrons
    • Day 8 _ Gradient Decent Types : Batch, Stochastic and Mini-Batch
    • Day 7 _ Gradient Decent in Machine Learning
    • Day 6 _ Why the Normal Equation Works Without Gradient Descent
    • Day 5 _ Mathematical Explanation behind SGD Algorithm in Machine Learning
    • Day 4 __ Regression & Classification with MNIST
    • Day 3 _ Model vs Instance Models _Train-Test Split _ ML
    • Day 2 _ integrate ML into iOS Apps
    • Day 1 _ Machine learning (ML) Overview
  • About us
    • Contact us
Do Not Forget to Check our IOS Apps in Applications section :) Thanks for your support in advance
Privacy Policies
  • INGOAMPT Privacy Policy
  • FlashCardINGOAMPT App Privacy Policy
  • Background Img Remove App Privacy Policy
  • Planner ToDo App Privacy Policy
  • Voice Edit App Privacy Policy
  • Text & Speech App Privacy Policy
Quick Links
  • Home
  • Published Applications
  • Education
© 2024 INGOAMPT. All rights reserved.

Artificial Intelligence ( Machine Learning & Deep Learning) + iOS apps with Deep learning integration all by INGOAMPT Dismiss

don't miss our new posts. Subscribe for updates

We don’t spam! Read our privacy policy for more info.

Check your inbox or spam folder to confirm your subscription.

We noticed you're visiting from United States (US). We've updated our prices to United States (US) dollar for your shopping convenience. Use Euro instead. Dismiss