Mastering TensorFlow: Using TensorBoard, Callbacks, and Model Saving in Keras

TensorFlow and Keras provide powerful tools for building, training, and evaluating deep learning models. In this blog post, we will explore three essential techniques:

• Using TensorBoard for visualization
• Utilizing callbacks to enhance model training
• Saving and restoring models

Using TensorBoard for Visualization

TensorBoard is an interactive visualization tool that helps you understand your model’s training dynamics. It allows you to view learning curves, compare metrics between multiple runs, and analyze training statistics.

Installation

!pip install -q -U tensorflow tensorboard-plugin-profile

Setting Up Logging Directory

We need a directory to save our logs. This directory will contain event files that TensorBoard reads to visualize the training process.

from pathlib import Path
from time import strftime

def get_run_logdir(root_logdir="my_logs"):
    return Path(root_logdir) / strftime("run_%Y_%m_%d_%H_%M_%S")

run_logdir = get_run_logdir()

Saving and Restoring a Model

Keras allows you to save the entire model (architecture, weights, and training configuration) to a single file or a folder.

Saving a Model

model.save("my_keras_model", save_format="tf")

Loading a Model

model = tf.keras.models.load_model("my_keras_model")

Saving Weights Only

model.save_weights("my_weights.h5")
model.load_weights("my_weights.h5")

Using Callbacks

Callbacks in Keras allow you to perform actions at various stages of training (e.g., saving checkpoints, early stopping).

ModelCheckpoint

Save the model at regular intervals.

checkpoint_cb = tf.keras.callbacks.ModelCheckpoint("my_checkpoints.weights.h5", save_weights_only=True)

EarlyStopping

Stop training when a monitored metric has stopped improving.

early_stopping_cb = tf.keras.callbacks.EarlyStopping(patience=10, restore_best_weights=True)

TensorBoard

Log data for TensorBoard.

tensorboard_cb = tf.keras.callbacks.TensorBoard(log_dir=run_logdir, profile_batch=(100, 200))

Full Code with Techniques Applied

Here is the full code incorporating all the techniques discussed:

# Install TensorFlow and TensorBoard plugin<br />
!pip install -q -U tensorflow tensorboard-plugin-profile</p>
<p># Import necessary libraries<br />
import tensorflow as tf<br />
import matplotlib.pyplot as plt<br />
import pandas as pd<br />
from pathlib import Path<br />
from time import strftime</p>
<p># Load Fashion MNIST dataset<br />
fashion_mnist = tf.keras.datasets.fashion_mnist<br />
(X_train_full, y_train_full), (X_test, y_test) = fashion_mnist.load_data()</p>
<p># Split data into training, validation, and test sets<br />
X_train, y_train = X_train_full[:-5000], y_train_full[:-5000]<br />
X_valid, y_valid = X_train_full[-5000:], y_train_full[-5000:]</p>
<p># Scale pixel values to the 0-1 range<br />
X_train, X_valid, X_test = X_train / 255.0, X_valid / 255.0, X_test / 255.0</p>
<p># Class names for Fashion MNIST<br />
class_names = ["T-shirt/top", "Trouser", "Pullover", "Dress", "Coat", "Sandal", "Shirt", "Sneaker", "Bag", "Ankle boot"]</p>
<p># Display the first few images and labels<br />
plt.figure(figsize=(10,10))<br />
for i in range(25):<br />
plt.subplot(5, 5, i + 1)<br />
plt.xticks([])<br />
plt.yticks([])<br />
plt.grid(False)<br />
plt.imshow(X_train[i], cmap=plt.cm.binary)<br />
plt.xlabel(class_names[y_train[i]])<br />
plt.show()</p>
<p># Set random seed for reproducibility<br />
tf.random.set_seed(42)</p>
<p># Build the model<br />
model = tf.keras.Sequential([<br />
tf.keras.layers.Flatten(input_shape=[28, 28]),<br />
tf.keras.layers.Dense(300, activation="relu"),<br />
tf.keras.layers.Dense(100, activation="relu"),<br />
tf.keras.layers.Dense(10, activation="softmax")<br />
])</p>
<p># Compile the model<br />
model.compile(loss="sparse_categorical_crossentropy", optimizer="sgd", metrics=["accuracy"])</p>
<p># Define log directory for TensorBoard<br />
def get_run_logdir(root_logdir="my_logs"):<br />
return Path(root_logdir) / strftime("run_%Y_%m_%d_%H_%M_%S")</p>
<p>run_logdir = get_run_logdir()</p>
<p># Define callbacks with corrected checkpoint filename<br />
checkpoint_cb = tf.keras.callbacks.ModelCheckpoint("my_checkpoints.weights.h5", save_weights_only=True)<br />
early_stopping_cb = tf.keras.callbacks.EarlyStopping(patience=10, restore_best_weights=True)<br />
tensorboard_cb = tf.keras.callbacks.TensorBoard(log_dir=run_logdir, profile_batch=(100, 200))</p>
<p># Train the model with callbacks<br />
history = model.fit(X_train, y_train, epochs=30, validation_data=(X_valid, y_valid), callbacks=[checkpoint_cb, early_stopping_cb, tensorboard_cb])</p>
<p># Start TensorBoard<br />
%load_ext tensorboard<br />
%tensorboard --logdir=./my_logs</p>
<p># Evaluate the model on the test set<br />
test_loss, test_acc = model.evaluate(X_test, y_test)<br />
print(f"Test accuracy: {test_acc:.4f}")</p>
<p># Plot training and validation accuracy and loss<br />
def plot_learning_curves(history):<br />
pd.DataFrame(history.history).plot(figsize=(8, 5))<br />
plt.grid(True)<br />
plt.gca().set_ylim(0, 1)  # set the vertical range to [0-1]<br />
plt.show()</p>
<p>plot_learning_curves(history)</p>
<p># Make predictions<br />
y_pred = model.predict(X_test)</p>
<p># Plot the first 25 test images, their predicted labels, and the true labels.<br />
# Color correct predictions in blue and incorrect predictions in red.<br />
plt.figure(figsize=(10,10))<br />
for i in range(25):<br />
plt.subplot(5, 5, i + 1)<br />
plt.xticks([])<br />
plt.yticks([])<br />
plt.grid(False)<br />
plt.imshow(X_test[i], cmap=plt.cm.binary)<br />
predicted_label = class_names[y_pred[i].argmax()]<br />
true_label = class_names[y_test[i]]<br />
color = 'blue' if predicted_label == true_label else 'red'<br />
plt.xlabel(f"{predicted_label} ({true_label})", color=color)<br />
plt.show()</p>
<p>

This blog post covers the essential techniques of using TensorBoard, callbacks, and model saving/restoring in Keras with TensorFlow. These techniques will enhance your model training process and provide valuable insights into your model's performance. Happy coding!


Check the results in the screenshots Bellow : 
Just run the code in Google Colab to see your Self but let’s see some images from the result of our given code which was a functional Keras mode which we discussed in our previous days post