Day 15 _ Sequential vs Functional Keras API Part 2 explanation

Let’s continue from day 14 which explained the Keras API and compare them

Part 1: Understanding Sequential vs. Functional API in Keras with a Simple Example

When building neural networks in Keras, there are two main ways to define models: the Sequential API and the Functional API. In this post, we’ll explore the differences between these two approaches using a simple mathematical example.

Sequential API

The Sequential API in Keras is a linear stack of layers. It’s easy to use but limited to single-input, single-output stacks of layers. Here’s a simple example to illustrate how it works.

Objective:

1. Multiply the input \( x \) by 2.
2. Add 3 to the result.

Mathematical Operations:

\[ y_1 = 2 \times x \]
\[ y_2 = y_1 + 3 \]

Let’s implement this using the Sequential API:

<br />
from keras.models import Sequential<br />
from keras.layers import Lambda</p>
<p># Define a simple sequential model<br />
model = Sequential()<br />
model.add(Lambda(lambda x: 2 * x, input_shape=(1,)))<br />
model.add(Lambda(lambda x: x + 3))</p>
<p>model.summary()<br />

Example Calculation:

• If \( x = 1 \):
• \( y_1 = 2 \times 1 = 2 \)
• \( y_2 = 2 + 3 = 5 \)

Functional API

The Functional API in Keras is more flexible and allows for the creation of complex models with multiple inputs and outputs. We’ll use the same mathematical operations to illustrate how it works.

Objective:

1. Multiply the input \( x \) by 2.
2. Add 3 to the result.

Mathematical Operations:

\[ y_1 = 2 \times x \]
\[ y_2 = y_1 + 3 \]

Let’s implement this using the Functional API:

<br />
from keras.models import Model<br />
from keras.layers import Input, Lambda</p>
<p># Define the input<br />
input_ = Input(shape=(1,))</p>
<p># Define the operations<br />
y1 = Lambda(lambda x: 2 * x)(input_)<br />
y2 = Lambda(lambda x: x + 3)(y1)</p>
<p># Create the model<br />
model = Model(inputs=input_, outputs=y2)</p>
<p>model.summary()<br />

Example Calculation:

• If \( x = 1 \):
• \( y_1 = 2 \times 1 = 2 \)
• \( y_2 = 2 + 3 = 5 \)

Summary

Both the Sequential and Functional APIs can achieve the same result in this simple example. However, the Sequential API is limited to linear stacks of layers, while the Functional API allows for more complex architectures. In the next part of this blog post, we’ll dive into a real neural network scenario to further illustrate the differences.

Part 2: Comparing Sequential vs. Functional API with a Real Neural Network Example

In this part, we’ll explore a more complex example involving multiple inputs and outputs to highlight the differences between the Sequential and Functional APIs in Keras.

Real Scenario: Neural Network with Multiple Inputs

Suppose we have two features, \( x_1 \) and \( x_2 \), which represent two different inputs. We want to create a neural network that processes these inputs separately and then combines their outputs to make a final prediction.

Sequential API

The Sequential API does not natively support multiple inputs and outputs. To illustrate, we’ll simplify the scenario by concatenating the inputs first and then processing them.

Sequential Model:

1. Concatenate \( x_1 \) and \( x_2 \).
2. Process the concatenated input through a Dense layer.
3. Produce the final output through another Dense layer.

Implementation:

<br />
from keras.models import Sequential<br />
from keras.layers import Dense, InputLayer</p>
<p># Simulate combined input<br />
model = Sequential()<br />
model.add(InputLayer(input_shape=(2,)))<br />
model.add(Dense(units=4, activation='relu'))<br />
model.add(Dense(units=1, activation='sigmoid'))</p>
<p>model.compile(optimizer='adam', loss='binary_crossentropy')<br />
model.summary()<br />

Here, we concatenate \( x_1 \) and \( x_2 \) manually before feeding them into the model.

Functional API

The Functional API allows us to directly work with multiple inputs and process them separately before combining the results.

Functional Model:

1. Process \( x_1 \) through a Dense layer.
2. Process \( x_2 \) through another Dense layer.
3. Combine the processed outputs.
4. Produce the final output through a Dense layer.

Implementation:

<br />
from keras.models import Model<br />
from keras.layers import Input, Dense, concatenate</p>
<p># Define inputs<br />
input_1 = Input(shape=(1,))<br />
input_2 = Input(shape=(1,))</p>
<p># Process each input separately<br />
y1 = Dense(units=4, activation='relu')(input_1)<br />
y2 = Dense(units=4, activation='relu')(input_2)</p>
<p># Combine the processed inputs<br />
combined = concatenate([y1, y2])</p>
<p># Final output layer<br />
output = Dense(units=1, activation='sigmoid')(combined)</p>
<p># Create the model<br />
model = Model(inputs=[input_1, input_2], outputs=output)</p>
<p>model.compile(optimizer='adam', loss='binary_crossentropy')<br />
model.summary()<br />

Summary of Operations

Sequential API:

• Input: \([x_1, x_2]\)
• Dense layer: \( y = \text{ReLU}(W_1 \cdot [x_1, x_2] + b_1) \)
• Output layer: \( z = \text{sigmoid}(W_2 \cdot y + b_2) \)

Functional API:

• Input 1: \( x_1 \)
• Dense layer: \( y_1 = \text{ReLU}(W_1 \cdot x_1 + b_1) \)
• Input 2: \( x_2 \)
• Dense layer: \( y_2 = \text{ReLU}(W_2 \cdot x_2 + b_2) \)
• Combine: \( y = \text{concatenate}([y_1, y_2]) \)
• Output layer: \( z = \text{sigmoid}(W_3 \cdot y + b_3) \)

Example Calculation

Assume \( x_1 = 1 \), \( x_2 = 2 \), and weights and biases are initialized as follows for simplicity:

• \( W_1 = W_2 = [1, 1, 1, 1] \)
• \( b_1 = b_2 = [0, 0, 0, 0] \)
• \( W_3 = [1, 1, 1, 1, 1, 1, 1, 1] \)
• \( b_3 = 0 \)

Sequential API:

• \( y = \text{ReLU}([1, 1, 1, 1] \cdot [1, 2] + [0, 0, 0, 0]) = \text{ReLU}([3, 3, 3, 3]) = [3, 3, 3, 3] \)
• \( z = \text{sigmoid}([1, 1, 1, 1] \cdot [3, 3, 3, 3] + 0) = \text{sigmoid}(12) \approx 1 \)

Functional API:

• \( y_1 = \text{ReLU}([1, 1, 1, 1] \cdot 1 + [0, 0, 0, 0]) = [1, 1, 1, 1] \)
• \( y_2 = \text{ReLU}([1, 1, 1, 1] \cdot 2 + [0, 0, 0, 0]) = [2, 2, 2, 2] \)
• Combine: \( y = \text{concatenate}([1, 1, 1, 1, 2, 2, 2, 2]) = [1, 1, 1, 1, 2, 2, 2, 2] \)
• \( z = \text{sigmoid}([1, 1, 1, 1, 1, 1, 1, 1] \cdot [1, 1, 1, 1, 2, 2, 2, 2] + 0) = \text{sigmoid}(12) \approx 1 \)

Conclusion

The Functional API provides greater flexibility for handling complex architectures involving multiple inputs and outputs, as demonstrated in this example. It allows you to define models where inputs are processed through separate paths and combined in various ways, which is not directly possible with the Sequential API.

Let’s even compare them in an image