How to Use the Keras Functional API?

A practical guide on the Keras Functional API for building complex deep learning models

Freepik

Keras is a popular high-level deep learning framework that provides a user-friendly interface to build and train neural networks. One of the key components of Keras is the Functional API, which allows you to define complex models with more flexibility than the Sequential model.

In this blog, we’ll explore the Keras Functional API and learn how to use it to build complex deep learning models. Let’s take a look at the topics we’ll cover. First, we’ll briefly discuss what the Keras functional API is. Next, we’ll load the dataset and perform data preprocessing. After that, we’ll learn how to build a model using Keras functional API. Finally, we’ll handle how to make predictions. Let’s dive in!

What is the Keras Functional API?

You know, deep learning models consist of layers. So, the Keras functional API is a way to build graphs of layers. This API allows you to easily build complex deep learning models that are more flexible than the Keras sequential API.

You can build deep learning models that have multiple inputs and outputs, share layers, and have non-linear connectivity between layers with this approach. This flexibility makes it a powerful tool for building a wide range of neural network architectures, including multi-input and multi-output models, directed acyclic graphs, and residual networks.

Cool, we briefly discussed what the Keras functional API is. Now, let’s take a look at how to use this API. To show this, let me load the dataset.

Data Loading

The dataset we are going to use is the California housing dataset. This dataset contains 8 features. All features are also numerical features encoded as floating numbers. Let’s load this dataset with Scikit-Learn. To do this, let’s import the fetch_california_housing function and then load the dataset with this function.

from sklearn.datasets import fetch_california_housing
housing = fetch_california_housing()

Nice, we loaded the dataset. Let’s move on to perform data preprocessing.

Data Preprocessing

Data preprocessing is an essential step in preparing data for model training. Before building the model, what we’re going to do first is to split the dataset. We build the model with the training set and evaluate the model with the test set.

Let’s split the dataset into training and test sets. To do this, we can use the train_test_split function.

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
     housing.data, housing.target, random_state=42)

Cool, the dataset is ready to build the model. Let’s move on to build the model.

Model Building

So far, we loaded the dataset and then created the training and test sets. Now, let’s handle how to build the model with Keras functional API. To show this, let’s import TensorFlow and then set TensorFlow’s random seed to make the results reproducible.

import tensorflow as tf
tf.random.set_seed(42)

Before we train the model, let’s create the layers that we will use to build the model. This is the model we will create.

Resource: Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow

As you can see, the model consists of two input layers, two normalization layers, two hidden layers, a concatenation layer, and lastly an output layer.

Consider a scenario where we want to go five attributes through one path and six attributes through another path. To show this, let’s start creating the wide input layers.

input_wide = tf.keras.layers.Input(shape=[5])
input_deep = tf.keras.layers.Input(shape=[6])

Cool, we created the input layers. Let’s go ahead and create the normalization layers. These layers work like StandardScaler in Scikit-Learn These layers help us scale data. Since we have two inputs, let’s create two normalization layers.

norm_layer_wide = tf.keras.layers.Normalization()
norm_layer_deep = tf.keras.layers.Normalization()

Awesome, we created the normalization layers. Note that, to use these normalization layers, we need to call the adapt method before calling the fit. Now, let’s pass our inputs to these normalization layers.

norm_wide = norm_layer_wide(input_wide)
norm_deep = norm_layer_deep(input_deep)

Note that data passing through the wide normalization layer goes directly to the concat layer. The other data, passing through the deep normalization layer, goes to the hidden 1 layer. Now, let’s create the first hidden layer.

hidden1 = tf.keras.layers.Dense(30, activation="relu")(norm_deep)
hidden2 = tf.keras.layers.Dense(30, activation="relu")(hidden1)

Cool, we’ve created our hidden layers. Now, let’s create a layer to concatenate the data passing through the hidden layers with the data passing through the wide normalization layer.

concat = tf.keras.layers.concatenate([norm_wide, hidden2])

Nice, we also created the concat layer. Now, let’s create our output layer. Since our problem is regression, we are going to set a neuron in this layer.

output = tf.keras.layers.Dense(1)(concat)

Awesome, we created our all layers. Now, let’s create our Keras model and pass the inputs and outputs to it.

model = tf.keras.Model(inputs=[input_wide, input_deep], outputs=[output])

Ok, we created the architecture of our model. We can display the input and output shapes of each layer in the plotted graph. Let me show you this.

keras.utils.plot_model(model, "my_first_model_with_shape_info.png", show_shapes=True)

The model architecture (Image by Author)

You can see the architecture of our model in this figure. Let’s move on to compile the model.

Model Compiling

Before training the model, we need to compile our model. While compiling, we specify a loss function, an optimizer, and metrics we will use. First, we specify our optimizer and then compile our model with the compile method.

optimizer = tf.keras.optimizers.Adam(learning_rate=1e-3)
model.compile(
   loss="mse", optimizer=optimizer, metrics=["RootMeanSquaredError"])

Cool, we compile our model with the compile method. Now, let’s create our input data. Let’s assign the first 5 attributes to a variable and the attributes after 2 to a variable. First, let’s create our training and test sets.

X_train_wide, X_train_deep = X_train[:, :5], X_train[:, 2:]
X_test_wide, X_test_deep = X_test[:, :5], X_test[:, 2:]

Next, let’s use the adapt method to find the mean and standard deviation values that we will use for the normalization layer.

norm_layer_wide.adapt(X_train_wide)
norm_layer_deep.adapt(X_train_deep)

Let’s train our model by calling the fit method. To use validation data, we’re going to use the validation_split parameter.

history = model.fit(
            (X_train_wide, X_train_deep), 
             y_train, 
             epochs=20, 
             validation_split=0.2))

When we run this command, model training starts.

Note that at the end of each epoch, the loss and evaluation metrics are calculated. We see that the loss values are decreasing. This means that the model learns from the data.

It is time to evaluate our model on the test data. For this, let’s call it the evaluate method.

test_scores = model.evaluate((X_test_wide, X_test_deep), y_test)

# Output:
loss: 0.3226 - root_mean_squared_error: 0.5679

Here you can see the loss and metric values. Now, let’s take some data from the test dataset and evaluate this data according to our model.

X_new_wide, X_new_deep = X_test_wide[:3], X_test_deep[:3]

Nice, we got our sample data. Now, let’s predict this data according to our model by calling the predict method.

y_pred = model.predict((X_new_wide, X_new_deep))
y_pred

# Output:
array([[0.36645722],
       [1.1630366 ],
       [3.6675177 ]], dtype=float32)

Okay, so we’ve made the predictions of the data. Let’s take a look at the actual values.

y_test[:3]

# Output:
array([0.477  , 0.458  , 5.00001])

The predictions aren’t too bad. You can get better predictions by fine-tuning the hyperparameters of the model.

Conclusion

The Keras Functional API is a powerful tool for building complex and flexible deep learning models. With the Functional API, you can define models with multiple inputs and outputs, share layers, and have non-linear connectivity between layers. This flexibility allows you to build a wide range of neural network architectures, from simple to more complex models.

In this blog post, we explored the basics of the Keras Functional API, including defining input and output layers, and building complex models using the Functional API.

The Keras Functional API offers a lot of flexibility, which can be overwhelming for beginners. However, with practice and patience, you can become proficient in building complex models using the Functional API.

That’s it. Thanks for reading. You can find the link to the notebook used in this blog here. Let’s connect YouTube | Medium | Twitter | Instagram.

Twitter Sentiment Analysis: FIFA World Cup 2022

7 Steps to Become a Machine Learning Engineer

Resource

• Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow

Level Up Coding

Thanks for being a part of our community! Before you go:

• 👏 Clap for the story and follow the author 👉
• 📰 View more content in the Level Up Coding publication
• 💰 Free coding interview course ⇒ View Course
• 🔔 Follow us: Twitter | LinkedIn | Newsletter

🚀👉 Join the Level Up talent collective and find an amazing job