Deploying and scaling Node.js on Google Kubernetes Engine with Continuous Integration

Table of Contents

• Introduction
• Setup the Node.js Server
• Setting up Google Cloud
• Dockerize Node.js
• Deploy an Image to the Container Registry
• Create Kubernetes Cluster
• Reserve a Static IP Address
• Create Kubernetes Services
• Kubernetes Autoscaling
• Setup Google Key Management Service
• Setup Cloud Build

Introduction

This post will cover how to dockerize a Node.js server to deploy on a Kubernetes cluster using Google Kubernetes Engine (GKE). We will be setting up a Node.js server, creating a Dockerfile to define the container configuration, create Kubernetes service files to define our required Kubernetes resources and setting up Google Cloud Build for automated continuous integration (CI). I uploaded my own code to GitHub here in case you run into any issues along the way, so make sure to keep it handy!

If you’ve completed my tutorial on dockerizing and deploying Node.js using Google Compute Engine, or you already have a dockerized Node.js app, you can skim through the first few steps of this tutorial as they are mostly repeats.

If you have your own Node.js app you’d like to use that’s fine too, although it may be easier to complete this tutorial with a simple application to avoid any unnecessary errors. For the purpose of this tutorial, I’m going to use a Node boilerplate script I’ve written — https://github.com/francescov1/node-boilerplate-script.

Note: if you are using your own Node.js code, ensure it is setup to run on port 3000 as I will be using that port for all Docker and Google Cloud configuration.

Setup the Node.js Server

If you are using your own Node.js app, skip this step. We’re going to clone the node-boilerplate-script repo, make a new folder and run the script.

git clone https://github.com/francescov1/node-boilerplate-script.git
mkdir gke-tutorial
cp node-boilerplate-script/node-init.sh gke-tutorial/
cd gke-tutorial/
bash node-init.sh

After a few moments, it should be complete. Run npm start and navigate to http://localhost:3000/api/examples/hello, where you should see “hello world”.

Setting up Google Cloud

Next we are going to setup Google Cloud. If you don’t have a Google Cloud account, check out https://cloud.google.com/free to learn more and for instructions on creating your account. Upon signup, you are given $300 of free credits, so don’t worry about needing to pay for the resources you create in this tutorial.

Create a new project in the Google Cloud console. If you’ve never done this before, see https://cloud.google.com/resource-manager/docs/creating-managing-projects. We will call our project gke-tutorial.

Next head over to https://cloud.google.com/sdk/docs/#install_the_latest_cloud_tools_version_cloudsdk_current_version to download the Google Cloud CLI (if you haven’t already), which we will use to deploy Kubernetes resources.

Once it is installed, run gcloud init. This will guide you through authenticating the CLI and choosing your project.

Dockerize Node.js

We can now setup Docker for our Node.js server before deploying it. Docker allows us to containerize our application in an easily portable and scalable format. I won’t go into detail on Docker or its benefits, so if you are interested check out https://dzone.com/articles/top-10-benefits-of-using-docker.

I’ll be using Atom, but feel free to use any other text editor. Create the Dockerfile and .dockerignore and open the working directory in your text editor.

touch Dockerfile .dockerignore

Add the following code to your .dockerignore:

node_modules
npm-debug.log

The above defines files and folders to ignore when building your Docker image. Add the following code to your Dockerfile:

# version of node to use
FROM node:8
# define working directory for docker
WORKDIR /usr/src/app
# copy all our source code into the working directory
COPY . .
# install npm dependencies and pm2
RUN npm install --only=production && npm install -g pm2
# expose port 3000 for our server to run on
EXPOSE 3000
# command to start our server
CMD [ "pm2-runtime", "start", "index.js" ]

Pm2 is a process manager that we will use to boot up our server. It comes bundled with a ton of cool features such as clustering and load balancing. Learn more here https://pm2.io/runtime. The CMD [ “pm2-runtime", “start", “index.js" ]line starts our Node.js server.

Deploy an Image to the Container Registry

We can now deploy a Docker image to the Google Container Registry, where it can be accessed by other services on Google Cloud. Run the following command in the root:

gcloud builds submit --tag gcr.io/<project-id>/gke-tutorial-image .

If you are prompted to enable the cloudbuild API, enter yes.

Create Kubernetes Cluster

We are now going to create a Kubernetes cluster in Google Cloud. Kubernetes can be quite daunting at first glance, so here’s a few articles to check out for a solid general overview: Kubernetes overview, Kubernetes in Google Cloud

Enter the following command in your terminal:

gcloud container clusters create gke-tutorial-cluster --disk-size 10 --num-nodes 1 --enable-autoscaling --min-nodes 1 --max-nodes 5 --zone us-central1-a

This will create a Kubernetes cluster called gke-tutorial-cluster with a disk size of 10GB and a single node. It also enables autoscaling, and sets the minimum and maximum number of nodes. The autoscaling feature will be discussed more later in the post.

It will take the cluster a few minutes to spin up. Once it’s complete, head over to https://console.cloud.google.com/kubernetes/list to see the cluster you just deployed.

Reserve a Static IP Address

We will now reserve a static IP address that will be used to access our application. Enter the following command:

gcloud compute addresses create gke-tutorial-ip

When prompted for the region, enter 1 (which indicates global) and hit enter.

Create Kubernetes Services

Next we need to define the specifications of a few Kubernetes services. The first thing we will do is install the Kubernetes CLI (kubectl). Run:

gcloud components install kubectl

We will be creating 4 files; ingress.yaml which will create a doorway into our app for external traffic by creating an HTTP Load Balancer, deployment.yaml which will provide specifications for our pods and our Docker image, service.yaml which will create a Node port that will receive traffic from the HTTP Load Balancer and send it to our Node.js server, and podscaler.yaml which will define horizontal pod scaling configuration (again more on the autoscaling later).

Create the following folder and files:

mkdir k8s
cd k8s
touch deployment.yaml ingress.yaml service.yaml podscaler.yaml

Now open deployment.yaml and add the following code (make sure to replace <project-id> in the image definition with your own:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gke-tutorial-deployment
  namespace: default
  labels:
    app: gke-tutorial-deployment
spec:
  replicas: 1 # number of pods
  template:
    metadata:
      labels:
        app: gke-tutorial-deployment
    spec:
      containers:
        - name: gke-tutorial-image
          image: gcr.io/<project-id>/gke-tutorial-image
          imagePullPolicy: Always
          ports:
            - containerPort: 3000
              protocol: TCP

For service.yaml, enter:

apiVersion: v1
kind: Service
metadata:
  name: gke-tutorial-deployment
spec:
  selector:
    app: gke-tutorial-deployment
  type: NodePort
  ports:
    - protocol: TCP
      port: 3000
      targetPort: 3000

which creates a doorway into our Docker container. For ingress.yaml, enter:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gke-tutorial-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: "gke-tutorial-ip"
spec:
  backend:
    serviceName: gke-tutorial-deployment
    servicePort: 3000

This will create the HTTP load balancer and point it to the NodePort service. Finally, for podscaler.yaml enter:

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: gke-tutorial-hpa
  namespace: default
  labels:
    app: gke-tutorial-deployment
spec:
  scaleTargetRef:
    kind: Deployment
    name: gke-tutorial-deployment
    apiVersion: apps/v1beta1
  minReplicas: 1
  maxReplicas: 5
  metrics:
    - type: Resource
      resource:
        name: cpu
        targetAverageUtilization: 80

Now we can deploy these services to our Kubernetes cluster using the Kubernetes CLI. Just before we do that, we need to configure kubectl with our cluster deployed in Google Cloud. Run:

gcloud container clusters get-credentials gke-tutorial-cluster --zone us-central1-a

Now run the following command to deploy the services:

kubectl apply -f k8s/

And all the services will deploy! If you navigate to the Kubernetes Engine in the Google cloud console (https://console.cloud.google.com/kubernetes) you can check out the Services by clicking it in the left sidebar, where you should see a NodePort service and an Ingress service being created. Under Workloads you will see your deployment.

It will take a few minutes to get everything up and running (the ingress service usually takes the longest to get going) so once the ingress service’s status changes from “Creating ingress” to “Ok”, enter the following command to get the external IP address we created earlier:

gcloud compute addresses list

This is the address that was attached to the load balancer the ingress service created. Enter this address into your browser and you should see My Node.js API.

Congrats! We’ve now setup our Node.js server in Kubernetes. Next, we will implement Continuous Integration (CI) using Google Cloud Build. This will automate our deployment process so that every time we make new changes to the app they will be deployed automatically.

Kubernetes Autoscaling

Before we continue, I’m going to do a short overview of how the autoscaling that we’ve configured works, as it can be slightly confusing at first. You may have noticed that we defined two autoscaling configurations (the autoscaling flags when creating the Kubernetes cluster and the podscaler.yaml service). If you don’t understand any of this terminology definitely check out the Kubernetes overview articles I mentioned at the beginning of the Kubernetes section.

I’ll start with explaining the podscaler.yaml. This sets up what we call horizontal pod scaling (HPA). A pod is essentially a tightly-coupled group of containers that share the same hardware. Each node (or machine) can run multiple pods depending on the resources available. The HPA schedules more pods if the current pods are being over-utilized and removes pods if they are being under-utilized, but it will not scale the number of nodes. The nodes are simply VMs controlled by Google Cloud so this needs to be configured through that, rather than Kubernetes (so now you see where the cluster autoscaler comes in).

Now adding more pods isn’t going to help us out too much if we aren’t actually allocating more VMs to our cluster. That’s why we also use the cluster autoscaler (CA). This configuration, which we defined when creating the cluster, autoscales the number of nodes in our cluster as needed.

So together these two services work as follows: as more resources are needed, additional pods will be scheduled. Once the number of pods scheduled requires more nodes, the CA will allocate those additional nodes. The pod scaler will also move pods between nodes to maximize the resources used and limit the number of nodes required. If a node is not required anymore, the CA will remove it from the cluster.

If you still don’t understand or would like to learn more, check out this great article: https://medium.com/magalix/kubernetes-autoscaling-101-cluster-autoscaler-horizontal-pod-autoscaler-and-vertical-pod-2a441d9ad231.

Setup Google Key Management Service

Before we configure Cloud Build, we want to setup Google Key Management Service (KMS) so that we can encrypt sensitive keys or values in our repository and then decrypt them during the deployment process (check out https://cloud.google.com/kms) for more info.

Head to https://console.developers.google.com/apis/library/cloudkms.googleapis.com and click Enable to enable the API. Wait a minute then enter the following command (if you get an error, wait a couple more minutes and try again):

gcloud kms keyrings create gke-tutorial-keyring --location global

This creates a keyring called gke-tutorial-keyring. A keyring is essentially a group of keys, each of which can be used to encrypt and decrypt files. Now well create a key called default:

gcloud kms keys create default --location global --keyring gke-tutorial-keyring --purpose encryption

And finally we can encrypt our .env file (right now there are no sensitive values in it but if we wanted to connect to any APIs, we would store our credentials in here.

gcloud kms encrypt --location global \
  --keyring gke-tutorial-keyring --key default \
  --plaintext-file .env \
  --ciphertext-file .env.enc

You should now have a .env.enc file in your repository. This file is safe to check into version control as it is encrypted (try opening it, it will look like total gibberish).

Setup Cloud Build

To automate our code deployments, we’ll need to create a GitHub repository. Login to your GitHub (or create an account) and create an empty repository (call it whatever, mine is gke-tutorial). Copy the HTTPS repo link then head back to the command line and enter the following commands (ensure to replace <repo-url>):

git init
git add .
git commit -m "Initial commit"
git remote add origin <repo-url>
git push -u origin master

Now head over to the Cloud Build section of the Google Cloud Console (https://console.cloud.google.com/cloud-build), select Triggers on the left menu bar and click Create trigger.

Select GitHub, click the consent checkbox and click Continue. You should now be prompted to login to your GitHub account. Once done, you will be presented with a list of your repositories. Select the repository we just created and click Continue. Under Branch (regex), enter master, and under Build configuration, select Cloud Build configuration file (yaml or json).

We will also add a few substitution variables so that if we ever want to update the configuration in the future we won’t have to change our build files. Under substitution variables, click Add item 5 times and enter the variables shown beside. Finally, click Create trigger.

The last step is to create our cloudbuild.yaml file, which will tell Cloud Build how to build and deploy our application. Create the file:

touch cloudbuild.yaml

and add the following code:

steps:
  # decrypt .env
  - name: gcr.io/cloud-builders/gcloud
    id: "Decrypt environment variables"
    args:
      - kms
      - decrypt
      - --ciphertext-file=.env.enc
      - --plaintext-file=.env
      - --location=global
      - --keyring=$_KEYRING
      - --key=$_KEY
  # pull previous image to speed up docker build
  - name: "gcr.io/cloud-builders/docker"
    id: "Pull"
    entrypoint: "bash"
    args:
      - "-c"
      - |
        docker pull gcr.io/$PROJECT_ID/$_IMAGE:latest || exit 0
  # build docker image
  - name: "gcr.io/cloud-builders/docker"
    id: "Build"
    args:
      [
        "build",
        "-t",
        "gcr.io/$PROJECT_ID/$_IMAGE:$SHORT_SHA",
        "--cache-from",
        "gcr.io/$PROJECT_ID/$_IMAGE:latest",
        ".",
      ]
  # push image to container registry
  - name: "gcr.io/cloud-builders/docker"
    id: "Push"
    args:
      - "push"
      - "gcr.io/$PROJECT_ID/$_IMAGE:$SHORT_SHA"
  # set image in deployment.yaml
  - name: "gcr.io/cloud-builders/gcloud"
    id: "Set image name"
    entrypoint: /bin/sh
    args:
      - "-c"
      - |
        sed -i "s/image: IMAGE/image: gcr.io\/$PROJECT_ID\/$_IMAGE:$SHORT_SHA/g" k8s/deployment.yaml
  # get kubernetes credentials
  - name: "gcr.io/cloud-builders/gcloud"
    id: "Authenticate kubernetes"
    args:
      [
        "container",
        "clusters",
        "get-credentials",
        "$_CLUSTER",
        "--zone",
        "$_ZONE",
      ]
  # deploy changes to kubernetes config files
  - name: "gcr.io/cloud-builders/kubectl"
    id: "Deploy"
    args: ["apply", "-f", "k8s/"]
  # add latest tag to new image
  - name: "gcr.io/cloud-builders/gcloud"
    id: "Tag"
    args:
      [
        "container",
        "images",
        "add-tag",
        "gcr.io/$PROJECT_ID/$_IMAGE:$SHORT_SHA",
        gcr.io/$PROJECT_ID/$_IMAGE:latest,
      ]

There’s a lot going on in the file above so let’s walk through each step defined. In each step, the name argument defines the CLI to use to run the command. The gcr.io/cloud-builders/… names refer to Google’s preinstalled packages (see https://cloud.google.com/cloud-build/docs/cloud-builders). If you need to use any other software packages during your build you will need to add a step to install them (using apt-get install most of the time). Another thing to note is any variables that is provided by Google Cloud by default can be accessed using $ in front of the variable name. Any substitution variables we define ourselves in the build trigger must also include an underscore before the variable name. You can read more here: https://cloud.google.com/cloud-build/docs/configuring-builds/substitute-variable-values.

The first step simply uses the opposite command that we used to encrypt our .env to decrypt it. The service account that performs builds will require additional permissions for this, so after the cloudbuild.yaml explanation we will add these. Next, we pull the latest Docker image from the Container Registry. Since Docker images are built in layers, Docker can use unchanged layers from previous images when building. This can speed up our build drastically especially when installing dependencies or making a minor update.

The next step builds our Docker image and names it. The SHORT_SHA is the first 7 characters of our commit SHA, which will differentiate each of our builds from each other. After that we push the Docker image to the container registry. The next step searches for the string “image: IMAGE” in our k8s/deployment.yaml file and replaces it with “image: gcr.io/<project-id>/<image>:<short-sha>”. This is how we will tell our Kubernetes deployment to use the new Docker image we just pushed. It also means we will have to change that line in our k8s/deployment.yaml file to match this.

Now we configure the Kubernetes credentials just like how we did manually at the beginning of this post. This will require another permission which we will add after. Next we deploy our Kubernetes services (again how we did manually) and finally we tag the image we just pushed with the tag latest, which ensures we can properly pull it in the next build.

Let’s go make those few changes mentioned in the walkthrough above (give our service account KMS and Kubernetes permissions and fill in a generic image name to be switched during builds).

Go to the IAM section of the console (https://console.cloud.google.com/iam-admin/iam) and find the member with Role Cloud Build Service Account. Click the edit icon on the side of that member and click Add Another Role. Search kms and select Cloud KMS CryptoKey Decrypter then search kubernetes and click Kubernetes Engine Developer.

Now head over to the k8s/deployment.yaml and replace the image name so that Cloud Build can fill it in every time. It should now look like this:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gke-tutorial-deployment
  namespace: default
  labels:
    app: gke-tutorial-deployment
spec:
  replicas: 1 # number of pods
  template:
    metadata:
      labels:
        app: gke-tutorial-deployment
    spec:
      containers:
        - name: gke-tutorial-image
          # this value is replaced during cloud build
          image: IMAGE
          imagePullPolicy: Always
          ports:
            - containerPort: 3000
              protocol: TCP

We can finally try out our new CI setup. We’ll make a small change to our code so that we can see the change reflected once deployed. Go into index.js and change the line app.all("*", (req, res) => res.status(200).send(“My Node.js API")); to app.all("*", (req, res) => res.status(200).send("My updated Node.js API”));.

Now navigate to the Cloud Build history (https://console.cloud.google.com/cloud-build/builds). Here is where you will see your builds. Leave this page open then go back to your command line and commit our new changes:

git add .
git commit -m "Cloud build setup"
git push

Click Refresh on the Cloud Build history page and click the new build (there should be a loading icon beside it). You can then scroll down and watch the logs of the build. If your build ever fails, here is where you will find the errors causing the failure. Once the build succeeds, wait a few seconds then enter the external IP address in your browser. You should now see “My updated Node.js API”.

Aaannd done! 🙌 Feel free to let me know any feedback or questions below, I would be happy to help. Remember to check out the GitHub repo if you have any issues.