CI/CD With AWS CodePipeline

How To Create a Deployment Pipeline for a CDK App Using AWS CodePipeline and GitHub

Published in

Level Up Coding

8 min readMar 20, 2022

This is another ‘how to do’ article which will describe the process of creating a deployment pipeline using AWS code pipeline. Building, testing and deployment of projects are no longer carried out manually, now. With CI/CD techniques offered by various Cloud Service Providers, the entire deployment process of any application has been simplified and smoothen with minimum human intervention.

AWS Codepipeline is one such service offered by AWS to automate the release pipeline of your project. In this article, we will automate the deployment pipeline of a CDK project with AWS Codepipeline integrated into the GitHub repository.

[Find the source code at the end of this article]

What is AWS Code Pipeline?

According to AWS itself, AWS CodePipeline is a fully managed continuous delivery service that facilitates the automation of the release pipelines for fast and reliable application and infrastructure updates. You can automate the entire build, test and deployment processes of your CDK application when committing a change to the code base (You can define the particular trigger in the release model). Therefore Codepipeline is a construct library module for continuous delivery of AWS CDK applications.

It is an always free service but under AWS free tier you are provided with 01 active pipelines per month and beyond that falls under the pay-as-you-go option. The specialty with CodePipeline is that it can be integrated with AWS CodeCommit, CodeStar or third-party services such as GitHub and own custom plugins.

Deployment Pipeline Using AWS CodePipeline, CDK and GitHub

We will implement the CI/CD pipeline as illustrated in figure 01.

Figure 01: High-Level Architecture of the Deployment Pipeline (Source: designed by Author with draw.io)

The codebase of the CDK project, the source code for lambda functions and the release pipeline is included in the GitHub repository. The pipeline which is depicted in the centre cage of figure 01 will be created within the AWS account, and it will be defined through the CDK code in the repo.

We will be setting it up such that each time we commit a change to the GitHub repo, it will publish to an AWS lambda function in a test environment. The next major step in the pipeline is that it will require manual approval which you have to review the changes and approve before deploying the code into the production environment. The production stage also employs a lambda function.

Step: 01 — Pre-requisites

There are a few prerequisites you need to have created or installed (follow the links) which I am not going to explain in this article. It is will be easy for you to follow the CDK workshop pre-requisites steps to configure your local environment. (In Windows, if the terminal commands are not working with PowerShell, use Command Prompt).

Note: If you are unfamiliar with CDK, it is highly recommend to follow the CDK workshop (free) and refer this article, next.

Install AWS-CLI
Create AWS account and configure credentials
Install NodeJS
Install CDK (globally)
Create an empty GitHub repository and clone it

After you completed the pre-requisites, you can follow the rest of the following steps to set up your first AWS deployment pipeline.

Step 02: Setup the CDK Project

1. Setup the Initial Code Base of CDK Project

Create a folder and give the name of your preference (‘aws-codepipeline-project’ in mine). Open the VS Code (or any code editor) in the folder and run the following commands to initialize the CDK application.

cdk init app --language typescript

(Yes, I will be using typescript to implement the code base of the CDK project.)

Figure 02: Initial folder structure of the CDK Project (Source: Screenshot by Author)

Once the project is initialized successfully, navigate to the ‘bin’ directory and open the file ‘your-root-directory-name.ts’ and make the following modifications. I have added some comments to explain the code and where you need to do the modifications.

bin/your-project-name.ts

Next, modify the ‘your-root-directory-name-stack.ts’ in the lib directory as follows.

lib/your-project-name-stack.ts

In the class AwsCodepipelineProjectStack, we create a new code pipeline construct. You can name the with a suitable name (CDKTestPipeline in mine). Next, add a new synthesis ‘ShellStep’ to point to our GitHub repository where the CDK code base is located. In there, the pipeline execution will occur upon any changes to the main branch of our repository. Finally, add the commands for build steps in the pipeline:

npm ci - (npm clean install), which is similar to npm install that is to be used in automated environments.
npm run build - to allow us to perform any necessary building/prep tasks for the project.
npx cdk synth - to synthesize whatever we have in the cloud formation stack to generate the self mutating pipeline.

2. Commit the Changes to the Remote GitHub Repository

Before committing the changes to the remote repository, first, check the configurations.

Set the remote origin.

git remote add origin https://github.com/aLLUPS/aws-codepipeline-project.git

If you are missing a .gitignore file in your CDK project, then make sure to add one and include the ‘node_modules’ in it.

Then commit the changes and push the codebase to the remote repository.

Step 03: Provide Access and Configurations in AWS

1. Generate GitHub Personal Access Token

Now we need to create a connection between our GitHub repository and the AWS account. For this, we will require an access token that has permission for AWS with the right access to our GitHub repository.

To do this go to Developer settings in your GitHub account and, under Personal access tokens, ‘generate new token’ (Figure 03). Provide the new token with a suitable name and make sure to mark the checkboxes: repo, and admin:repo_hook. Then click ‘generate token’. Once the token is generated, save it somewhere because you will no longer be able to view it.

Figure 03: Generate GitHub personal access token (Source: screenshot by Author)

2. Configure AWS Account

Go to your AWS management console and search for ‘Secrets Manager’. This is a 30-day free trial service and after that, you will have to pay $0.40 per secret per month and $0.05 per 10,000 API calls.

Next, go to ‘store new secret’ and ‘select other type of secret’. Then select the ‘plain text’ option under ‘key/value pairs’ and paste the GitHub Personal Access Token. Then give it a suitable name (‘github-token’ in mine). Finally, ‘Store’ the key.

Note: you must store the ‘secret’ in the same region you configured your AWS credentials in the pre-requisite step 2. (i.e. in the AWS management console first select the region, and then store the secret).

Figure 04: Store the GitHub Personal Access Token in ‘AWS Secrets Manager’ (Source: screenshot by Author)

Step 04: Deployment of the Pipeline With Basic Stages

1. Bootstrap the CDK project in Local Repository

If you are running a CDK project for the first time in your local machine another important step before deploying the CDK project to codepipeline, is to bootstrap it. Run the following commands in the command prompt opened in the root directory to bootstrap your project environment.

cdk bootstrap

To successfully run this command, first, you must have configured the credentials correctly in the pre-requisite step 2. The region should be the same as the region given under the environment variables in bin/your-project-name.ts file.

If the bootstrapping process is succeeded you will be able to see the CloudFormation stack created in your AWS account (Figure 05).

Figure 05: CloudFormation Stack of the CDK project (Source: screenshot by Author)

2. Deploy the CDK project

This step will create the initial version of our code pipeline and other infrastructures required to deploy the project. For that run the following command in Command prompt.

cdk deploy

When you run the deploy command it will ask whether you need to create the pipeline and other necessary infrastructures and enter ‘yes’ to proceed with the deployment. If you fail to deploy, most probably it might be due to a problem with your GitHub access token, storing the secret in a different region etc. Refer to the following troubleshooting options given in the AWS CDK Documentation. fIf the deployment is successful you can see the completed CloudFormation stack and the pipeline created (Figure 06).

Figure 07: The deployment pipeline is successfully created (Source: screenshot by Author)

In figure 07, you can see that the initial pipeline of our CDK project consists of 03 major stages: Source, Build and UpdatePipeline.

Figure 07: The steps in the initial pipeline (Source: screenshot by Author)

Step 05: Configure More Steps Into the Pipeline

1. Initializing Lambda

Go to the lib directory and create new file ‘lambda-stack.ts’ as follows:

lib/lambda-stack.ts

We will have a test and production environment for our lambda functions. And we want to set an environment variable inside our lambda function (stageName) so that it knows what stage it is in. For example, imagine that we have two databases as test and production. Here, the test version of our lambda function knows that it is in the test and it will connect to a test database and the production one will know that it is in the production stage and will connect to the production database.

Next, create a new directory called ‘lambda’ inside the ‘lib’ directory and a new file, ‘handler.ts’ inside it. Because in the lambda-stack.ts we defined that we are using a handler file with an entry point function named ‘handler’ which is in the directory ‘lambda’.

lib/lambda/handler.ts

2. Setup the Stages

The next step is to set up the stages. Create a new file called ‘stage.ts’ in the lib directory and modify it as follows. Here we are creating a new lambda stack from the LambdaStack and should pass the stageName.

lib/stage.ts

In order to get the ‘stageName’ to stage.ts, we should add the following lines of codes to the ‘lib/your-project-name-stack.ts’ file. In here also, make sure to use the same account ID and ‘aws-region’ that you have configured earlier. Also, after the testing stage, I have added a manual review step also (line 29).

lib/your-project-name-stack.ts

3. Commit the changes to update the pipeline

Finally, commit the code changes to the remote GitHub repository. Once the repo is updated, in the codepipeline you will observe the new stages added: the testing, manual approval and production, other than the initial stages, Source, Build and UpdatePipeline. The execution of the pipeline will be paused until you provide the manual approval to the changes (Figure 08).