Serverless Caching Strategies — Part 3 (Lambda Runtime) 🚀
How to use serverless caching strategies within your solutions, with code examples and visuals, written in TypeScript and the CDK, and with associated code repository in GitHub. Part 3 covering caching in the Lambda runtime environment.
Introduction
This is Part 3 of a number of articles covering serverless caching strategies on AWS, and why you should use them. The Github repo can be found here https://github.com/leegilmorecode/serverless-caching.
This part is going to cover caching at the Lambda runtime environment.
🔵 Part 1 of this article covered caching at the API layer using Amazon API Gateway.
🔵 Part 2 of this article looks at caching at the database level using DynamoDB DAX.
🔵 Part 3 of this article will look at caching within the Lambda runtime environment itself.
🔵 Part 4 of this article will look at caching at the AppSync level.
🔵 Part 5 of this article will look at caching at the CDN level with CloudFront.
This article is sponsored by Sedai.io
Quick recap 👨🏫
The following image below shows some of the areas you are able to cache within Serverless solutions:
What are we building? 🏗️
As described in Part 1 of the series, this is what we are building out; and we are focusing on the area highlighted in pink within this article:
Serverless Blog ✔️
The serverless blog has the following flow:
⚪ A CloudFront distribution caches the React website which has an S3 bucket as its origin. We can cache the web app at this level.
⚪ The React app utilises a GraphQL API for accessing its data using AWS AppSync. For certain endpoints we may look to utilise AppSync caching.
⚪ The AppSync API resolves to DynamoDB through Lambda for its data, and we are using DAX as a cache sitting in front of the database. We can utilise DAX to cache at the database level here.
AWS News Blog ✔️
The AWS News blog has the following flow:
⚪ A CloudFront distribution caches the React website which has an S3 bucket as its origin. We can cache the web app at this level.
⚪ The React app utilises a REST API for its data using Amazon API Gateway. We have caching here at the API level within API Gateway.
⚪ For cache misses we use a Lambda function to retrieve the data from a Serverless Aurora database. We can also cache certain data within the lambda itself in this scenario.
💡 Note: this is the minimal code and architecture to allow us to discuss key architecture points in the article, so this is not production ready and does not adhere to coding best practices. (For example no authentication on end points). I have also tried not to split out the code too much so example files are easy to view with all dependencies in one file.
How can we cache within Lambda itself? ✔️
There are a few ways in which we can cache within the lambda itself:
🔵 Cache within the Lambda runtime environment (in memory).
🔵 Cache within the /tmp folder on the Lambda.
🔵 Caching of secrets using the AppConfig Lambda Layer.
💡 Note: Caching of secrets using lambda layers/extensions and AppConfig is already covered in this article below.
Cache within the Lambda runtime environment ✔️
Once a Lambda container has been initialised and ran for the first time, it will persist for further invocations (and remains warm for up to around 15 minutes of inactivity when attached to a VPC, and around 5 minutes when not).
Anything which is created outside of the Lambda handler itself will remain in memory for the duration the Lambda container is warm, across further invocations.
We can use these two features above to allow us to populate a variable outside of the Lambda handler on the first run; in our example populating with data (blogs) from a database call; and then subsequent calls to the Lambda container will read from memory (instead of reaching back out to the database every time).
Lets see a basic annotated code example below:
In our scenario, we are happy that the list of articles rarely changes (say once per month), and caching in this manner may be more cost effective than caching at the API Gateway level, or continuously reading from the database on every lambda invocation. There is also a massive performance gain.
We do need to consider however, that when a new blog post is published it may take a little time for these warm lambdas to die, and for the in memory caches to be rehydrated again.
💡 Note: We could certainly work around this by adding a TTL in the in-memory cache, and rehydrating after a given period of time.
“Caching in this manner may be more cost effective than caching at the API Gateway level”
This is where we need to consider all of the areas where we can cache across our Serverless solutions, and choose the right one based on cost, complexity, business requirements, and performance.
💡 Note: A real World example from the past was a Lambda caching the access token from a client-credentials grant flow (Machine to Machine flow). This access token would have an expiry of around 24 hours, so we didn’t need to generate the token with every request, and we cached the token in memory instead.
“This is where we need to consider all of the areas where we can cache across our Serverless solutions, and choose the right one based on cost, complexity, and performance.”
Cache within the /tmp folder on the Lambda ✔️
Similar to above, we are able to populate the /tmp folder on a Lambda with up to 512MB in size, and then this ephemeral file storage will be persisted across further invocations.
“The Lambda execution environment provides a file system for your code to use at /tmp. This space has a fixed size of 512 MB. The same Lambda execution environment may be reused by multiple Lambda invocations to optimize performance. The /tmp area is preserved for the lifetime of the execution environment and provides a transient cache for data between invocations. Each time a new execution environment is created, this area is deleted.
Consequently, this is intended as an ephemeral storage area. While functions may cache data here between invocations, it should be used only for data needed by code in a single invocation. It’s not a place to store data permanently, and is better-used to support operations required by your code.” — https://aws.amazon.com/blogs/compute/choosing-between-aws-lambda-data-storage-options-in-web-apps/
In our example, we pull some static objects from Amazon S3 (logo image files), and then write them to the tmp folder on the Lambda. On further invocations of the Lambda, the logo files remain in tmp, and we no longer need to reach out to S3 after the first time (per Lambda instance).
In this scenario we can increase the speed of the Lambdas and reduce cost dramatically; however as like above, the files may be outdated and won’t change until the Lambda is a fresh container.
The following table below shows the relative speeds when it comes to storage types with Lambda:
“In this scenario we can increase the speed and reduce cost dramatically; however as like above, the files may be outdated and won’t change until the Lambda is a fresh container again.”
Lets see a basic annotated code example below from our repo where we are caching the logo files from S3 into the tmp directory:
Like we discussed above with in memory caching, this is where we need to consider all of the areas where we can cache across our Serverless solutions, and choose the right one based on cost, complexity, and performance (as well as limitations, such as the maximum temp storage size of 512 MB).
💡 Note: A real World example from the past was a Lambda caching JSON structures which could be amended by clients rarely, so we didn’t want the inflexibility of storing the JSON in code meaning developers needed to make the change, and didn’t want to also pull this from an object or file store on every request.
Getting Started! ✔️
To get started, clone the following repo with the following git command:
git clone https://github.com/leegilmorecode/serverless-cachingThis will pull down the example code to your local machine.
Deploying the solution! 👨💻
🛑 Note: Running the following commands will incur charges on your AWS accounts, and some services are not in free tier.
In the ‘aws-blog’ folder of the repo run the following command to install all of the dependencies:
npm iOnce you have done this, run the following command to deploy the solution:
npm run deploy🛑 Note: Remember to tear down the stack when you are finished so you won’t continue to be charged, by using ‘npm run remove’.
💡 Note: We use a CustomResource as part of the deploy to create the blogs table and populate it with some dummy data, so you can use it straight away.
Testing the solution 🎯
Now that we have the solution deployed, you can test the endpoints using the postman file which can be found here: aws-blog/postman/serverless-caching-aws-blogs.postman_collection.json
💡 Note: You will also need to put some example images in the S3 bucket which is created.
How can we determine that the caching is working?
Lets have a look at the logs to see the caching working in our two scenarios:
Cache in memory ✔️
The first scenario we are going to cover is caching the blogs in memory, which we can see below:
We can see from the logs below that:
- With the first call (and the cache being empty) the lambda took 549ms to complete (mainly due to the database calls and cold start)
- The second call to the same lambda had the blog posts cached in memory, and the lambda took around 21ms to complete, and no call made to the database at all.
This is a huge saving on latency and an increase in performance, and we have not needed to make further database calls which would save costs when using a database such as DynamoDB.
Cache in temp ✔️
Next we will focus on caching files in the tmp folder on the Lambda itself, specifically pulling images (logos) from an S3 bucket. We can see from the logs below that:
- With the first call (and the cache being empty) the lambda took 197ms to complete (mainly due to the call to S3 to pull down the logo from the S3 bucket and the cold start)
- The second call to the same lambda had the logo image cached in the
tmpdirectory in the Lambda runtime environment, and the lambda took around 4ms to complete on average.
This is a huge saving on latency and increase of performance, and we have not needed to make further calls to S3 which would save on costs too.
The logs below shows the caching taking place:
What are the advantages and disadvantages?
So now that we have covered how we can cache within our Lambdas; what are the advantages and disadvantages?
Advantages
🔵 Potential cost savings.
🔵 Potential increase in performance and reduction in latency.
Disadvantages
🔵 The caching can remain in place as long as the Lambda is warm, so non-deterministic. (You can get around this with storing a TTL timestamp in the cache itself however).
🔵 Depending on the throughput and scaling of Lambda there may be more efficient caching services to use.
Summary
I hope you found that useful! Please continue to join me on this caching journey through parts 4 and 5!
Go and subscribe to my Enterprise Serverless Newsletter here for more of the same content:
Wrapping up 👋
Please go and subscribe on my YouTube channel for similar content!
I would love to connect with you also on any of the following:
https://www.linkedin.com/in/lee-james-gilmore/
https://twitter.com/LeeJamesGilmore
If you found the articles inspiring or useful please feel free to support me with a virtual coffee https://www.buymeacoffee.com/leegilmore and either way lets connect and chat! ☕️
If you enjoyed the posts please follow my profile Lee James Gilmore for further posts/series, and don’t forget to connect and say Hi 👋
Please also use the ‘clap’ feature at the bottom of the post if you enjoyed it! (You can clap more than once!!)
About me
“Hi, I’m Lee, an AWS Community Builder, Blogger, AWS certified cloud architect and Principal Software Engineer based in the UK; currently working as a Technical Cloud Architect and Principal Serverless Developer, having worked primarily in full-stack JavaScript on AWS for the past 5 years.
I consider myself a serverless advocate with a love of all things AWS, innovation, software architecture and technology.”
*** The information provided are my own personal views and I accept no responsibility on the use of the information. ***
You may also be interested in the following:
