Schedule your Lambda functions with boto3 (CRON)
This is a quick tutorial on how to use Python and boto3 to programmatically schedule the execution of AWS Lambda functions. That’s super useful if you want to add/remove rules after you deployed your function, for example, based on input from your users.
If you only need to set up a schedule once, have a look at the awesome Serverless tool instead.
Here at textcloud, we’re building the majority of your backend on the shoulders of AWS Lambda. This allows us to provide fast and robust workflow automation to our users, no matter if they run a simple workflow once a month, or need to trigger complex scraping and machine learning jobs every five minutes.
Today we included a scheduling feature to trigger a workflow on a certain interval. For example, if you want to get the latest relevant job offers from UpWork once a day per email.
The gory details
The goal: EventBridge Rule → Lambda invocation
We can easily schedule Lambda functions with AWS EventBridge rules. Each rule receives a schedule either in CRON format or using the ‘rate()’ syntax. A rule can have multiple targets, these are the resources you want to trigger whenever the rule becomes active. In our case, each rule should trigger a single Lambda function.
Easy peasy, right?
The problem: Rules silently fail
This all looked so easy in the beginning. We’ve added a rule to EventBridge and created a target that sends an input to the ARN of our Lambda function.
But then: Nothing.
The logs of the function are empty, and each rule invocation seems to fail. But why?
The first part of the solution: Permissions
This smells a lot like a problem with permissions. Maybe EventBridge is not allowed to call a Lambda function!
Let’s create a role for Lambda and give it the ‘CloudWatchEventsFullAccess’ and ‘AWSLambda_FullAccess’ permissions. Keep in mind to regularly check the Access Advisor to see what permissions are not needed!
Now we need to add another trusted entity to the role: ‘events.amazonaws.com’. The trust policy should look something like this:
We copy the role ARN and add it to our Python code:
Looks pretty nice now! Let’s remove the old rule and add it again and keep watching the logs of our Lambda function.
Nothing! How can that be? We already felt like AWS wizards!
The second part of the solution: Add it to Lambda
I spare you the hours of digging through documentation, swearing, crying, and reading tutorials.
Turns out we need to explicitly tell our Lambda function that the EventBridge rule is allowed to trigger it, even though it has full access permissions to Lambda!
And this should look like this:
And it works!
The full code for the lazy ones
That’s it!
Happy coding! Or if AWS already reduced you to tears, check out textcloud and see how workflow automation + natural language processing helps your company save time and money by automating complex jobs :)
