AWS Lambda as a serverless cronjob – Scheduling

posted in: aws, lambda, serverless | 0

CloudWatch events schedule can be used as an event source to trigger AWS Lambda functions. This can be used as a serverless alternative for automating cronjobs/scheduled tasks. We need to bear in mind that this would be suitable for simple tasks as AWS Lambda has it’s limitations. But, being a serverless option, for simple tasks using this approach can eliminate the overhead of managing and maintaining infrastructure. For instance, if I had to run a cronjob every day, I would perhaps fire up an EC2 instance and setup a cronjob. But, this entails the responsibility of keeping the instance updated, installing security updates and patches, securing the instance etc.

AWS Lambda function

Here is a simple AWS Lambda function that runs every minute.

import datetime
def lambda_handler(event, context):
    print("Scheduled AWS Lambda Task")

Deploy lambda function and schedule invocation

I will package the lambda function and deploying the lambda function code using AWS S3 buckets. This bash script to quickly create and deploy the lambda function, create the cloudwatch event, define the rules for the event and use the lambda function as a target for the event.

rm -r venv
#create virtual environment
virtualenv -p /usr/bin/python2.7 venv
source venv/bin/activate
#python modules, if any
#pip install my_module
zip -9
cd venv/lib/python2.7/site-packages/
zip -r9 ../../../../ *
cd ../../../../

This snippet here will create a virtual environment and package the lambda function into a zip archive. Now, this zip file should be uploaded to s3.

#copy the zip file to s3
aws s3 cp s3://my-bucket/key1/

Once the lambda function is package and uploaded to s3. We can create the lambda function.

aws lambda create-function \
      --region us-east-1 \
      --runtime python2.7 \
      --timeout 10 \
      --description my-lambda-function \
      --role arn:aws:iam::111111111111:role/lambda-s3-role \
      --handler handler.lambda_handler \
      --function-name my-lambda-function \
      --code S3Bucket=my-bucket,S3Key=key1/

Now, we can create a cloudwatch event and use it as a source to trigger the lambda function on a schedule. Here I use a schedule of the lambda function triggered once every minute.

aws lambda add-permission \
      --function-name my-lambda-function \
      --statement-id my-statementid \
      --action 'lambda:InvokeFunction' \ 
      --principal \
      --source-arn arn:aws:events:us-east-1:111111111111:rule/every-minute-schedule-rule \
      --region us-east-1
aws events put-rule \
      --name every-minute-schedule-rule \ 
      --schedule-expression 'rate(1 minute)' \
      --region us-east-1
aws events put-targets \
      --rule every-minute-schedule-rule \ 
      --targets '{"Id":"1","Arn":"arn:aws:lambda:us-east-1:111111111111:function:my-lambda-function"}' \
      --region us-east-1

The schedule event rules accept either cron expressions or rate expressions. In this example I have used the rate expression for the schedule of once every minute.

The CloudWatch logs for the lambda function will now start displaying the output of the lambda function invocation every minute.

Leave a Reply