AWS Lambda – CloudWatch event to S3 action

AWS Lambda is a great tool to enhance your messaging and alerting without creating more infrastructure to manage.

Use Case

I have CloudWatch Log Alarms (originating from CloudTrail events) that I want to act upon by uploading data to S3.

Lambda Function

Create a Lambda function with these attributes.

Code

console.log('Loading function');

exports.handler = function(event, context) {
    console.log('Received event:', JSON.stringify(event, null, 2));
    var AWS = require('aws-sdk');

    var mytimestamp = Date.now();

    var alarm_msg = event.Records[0].Sns.Message;
    var bucketname = 'YOUR_BUCKET_NAME'
    var myfilename = 'ScottTest_' + mytimestamp

    var s3bucket = new AWS.S3({params: {Bucket: bucketname}});
    s3bucket.createBucket(function() {
        var params = {Key: myfilename, Body: 'Message Part 1. Here is the message payload'+alarm_msg+'MORETHINGS'};
        s3bucket.upload(params, function(err, data) {
            if (err) {
               console.log("Error uploading data: for filename" + myfilename, err);
            } else {
               console.log("Successfully uploaded data to " + bucketname + "/" + myfilename);
            }
        });
    });

};

Configuration

  • Runtime: NodeJS
  • Handler: index.handle
  • Role: lambda_s3_exec_role
  • Advanced: Timeout: 10

Save your Lambda Function.

 

SNS Topic

Create a new topic with a sensible name, like CloudWatchToLambdaTopic.

Create Subscription

  • Protocol: AWS Lambda
  • EndPoint: The lambda function you just made

 

Back to Lambda

Configuration

  • Event Sources: You should see your SNS Connection here

Test Data

Use this JSON blob as test data.

{
    "Records": [
        {
            "EventSource": "aws:sns",
            "EventVersion": "1.0",
            "EventSubscriptionArn": "arn:aws:sns:u****”,
            "Sns": {
                "Type": "Notification",
                "MessageId": “2***”,
                "TopicArn": “a***”,
                "Subject": "ALARM: \"CloudTrail-MeepCows!***”,
                "Message": "{\"AlarmName\”:***”,
                "Timestamp": "2015-08-13****”,
                "SignatureVersion": "1",
                "Signature": “B****”,
                "SigningCertUrl": "https:***”,
                "UnsubscribeUrl": "https:***”,
                "MessageAttributes": {}
            }
        }
    ]
}

S3

You should see your S3 object uploaded to the AWS. You can look in the console down below to see if there are any errors.

CloudWatch

In CloudWatch -> Logs, you should see a stream named /aws/lambda/YOUR_LAMBDA_FUNCTION. This will have the same output as what you could see from the above step. Here it is for debugging.

If you have a CloudTrail Log group, you can click on one of you existing filters. You can update Alarms to read from the SNS group you created above.

What is Supposed to Happen

A Cloudtrail event fires. Your CloudWatch filter catches it, which in turn fires it’s alarm notifying your SNS Topic. Your Lambda function is subscribed to that SNS topic, and it fire its code. The Lambda function code uploads a S3 file.

Power

This is a powerful concept, you can make all sorts of alarms and messaging with this structure. This is the easy way. The next step would be to read the CloudTrail S3 dumps from Lambda and have even more information to message.

 

Leave a Reply

Your email address will not be published. Required fields are marked *