Preface

If you want to enable “HTTP flood protection” or “Scanner & Probe protection” in AWS WAF Automations, both of them have these 2 log parser options:

1. AWS Lambda log parser
2. Amazon Athena log parser

Now, you may want to know what are the differences between them? Basically, you can choose from 2 aspects:

1. Do you need to customize URL path or block period or threshold?
2. If you worry that malicious IP addresses are distributed in different log files so that it’s hard to analyze?

Lambda log parser supports customized block rules like URL path, block period, and threshold via simply modifying a JSON file, while Athena log parser needs to modify Python code.

Besides, Lambda log parser is limited to its analysis method, it can only parse 1 file at a time. This means it cannot store the last analyzed file content so that if malicious IP addresses are distributed in many log files, it’s impossible to find them out. While Athena is naturally good at analyzing lots of log files.

Comparison

Here I list some basic differences of Lambda log parser & Athena log parser.

• Lambda Log Parser
  • Trigger
    • Target S3 bucket has event notification
      • Event type: All object create events
      • Filter: AWSLogs/, gz
      • Destination: AWSWAFSecurityAutomations-LogParser-xxxxxx
    • Configurable
      • Yes, in {bucket}/AWSWAFSecurityAutomations-app_log_conf.json
• Athena Log Parser
  • Trigger
    • (if enabled) Move log files to another S3 bucket directory (AWSWAFSecurityAutomations-LambdaAddAthenaPartition-xxxxx)
      • Target S3 bucket has event notification
        • Event type: All object create events
        • Filter: AWSLogs/, gz
        • Destination: AWSWAFSecurityAutomations-LogParser-xxxxxx
      • CloudWatch Events every 1 hour
    • Parse log files in S3 bucket (AWSWAFSecurityAutomations-LambdaAthenaAppLogParser)
      • Target S3 bucket has event notification
        • Event type: All object create events
        • Filter: AWSLogs/, gz
        • Destination: AWSWAFSecurityAutomations-LogParser-xxxxxx
      • CloudWatch Events every 5 minutes
    • Configurable
      • Hard, you have to modify build_athena_query_for_waf_logs() function in Lambda to change Athena Query

How to enable

1. Enable logging in ALB or CloudFront, choose a bucket to store log files
2. Update “AppAccessLogBucket” in CloudFormation stack and enable log parser
   1. If you have customized rules in that Web ACL, those settings could be overwritten. Make sure you backup (download) those configurations and update them into your CloudFormation template
3. (Optional) If you enable Lambda log parser, you can modify the configuration file in your S3 bucket.

References

1. AWS WAF Automations Implementation Guide, pp. 23-30

• ← Previous Post
• Next Post →