InSpec 2.0 added builtin support for scanning public cloud resources in AWS and Azure (https://blog.chef.io/2018/02/20/announcing-inspec-2-0/). With the addition of these new features it only made sense to be able to perform scanning of the cloud environment from within the environment and a serverless option like lambda made a lot of sense.
- Simplified IAM permission model for multiple AWS accounts
- All the benefits of serverless architecture
- Credential/Access key management is non-existant since an IAM role is used to access AWS
InSpec is written in Ruby which created an interesting problem given that AWS has not added official support for Ruby as a language that AWS Lambda can utilize.
There are a number of solutions such as using JRuby, Traveling Ruby and others but the most effective solution was covered in the post below.
The solution details compiling a version of Ruby from source and installing the desired gems in a Amazon Linux instance then bundling it up to run in the Lambda function. While this solution would work in most situations I found that the version of openssl in the Amazon Linux instance is not the same that is available on the Lambda functions.
To overcome this challenge an environment that matched that of AWS lambda was required to compile ruby with the correct openssl version. The LambCI Docker container (https://github.com/lambci/docker-lambda) was exactly the environment that was needed to properly compile ruby with the correct version of openssl.
Below is the Dockerfile used to compile ruby 2.5.1, install InSpec and package up the deployment files needed to run it on AWS lambda.
All the code in this post can be found in a github repo. https://github.com/martezr/serverless-inspec
FROM lambci/lambda:build-python3.6 ENV RUBY_VERSION 2.5.1 ENV GEM_INSTALLED inspec RUN yum -y install zlib-devel gcc zip RUN curl -sL https://cache.ruby-lang.org/pub/ruby/2.5/ruby-$RUBY_VERSION.tar.gz | tar -zxv WORKDIR ruby-$RUBY_VERSION RUN ./configure --prefix=/var/task/customruby --disable-werror --disable-largefile --disable-install-doc --disable-install-rdoc --disable-install-capi --without-gmp --without-valgrind RUN make RUN make install RUN /var/task/customruby/bin/gem install $GEM_INSTALLED COPY lambda.py /var/task WORKDIR /var/task # Create a lambda deployment package RUN zip -r lambda.zip customruby/ lambda.py
In addition to the Dockerfile a build script is used to copy the deployment package from the docker container to the local host to allow uploading to the S3 bucket for running from Lambda.
#!/bin/bash # Build Docker Image docker build -t rubylambda . # Run Docker Images docker run --name rubylambda rubylambda bash # Copy Lambda zip file rm -Rf lambda.zip docker cp rubylambda:/var/task/lambda.zip . # Cleanup Docker Containers docker rm rubylambda
Since AWS Lambda doesn't support natively running Ruby we need to use a supported language to call our ruby code and in this case Python is the language of choice. Below is the snippet of code that executes InSpec when the Lambda function is triggered.
import time import os from subprocess import Popen, PIPE, STDOUT def lambda_handler(event, context): github_repo = os.environ['GITHUB_REPO'] # Specify region to scan if os.environ['INSPEC_AWS_REGION']: aws_region = os.environ['INSPEC_AWS_REGION'] else: aws_region = os.environ['AWS_REGION'] cmd = '/var/task/customruby/bin/inspec exec --no-color ' + github_repo + ' -t aws://' + aws_region p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True) output = p.stdout.read() print(output.decode('utf-8')) return if __name__ == '__main__': lambda_handler('event', 'handler')
The Lambda function requires permissions to scan/inspect the AWS environment as well as write logs of executions to CloudWatch. The two AWS managed policies provide the necessary IAM permissions to run InSpec against an AWS environment.
- ReadOnlyAccess (AWS Managed Policy)
- AWSLambdaBasicExecutionRole (AWS Managed Policy)
The Lambda function requires a number of environment variables to support manipulating the code on the fly without making changes to the underlying code.
- HOME (Required) - The working directory for the Lambda function. This must be set to
/tmpto allow InSpec to write to the directory.
- GITHUB_REPO - The Github repository of the InSpec profile to execute
- INSPEC_AWS_REGION - The AWS region that InSpec will scan
The CloudFormation template below deploys the following components to bootstrap an AWS account with the InSpec lambda function.
- IAM Role: IAM role for running the lambda function
- Lambda Function: Lambda function for running InSpec against the AWS environment
- S3 Bucket: An S3 bucket for storing the Lambda code
AWSTemplateFormatVersion: 2010-09-09 Description: 'CloudFormation Stack to deploy serverless InSpec' Parameters: GithubRepo: Type: String Default: https://github.com/martezr/serverless-inspec-profile.git Description: Enter the Github repository that contains the InSpec AWS profile. InSpecAWSRegion: Type: String Default: us-east-1 Description: Enter the AWS region for InSpec to scan. S3BucketName: Type: String Description: Enter the name of the S3 bucket where the lambda deployment package is stored. Resources: InSpecLambdaRole: Type: 'AWS::IAM::Role' Properties: RoleName: 'InSpecLambdaRole' AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - lambda.amazonaws.com Action: - 'sts:AssumeRole' ManagedPolicyArns: - 'arn:aws:iam::aws:policy/ReadOnlyAccess' - 'arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole' InSpecLambda: Type: 'AWS::Lambda::Function' Properties: FunctionName: InSpecLambda Description: InSpec Compliance Lambda Timeout: '45' Environment: Variables: HOME: /tmp GITHUB_REPO: !Ref GithubRepo INSPEC_AWS_REGION: !Ref InSpecAWSRegion Handler: lambda.lambda_handler Runtime: python3.6 Role: 'Fn::GetAtt': - InSpecLambdaRole - Arn Code: S3Bucket: !Ref S3BucketName S3Key: lambda.zip InSpecLambdaS3Bucket: Type: AWS::S3::Bucket BucketName: !Ref S3BucketName
GitHub Serverless-InSpec Example Profile
InSpec Tutorial Part #5