Use AWS CodePipeline CI/CD to automate Unit Test and Deploy Lambda Function

Post views: 817

In this guide, I will walk you through setting up a Continuous Integration and Continuous Deployment (CI/CD) pipeline. This pipeline will automate unit testing and deploy your code to an AWS Lambda function using AWS CodePipeline, CodeBuild and CloudFormation.

Source Stage

Our journey begins with a GitHub repository, which stores our source code. Any time changes are pushed to this repository, the pipeline is automatically triggered.

Build & Test Stage

Next, AWS CodeBuild steps in to compile the source code and run unit tests. Upon successful completion, the build artifacts are stored in an S3 bucket.

Deploy Stage

Finally, AWS CloudFormation takes over, retrieving the artifacts (source code) from the S3 bucket and deploying the function to AWS Lambda.

Step 1: Create Lambda Function

As an example, I create a simple Lambda Function that takes a city name as a parameter and returns the current temperature. To fetch the weather information, I signup for a free account at OpenWeather.org to use their API. Following is the API call to get the weather data:

http://api.openweathermap.org/data/2.5/weather?q=city_name&appid=api_key&units=metric

This is my Lambda function:

I use the following event data to test my function and get a response that shows the function is working:

{
  "city": "Vancouver"
}

I create a GitHub repository and upload my code with following commands:

git add.; git commit -m "Add Lambda Function Python code"; git push

Step 2: Create CodeBuild Project

Go to AWS CodeBuild console and click on Create project. To enable CodeBuild to retrieve source code from GitHub, setup a connection using OAuth and choose the source code repository. This connection allows CodeBuild to automatically fetch and build the source code whenever changes are pushed to the specified GitHub repository.

Since my Lambda function is written in Python and there is no Python runtime available in the standard environment options, I use a standard runtime and specify my runtime by using a buildspec file.

The buildspec file ensures that CodeBuild uses the specified Python runtime to build and test my Lambda function code. This file is also where I add automated unit tests. Here’s how I set up my buildspec.yml file:

version: 0.2
phases:
  install:
    runtime-versions:
      python: 3.10   # Specify the Python version you need
  build:
    commands:
      - echo Testing the Lambda functon...
      - python -m unittest discover tests
artifacts:
  files:
    - '**/*'    # Include all files in the build output

Explanation:

install phase:
- runtime-versions specifies the Python version.
build phase:
- python -m unittest discover test – a Python testing framework that supports test discovery by automatically discover and run all test methods with a project directory, in this case the ‘test‘ folder.
artifacts:
- Specifies which files to include in the build output.

Here’s a simple unit test example. It checks two things: whether the function returns a success code of 200, and whether the response includes the same city name I provided along with its current temperature:

def unit_test(self):
    event={'city':'Vancouver'}
    result = lambda_function.lambda_handler(event, None)
    self.assertEqual(result['statusCode'], 200)
    self.assertEqual(result['headers']['Content-Type'], 'application/json')
    self.assertIn(event['city'], result['body'])
    self.assertIn('current_temperature', result['body'])

CodeBuild saves the code as a zip file in S3 bucket, which is a crucial step. Later on, CloudFormation will use this stored code as a source to deploy it to Lambda.

You can set up webhook events to tell CodeBuild to rebuild your code whenever there’s a push request on GitHub. However, I prefer to add this to my CodePipeline instead and have it trigger CodeBuild automatically.

Step 3: Create CloudFormation Stack

To deploy my Python function code to AWS Lambda using CloudFormation, I need to prepare a CloudFormation template that specifies the location of the source code zip file stored in the S3 bucket. Here’s an example of how my cloudformation_lambda.yml template looks:

Resources:
  LambdaFunction:
    Type: 'AWS::Lambda::Function'
    Properties:
      FunctionName: myWeatherLambda
      Handler: index.handler
      Runtime: python3.10
      Role: !GetAtt LambdaFunctionRole.Arn
      MemorySize: 1024
      Code: 
        S3Bucket: wallace-bucket-for-codebuild
        S3Key: myWeatherLambdaCodeBuild/codebuild.zip
                  
  LambdaFunctionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service:
            - lambda.amazonaws.com
          Action:
          - sts:AssumeRole
      Path: "/"
      Policies:
      - PolicyName: AppendToLogsPolicy
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
            - logs:CreateLogGroup
            - logs:CreateLogStream
            - logs:PutLogEvents
            Resource: "*"

The ‘S3Bucket ‘ parameter specifies the S3 bucket name and the ‘S3Key‘ parameter indicates where the Lambda function code zip file is stored. The template also creates a Lambda function role that grants the Lambda function permission to write logs.

Go to CloudFormation console and click on Create stack. I use the cloudformation_lambda.yml template I prepared to create a stack:

Stage 4: Create Service role for CodePipeline

A service role is needed to grant permission to CodePipeline to access the S3 bucket, deploy CloudFormation stacks, create Lambda function role and other necessary actions. Here’s how to create the IAM role, go to IAM console and click on Roles and Create role.

Edit the trust relationships to change the service from ‘ec2.amazonaws.com‘ to ‘cloudformation.amazonaws.com‘

Create an inline policy to grant permission to the following:

S3: Allows GetObject, PutObject, and related actions for all buckets.
IAM: Permissions for role management, including creation, deletion, and policy attachment.
Lambda: Allows invoking Lambda functions and listing available Lambda functions, creation, deletion and updates.
CloudFormation: Allows managing CloudFormation stacks, such as CreateStack, DeleteStack, etc.
CodeBuild: Allows BatchGetBuilds and StartBuild actions for CodeBuild projects.

{
	"Statement": [
		{
			"Action": [
				"s3:GetObject",
				"s3:GetObjectVersion",
				"s3:GetBucketVersioning",
				"s3:PutObject"
			],
			"Resource": [
				"arn:aws:s3:::*"
			],
			"Effect": "Allow"
		},
		{
			"Action": [
				"iam:CreateRole",
				"iam:DeleteRolePolicy",
				"iam:GetRole",
				"iam:GetRolePolicy",
				"iam:PassRole",
				"iam:DetachRolePolicy",
				"iam:DeleteRolePolicy",
				"iam:DeleteRole",
				"iam:AttachRolePolicy",
				"iam:PutRolePolicy",
				"iam:PassRole"
			],
			"Resource": [
				"arn:aws:iam::*:role/*"
			],
			"Effect": "Allow"
		},
		{
			"Action": [
				"lambda:GetFunction",
				"lambda:GetFunctionConfiguration",
				"lambda:CreateFunction",
				"lambda:DeleteFunction",
				"lambda:InvokeFunction",
				"lambda:ListFunctions"
			],
			"Resource": [
				"arn:aws:lambda:ca-central-1:339713082608:function*"
			],
			"Effect": "Allow"
		},
		{
			"Action": [
				"cloudformation:CreateStack",
				"cloudformation:DeleteStack",
				"cloudformation:DescribeStacks",
				"cloudformation:UpdateStack",
				"cloudformation:CreateChangeSet",
				"cloudformation:DeleteChangeSet",
				"cloudformation:DescribeChangeSet",
				"cloudformation:ExecuteChangeSet",
				"cloudformation:SetStackPolicy",
				"cloudformation:ValidateTemplate"
			],
			"Resource": [
				"arn:aws:cloudformation:ca-central-1:339713082608:stack*"
			],
			"Effect": "Allow"
		},
		{
			"Action": [
				"codebuild:BatchGetBuilds",
				"codebuild:StartBuild"
			],
			"Resource": "arn:aws:codebuild:ca-central-1:339713082608:project*",
			"Effect": "Allow"
		}
	],
	"Version": "2012-10-17"
}

Stage 5: Create CodePipeline

Now, the final step is to integrate all the components by creating a CodePipeline. Here’s how to set it up:

Go to CodePipeline console and click on Create pipeline. Set the source provider to GitHub and specify the repository that contains the source code.

A git push to the repository will trigger the pipeline:

To configure the build stage, use AWS CodeBuild as the build provider with the CodeBuild project created in Step 2.

Use AWS CloudFormation as the deploy provider and specify the CloudFormation template prepared in Step 3 in the File name.

For role name, use the IAM service role created in Step 4.

With the CodePipeline set up, push a change to the GitHub repository to see the pipeline in action.

Once the pipeline executes, go to the Lambda console to verify if the Lambda function has been created successfully.

Thank you for reading! You can find my GitHub repo here. I hope this guide gives you a clear understanding of how to set up a CI/CD pipeline with AWS CodePipeline. Stay tuned for more exciting AWS guides and insights.

MY CLOUD JOURNEY