Two Lambdas One Stack

Use CDK to deploy, re-deploy, destroy, and communicate between two Lambdas in a single stack?

AWSLambdaCDKPython

To build a fully-fledged serverless-based product on AWS, you’re going to need more than one Lambda. This article explains how to:

  1. Create two Lambdas at once in a Python CDK stack.
  2. Deploy them separately.
  3. Destroy them separately.
  4. Have them trigger each other.

If you’re here, I’m assuming you already have the Python CDK set up and ready to go. If not, you can read this tutorial to prepare a single Lambda.

Initial Python Lambda

The Simplest AWS Lambda in Python

If you know the basics already, and already have Python CDK installed, you can download the starter code from my GitHub here:

GitHub — Connor56/cdk-python-stack: A basic CDK stack with a single Lambda in it.

The SSH command to clone is:

bash
git clone git@github.com:Connor56/cdk-python-stack.git

Now, let’s get into the new stuff.

Step 1 — Add a Second Lambda Function

If you followed the tutorial above, or downloaded the code from GitHub. You should see the following directory structure in your project:

bash
📁 ./

Initially, we’ll keep the second lambda function simple. So, add another file inside the /lambda directory called lambda_function_2.py. Open the file in an editor and paste in the following:

python
import json
from typing import Dict, Any

def lambda_handler(event: Dict[str, Any], context: Any) -> Dict[str, Any]:
    """
    AWS Lambda handler
    :param event: dict, the event data sent by the requester
    :param context: LambdaContext, runtime information
    :return: dict,
    """

    body = {"message": "Hello Lambda the 2nd!"}

    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(body),
    }

This is a copy of the lambda_function.py, except now we’re returning "Hello Lambda the 2nd!".

Step 2 — Add the Second Lambda to the Stack

If you followed my tutorial above, your CDK stack will look something like this:

python
from aws_cdk import (
    Stack,
    aws_lambda as _lambda,
)
from constructs import Construct

class MyFirstPythonStackStack(Stack):
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        # Base lambda function set up
        fn = _lambda.Function(
            self,
            "MyFunction",  # The name of the function in Cloud Formation
            runtime=_lambda.Runtime.PYTHON_3_11,  # You runtime environment
            code=_lambda.Code.from_asset("lambda"),  # Your code directory
            handler="lambda_function.lambda_handler",  # Your <module>.<handler-function>
        )

        # Create a function URL for the Lambda function
        fn_url = fn.add_function_url(
            auth_type=_lambda.FunctionUrlAuthType.NONE,  # Public access, no Auth required
            cors=_lambda.FunctionUrlCorsOptions(
                allowed_origins=[
                    "*"
                ],  # Allow the request to come from any origin
                allowed_methods=[
                    _lambda.HttpMethod.ALL
                ],  # Allow the request to have any http method
                allowed_headers=["*"],  # Allow any headers in the request
            ),
        )

        # Output the function URL
        from aws_cdk import CfnOutput

        # Print the URL address to the terminal when you deploy
        CfnOutput(
            self,
            "FunctionUrl",
            value=fn_url.url,
            description="URL for the Lambda function",
        )

To add another Lambda function, you could copy and paste the fn, fn_url, and CfnOutput sections:

python
from aws_cdk import (
    Stack,
    aws_lambda as _lambda,
)
from constructs import Construct

class MyFirstPythonStackStack(Stack):
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        # Base lambda function set up
        fn = _lambda.Function(
            self,
            "MyFunction",  # The name of the function in Cloud Formation
            runtime=_lambda.Runtime.PYTHON_3_11,  # You runtime environment
            code=_lambda.Code.from_asset("lambda"),  # Your code directory
            handler="lambda_function.lambda_handler",  # Your <module>.<handler-function>
        )

        # Create a function URL for the Lambda function
        fn_url = fn.add_function_url(
            auth_type=_lambda.FunctionUrlAuthType.NONE,  # Public access, no Auth required
            cors=_lambda.FunctionUrlCorsOptions(
                allowed_origins=[
                    "*"
                ],  # Allow the request to come from any origin
                allowed_methods=[
                    _lambda.HttpMethod.ALL
                ],  # Allow the request to have any http method
                allowed_headers=["*"],  # Allow any headers in the request
            ),
        )

        # Second lambda function set up
        fn2 = _lambda.Function(
            self,
            "MyFunction2",  # The name of the function in Cloud Formation
            runtime=_lambda.Runtime.PYTHON_3_11,  # You runtime environment
            code=_lambda.Code.from_asset("lambda"),  # Your code directory
            handler="lambda_function_2.lambda_handler",  # Your <module>.<handler-function>
        )

        # Create a function URL for the second Lambda function
        fn_url2 = fn2.add_function_url(
            auth_type=_lambda.FunctionUrlAuthType.NONE,  # Public access, no Auth required
            cors=_lambda.FunctionUrlCorsOptions(
                allowed_origins=[
                    "*"
                ],  # Allow the request to come from any origin
                allowed_methods=[
                    _lambda.HttpMethod.ALL
                ],  # Allow the request to have any http method
                allowed_headers=["*"],  # Allow any headers in the request
            ),
        )

        # Output the function URL
        from aws_cdk import CfnOutput

        # Print the URL address to the terminal when you deploy
        CfnOutput(
            self,
            "FunctionUrl",
            value=fn_url.url,
            description="URL for the Lambda function",
        )

        # Print the URL address to the terminal when you deploy
        CfnOutput(
            self,
            "FunctionUrl2",
            value=fn_url2.url,
            description="URL for the Lambda function",
        )

Because we put the new Lambda’s file in the same /lambda directory, we can keep code=_lambda.Code.from_asset(“lambda”) from the original function definition. We’ve also left the runtime unchanged, and the function URL definition is identical; only the assigned variable is different: fn_url2.

The important changes from the _lambda.Function definition are:

  1. The function name, now "MyFunction2".
  2. The handler path, now "lambda_function_2.lambda_handler".

These changes give the function a separate name in the stack, and tell the stack to look inside of lambda_function_2.py for the new request handler.

Making it Concise

For this simple example, you could be more concise by leveraging the Code part of Infrastructure-as-Code. Build the Lambdas in a for loop:

python
from aws_cdk import (
    Stack,
    aws_lambda as _lambda,
    CfnOutput,
)
from constructs import Construct

class MyFirstPythonStackStack(Stack):
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)
    
        # Define the changing parts of the Lambda functions
        functions = [
            {
                "id": "MyFunction",
                "handler": "lambda_function.lambda_handler",
            },
            {
                "id": "MyFunction2",
                "handler": "lambda_function_2.lambda_handler",
            },
        ]
        

        # Loop through the Lambda initialisations
        for cfg in functions:
  
            fn = _lambda.Function(
                self,
                cfg["id"],
                runtime=_lambda.Runtime.PYTHON_3_11,
                code=_lambda.Code.from_asset("lambda"),
                handler=cfg["handler"],
            )

            fn_url = fn.add_function_url(
                auth_type=_lambda.FunctionUrlAuthType.NONE,
                cors=_lambda.FunctionUrlCorsOptions(
                    allowed_origins=["*"],
                    allowed_methods=[_lambda.HttpMethod.ALL],
                    allowed_headers=["*"],
                ),
            )

            CfnOutput(
                self,
                f"{cfg['id']}Url",
                value=fn_url.url,
                description=f"URL for the {cfg['id']} Lambda function",
            )

This produces the same stack and is easier to read.

Step 3 — Deploy the Stack

Initial deployment works as usual. Deploy the stack with this command:

typescript
cdk deploy

You’ll be given a set of IAM security statements. Review them; if they look okay to you (they should), accept and deploy. You’ll be given two URLS as output:

bash
Outputs:
MyFirstPythonStackStack.MyFunction2Url = https://gef75e3jncnhzhahb4bimsmv7e0d123p.lambda-url.eu-west-2.on.aws/
MyFirstPythonStackStack.MyFunctionUrl = https://ajslkhslu3298743kjslkshdkjsa23oi.lambda-url.eu-west-2.on.aws/

Check the deployments working by running a curl command for both endpoints:

xml
curl <your-url-here>

Step 4 — Deploy and Destroy Separately

To make changes in a single stack, your only real option is to trust cdk synth to generate accurate *Change Sets*. A Change Set is a set of actions that AWS needs to take to align your deployed infrastructure with your definition in code.

It works like this:

  1. CDK synthesises a CloudFormation template from your code.
  2. CDK compares the template to what’s currently deployed.
  3. It uses the differences to make a Change Set.
  4. AWS uses the Change Set to alter your infrastructure to match your code.

For example, deploy your current stack. Now delete the second Lambda function and run:

typescript
cdk diff

This will generate a change set and print it to the screen. You should see something like this:

bash
Stack MyFirstPythonStackStack
IAM Statement Changes
┌───┬────────────────────────────────┬────────┬──────────────────────────┬──────────────────────────────┬───────────┐
│   │ Resource                       │ Effect │ Action                   │ Principal                    │ Condition │
├───┼────────────────────────────────┼────────┼──────────────────────────┼──────────────────────────────┼───────────┤
│ - │ ${MyFunction2.Arn}             │ Allow  │ lambda:InvokeFunctionUrl │ *                            │           │
├───┼────────────────────────────────┼────────┼──────────────────────────┼──────────────────────────────┼───────────┤
│ - │ ${MyFunction2/ServiceRole.Arn} │ Allow  │ sts:AssumeRole           │ Service:lambda.amazonaws.com │           │
└───┴────────────────────────────────┴────────┴──────────────────────────┴──────────────────────────────┴───────────┘
IAM Policy Changes
┌───┬────────────────────────────┬────────────────────────────────────────────────────────────────────────────────┐
│   │ Resource                   │ Managed Policy ARN                                                             │
├───┼────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ - │ ${MyFunction2/ServiceRole} │ arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole │
└───┴────────────────────────────┴────────────────────────────────────────────────────────────────────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)

Resources
[-] AWS::IAM::Role MyFunction2/ServiceRole MyFunction2ServiceRole07F5EE0E destroy
[-] AWS::Lambda::Function MyFunction2 MyFunction2F22264CA destroy
[-] AWS::Lambda::Url MyFunction2/FunctionUrl MyFunction2FunctionUrlB02WBD63 destroy
[-] AWS::Lambda::Permission MyFunction2/invoke-function-url MyFunction2invokefunctionurlB04E44E5 destroy

Outputs
[-] Output MyFunction2Url: {"Description":"URL for the MyFunction2 Lambda function","Value":{"Fn::GetAtt":["MyFunction2FunctionUrlB0E48D63","FunctionUrl"]}}

All of the second Lambda function’s resources are set to be deleted. Run this with:

typescript
cdk deploy

Attempt to hit the second Lambda with curl and you won’t be able to, whilst the first will remain. You have successfully destroyed the second Lambda, whilst leaving the first alone.

This applies to any other changes you make. For example, uncomment your second Lambda definition and alter the returned message from the first Lambda. Then run:

css
cdk deploy --hot-swap

This will redeploy the second Lambda and alter the returned value of the first.

Is This Separate?

Yes and No. It’s separate in the sense that each Lambda is its own entity; when you apply a Change Set that destroys one of the Lambdas, it is gone. However, as the Lambdas are part of the same stack, and cdk diff is stack wide, their deployment times increase as the size of the stack increases.

For our small example, this isn’t an issue. But it can become painful on large projects.

Step 5— Make the Lambdas Talk

You can allow one Lambda to call another by granting it the invoke permission in the Stack:

python
from aws_cdk import (
    Stack,
    aws_lambda as _lambda,
    aws_iam as iam,
    CfnOutput,
)
from constructs import Construct

class MyFirstPythonStackStack(Stack):
    """
    Two Lambda functions and dedicated URLS. One function with
    permission to invoke the other.
    """
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        # Create a function
        fn = _lambda.Function(
            self,
            "MyFunction",
            runtime=_lambda.Runtime.PYTHON_3_11,
            code=_lambda.Code.from_asset("lambda"),
            handler="lambda_function.lambda_handler",
        )

        fn_url = fn.add_function_url(
            auth_type=_lambda.FunctionUrlAuthType.NONE,
            cors=_lambda.FunctionUrlCorsOptions(
                allowed_origins=["*"],
                allowed_methods=[_lambda.HttpMethod.ALL],
                allowed_headers=["*"],
            ),
        )

        # Create a function
        fn2 = _lambda.Function(
            self,
            "MyFunction2",
            runtime=_lambda.Runtime.PYTHON_3_11,
            code=_lambda.Code.from_asset("lambda"),
            handler="lambda_function_2.lambda_handler",
        )

        fn_url2 = fn2.add_function_url(
            auth_type=_lambda.FunctionUrlAuthType.NONE,
            cors=_lambda.FunctionUrlCorsOptions(
                allowed_origins=["*"],
                allowed_methods=[_lambda.HttpMethod.ALL],
                allowed_headers=["*"],
            ),
        )

        # Grant fn permission to invoke fn2
        fn.add_to_role_policy(
            iam.PolicyStatement(
                actions=["lambda:InvokeFunction"],
                resources=[fn2.function_arn],
            )
        )

        # This can also be done with
        # fn.grant_invoke(fn2)
        # Expose the second function's ARN to the first function
        fn.add_environment("PEER_FN_ARN", fn2.function_arn)

        CfnOutput(
            self,
            "MyFunctionUrl",
            value=fn_url.url,
            description=f"URL for the MyFunction Lambda function",
        )

        CfnOutput(
            self,
            "MyFunction2Url",
            value=fn_url2.url,
            description=f"URL for the MyFunction2 Lambda function",
        )

The additions to our stack are:

  • fn.add_to_role_policy(...)
  • fn.grant_invoke(fn2)
  • fn.add_environment(...)

Either of the first two methods grants the first function an IAM permission that allows it to call the second function. The final addition appends an environment variable to the function at runtime that tells it the Amazon Resource Name (ARN) of the secondary function. The first function can use this to call the second via the boto3 client.

Calling the Second Lambda

To call the second Lambda in the first function, we add the AWS client for Python, boto3, to the first Lambda’s code:

python
import json
from typing import Dict, Any
import boto3
import os

# Set up the client to communicate with AWS and the other Lambda function
client = boto3.client("lambda")


def lambda_handler(event: Dict[str, Any], context: Any) -> Dict[str, Any]:
    """
    AWS Lambda handler

    :param event: dict, the event data sent by the requester
    :param context: LambdaContext, runtime information
    :return: dict,
    """

    body = json.loads(event["body"])

    # Standard Response
    if "calledBy" in body:
        return {
            "statusCode": 200,
            "headers": {"Content-Type": "application/json"},
            "body": json.dumps({"message": f"Hello, {body['calledBy']}!"}),
        }

    # Call secondary Lambda function to get response
    response = client.invoke(
        FunctionName=os.environ["PEER_FN_ARN"],
        InvocationType="RequestResponse",
        Payload=json.dumps({"calledBy": "MyFunction"}),
    )

    # Parse secondary Lambda function's response
    result = json.loads(response["Payload"].read())

    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(result),
    }

You can see that if the event passed to the function contains a key "calledBy" it will return a direct message. If it does not, it will call the second Lambda and return its response.

Deploy these changes with:

bash
cdk deploy

Then call the first function with and without the calledBy input in the event. You will see two separate responses, one from the first function and the other from the second, via the first.

You can do this from a Python script with the requests library:

python
import requests

first_function_response = requests.post(
    "https://3wnkxylytumpl5t4qf23n4vcbi0kngwl.lambda-url.eu-west-2.on.aws/",
    json={"calledBy": "MyFunction"}
)

print(first_function_response.content)

second_function_response = requests.post(
    "https://3wnkxylytumpl5t4qf23n4vcbi0kngwl.lambda-url.eu-west-2.on.aws/",
    json={"calledBy": "MyFunction"}
)

print(second_function_response.content)


Step 6 — Total Destruction

Congratulations! You’ve reached the end of the tutorial. Before you go, don’t forget to completely destroy your stack to avoid any unwanted costs. You can do this with:

bash
cdk destroy

Final Words

I hope this method worked for you! If not, post your questions and I’d be happy to help. This is just one of the possible methods for deploying two Lambdas; there’s another I’ll cover in a separate article. Stay tuned for that.

Happy DevOpsing!