Two Lambda Stacks One App

How can I use CDK deploy, re-deploy, destroy, and communicate between two Lambdas in a single application?

AWSLambdaCDKPython

Creating your infrastructure in a single stack can be cumbersome. Deployment times increase with the size of the stack, so that minor alterations to a Lambda function go from seconds to minutes to deploy. If this is slowing you down, it’s time to think about separating your application into separate stacks.

This tutorial will show you how to:

  1. Create two Lambdas at once in a Python CDK app.
  2. Deploy them separately.
  3. Destroy them separately.
  4. Have them trigger each other.

If you’re here, I’m assuming you already have the Python CDK set up and ready to go. If not, you can read this tutorial to prepare a single Lambda.

Initial Python Lambda

The Simplest AWS Lambda in Python

If you know the basics already, and already have Python CDK installed, you can download the starter code from my GitHub here:

GitHub - Connor56/cdk-python-stack: A basic CDK stack with a single Lambda in it.

The SSH command to clone is:

bash
git clone git@github.com:Connor56/cdk-python-stack.git

Now, let’s get into the new stuff.

Step 1 — Add a Second Lambda Function

If you followed the tutorial above, or downloaded the code from GitHub. You should see the following directory structure in your project:

bash
📁 ./

Initially, we’ll keep the second lambda function simple. So, add another file inside the /lambda directory called lambda_function_2.py. Open the file in an editor and paste in the following:

python
import json
from typing import Dict, Any


def lambda_handler(event: Dict[str, Any], context: Any) -> Dict[str, Any]:
    """
    AWS Lambda handler

    :param event: dict, the event data sent by the requester
    :param context: LambdaContext, runtime information
    :return: dict,
    """

    body = {"message": "Hello Lambda the 2nd!"}

    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(body),
    }

This is a copy of the lambda_function.py, except now we’re returning "Hello Lambda the 2nd!".

Step 2 — Create a Reusable Stack

If you followed my tutorial above, your CDK stack will look something like this:

python
from aws_cdk import (
    Stack,
    aws_lambda as _lambda,
)
from constructs import Construct


class MyFirstPythonStackStack(Stack):

    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        # Base lambda function set up
        fn = _lambda.Function(
            self,
            "MyFunction",  # The name of the function in Cloud Formation
            runtime=_lambda.Runtime.PYTHON_3_11,  # You runtime environment
            code=_lambda.Code.from_asset("lambda"),  # Your code directory
            handler="lambda_function.lambda_handler",  # Your <module>.<handler-function>
        )

        # Create a function URL for the Lambda function
        fn_url = fn.add_function_url(
            auth_type=_lambda.FunctionUrlAuthType.NONE,  # Public access, no Auth required
            cors=_lambda.FunctionUrlCorsOptions(
                allowed_origins=[
                    "*"
                ],  # Allow the request to come from any origin
                allowed_methods=[
                    _lambda.HttpMethod.ALL
                ],  # Allow the request to have any http method
                allowed_headers=["*"],  # Allow any headers in the request
            ),
        )

        # Output the function URL
        from aws_cdk import CfnOutput

        # Print the URL address to the terminal when you deploy
        CfnOutput(
            self,
            "FunctionUrl",
            value=fn_url.url,
            description="URL for the Lambda function",
        )

This can be made into a reusable, parameterisable stack. I’ve called mine FunctionStack:

python
from aws_cdk import (
    Stack,
    aws_lambda as _lambda,
    CfnOutput,
)
from constructs import Construct

class FunctionStack(Stack):
    """
    Parameterisable CDK stack used to produce a single Lambda.
    """
    
    def __init__(self, scope: Construct, id: str, handler: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        fn = _lambda.Function(
            self,
            "Fn",
            runtime=_lambda.Runtime.PYTHON_3_11,
            code=_lambda.Code.from_asset("lambda"),
            handler=handler,
        )

        url = fn.add_function_url(
            auth_type=_lambda.FunctionUrlAuthType.NONE,
            cors=_lambda.FunctionUrlCorsOptions(
                allowed_origins=["*"],
                allowed_methods=[_lambda.HttpMethod.ALL],
                allowed_headers=["*"],
            ),
        )

        CfnOutput(
            self,
            "Url",
            value=url.url,
            description=f"URL for {id}",
        )

It contains everything required to set up a Lambda with an accompanying URL and output. Because it inherits from the Stack class, CDK will interpret any instantiation of it as its own logically separate stack.

This stack can be used in ./app.py file, in the root directory, to produce two separate FunctionStack:

python
import aws_cdk as cdk

from my_first_python_stack.my_first_python_stack_stack import FunctionStack

app = cdk.App()

function_2 = FunctionStack(
    app,
    "MyFunction2Stack",
    handler="lambda_function_2.lambda_handler",
)

FunctionStack(
    app,
    "MyFunctionStack",
    handler="lambda_function.lambda_handler",
    target_function_arn=function_2.function_arn,
)

app.synth()

Step 3 — Deploy the Stacks

This gives you two stacks in your CDK application. If you attempt to deploy normally:

bash
cdk deploy

You’ll get an error like this:

bash
Since this app includes more than a single stack, specify which stacks to use (wildcards are supported) or specify `--all`
Stacks: MyFunctionStack · MyFunction2Stack

As the error states, you can fix this by deploying with:

bash
cdk deploy --all

This will sequentially deploy the stacks. This takes roughly 40 seconds a stack. To add concurrency, you can use the following command instead:

bash
cdk deploy --all --concurrency 2 --require-approval never

The --require-approval never flag prevents the IAM security changes from being shown to you, which is necessary for concurrency to work on deployment. If you’re worried about the changes, you can check the diff beforehand with:

bash
cdk diff

You will see outputs like the following after each stack has deployed:

bash
Outputs:
MyFunctionStack.Url= https://gef75e3jncnhzhahb4bimsmv7e0d123p.lambda-url.eu-west-2.on.aws/
MyFunctionStack2.Url = https://ajslkhslu3298743kjslkshdkjsa23oi.lambda-url.eu-west-2.on.aws/

Check the deployments working by running a curl command for both endpoints:

bash
curl <your-url-here>

Step 4 — Deploy Separately

Because your stacks are literally separate entities with their own CloudFormation templates, you can deploy your stacks separately with:

bash
cdk deploy YourStackNameHere

This will deploy the stack of the given name only. If you’ve made small alterations to one of your Lambda function stacks, you can redeploy it with:

css
cdk deploy --hot-swap

If your stacks are independent, this will deploy in seconds, no matter how many Lambda stacks you have.

Step 5 — Destroy the Stacks

If you want to destroy a single stack, you can do so with:

typescript
cdk destroy YourStackName

If you have multiple stacks, you can destroy them all with:

css
cdk destroy --all

Unfortunately, there’s currently no built-in way to destroy stacks concurrently, so if you’ve deployed a lot of them, you’ll be waiting a while.

Step 6 — Make the Lambdas Talk

To allow one Lambda stack permission to invoke the function of another, you have to allow the FunctionStack to be parameterised by the other. This means accepting the Amazon Resource Identifier (ARN) of another function in the stack definition.

FunctionStack can be extended to do this:

python
from aws_cdk import (
    Stack,
    aws_lambda as _lambda,
    aws_iam as iam,
    CfnOutput,
)
from constructs import Construct


class FunctionStack(Stack):
    """
    Parameterisable CDK stack used to produce a single Lambda.
    """

    def __init__(
        self,
        scope: Construct,
        id: str,
        handler: str,
        target_function_arn: str | None = None,
        **kwargs,
    ) -> None:
        super().__init__(scope, id, **kwargs)

        fn = _lambda.Function(
            self,
            "Fn",
            runtime=_lambda.Runtime.PYTHON_3_11,
            code=_lambda.Code.from_asset("lambda"),
            handler=handler,
        )


        # Expose the function for use in app.py
        self.fn = fn

        url = fn.add_function_url(
            auth_type=_lambda.FunctionUrlAuthType.NONE,
            cors=_lambda.FunctionUrlCorsOptions(
                allowed_origins=["*"],
                allowed_methods=[_lambda.HttpMethod.ALL],
                allowed_headers=["*"],
            ),
        )

        # Add permission to invoke the target function if one is provided
        if target_function_arn is not None:
            fn.add_to_role_policy(
                iam.PolicyStatement(
                    actions=["lambda:InvokeFunction"],
                    resources=[target_function_arn],
                )
            )
            
            # Add environment variable for the target function ARN
            fn.add_environment("PEER_FN_ARN", target_function_arn)

        CfnOutput(
            self,
            "Url",
            value=url.url,
            description=f"URL for {id}",
        )

target_function_arn has been added to the class initialisation. With the resource identifier, you can grant your Lambda permission to invoke the other in code.

In detail, the other additions to our stack are:

  • self.fn = fn
  • fn.add_to_role_policy(...)
  • fn.add_environment(...)

The first addition sets the Lambda function as an attribute of the class. This allows the function to be accessed in ./app.py, which is required for passing the ARN. The next addition grants an IAM permission that lets our Lambda stack invoke another function. The final addition appends the ARN of the secondary function to the first function as an environment variable, allowing it to call it with the boto3 client.

Update the app.py

To use this updated stack, the app.py must be updated to the following:

python
import aws_cdk as cdk

from my_first_python_stack.my_first_python_stack_stack import FunctionStack

app = cdk.App()

function_2 = FunctionStack(
    app,
    "MyFunction2Stack",
    handler="lambda_function_2.lambda_handler",
)

FunctionStack(
    app,
    "MyFunctionStack",
    handler="lambda_function.lambda_handler",
    target_function_arn=function_2.fn.function_arn,
)

app.synth()

I have created the second stack first so that it’s internal function_arn can be passed on to the first stack.

Update the First Lambda to Call the Second

Finally, we need to update the first Lambda function so that it can call the second one:

python
import json
from typing import Dict, Any
import boto3
import os

# Set up the client to communicate with AWS and the other Lambda function
client = boto3.client("lambda")


def lambda_handler(event: Dict[str, Any], context: Any) -> Dict[str, Any]:
    """
    AWS Lambda handler

    :param event: dict, the event data sent by the requester
    :param context: LambdaContext, runtime information
    :return: dict,
    """

    body = json.loads(event["body"])

    # Standard Response
    if "calledBy" in body:
        return {
            "statusCode": 200,
            "headers": {"Content-Type": "application/json"},
            "body": json.dumps({"message": f"Hello, {body['calledBy']}!"}),
        }

    # Call secondary Lambda function to get response
    response = client.invoke(
        FunctionName=os.environ["PEER_FN_ARN"],
        InvocationType="RequestResponse",
        Payload=json.dumps({"calledBy": "MyFunction"}),
    )

    # Parse secondary Lambda function's response
    result = json.loads(response["Payload"].read())

    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(result),
    }

You can see that if the event body passed to the function contains a key "calledBy" it will return a direct message. If it does not, it will call the second Lambda and return its response.

Deploy these changes with:

bash
cdk deploy --hot-swap --all --concurrency 2 --require-approval never

Then call the first function with and without the calledBy input in the event. You will see two separate responses, one from the first function and the other from the second (via the first).

You can do this from a Python script with the requests library:

python
import requests

first_function_response = requests.post(
    "https://3wnkxylytumpl5t4qf23n4vcbi0kngwl.lambda-url.eu-west-2.on.aws/",
    json={"calledBy": "MyFunction"}
)

print(first_function_response.content)

second_function_response = requests.post(
    "https://3wnkxylytumpl5t4qf23n4vcbi0kngwl.lambda-url.eu-west-2.on.aws/",
    json={"calledBy": "MyFunction"}
)

print(second_function_response.content)

Final Words

I hope this works nicely for you! If not, post your questions and I’d be happy to help. Realistically, this method is only helpful if you're using so many AWS resources that a single stack is punishingly slow to deploy. If you’re project is small, use a single stack. You can see how here:

Two Lambdas One Stack

Happy DevOpsing!