Manage a Python Lambda’s Packages

How do you manage a Python Lambda’s deployed packages in a CDK stack?

May 5, 2025

AWSLambdaPythonCDK

This tutorial will show you two methods for managing your Python Lambda’s packages:

Bundling directly in the Lambda function.
Bundling in a layer and attaching it.

You will learn how to bundle Numpy into a Lambda function, deploy it in a CDK stack, test it, and then tear it down.

Step 0 — Prerequisites

This article assumes you have the following:

An AWS Account.
The Python CDK installed.
Docker Desktop installed.
A CDK stack with a Python Lambda in it.

If you have none of the above, you’ll need to get them for this article to be useful. The article is part of a series that explains how to set up Python Lambda functions in CDK. So if you’d like tutorials on how to get to this point, you can click back through the previous articles.

This is the previous one:

Store Unencrypted Secrets in Parameter Store With CDK

The code associated with the series is on my GitHub. This article follows on from this repo and branch:

GitHub - Connor56/cdk-python-stack at part-3/managing-secrets-with-parameter-store

If you’d like to follow along, clone the repository and checkout the branch part-3/managing-secrets-with-parameter-store. This is the starting point of this article. You can see it finished in either:

part-4/managing-a-lambdas-python-packages/bundle-method
part-4/managing-a-lambdas-python-packages/layers-method

Step 1 — Define Your Lambda’s Packages

There are two ways to do this:

Add a requirements.txt file to each of your Lambda directories and add a bundling step to your Lambda’s stack.
Add a Lambda Layer to the CDK stack to install the packages once and use them on multiple Lambdas simultaneously.

I’ll show you how to use both methods.

Method 1 — Bundling and requirements.txt

If your Lambdas don’t share many packages, or if you only have a few, it makes sense to use this method.

To start, place a requirements.txt file in your Lambda’s directory. If you’re starting from my GitHub code, your overall directory structure will look like this:

📁 ./
├── 📄 .env.example
├── 📄 .gitignore
├── 📄 README.md
├── 📄 app.py
├── 📄 cdk.json
├── 📄 poetry.lock
├── 📄 pyproject.toml
├── 📁 lambda/
│   ├── 📄 lambda_function.py
│   └── 📄 lambda_function_2.py
├── 📁 my_first_python_stack/
│   ├── 📄 __init__.py
│   ├── 📄 my_first_python_stack_stack.py
├── 📁 tests/
│   ├── 📄 __init__.py
│   ├── 📁 unit/
│   │   ├── 📄 __init__.py
│   │   └── 📄 test_my_first_python_stack_stack.py

Go into the /lambda directory from the root and add a requirements.txt file:

plaintext

numpy==2.2.5

For this tutorial, we’re going to install numpy as it’s a small, commonly used number crunching package that includes C binaries. This is to show it’s possible to use Python packages that come with compiled code.

Next, update the Lambda function’s bundling procedure. If you’re using my code, we’ll use the fn function variable from my_first_python_stack_stack.py. Initially, it looks like this:

python

# Create a function

To update the bundling, add the bundling argument to .from_asset:

python

from aws_cdk import BundlingOptions # <-- Add this to your aws imports

You can see we’ve added a new class BundlingOptions. This contains the configuration we want to use for bundling our function’s dependencies. For the configuration we have:

image — This is the Docker image we’re going to run our bundling command on.
command — This is the command the Docker image will run when bundling.

command is a collection of arguments that will be executed in the Docker container’s shell. It starts with bash -c, which tells the shell everything after bash -c should be interpreted as a bash script. This allows you to run multiple commands one after the other, using the && operator.

To prepare the Python packages we need, the Docker container runs:

bash

pip install -r requirements.txt -t /asset-output

For our example, this install numpy==2.2.5 into the /asset-output directory. This is a special directory in the bundling procedure; anything in it when bundling ends is zipped and used in the Lambda function. There’s a file size limit of 250MB of unzipped code.

The final part of the command:

bash

cp -r . /asset_output

copies the remaining files in the /lambda directory, i.e. the actual lambda handlers and functions, etc., so that they are also deployed to the Lambda’s deployment code. If you’re following my code, your final CDK stack will look like this:

python

from aws_cdk import (
    Stack,
    aws_lambda as _lambda,
    aws_ssm as ssm,
    aws_iam as iam,
    CfnOutput,
    BundlingOptions,
)
from constructs import Construct
import os
from dotenv import load_dotenv

load_dotenv("../.env")


class MyFirstPythonStackStack(Stack):
    """
    Two Lambda functions and dedicated URLS. One function with
    permission to invoke the other.
    """

    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        # Create a function
        fn = _lambda.Function(
            self,
            "MyFunction",
            runtime=_lambda.Runtime.PYTHON_3_11,
            code=_lambda.Code.from_asset(
                "lambda",
                bundling=BundlingOptions(
                    image=_lambda.Runtime.PYTHON_3_11.bundling_image,
                    command=[
                        "bash",
                        "-c",
                        # install into /asset-output and copy your code
                        "pip install -r requirements.txt -t /asset-output && "
                        "cp -ru . /asset-output",
                    ],
                ),
            ),
            handler="lambda_function.lambda_handler",
        )

        fn_url = fn.add_function_url(
            auth_type=_lambda.FunctionUrlAuthType.NONE,
            cors=_lambda.FunctionUrlCorsOptions(
                allowed_origins=["*"],
                allowed_methods=[_lambda.HttpMethod.ALL],
                allowed_headers=["*"],
            ),
        )

        # Create a parameter store secret
        secret = ssm.StringParameter(
            self,
            id="SomeRandomKey",
            parameter_name="SomeRandomKey",
            description="A random key to test",
            string_value=os.getenv("SomeRandomKey"),
        )

        # Grent read access to the first function
        secret.grant_read(fn)

        # Create a function
        fn2 = _lambda.Function(
            self,
            "MyFunction2",
            runtime=_lambda.Runtime.PYTHON_3_11,
            code=_lambda.Code.from_asset("lambda"),
            handler="lambda_function_2.lambda_handler",
        )

        fn_url2 = fn2.add_function_url(
            auth_type=_lambda.FunctionUrlAuthType.NONE,
            cors=_lambda.FunctionUrlCorsOptions(
                allowed_origins=["*"],
                allowed_methods=[_lambda.HttpMethod.ALL],
                allowed_headers=["*"],
            ),
        )

        # Grant fn permission to invoke fn2
        fn.add_to_role_policy(
            iam.PolicyStatement(
                actions=["lambda:InvokeFunction"],
                resources=[fn2.function_arn],
            )
        )
        # This can also be done with
        # fn.grant_invoke(fn2)

        # Expose the second function's ARN to the first function
        fn.add_environment("PEER_FN_ARN", fn2.function_arn)

        CfnOutput(
            self,
            "MyFunctionUrl",
            value=fn_url.url,
            description=f"URL for the MyFunction Lambda function",
        )

        CfnOutput(
            self,
            "MyFunction2Url",
            value=fn_url2.url,
            description=f"URL for the MyFunction2 Lambda function",
        )

That’s all you need to bundle packages with your Lambda. Nice, huh? The only problem is that it’s slow, and you must do it every time you redeploy. A small alteration to your function might have been hot-swapped in seconds, but now it’s taking over a minute. Perplexing… Let’s talk about Layers.

Method 2 — Lambda Layers

Lambda layers are additions to your function. Think of them like runtime add-ons or plugins. They consist of a zip folder, just like your normal function, with useful functionality in them. At runtime, they are copied into the /opt directory of your Lambda and can be used by your main code.

You define them separately in CDK, meaning they can be deployed separately. To create them, you bundle them like in Method 1, except this time, you only need to do it once. Most importantly, your handler code is separate so hot-swapping is back on the menu!

If you’re following along on GitHub, you’ll have the following folder structure:

📁 ./
├── 📄 .env.example
├── 📄 .gitignore
├── 📄 README.md
├── 📄 app.py
├── 📄 cdk.json
├── 📄 poetry.lock
├── 📄 pyproject.toml
├── 📁 lambda/
│   ├── 📄 lambda_function.py
│   └── 📄 lambda_function_2.py
├── 📁 my_first_python_stack/
│   ├── 📄 __init__.py
│   ├── 📄 my_first_python_stack_stack.py
├── 📁 tests/
│   ├── 📄 __init__.py
│   ├── 📁 unit/
│   │   ├── 📄 __init__.py
│   │   └── 📄 test_my_first_python_stack_stack.py

Add a new directory to the root called /layer and add the same requirements.txt file from Method 1 in there:

📁 ./
├── 📄 .env.example
├── 📄 .gitignore
├── 📄 README.md
├── 📄 app.py
├── 📄 cdk.json
├── 📄 poetry.lock
├── 📄 pyproject.toml
├── 📁 lambda/
│   ├── 📄 lambda_function.py
│   └── 📄 lambda_function_2.py
├── 📁 layer/
│   └── 📄 requirements.txt
├── 📁 my_first_python_stack/
│   ├── 📄 __init__.py
│   ├── 📄 my_first_python_stack_stack.py
├── 📁 tests/
│   ├── 📄 __init__.py
│   ├── 📁 unit/
│   │   ├── 📄 __init__.py
│   │   └── 📄 test_my_first_python_stack_stack.py

Add a Layer to your CDK stack:

python

from aws_cdk import BundlingOptions # <-- Add this to your aws imports

BundlingOptions is used in exactly the same way as Method 1, except this time to create a Layer Construct. The only important, non-obvious addition is: compatible_runtimes, which defines the Lambda runtimes that can operate with this layer. This has been set to PYTHON_3_11 to match the Lambda function.

Add the layer to your Lambda function with:

python

# Create a function

If you’re following my GitHub code, your final CDK stack should look something like this:

python

from aws_cdk import (

Step 2 — Use Numpy in Your Lambda

To check this is working, update your Lambda function to use numpy. If you’re using the GitHub code, your function will look like this:

python

import json
from typing import Dict, Any
import boto3
import os

# Set up the client to communicate with AWS and the other Lambda function
client = boto3.client("lambda")
ssm = boto3.client("ssm")


def lambda_handler(event: Dict[str, Any], context: Any) -> Dict[str, Any]:
    """
    AWS Lambda handler

    :param event: dict, the event data sent by the requester
    :param context: LambdaContext, runtime information
    :return: dict,
    """

    body = json.loads(event["body"])

    # Standard Response
    if "calledBy" in body:
        return {
            "statusCode": 200,
            "headers": {"Content-Type": "application/json"},
            "body": json.dumps({"message": f"Hello, {body['calledBy']}!"}),
        }

    # Call secondary Lambda function to get response
    response = client.invoke(
        FunctionName=os.environ["PEER_FN_ARN"],
        InvocationType="RequestResponse",
        Payload=json.dumps(
            {
                "calledBy": "MyFunction",
            }
        ),
    )

    # Parse secondary Lambda function's response
    result = json.loads(response["Payload"].read())

    some_random_key = ssm.get_parameter(Name="SomeRandomKey")

    result["some_random_key"] = some_random_key["Parameter"]["Value"]

    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(result),
    }

Update by creating a numpy.array and performing a dot product with:

python

import numpy as np

# ...(ignore rest of file)...

numpy_array = np.array([1, 2, 3])
print(numpy_array @ numpy_array)

Add the result to the response returned from the Lambda, and your final Lambda function will look like this:

python

import json
from typing import Dict, Any
import boto3
import os
import numpy as np

# Set up the client to communicate with AWS and the other Lambda function
client = boto3.client("lambda")
ssm = boto3.client("ssm")


def lambda_handler(event: Dict[str, Any], context: Any) -> Dict[str, Any]:
    """
    AWS Lambda handler

    :param event: dict, the event data sent by the requester
    :param context: LambdaContext, runtime information
    :return: dict,
    """

    body = json.loads(event["body"])

    # Standard Response
    if "calledBy" in body:
        return {
            "statusCode": 200,
            "headers": {"Content-Type": "application/json"},
            "body": json.dumps({"message": f"Hello, {body['calledBy']}!"}),
        }

    # Call secondary Lambda function to get response
    response = client.invoke(
        FunctionName=os.environ["PEER_FN_ARN"],
        InvocationType="RequestResponse",
        Payload=json.dumps(
            {
                "calledBy": "MyFunction",
            }
        ),
    )

    # Parse secondary Lambda function's response
    result = json.loads(response["Payload"].read())
  
    # --- The Numpy section ---
    numpy_array = np.array([1, 2, 3])
    dot_product = numpy_array @ numpy_array

    result["numpy_array"] = numpy_array.tolist()
    result["dot_product"] = int(dot_product) # <-- convert to regular int for JSON
    # -------------------------

    some_random_key = ssm.get_parameter(Name="SomeRandomKey")

    result["some_random_key"] = some_random_key["Parameter"]["Value"]

    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(result),
    }

Step 3 — Deploy and Test

Check if it has worked by running:

bash

cdk deploy

If you’re using my GitHub code, you’ll see an IAM statement similar to the following:

bash

IAM Statement Changes
┌───┬───────────────────────────────────────────────────────────────────────────────────────────────┬────────┬────────────────────────────────────────────────────────────────────────────────┬───────────────────────────────┬───────────┐      
│   │ Resource                                                                                      │ Effect │ Action                                                                         │ Principal                     │ Condition │      
├───┼───────────────────────────────────────────────────────────────────────────────────────────────┼────────┼────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────┼───────────┤      
│ + │ ${MyFunction2.Arn}                                                                            │ Allow  │ lambda:InvokeFunction                                                          │ AWS:${MyFunction/ServiceRole} │           │      
│ + │ ${MyFunction2.Arn}                                                                            │ Allow  │ lambda:InvokeFunctionUrl                                                       │ *                             │           │      
├───┼───────────────────────────────────────────────────────────────────────────────────────────────┼────────┼────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────┼───────────┤      
│ + │ ${MyFunction2/ServiceRole.Arn}                                                                │ Allow  │ sts:AssumeRole                                                                 │ Service:lambda.amazonaws.com  │           │      
├───┼───────────────────────────────────────────────────────────────────────────────────────────────┼────────┼────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────┼───────────┤
│ + │ ${MyFunction.Arn}                                                                             │ Allow  │ lambda:InvokeFunctionUrl                                                       │ *                             │           │      
├───┼───────────────────────────────────────────────────────────────────────────────────────────────┼────────┼────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────┼───────────┤      
│ + │ ${MyFunction/ServiceRole.Arn}                                                                 │ Allow  │ sts:AssumeRole                                                                 │ Service:lambda.amazonaws.com  │           │      
├───┼───────────────────────────────────────────────────────────────────────────────────────────────┼────────┼────────────────────────────────────────────────────────────────────────────────┼───────────────────────────────┼───────────┤      
│ + │ arn:${AWS::Partition}:ssm:${AWS::Region}:${AWS::AccountId}:parameter/${SomeRandomKeyF0B263F7} │ Allow  │ ssm:DescribeParameters                                                         │ AWS:${MyFunction/ServiceRole} │           │      
│   │                                                                                               │        │ ssm:GetParameter                                                               │                               │           │      
│   │                                                                                               │        │ ssm:GetParameterHistory                                                        │                               │           │      
│   │                                                                                               │        │ ssm:GetParameters                                                              │                               │           │      
└───┴───────────────────────────────────────────────────────────────────────────────────────────────┴────────┴────────────────────────────────────────────────────────────────────────────────┴───────────────────────────────┴───────────┘      
IAM Policy Changes
┌───┬────────────────────────────┬────────────────────────────────────────────────────────────────────────────────┐
│   │ Resource                   │ Managed Policy ARN                                                             │
├───┼────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ + │ ${MyFunction2/ServiceRole} │ arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole │
├───┼────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ + │ ${MyFunction/ServiceRole}  │ arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole │
└───┴────────────────────────────┴────────────────────────────────────────────────────────────────────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)

Accept these changes with a y and let the stack deploy. Once its done you’ll see two output URLs. Copy the one for the first Lambda function and create a new Python script with the following in it:

python

import requests

response = requests.post("<Your-Url-Here>", json={"calledBy1": "MyFunction"})
print(response.content)

You should get a response like the following:

python

{
    "statusCode": 200,
    "headers": {"Content-Type": "application/json"},
    "body": '{"message": "Hello, this is Lambda 2!"}',
    "numpy_array": [1, 2, 3],
    "dot_product": 14,
    "some_random_key": "laksjdflaskfjsadf0--a-sdfashdkf--27asjkhfskjhlaksdfh",
}

Voila! Your numpy array and its vector magnitude (dot product with itself) have been returned.

Step 4 — Tear Down the Stack

This stack is all serverless, so it’s relatively safe to leave it deployed. However, to be certain you don’t get charged, you can remove it from your account with:

bash

cdk destroy

Final Words

There you go! You can now add any Python package to your Lambda… as long as they add up to under 250MB! To read more about building infrastructure with CDK, subscribe to this publication. I’m in the middle of writing a set of tutorials that teach you how to create a backend for 99% of all use cases.

If you’d like to see the completed tutorial code for either method, you can view it below.

Method 1:

GitHub - Connor56/cdk-python-stack at part-4/managing-a-lambdas-python-packages/bundle-method

Method 2:

GitHub - Connor56/cdk-python-stack at part-4/managing-a-lambdas-python-packages/layers-method