Install Python Package in AWS Lambda

AWS Lambda is a serverless computing service. It lets you run code without managing servers. Python is a popular choice for Lambda functions. But installing external packages can be tricky.

This guide will show you how to install Python packages in AWS Lambda. You'll learn different methods to include dependencies in your Lambda functions.

Why Install Python Packages in AWS Lambda?

Lambda functions often need external libraries. The default environment includes only basic Python packages. You must manually add any extra dependencies.

Common use cases include:

• Data processing with Pandas
• API calls with Requests
• Database connections

Method 1: Using Lambda Layers

Lambda Layers is the recommended way to add packages. It separates your code from dependencies. This makes your deployment package smaller.

Follow these steps:

1. Create a virtual environment
2. Install packages in a specific folder
3. Zip the folder
4. Create a Lambda Layer
5. Attach the layer to your function

Here's how to create the package directory:

mkdir python
pip install requests -t python/
zip -r layer.zip python

Upload layer.zip as a new Lambda Layer. Then attach it to your function.

Method 2: Deploying with Deployment Package

You can bundle packages directly with your code. This creates a larger deployment package but is simpler for small projects.

Steps:

1. Install packages in a local directory
2. Add your Lambda function code
3. Zip everything together
4. Upload to AWS Lambda

Example commands:

pip install requests -t .
zip -r function.zip .

Remember AWS Lambda has a 50MB limit for direct uploads. For larger packages, use S3.

Method 3: Using Docker Containers

Lambda now supports container images. This lets you build custom environments. It's similar to installing Python packages in Docker.

Steps:

1. Create a Dockerfile
2. Build the image
3. Push to ECR
4. Create Lambda function from image

Example Dockerfile:

 
FROM public.ecr.aws/lambda/python:3.8

# Install packages
RUN pip install requests pandas

# Copy function code
COPY app.py ${LAMBDA_TASK_ROOT}

# Set the handler
CMD ["app.lambda_handler"]

Working with Requirements.txt

For complex projects, use requirements.txt. This is similar to installing with requirements.txt locally.

First create the file:

echo "requests==2.25.1" > requirements.txt
echo "pandas==1.2.3" >> requirements.txt

Then install packages:

pip install -r requirements.txt -t .

Handling Binary Dependencies

Some packages need binary compilation. These must be built on Amazon Linux for Lambda compatibility.

Options:

• Use AWS Cloud9 (Amazon Linux environment)
• Build in a Docker container
• Use pre-built wheels

For example, to build NumPy:

docker run -v $(pwd):/output -it amazonlinux bash
yum install -y python3-devel gcc
pip install numpy -t /output

Testing Your Lambda Function

Always test your function after adding packages. Common issues include:

• Missing dependencies
• Incorrect package versions
• Binary compatibility problems

Use Lambda's test feature or local testing tools.

Best Practices

Follow these tips for better results:

• Keep packages updated
• Minimize package size
• Use layers for shared dependencies
• Consider Pipenv for dependency management

Conclusion

Installing Python packages in AWS Lambda requires careful planning. Layers are the cleanest solution for most cases. For complex needs, container images offer more flexibility.

Remember to test thoroughly after adding dependencies. Proper package management will make your Lambda functions more reliable and maintainable.