Optimizing Performance and Cost: Migrating an Express.js Application from EC2 to AWS Lambda

Introduction:

In a recent project, our team worked on optimizing a Node.js application that was originally hosted on an EC2 instance. The application experienced significant performance challenges, with response times exceeding 5 seconds per request. To enhance performance and reduce operational costs, we transitioned the application to Express.js on AWS Lambda. This migration not only brought response times down to under 1 second but also introduced a more scalable and cost-efficient architecture.

Why the Migration Was Necessary:

Our decision to move away from EC2 was driven by several key factors:

• Performance Bottlenecks: The existing EC2 infrastructure struggled to meet performance expectations, leading to slow response times.
• Faster API Responses: A target of under 1 second response time was essential for improving user experience.
• Cost Optimization: Running a dedicated EC2 instance was expensive, particularly during off-peak hours when resources were underutilized.
• Scalability Needs: AWS Lambda’s serverless nature allows for automatic scaling without manual intervention.

The Migration Process

Deploying Express.js on AWS Lambda

To implement the transition smoothly, we:

• Used AWS API Gateway to trigger the Express.js Lambda function.
• Containerized the application using AWS Lambda layers for better dependency management.
• Leveraged Lambda’s auto-scaling to enhance efficiency and eliminate manual scaling efforts.

Optimizing File Storage with S3 and CDN

Initially, application files were stored in Amazon S3 and served via a CDN to reduce latency. However, data transfer out (DTO) costs became a concern, particularly due to serving files from a private subnet using a NAT gateway.

Addressing Cost and Performance Challenges with Redis

Identifying the Issue:

• High DTO charges resulted from NAT gateway usage in the private subnet.
• CDN requests added to the overall expense, further impacting cost efficiency.

The Solution: Implementing Redis in a Private Subnet

To optimize costs and performance, we:

• Deployed Redis within the same private subnet to serve as a caching layer.
• Modified the Lambda function to first check Redis for cached files before fetching from S3.
• Stored frequently accessed files in Redis, ensuring near-instant responses.

The Outcome:

• DTO costs were eliminated as requests remained within the private subnet.
• CDN input/output costs dropped to zero.
• Latency was significantly reduced, with Redis delivering sub-millisecond response times.

Monitoring and Performance Metrics

Post-migration, we implemented monitoring to track the impact of our changes. Key performance indicators included:

• Concurrent executions: Ensuring seamless auto-scaling.
• Invocation counts: Tracking Lambda function calls.
• Error rates (5XX, 4XX): Identifying and addressing failed requests.
• Success rates (2XX): Measuring successful responses.
• Response times: Reduced from 5 seconds (EC2) to under 1 second (Lambda + Redis).

Conclusion

The migration from EC2 to AWS Lambda transformed the application’s performance and cost efficiency. By implementing Redis within a private subnet, we eliminated DTO charges, reduced CDN costs, and improved response times. This shift has enabled a faster, more scalable, and cost-effective solution, ensuring a seamless user experience while optimizing cloud infrastructure spending.

This project highlights the power of serverless computing, caching strategies, and cost-efficient architecture in modern cloud environments. Have you considered moving your workloads to AWS Lambda? We’d love to hear about your experiences and challenges!

Contact us today for a FREE consultation: sales@cloud.in or call at +91-020-66080123

Blog is written by Numan Gharte ( Cloud Engineer @Cloud.in)