Serverless

Overview

Serverless is an architecture that hands infrastructure management, such as server provisioning, scaling, patching, and handling failures, to a cloud provider. Developers focus on writing application code and don't need to think about when, on which servers, or on how many servers that code runs. It doesn't mean "there are no servers" but "you don't manage the servers."

Pay-per-use pricing and cost structure

A major feature of serverless is its pay-per-use model, where you're charged only for the requests that actually happen. Traditional servers keep running 24 hours a day, so they cost money even late at night when traffic is zero. With serverless, if there are no requests, you aren't charged for that execution either.

For individual developers and small services, this is a big advantage. For a service with a few thousand requests a month, the number of requests fits within the AWS Lambda free tier (1 million requests a month). Even if the service grows and traffic increases, it scales out automatically, so the burden of planning capacity in advance is small.

Cold starts and how to deal with them

One of the challenges of serverless is the cold start. If there are no requests for a certain period, the execution environment is discarded, and the environment has to be rebuilt on the next request. The time this initialization takes ranges from under 100 milliseconds to more than 1 second.

There are several ways to reduce the impact of cold starts, such as choosing the runtime (startup speed differs by type), reducing the size of the deployment package, and using Provisioned Concurrency (a feature that reserves execution environments in advance). Services that need to respond in real time to user actions, like a question box service, call for a design that keeps cold starts from being noticeable.

The serverless setup of a question box service

This question box service is built serverless from the frontend to the backend. The frontend deploys Next.js on Lambda using OpenNext, and the API is served through Lambda Function URLs. It uses DynamoDB for the database, SES for sending authentication tokens, and Bedrock for content filtering.

In this setup, there's no work managing the servers themselves, and infrastructure costs rise and fall with traffic. If you kept an EC2 instance running all the time, you'd pay for the hours it runs even with no traffic, but a serverless setup makes it easier to keep costs down while traffic is low.