I use AWS a ton but Lambda still astounds me. Throw some code in a function, send 1m requests as fast as you can.
It ate up all available file descriptors on my little t3 box and still ran 18k RPS with a p99 of 0.3479s. Not many services can go from 0 to 18k RPS instantaneously with this p99.
Happy Lambda Init Billing day to those who celebrate. Fix your cold starts!
NEW: Lambda can now send up to 200mb payloads using response streaming! I assume this is mostly directed at LLM inference workloads, where chatbots can stream large amounts of data over the wire as it becomes available.
The faster your cold starts are, the cheaper these will be!
This includes full Datadog tracing/logs/metrics by the way.
There's no compromising on observability, not even cold starts.
Lambda now charges for init time, so it's useful to count sandboxes which are proactively initialized but never receive a request.
Here's what happens after a 10k request burst. Hundreds of sandbox shutdowns, along with 22 sandboxes which were spun up but never received a request.