AWS Lambda now supports up to 10 GB Ephemeral Storage
aws.amazon.comIf it's ephemeral to a function, it's not "storage". It's just core memory that may or may not be backed by physical "storage" media, and is organized as a tmpfs for the user's convenience.
One man’s swap-backed tmpfs is another man’s page-cache-backed filesystem. /s
I remember when I was breaking my head against the wall to fit puppeteer inside a lambda. Good times.
What was the use case?
I was working for a programmatic advertising demand side platform. We had customers that submitted all sorts of ads to us and the lambda was in charge of rendering the ad and performing some validations on it. We accomplished it in the end using layers. It was in 2017, but it feels like a lifetime ago.
this is fantastic for anyone wanting to run Chrome inside of Lambda (one of the biggest challenges is getting Chrome to fit)
What's the use case? I see a sibling commenter mentioned puppeteer which I'm aware is essentially Chrome. Are you too referring to running puppeteer in Lambda? If so, what is the purpose?
Puppeteer-core can already fit in a lambda around 60mb and can be used for web scraping, taking screenshots, or performance testing.
How’s the overhead? Are you launching full chrome every lambda invocation?
In one lambda I can load chrome, visit a website, take a screenshot and save it to S3 in around 2s with around 280mb memory used. You could keep chrome open and reuse it if available, but you can't count on it with lambda being stateless.
I use this Chromium binary https://www.npmjs.com/package/chrome-aws-lambda
> lambda being stateless
/tmp is ephemeral in the sense that it dies with the Lambda instance, but it is also persistent between multiple serial invocations landing on that instance, even if the instance is frozen and thawed between some of those invocations.
The process memory state on the instance (set up during the Init phase) is also available between invocations.
https://docs.aws.amazon.com/lambda/latest/dg/runtimes-contex...
You can't count on that freezing and thawing process, it's not guaranteed at all. The whole point of Lambda/FaaS in general is that each invocation might be truly ephemeral, depending on factors that are largely beyond any single user's control.
this package requires using JS for your stack. your options are much slimmer for other languages.
Does that work outside of JS?
Chrome headless
scalable pdf rendering
This is also useful for deploying a big static website with AWS CDK which currently uses a Lambda and had the limit of 512mb (it was already possible to pass it by using an attached storage but this makes it easier).