AWS Developer Forums: S3 Block Devices

Did anyone not try some form of S3 hackery back in 2007?

We did for streaming media, due to a much better ratio of overhead to blocks for the content. And actually, we didn’t use blocks so much as multi-part byte ranges (RANGE header in GET request).

Importantly, we were also able to skip the OS, put a shim directly in the media server that would retrieve blocks or byte ranges prior to the server needing to send them. This way, our distributed edges didn’t need to use local disk for anything but the “hot” content, while “cold” content could source from S3 — while still supporting arbitrary seek by the viewer.

To an extent, think of the media server plus S3 byte range shim as “CloudFront for Streaming Video”, back in 2007. (Later, HLS would make this all much easier, enabling traditional CDNs to take on video delivery as well.)

While it worked a treat, didn’t take long to deprecate for most content, as we needed to be able to sell petabytes VOD media storage for less than AWS charged for S3. We ended up using it as a pre-warm tier, such as for a VOD movie launch, to relieve work from hub data centers.

This thread is from before EBS (Elastic Block Store) existed, and long before S3 Storage Gateway existed. It doesn't seem to have much relevance today.

Please mark as 2007

I've always understood that trying to use S3 as a block store (or as anything other than an object store) is an extremely bad idea. Has this changed?