Settings

Theme

100+ Lines of Code to Implement Kafka on S3

github.com

7 points by TopRainbowT 2 years ago · 2 comments

Reader

chenyang 2 years ago

Just like warpstream or confluent freight cluster, but AutoMQ is source available, you may find the key s3 wal implementation here: https://github.com/AutoMQ/automq/blob/main/s3stream/src/main...

TopRainbowTOP 2 years ago

AutoMQ claims this is the best design for streaming storage on the cloud: Shared WAL + Shared Object. This new shared storage architecture for streaming systems can leverage the advantages of different cloud storage services. They use only a few lines of code to implement features like building a Kafka service on S3. This blog shows how they achieve this. What do you think?

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection