Member-only story
A Dummy’s Guide to Distributed Queues
If you’ve ever wondered what Kafka, Heron, real-time streaming, SQS or RabbitMQ are all about, then this article is for you. I’ll discuss in detail why we need a queue for today’s modern software architecture, what are some common technologies used, and how queues are commonly used in the industry. If you enjoy this article, I have a course on scaling distributed systems where I discuss these topics in more detail.
OK, let’s get into it!
First and foremost, why do you need a queue/message broker?
The story of how a queue saved the lemonade stands
Imagine that you are running a lemonade 🍋 stand, and you built out a nifty little web-app that keeps track of how often your clients return to your lemonade stand.
Your web-app has an endpoint, say yourlemonade.com/traffic, and every time you click on a button, the traffic count goes up by 1. Beautiful.
As the traffic to your lemonade stand increases, you click the button more and more. Well, since you’re living in a relatively small neighborhood, you only get 10–20 people per day. Your sales proceed as usual, the web-app handles the traffic just fine, and everything is fine and dandy. Perfect.
