Coursify

System Design for Software Engineers

Pub/Sub vs. Message Queuing: Patterns of Distribution

Pub/Sub vs. Message Queuing: Patterns of Distribution

While both patterns involve sending messages asynchronously, they serve different architectural purposes. Choosing the right one depends on whether you want a message to be processed by exactly one consumer or every interested consumer.

1. Message Queuing (Point-to-Point)

In a traditional queue, one message is delivered to one consumer. If multiple consumers are listening to the same queue, they will compete for messages (Competing Consumers pattern).

  • Behavior: Work is distributed. Each message is processed exactly once by whichever consumer is free first.
  • Analogy: A ticket counter. Each customer (message) is served by one clerk (consumer).
  • Use Case: Task processing, order fulfillment, image resizing.

2. Publish/Subscribe (Pub/Sub)

In a Pub/Sub model, a producer "publishes" a message to a Topic. Multiple services can "subscribe" to that topic. When a message is published, every subscriber receives a copy of that message.

  • Behavior: Information is broadcast. One message triggers multiple independent actions.
  • Analogy: A newspaper. One issue (message) is sent to every subscriber (consumer).
  • Use Case: Real-time notifications, updating multiple microservices after a user signs up, logging/auditing.

Designing a Fan-Out Architecture

  1. 1
    Step 1

    Suppose your 'Order Service' just completed a sale. This is a significant event that multiple other parts of your system need to know about.

  2. 2
    Step 2

    Instead of the Order Service calling 5 other APIs, it simply publishes a NewOrder message to a Pub/Sub Topic (e.g., using Google Cloud Pub/Sub or AWS SNS).

  3. 3
    Step 3

    Each downstream service creates its own subscription to the topic:

    • Shipping Service: Subscribes to start the delivery process.
    • Inventory Service: Subscribes to decrement stock counts.
    • Email Service: Subscribes to send a receipt to the user.
    • Analytics Service: Subscribes to update sales dashboards.
  4. 4
    Step 4

    All 4 services receive the message at the same time and process it independently. If the Email Service is slow, it doesn't delay the Shipping Service.

  5. 5
    Step 5

    For high-reliability, each subscriber often has its own Queue behind the subscription. This combines the broadcast power of Pub/Sub with the reliability/retries of Message Queuing.

Comparison Table

FeatureMessage QueuePub/Sub
DeliveryPoint-to-Point (1:1)Broadcast (1:Many)
Consumer RelationshipCompetingIndependent
Best ForTask DistributionEvent Notification
Primary GoalEfficiency / ThroughputDecoupling / Extensibility

Common Mistakes

  • Using Pub/Sub for Tasks: Publishing a 'ResizeImage' message to a topic where 3 identical workers are subscribed. All 3 workers will resize the same image, wasting CPU. Use a Queue instead.
  • Tightly Coupled Subscriptions: Designing subscribers that depend on each other. Pub/Sub works best when subscribers are completely independent.
  • Ignoring Delivery Guarantees: Assuming Pub/Sub is always "at-least-once." Some simple Pub/Sub systems (like Redis Pub/Sub) are "fire-and-forget" and will lose messages if a subscriber is offline.

Recap

  • Message Queues are for Work Distribution (one message, one worker).
  • Pub/Sub is for Event Distribution (one message, many workers).
  • Fan-out is the pattern of using Pub/Sub to trigger multiple downstream actions.
  • You can combine both by putting a queue behind every subscription.

Knowledge Check

Question 1 of 3
Q1Single choice

If you have a 'Payment Service' that needs to trigger both an 'Invoice Service' and a 'Reward Points Service' simultaneously, which pattern should you use?

Previous

Next