Event-Driven Microservices with NATS JetStream
Selecting a messaging system for microservice communication involves balancing throughput, footprint complexity, and persistence guarantees. Heavy enterprise message brokers (such as Apache Kafka or RabbitMQ) require significant memory footprints, complex Zookeeper/KRaft clusters, and dedicated operations management.
NATS JetStream is a lightweight, ultra-fast, cloud-native messaging engine built into the single NATS binary (written in Go). JetStream extends core NATS publish/subscribe with built-in Message Persistence, At-Least-Once / Exactly-Once Delivery, Durable Consumers, and Distributed Key-Value Stores. With a sub-20MB binary memory footprint, NATS JetStream processes millions of messages per second with microsecond latency. This guide details JetStream stream subjects, consumer ACK policies, embedded KV storage, and Kubernetes cluster deployments.
Mental Model: Core NATS Pub/Sub vs JetStream Persistent Message Streams
Core NATS operates as an in-memory 'fire-and-forget' publish-subscribe system. If no subscriber is actively connected when a message is published, the message is dropped. While ideal for high-frequency metrics or live presence pinging, core pub/sub cannot guarantee delivery for critical business events.
NATS JetStream adds built-in message persistence and state management.
JetStream captures messages matching defined subject wildcards (e.g., ORDERS.*) and writes payloads to file or RAM storage streams. Subscribers create Durable Consumers that track message offset progress, allowing microservices to replay past events or recover offline state safely. For event-driven messaging patterns, review building event driven microservices kafka schema registry and building high throughput apis go gin framework.
Quick reference
- Single lightweight Go binary with sub-20MB RAM footprint executing pub/sub and persistence.
- JetStream adds message persistence, replay buffers, and consumer offset tracking.
- Delivers microsecond-level message dispatch latency across distributed microservices.
- Supports subject wildcard routing (e.g., ORDERS.created, ORDERS.shipped).
- Powers cloud-native infrastructure at Siemens, Rivian, MasterCard, and Synadia.
Remember this
Use NATS JetStream for lightweight, low-latency persistent message streaming between microservices.
Streams, Consumers, & Acknowledgement Policies (Explicit vs All)
JetStream introduces key architectural abstractions:
1. Stream Configuration: Defines message retention policies, limits (max age, max bytes), and storage backends (File vs Memory):
1js.AddStream(&nats.StreamConfig{2 Name: "ORDERS",3 Subjects: []string{"ORDERS.*"},4 Storage: nats.FileStorage,5 Replicas: 3,6})2. Durable Consumers & ACK Policies: Microservices pull or push messages from streams. JetStream supports three Acknowledgement (ACK) modes:
- AckExplicit: Consumer must acknowledge each message individually.
- AckAll: Acknowledging message #50 implicitly acknowledges messages #1 through #49.
- AckNone: No acknowledgement required (high-speed processing).
Quick reference
- StreamConfig maps subject wildcards (ORDERS.*) to persistent File or Memory storage engines.
- Replicas parameter replicates stream messages across 3 NATS cluster nodes via Raft.
- Durable Consumers maintain consumer offsets across application restarts or deployments.
- AckExplicit guarantees at-least-once processing by requiring explicit worker ACK responses.
- MaxDeliver parameters route un-acknowledged failing messages to Dead-Letter subjects.
Remember this
Configure durable consumers with AckExplicit policies to guarantee at-least-once message delivery.
Distributed Key-Value & Object Stores Built into NATS
Beyond message streaming, NATS JetStream includes built-in Distributed Key-Value (KV) and Object Store engines built directly on top of JetStream streams:
- NATS KV Store: Exposes a low-latency key-value API (kv.Put("feature-flag", []byte("true"))) with watch listeners (kv.Watch()) that push real-time key mutation updates to microservices.
- NATS Object Store: Handles large file chunking (e.g., 50MB PDF invoice storage) across NATS cluster nodes, eliminating the need to deploy separate Redis or S3 infrastructure for microservice asset caching.
Quick reference
- NATS KV Store provides real-time key-value storage with reactive mutation watch listeners.
- NATS Object Store streams large binary files in chunked frames across NATS cluster nodes.
- Eliminates third-party infrastructure dependencies (like Redis or MinIO) for simple microservices.
- Built-in revision history tracks key modification history and tombstone deletions.
- Secured using NATS Decentralized User Authentication (NKEYs & JWT tokens).
Remember this
Use NATS KV and Object Stores to simplify microservice architecture without extra infrastructure.
Deploying High-Availability NATS Clustered Superclusters in Kubernetes
Production NATS deployments run as Clustered Superclusters using the NATS Helm Chart or Kubernetes Operator:
1. Local Raft Clustering: A 3-node or 5-node NATS cluster forms a Raft consensus group, replicating JetStream streams and KV stores across availability zones. 2. Superclusters (Multi-Region Gateways): Connects distinct NATS clusters across geographic regions (AWS us-east-1 and eu-west-1) via encrypted gateway connections, enabling global event routing with local message delivery speed.
Quick reference
- Raft consensus protocol manages leader election and stream data replication across 3+ nodes.
- NATS Helm Chart deploys statefulsets with automated peer discovery via Kubernetes DNS.
- Superclusters route messages across multi-cloud regions over encrypted TLS gateway links.
- Leaf Nodes allow edge devices or remote branch offices to cache events offline and sync on reconnect.
- NATS CLI (
nats stream ls,nats sub) provides powerful cluster administration capabilities.
Remember this
Deploy 3-node NATS Raft clusters and Gateway Superclusters for multi-region fault tolerance.
Key takeaway
To test NATS JetStream, launch a server via Docker (docker run -p 4222:4222 nats -js). Use the NATS CLI (nats stream add ORDERS) to create a stream and publish events (nats pub ORDERS.created '{"id": 101}').
Related Articles
Explore this topic