apache/pulsar

Apache Pulsar is worth considering when you need one messaging system to cover both work queues and event streaming, and you accept running a JVM-based cluster with its own storage layer. It is not a drop-in for a small project; the operational surface is the main decision point, not the feature list.

The problem

A team starts with a queue for background jobs, then adds a separate log for event replay, then needs cross-region copies of some topics. Each system brings its own client libraries, retention rules and administration. Pulsar consolidates these into one cluster: producers publish to topics, consumers either share a subscription (queue semantics) or each read the full stream, and the broker tracks consumer cursor positions so replay is a configuration choice rather than a rebuild. The result is one provisioning API and one tenancy model instead of several.

What's different

Pulsar separates serving from storage. Brokers handle connections, topic ownership and dispatch, while durable data sits in BookKeeper bookies; the README also lists geo replication, transparent partitioned topics, batching and a load balancer as core features project instructions. In practice this means a broker is mostly a caching and routing layer, so expanding capacity for connections and capacity for storage can be planned separately. Multi-tenancy, authentication, authorization and quotas are designed in from the start, which matters if you host messaging for several teams or customers rather than running one application's bus.

The trade-off is that the boundary between broker and storage is not free. A 2020 HN commenter noted the appeal of stateless brokers but argued that the surrounding JVM dependencies create administration overhead for companies that are not operating at very large scale comment; a reply pointed out that BookKeeper nodes remain stateful, so replacing one still involves data redistribution reply. Treat that as an architectural warning about where the operational work moves, not as a verdict on the current release.

Who should try it

Good fit: a platform team already running Kubernetes and expecting multiple tenants, durable replay, or geo-replication, where one system with queue and stream semantics reduces the number of moving parts to operate. Also a fit for organizations that need per-topic retention and cursor management without building it themselves.

Poor fit: a single application that just needs a lightweight job queue or a small pub-sub bus, or a team without someone who can own a stateful storage tier. A 2020 commenter said they prefer RabbitMQ in production and Redis for small projects comment; that reflects a different scale and operational appetite. Starting from the Docker image container image or a Helm chart Helm chart is documented, but those paths change installation, not the need to understand the broker/bookie split.

Requirements and limitations

Version and JDK are coupled: master targets JDK 21, 25 or 26, the 4.0+ line targets JDK 21, and older lines need JDK 17 or below version table. Upgrading Pulsar therefore often means upgrading the runtime first. Building needs a JDK and zip, plus the documented build tooling; the Gradle wrapper removes a separate Gradle install setup guide.

Ongoing upkeep is the larger cost. You are responsible for broker and BookKeeper capacity, topic and subscription cleanup, quota policy and client-version compatibility across the separate language clients client list. The README points to other repositories for connectors, adapters and clients, so those components do not necessarily move in lockstep with the core. There is no one-command way around planning for stateful storage nodes and their data movement.

HotRepo's take

HotRepo recorded one daily-board appearance on 2026-10-25 with 15,353 stars and +2 that day; the single dated snapshot cannot establish whether interest is rising, fading or sustained. The older HN thread (2020) shows discussion volume, not current consensus. Before committing, verify the JDK line for the exact release you plan to run and read the broker/BookKeeper operational guidance, since that architecture determines who must own the cluster.