apache/ozone

Project Overview

Apache Ozone is a scalable, redundant, distributed object store built for Hadoop and cloud-native environments. It is designed to hold billions of objects of varying sizes and runs well in containerized setups such as Kubernetes and YARN.

Key traits relevant to deployment decisions:

• Multi-protocol access: exposes both S3 and Hadoop File System APIs, so existing S3 tools and Hadoop workloads can point at the same store.
• Strong consistency: uses RAFT-based protocols rather than eventual consistency.
• Fault tolerance: a fully replicated design intended to survive multiple failures.
• Security: Kerberos authentication, native ACLs, Ranger integration, and TDE plus on-wire encryption.

It targets data analytics and object-store workloads, and is a top-level Apache Software Foundation project.

Quick Start

The fastest path is the Docker Compose sample cluster. You need Docker with the Compose plugin, and the AWS CLI if you want to exercise S3 access.

1. Fetch the sample Compose file:

curl -O https://raw.githubusercontent.com/apache/ozone-docker/refs/heads/latest/docker-compose.yaml

2. Start the cluster with three DataNodes:

docker compose up -d --scale datanode=3

The default replication factor is 1. Set OZONE_REPLICATION_FACTOR to change it.

3. Configure credentials. The sample cluster is unsecured, so arbitrary keys work:

export AWS_ACCESS_KEY_ID=testuser/scm@EXAMPLE.COM
export AWS_SECRET_ACCESS_KEY=c261b6ecabf7d37d5f9ded654b1c724adac9bd9f13e247a235e567e8296d2999

4. Create a bucket and upload a file through the S3 gateway on port 9878:

aws s3api --endpoint http://localhost:9878/ create-bucket --bucket=wordcount
ls -1 > /tmp/testfile
aws s3 --endpoint http://localhost:9878 cp --storage-class REDUCED_REDUNDANCY /tmp/testfile s3://wordcount/testfile

For a more realistic cluster, download the binary release from the Ozone download page, untar it, then run cd compose/ozone && docker compose up -d --scale datanode=3. The compose folder also contains secure, HA and MapReduce example configurations, and the binary package ships Kubernetes resource files.

To build from source, use Maven with mvn clean install -DskipTests; the README notes the Docker Compose cluster can then be started from hadoop-ozone/dist/target/ozone-*/compose/ozone.

Usage Constraints

• The Compose quick start runs with replication factor 1 and no security; it is a development/demo setup, not a production configuration. Production use requires planning replication, HA and Kerberos/ACL setup separately.
• S3 access requires pointing clients at the gateway endpoint (http://localhost:9878 in the sample) rather than AWS's default endpoint.
• Building from source requires Apache Maven and skips tests with the given command.
• The project is licensed under Apache 2.0.

HotRepo's take

On the Java daily list for 2026-11-12, Ozone held position 16 with 1,334 stars and 657 forks and no star change that day. A fork-to-star ratio near one half is unusual for an individual tool and fits a storage platform that teams deploy and adapt rather than simply star. The single recorded snapshot supports only that day's standing, not a growth trend.