klauspost/compress

A library worth considering whenever a Go service spends noticeable time compressing or decompressing data. For most teams the adoption cost is one changed import path, but the statistics quoted in the README are the author's benchmarks, not independent measurements for your data.

The problem

A Go service that compresses HTTP responses, writes zip archives or stores compressed blobs calls the standard library and hits a default throughput it did not choose. The usual fix is either to accept the overhead or to swap in a different compression format, which means changing serialization code, stored files and any consumer of that data. This project keeps the same formats and API surface, so a team can change the import path and keep reading and writing identical gzip, zlib, zip or flate data. The result is the same compressed bytes format with different speed characteristics.

What's different

There are three distinct offerings in one module. First, the deflate packages are documented as drop-in replacements for gzip, zlib and zip, with a typical speed of about 2x the standard library in the author's own numbers; users can also disable assembly or unsafe via build tags when a platform or policy requires it. Second, there is a pure-Go zstandard implementation plus S2, described as a high-performance replacement for Snappy, and a snappy package that is a drop-in replacement for github.com/golang/snappy. Third, lower-level entropy coders and decoders exist for narrower use cases, such as huff0, FSE, lzw and Microsoft XPRESS decoding for WIM/Compact OS data in xpress.

The main trade-off is that "drop-in" here means API-compatible, not universally identical in behaviour. The newest release shows the kind of edge cases that matter: reused zstd frame decoders now apply reset options, truncated Huffman extra bits now report unexpected EOF, and a caller-header bug was fixed in gzhttp release notes. Stateless gzip compression is explicitly called out as suboptimal for size and speed and intended only for very many mostly idle compressors. If you depend on byte-identical output, exact error strings or a particular memory profile, those are the things to check rather than assume.

Who should try it

Good fit: a Go backend that already compresses HTTP traffic with the standard gzip packages, or a service reading Snappy or zstd data where throughput matters and the format must stay the same. Also a good fit if you need a pure-Go zstd path without cgo, or want to strip assembly and unsafe with build tags for portability reasons. Poor fit: you need a different format entirely, need a written independent benchmark against your exact payloads before switching, or work outside Go, since this library is Go-only. Another limitation is that it does not make parallel gzip compression automatic; the README points to the separate pgzip project for multithreaded compression on big files.

Requirements and limitations

The package installer command is in the README and setup is trivial for an existing Go module. The real constraint is compatibility: the project supports the current Go version and two versions back project instructions, so older toolchains can leave you pinned to an older release. The optimized paths rely on assembly and unsafe by default; the noasm and nounsafe tags exist, but they are an escape hatch, and the README does not promise the same speed with them. Ongoing upkeep is release tracking rather than configuration. The project has a steady changelog: recent fixes include zstd decoder reset behaviour, flate truncated-input handling and a gzhttp transport header issue release notes. Expect to read such notes when bumping versions, especially if you use compressed data across service boundaries.

HotRepo's take

HotRepo began tracking this repository on 2026-05-10; the only recorded daily-board observation is 2026-10-27, when it had 5,702 stars and gained 3 stars that day. A single dated snapshot cannot show whether interest is rising or fading, so persistence cannot be judged from this evidence. For a decade-old library like this, one trending appearance matters less than its changelog discipline. Before switching, verify one compatibility point: whether your code depends on exact gzip/flate error behaviour or output bytes, since recent releases have changed the former.