pola-rs/polars

What it does

Polars is an analytical query engine for DataFrames and SQL, written in Rust. The DataFrame expressions and SQL strings share one optimizer and execution engine, so you can write either style or mix them in a single query readme:L47-L64. It targets data analysts, engineers and Python/Rust/Node.js/R users who need fast local analytics on medium-to-large datasets.

Core characteristics:

• Multi-threaded, vectorized execution, with optional NVIDIA GPU acceleration.
• Lazy queries optimized before execution.
• Out-of-core processing: operations that exceed RAM can spill to disk.
• Apache Arrow columnar format for zero-copy interoperability.
• Native extension through I/O and expression plugins readme:L60-L63.

Quick start

Python is the shortest entry point. Install the package:

pip install polars

Then build a lazy query and collect it:

import polars as pl

df = (
    pl.scan_parquet("orders.parquet")
    .filter(pl.col("status") == "shipped")
    .group_by("customer_id")
    .agg(
        pl.col("amount").sum().alias("total"),
        pl.len().alias("n_orders"),
    )
    .sort("total", descending=True)
    .collect()
)

This returns a DataFrame with one row per customer, sorted by total amount. scan_parquet creates a lazy scan, so the filter, aggregation and sort are optimized and executed in parallel readme:L67-L87. SQL is available through pl.sql(...), and its result is a regular LazyFrame, so you can continue with expression methods readme:L107-L123. See the SQL user guide for more.

Installation and build options

Beyond pip install polars, optional dependencies are controlled by feature flags described in the installation guide. If you need a bleeding-edge build, compile from source: install Rust, install maturin, then run a make build* target inside py-polars. The build targets trade compile time against runtime performance and debug symbols; LTS_CPU=1 is intended for older CPUs without AVX2 readme:L144-L176.

Constraints

• APIs are exposed for Python, Rust, Node.js and R, but the Python bindings wrap a separate crate named py-polars while both the Python package and module are named polars readme:L144-L176.
• Out-of-core execution does not cover every operation; unsupported queries fall back to the in-memory engine, which can exceed available memory comment by ritchie46, 2022-12-22.
• GPU acceleration is optional and limited to NVIDIA GPUs readme:L60-L63.
• The project is MIT licensed readme:L192-L194.

HotRepo's take

The repository had 40,316 stars and 3,235 forks at the recorded 2026-11-11 Rust daily snapshot, adding 17 stars that day and reaching 7th on the Rust daily list. Hacker News discussion around the 1.0.0 release framed it as more than a pandas alternative, with one commenter noting the in-memory limits before needing something like Spark keep moving higher comment by andenacitelli, 2024-07-02. That matches the project's emphasis on lazy optimization and disk spilling rather than only raw benchmark speed.