eliasbiondo/antirion

Antirion is worth a look if you already call more than one LLM provider and want the routing, spending caps and request history in your own infrastructure rather than in another vendor's dashboard. The README's setup path is deliberately short, but the feature surface implies real operational commitment.

The problem

An application that calls OpenAI, Anthropic and Gemini usually ends up with provider-specific keys scattered across services, ad hoc retry and fallback logic in each codebase, and spending that can only be reconstructed by reconciling several provider invoices. Antirion moves that per-call work to one endpoint: applications call the gateway's /v1 API, and the gateway selects a provider, applies keys and limits, and records the result. Afterwards a team can see spend, latency, errors and fallback attempts per model, key, team or person in a built-in console, instead of chasing each provider separately.

What's different

The gateway, its Admin API and the web console ship in one Rust binary or distroless image, with no separate UI deployment or sidecar. The README reports a benchmark on dedicated AWS c7i machines with one key checked per request: at 5,000 requests per second on 4 vCPUs, median gateway overhead was 0.53 ms and p99 0.95 ms, compared with 19.7 ms and 95.2 ms for Bifrost 2.2.4; LiteLLM 1.103.1 has no overhead figure in the excerpt. These are the author's measurements under a minimal setup with no logging or cache, so treat them as an indicator of the design goal rather than a guarantee for a configuration with guardrails and full request logs enabled.

The console is the other differentiator: request logs include route, attempts, tokens, cost, guardrail results, and an option to store content. That makes fallback between providers inspectable after the fact, rather than something you infer from application errors. The trade-off is that such a gateway becomes a shared dependency for every AI call; if it is unavailable, callers must handle that, and the README does not spell out a bypass path.

Who should try it

Good fit: a small platform team running several products against two or more providers that wants per-team or per-key budgets and a single place to see what was called and what it cost. The console's invite flow and per-person accounts also suit a team that needs shared visibility without sharing raw provider keys. Poor fit: a single-service project using one provider and one key, where the gateway adds a hop and an admin surface for little gain, and anyone who cannot run a container or chart. The README's documented path requires Docker and an OpenAI or Anthropic key to start; a Helm chart exists for Kubernetes deployments, but setup still assumes container infrastructure.

Requirements and limitations

The quick start needs Docker, an API key from OpenAI or Anthropic, and a volume for /var/lib/antirion. The first start prints an admin token once and stores it in the volume; losing it means recovering it from the container's data directory, so plan how that secret is retained. The standalone node uses an embedded store; the cluster path depends on PostgreSQL and Valkey and needs a bootstrap token of at least 32 characters plus an encryption key. Release images are published per vX.Y.Z tag, so upgrade cadence is something to decide deliberately. The README excerpt does not cover the full provider-support matrix, guardrail configuration or cost-accounting accuracy, so verify those against your own providers before committing.

HotRepo's take

HotRepo began tracking the project on 2026-10-24 and recorded 232 total stars, #23 on the NEW-REPOSITORY board (not the overall Trending board) for Rust on 2026-10-25. That is a single dated snapshot, with no comparable daily-board history, so persistence of interest cannot yet be judged. Before adopting, check the performance section and reproduce the benchmark with your own logging and guardrail settings enabled, since the published numbers use a minimal setup.