ifixai-ai/iFixAi

What iFixAi Does

iFixAi is an open-source auditor for operational alignment of AI agents. It builds a simulation of a business, sends an agent the requests a real day might bring—including requests that exceed its role—and has a judge from a different model provider grade each reply. The goal is to answer whether an agent does what it is supposed to do within the organization's intentions, goals and permissions, rather than only checking model capability or latency.

The inspections combine red teaming, governance, assurance and alignment checks. Each result pairs a pass or fail with evidence showing what was asked and what the agent did. Reports are written in JSON and Markdown, and the overall A–F grade comes from five pillars: fabrication, manipulation, deception, unpredictability and opacity. Three mandatory minimums (B01, B08, P01) cap the grade at D if failed. The default run executes 10 chat-based inspections; --full-run adds the remaining inspections for 61 total, setting aside checks that cannot apply.

It suits teams deploying customer-facing or tool-using agents who need an independent, repeatable check against business rules, and developers who want to gate agent changes in CI.

Quick Start

Python 3.10 or newer is required. The demo needs no API key and runs on a built-in mock agent with canned replies and grades.

python3 -m venv .venv && source .venv/bin/activate
pip install ifixai
ifixai author reference -o reference.yaml
ifixai validate reference.yaml
ifixai run --provider mock --eval-mode self --fixture reference.yaml

On Windows use py -m venv .venv and .venv\Scripts\activate; if the ifixai command is not found, run python -m ifixai.cli.main. A successful demo prints a scorecard and writes reports to ./ifixai-results/.

To audit a real agent, copy the reference profile to my-agent.json, edit it or let the coding-agent plugin generate it, then build and validate the fixture. Install the judge extra you will use and run the setup wizard before a dry run:

pip install "ifixai[openai]"
ifixai author my-agent.json -o my-agent.yaml
ifixai validate my-agent.yaml
ifixai setup
export AGENT_API_KEY=...
ifixai run --fixture ./my-agent.yaml --api-key "$AGENT_API_KEY" --dry-run
ifixai run --fixture ./my-agent.yaml --api-key "$AGENT_API_KEY"

The dry run shows the number of tests and judge calls and why some checks do not apply, then stops before billing. Any chat endpoint can be audited; OpenAI-compatible endpoints work directly, while other protocols require a short Python adapter described in testing your agent. Judges can come from OpenAI, Anthropic, Gemini, Azure, Bedrock, OpenRouter and others, selected from configured keys and never from the agent's own provider.

Claude Code and Codex users can install the plugin through their plugin marketplaces, while Cursor, VS Code, Windsurf, Zed and similar editors can install the skill with uvx ifixai install --agents cursor and then invoke /ifixai-skill. See the plugin guide and get started for the agent-driven workflow. In scripts or CI, the equivalent single command uses --provider http --endpoint <agent-url> and an optional --min-score.

Constraints

• Audits send real adversarial requests to the target endpoint, so point it at a staging copy rather than production data.
• The agent endpoint must accept chat requests; non-OpenAI-compatible APIs need a Python adapter.
• Judge keys for at least one supported provider are needed for real runs; the mock demo does not require any.
• The default run covers 10 inspections over chat, while action-level and recorded-trajectory checks require other suites or the trajectory-assurance workflow.
• The license is Apache-2.0.

HotRepo's take

iFixAi entered the all-language daily list in October 2026 with 2,143 and 2,109 new stars on consecutive days, reaching roughly 26,800–28,000 stars, and appeared at position 6 on the Python daily list on 2026-11-01 with 1,654 stars gained in that period. Those are short, non-contiguous observations, but they show sustained attention during that window. The Hacker News threads had minimal discussion (one thread with a single comment), so there is little public debate to weigh; the project's own claim of answering in under 120 seconds should be read as a product statement rather than an independently verified benchmark. The more concrete differentiator is the design itself: grading from a separate provider and requiring evidence for each inspection.