TngSHoO/agentcase-lab

What it does

AgentCase Lab is a portfolio MVP for AI product teams that need to debug agents after they fail. A run is broken into a replayable trace covering user input, plan, tools, memory, model calls and final output, so reviewers can separate prompt problems from retrieval, context, permission or hallucination issues. The app also compares task status, success rate, latency, token usage and cost before and after a fix.

Five Streamlit pages provide a metrics dashboard, filterable failure cases, trace playback, root-cause analysis and version comparison. Root-cause analysis runs a rule-based demo by default, with an optional OpenAI-compatible model call when credentials are configured.

The built-in 30 cases are explicitly simulated data, not real model rankings or production results README.

Quick start

Python 3.11 or newer is required. On Windows PowerShell:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -r requirements.txt
streamlit run app.py

Open http://localhost:8501. On first launch the app creates the SQLite database and loads the 30 demo cases startup steps. The README only documents this Windows path; macOS and Linux users need the equivalent python -m venv and activation commands.

To use live AI root-cause analysis, copy .env.example to .env and set AGENTCASE_API_KEY, AGENTCASE_API_BASE and AGENTCASE_MODEL to an OpenAI-compatible endpoint optional AI setup. Keep the key out of version control.

For tests, install the dev requirements and run pytest plus ruff check . test commands.

Constraints

• The product is a portfolio MVP with simulated case data; its metrics do not describe any production agent.
• Runtime state lives in a local SQLite database generated under data/, so results are local to a machine.
• Real LLM analysis depends on an external OpenAI-compatible service and a valid key; without it only the rule-based demo works.
• The development notes state that AI coding tools assisted implementation development notes.
• MIT licensed; reuse and commercial use must retain the author and license notice license.

HotRepo's take

The repository appeared on the HotRepo new-project list for Python on 2026-11-06, with 8 stars gained in a roughly two-hour window and 108 stars at the end of that observation, alongside 2 forks. No GitHub daily rank is recorded, and the available sampling spans only a few hours, so it supports a fact about that day rather than a trend. The design treats agent observability as a review workflow — trace, classify, compare — which fits product teams more than platform engineers building their own instrumentation. Since the shipped cases are simulated and the project is self-described as an MVP, the useful signal is the workflow and documentation structure rather than any benchmark result.