Skills for your AI assistant
Add a skill and Claude Code, Codex, Cursor or a similar assistant learns one more job.
Evals and monitoring
33 skillsTracing, evaluating and observing LLM apps, and managing prompts.
- skillevaluator287 installsRuns SkillEvaluator to score a SKILL.md, write test cases, explain reports and gate skills in CI.
- evaluating-llms-harness68 installsBenchmarks language models on standard tests like MMLU and GSM8K.
- dspy26 installsWrites and auto-optimizes DSPy programs for prompts and RAG pipelines.
- writing-instructions19 installsRewrites prompts and instructions so the model actually follows them.
- create-custom-grader8 installsConverts an existing benchmark, rubric, verifier or task file into a SkillEvaluator custom evaluation.
- agent-evaluation1 installsRuns one test agent on one benchmark case once, scores it privately and returns a single protocol result.
- agent-optimization1 installsImproves an agent's state using versioned scores and linked traces from a fixed benchmark.
- evolving-with-swarms1 installsImproves a program against your scorer by running cheap model swarms within a dollar cap.
- cli-evalCreates and runs evaluation suites, tracks benchmark progress, and compares model scores from the command line.
- cli-healthChecks server health, component status, and live metrics with a real-time dashboard from the command line.
- agent-evaluationBuilds an evaluation system for your agent, covering metrics, test environments, rubrics, failure analysis and regression checks.
- eval-dataset-designDesigns agent evaluation tasks and datasets, including validators, difficulty tiers, leak prevention and long-term upkeep.
- agent-prompt-quality-barChecks an agent system prompt against a fixed rubric of contracts, fallbacks and anti-patterns.
- setupSets up MLflow tracing for Claude Code.
- statusShows the current MLflow tracing configuration for Claude Code.
- evalsExplains the Kortix eval system for writing benchmark tasks, running trials and reading scores.
- agent-testingTests an agent before publishing it, suggesting questions to ask, failures to watch for and fixes to make.
- releasing-agenticos-resourcesTakes an agent through datasets, evals, published versions, staging, promotion and rollback.
- agent-observabilityAdds local hook-based metrics for tool latency, token usage, session cost and multi-agent traces.
- verify-secretsTests whether text marked secret can leak back through replies, errors, traces or written files.
- darwinian-evolver16 installsEvolves prompts, regex, SQL or code with Imbue's evolution loop.
- benchmark-design1 installsDesigns and calibrates a multi-case capability benchmark and sets a traceable formal baseline.
- cli-policy-auditInspects audit logs, access policies, telemetry, and request history for compliance from the command line.
- omni-usage-logsFilters call logs by provider, model, time, status and cost, then exports and aggregates token usage.
- ace-rsi-toolkitRuns the ACE method to build a playbook of rules from training traces and measure it on a benchmark.
- ape-rsi-toolkitReproduces the APE method to induce instructions from examples, measure each and keep the best.
- awm-rsi-toolkitRuns the AWM method to learn reusable workflows from an agent's successful runs and measure them on a benchmark.
- openskill-rsi-toolkitLearns reusable skills with OpenSkill by retrieving domain knowledge, refining them with virtual tests and exporting the latest valid version.
- opro-rsi-toolkitReproduces the OPRO method to propose instructions from scored history, measure each and keep the best.
- rrsi-rsi-toolkitRuns a recursive self-improvement experiment on an agent harness with edit budgets, leakage checks and pruning.
- skilladam-rsi-toolkitEvolves a reusable Markdown skill from execution traces and feedback using an issue tracker and edit budget.
- skillopt-rsi-toolkitImproves a reusable skill through trajectory reflection, bounded patches and selection on fixed evaluation tasks.
- trace-rsi-toolkitEvolves a modular skill bank from actor trajectories and curator distillation, then picks a checkpoint by score.