osolmaz/switch

switch is a TypeScript framework for turning a task into a state machine that a decision model navigates by answering typed questions. Decision models like Jev return probabilities for a finite set of options rather than generating text, so they cannot emit the tool calls a coding agent relies on. switch builds those choices in code and the model selects one per step until no further questions arise. It suits people building eval judges, games, or other narrow finite-choice agents around such a model.

How a run works

You implement a domain that supplies an initial state, a question function, a transition function and an optional view. Each step, the question function produces bool, choice or score questions, the model answers with probabilities, and the transition function produces the next state. A run ends when there are no questions, on request failure, or at an optional step limit. The picking helper builds a choice question from a list of next steps. An optional outcome reports how a case ended, and every state, question and answer is written to a trace.

A concrete use is an eval judge: instead of extracting every page of a 200-page document into context, the model reads files, looks at page images and searches terms one step at a time, taking in only what a rubric item needs. The repository ships this as a rubric judge example, alongside chess and WikiRace examples in the documentation.

Using it

The core package, model adapter, CLI helper and test decider install from npm; Node.js 22.18 or newer is required. @osolmaz/switch/models provides piAiDecider, which works with any model pi-ai lists as a classifier, including Jev, and reads the API key from the environment. A fake decider in @osolmaz/switch/testing lets tests run without model calls. The examples and the browser-based viewer are not on npm and require cloning the repository and installing its pnpm workspace.

To turn a domain into a program, write a harness whose setup returns a runner, then wrap it with commandLine to get run and replay commands. That helper is a Commander program, so extra commands can be added. run writes a run folder with each case's result, trace and requests, and resumes a stopped run where it stopped. replay re-runs a case from its trace without the model; --fork <n> repeats the first n steps and then lets a model continue per the domain documentation.

Limits and model fit

The README is direct about model quality: Jev is weak at chess and, as a judge, is not yet good enough to replace the strongest chat models. The stated path to a stronger narrow agent is fine-tuning an open decision model on data generated in one domain, which the author expects to be better and cheaper for that job. Token budgeting and replay details are documented in the domain docs.

The project is MIT licensed. HotRepo recorded 236 total stars on its NEW-REPOSITORY board for TypeScript on 2026-10-30; this is a dated snapshot of a new-repository board, not a daily growth measurement.