inference-optimisation16 installs
Measures and tunes local LLM serving speed, testing changes one at a time and keeping a record of results.
Guides an agent through measuring and speeding up a locally served LLM on your own hardware.
Measures and tunes local LLM serving speed, testing changes one at a time and keeping a record of results.