Posts tagged "model-routing"

Every Developer Gets Their Own Router. Nobody Else Gets Smarter.

There's a ton of focus on reducing token costs or using cheaper models. These are directionally correct but not focused on the right north star. Cost per task, particularly successful tasks that don't get reworked, is the only thing that matters.

Github and Manifest found that model routing was ineffective, but they were measuring the wrong thing. They focused on a simple prompt instead of expanding that prompt to better understand the exact nature of the task and the environment. In the very early data we have, we're seeing dramatically better results.

Model routing doesn't work if you only measure cost-per-token. It only works if you measure cost-per-task. Uber measured this and provides some great data points.

OpenRouter moves your inference call. Hokusai decides which call to make — on your objective, with the reasoning visible, and with a stake in the policy your data improves. A plain comparison of gateways and decision layers.