Posts about Case Studies

Github and Manifest found that model routing was ineffective, but they were measuring the wrong thing. They focused on a simple prompt instead of expanding that prompt to better understand the exact nature of the task and the environment. In the very early data we have, we're seeing dramatically better results.

Model routing doesn't work if you only measure cost-per-token. It only works if you measure cost-per-task. Uber measured this and provides some great data points.