Posts tagged "cost-per-task"

There's a ton of focus on reducing token costs or using cheaper models. These are directionally correct but not focused on the right north star. Cost per task, particularly successful tasks that don't get reworked, is the only thing that matters.

Github and Manifest found that model routing was ineffective, but they were measuring the wrong thing. They focused on a simple prompt instead of expanding that prompt to better understand the exact nature of the task and the environment. In the very early data we have, we're seeing dramatically better results.

Model routing doesn't work if you only measure cost-per-token. It only works if you measure cost-per-task. Uber measured this and provides some great data points.