Cost per task is the only hill worth climbing
There's a ton of focus on reducing token costs or using cheaper models. These are directionally correct but not focused on the right north star. Cost per task, particularly successful tasks that don't get reworked, is the only thing that matters.
Recent conversations about which models reduce cost per token are missing the point. The metric that matters to a business is cost per completed task.
It is good to see the market waking up to this.
Artificial Analysis now publishes cost per task as part of its Coding Agent Index. That is a meaningful change. Price per token is the price of a raw material. Cost per task is much closer to the cost of the finished product.
A model with a lower token price can still be the more expensive choice if it uses more tokens, makes more attempts, or produces work that needs more correction. Conversely, an expensive model can be economical when it completes a difficult task cleanly.
Once the unit of measurement becomes the task, most of the old model-pricing debate stops being useful.
But the public metric still has an important limitation. Artificial Analysis reports the average API cost of a benchmark attempt. A business needs to know the cost of a completed task whose output actually holds up.
The difference is rework, and rework is where margins disappear.
An attempt is not a completed task
The unitcostai coding-agent calculator gets this distinction right.
Its default scenario assumes a 72% acceptance rate. At that rate, producing 140 accepted tasks requires roughly 194 attempts. The calculator calls this a 1.39x rework multiplier.
That changes the economics considerably:
- Cost per attempt: $4.17
- Cost per accepted task: $5.80
Both numbers describe the same system. Only one describes what the customer is buying.
This is the agent equivalent of confusing manufacturing input cost with the cost of a finished unit. Scrap, defects, and rework still belong in cost of goods sold even when they do not appear in the final product.
The calculator makes acceptance rate a first-class input, which is exactly where it belongs. But somebody still has to enter that rate. A spreadsheet cannot tell you whether the work held up, how much corrective rework followed, or whether a different model would have produced a better business result.
That information has to come from the harness.
They built the calculator. We have the data.
Every production harness already generates evidence about what happened after a model was selected.
Did the task complete? Did the result hold up? How many additional attempts followed? Was a stronger model eventually needed? How much corrective rework accumulated before the task was accepted?
Most systems keep excellent records of token consumption and weak records of outcomes. That makes it easy to calculate the price of an attempt and surprisingly difficult to calculate the cost of useful work.
Hokusai is designed around the second number.
A Hokusai decision model predicts what is likely to hold up before the harness commits resources. It can return a risk score, recommend a model or advisor, and explain the reasoning behind that recommendation. The harness then reports the outcome, including whether the work held up and how much corrective rework followed.
Over time, acceptance rate stops being an assumption typed into a calculator. It becomes a measured property of specific decisions:
- Model A on this class of task
- Model B in this kind of repository
- An advisor used at this point in the workflow
- A cheaper worker operating below a particular risk score
- A stronger model reserved for cases where the expected improvement justifies its cost
That is the data required to improve cost per completed task.
The architecture is becoming clear
This does not mean using the cheapest model everywhere. That usually looks good only until rework is included.
The more interesting pattern is to separate the work. Use expensive capability where it has the most leverage and cheaper models for bounded execution that they can reliably handle.
UsageBox summarizes Anthropic's first-party multi-agent results. A Fable 5 orchestrator directing Sonnet 5 workers reached 96% of the all-Fable performance at 46% of the cost on BrowseComp. The split scored 86.8% instead of 90.8%, while costing $18.53 per problem instead of $40.56.
That is strong evidence for planner, worker, and advisor decomposition from a company that sells the models.
It is not yet a complete operating policy.
A benchmark can show that the architecture works in aggregate. It cannot tell an individual harness which tasks the cheaper worker can clear, which ones need a stronger planner, or where apparent savings will return later as corrective rework.
Those decisions depend on the harness's own task mix and reported outcomes.
This is a gross-margin problem
Tomasz Tunguz connected harness design directly to business performance in The Harness Margin Opportunity.
His example considers two companies selling the same $250,000 AI service. One uses a state-of-the-art model throughout the workflow. The other turns repeatable parts of the work into deterministic code and reserves expensive capability for the few decisions that need it.
The first spends $131,000 on inference. The second spends $37,000. After the same hosting, evaluation, and human costs, one operates at a 38% gross margin and the other at 75%.
Same customer. Same contract value. Same intended result. Very different business.
The difference also changes how quickly each company can grow. On a contract that costs $150,000 to win, Tunguz's lower-margin company takes nineteen months to earn back the acquisition cost. The higher-margin company takes ten.
Cost per completed task flows directly into gross margin, CAC payback, available operating capacity, and the amount a company can reinvest in growth.
This is why routing knowledge is more than an optimization feature. Knowing when a cheaper path will hold up becomes a business asset.
A competitor can copy the model list and reproduce the workflow diagram. It cannot immediately copy the outcome history that shows which route works for which task.
The cost advantage and the data advantage are the same thing.
One north star for every decision model
Hokusai will offer decision models for different parts of a harness, but they all serve the same business metric.
A router predicts which model is likely to complete a task successfully.
A risk model predicts where corrective rework is most likely.
An advisor helps the harness decide where additional capability is worth paying for.
Tool-selection and retrieval models improve other choices that affect whether the work holds up on the first path through the system.
Each decision changes either the total cost of the task or the probability that the task will be completed without another attempt. Cost per completed task captures both.
That makes it a much better north star than token spend. A token budget can be reduced by producing worse work. Cost per completed task gets worse when those savings create failures and rework.
It is also better than acceptance rate alone. A system can raise acceptance by using the most expensive model for everything. Cost per completed task forces capability and cost into the same business equation.
The missing number is inside the harness
The market now has an independent cost-per-task benchmark. It has calculators that account for acceptance and rework. Model vendors have published evidence that decomposed workflows can retain most frontier performance at a much lower task cost.
The remaining problem is measurement.
If a company cannot report acceptance rate by task type, model choice, risk score, and workflow decision, it does not know its cost per completed task. It knows its model bill.
Those are not the same thing.
The businesses that learn the difference will build better harnesses, earn better margins, and accumulate an advantage that becomes harder to copy with every completed task.
Cost per task is the only hill that matters.
Hokusai's job is to help every harness climb it.