Measure AI cost per finished job

A cheaper model can make a workflow more expensive. If it needs more retries, sends more cases to review, or produces work that someone must repair, the savings on the model bill may disappear elsewhere.

The useful denominator is an acceptable, completed job. For an invoice workflow, that might mean the correct supplier, amount, and accounting category have reached the accounting system. Generating a plausible JSON object is an intermediate step.

The July 2024 paper AI Agents That Matter argues for evaluating agent cost alongside accuracy. That is a helpful starting point for purchasing decisions: a benchmark score and a token price each leave part of the operating picture out.

Count the whole attempt

For a defined period, add model usage, paid tool calls, infrastructure attributable to the workflow, and human review or repair. Include spending on failed attempts. Divide by the number of jobs that met the acceptance criteria.

Keep implementation and migration costs visible separately. If you spread them across expected future volume, state that volume and time horizon. A forecast of ten thousand monthly jobs should not quietly become a fact in the spreadsheet.

Here is a hypothetical comparison, using an assumed review cost of $30 per hour. These are illustrative numbers, not vendor prices or Lumina client results.

Monthly measure	Workflow A	Workflow B
Model, tools, and infrastructure	$400	$250
Review and repair	12 hours	22 hours
Review cost	$360	$660
Acceptable completed jobs	800	800
Combined operating cost	$760	$910
Cost per completed job	$0.95	$1.14

Workflow B has the smaller infrastructure bill and the larger operating cost. The calculation also omits the cost of waiting for review; a time-sensitive process may need that measured separately.

Compare the same work

Use the same acceptance criteria and a comparable mix of inputs. Otherwise, a route handling clean forms can appear more efficient than one handling exceptions even when the underlying system is worse.

Separate ordinary jobs, ambiguous inputs, and tool failures. Record completion rate and time to completion alongside cost. Include the slowest portion of the workload: a low average can hide a queue that leaves a few customers waiting for hours.

Use a stable evaluation set before changing the route. The 30-case pilot checklist is one way to make the definition of acceptable work explicit.

Route work only when you can check the result

The 2023 FrugalGPT paper explored model cascades that balance cost and performance. Its historical results are evidence for investigating routing, not a savings forecast for a different application in 2026.

A practical route might try a lower-cost model on structured requests and escalate cases that fail validation. That requires a useful validator. Checking that an amount is numeric will not detect a believable amount copied from the wrong invoice.

Do not make the model's statement that it is confident your only gate. Use checks against source records, task-specific evaluations, and human review where correctness cannot be established automatically.

Put a ceiling on unfinished work

Define limits for attempts, tool calls, and elapsed time. When a limit is reached, preserve the job state and send it to a named owner. A silent retry loop can spend money without moving the task closer to completion.

Review the expensive failures by reason. Missing source data, a slow connector, and an unclear approval rule need different fixes. Changing the model may address none of them. The next cost reduction should target the part of the workflow that is consuming the time.