Stop 9 of 10 · Weekly

Cost and usage ledger

One wrapper that records tokens per tool and per client, a dated rates file the studio types by hand, a weekly report with the denominator that matters, and the question that kills a tool.

Six tools, one bill, no idea which

By stop 8 the studio has a checker, a diagnoser, a ticket writer, an eval harness and two helpers Mo built in an afternoon. They all call the same API with the same key. The invoice is one number. Nobody can say whether the diagnosis step is worth what it costs, which means nobody can delete anything.

One wrapper fixes that, and it is the only way any tool is allowed to call the API.

export async function call({ tool, clientId, runId, model, messages }) {
  const res = await anthropic.messages.create({ model, messages, max_tokens: 1024 });
  const u = res.usage;
  db.prepare(`insert into usage_log
    (at, run_id, tool, client_id, model, input_tokens, output_tokens, cache_read, cache_write)
    values (?, ?, ?, ?, ?, ?, ?, ?, ?)`).run(
      new Date().toISOString(), runId, tool, clientId, model,
      u.input_tokens, u.output_tokens, u.cache_read_input_tokens ?? 0, u.cache_creation_input_tokens ?? 0
    );
  return res;
}

tool and clientId are required arguments, so a call with no owner does not compile in review. runId ties every call in one night together, which is how you find the night a retry loop ran four hundred times. The two cache columns are there so you can see whether caching is doing anything, rather than believing it is.

Rates live in a dated file

rates.json holds one entry per model, typed by hand from claude.com/pricing, with the date it was read and the initials of whoever read it. The report multiplies tokens by rates at report time. A model with no entry fails the report by name.

The weekly report

The third line needs a definition the studio agrees once and then stops arguing about. Dany's: a finding whose ticket someone opened, commented on or closed within seven days.

Write scripts/usage-report.mjs. It reads usage_log for a date range and rates.json, and prints four sections: spend by tool, spend by client, cost per acted-on output by tool, and cache read share by tool.

An acted-on output is a finding whose ticket has any human activity within seven days of filing. Join through the findings table.

Fail with a named error if any model in the range has no entry in rates.json, and print the date on the rates file at the top of every report so nobody quotes a number without knowing which rates made it.

Here is my weekly tool report and the same report from four weeks ago: [paste both reports].

For each tool, answer three questions from the numbers alone. Did a person act on anything it produced this month. What does one acted-on output cost compared with the same tool four weeks ago. And would deleting it move any number a client can see.

Then rank the tools by the case for keeping them, and name the one you would delete first with the sentence I should put in the commit message.

Dany deleted the report-summarising helper Mo built in an afternoon: forty runs, three outputs anyone opened, and the cost per acted-on output was the highest of the six. The checker, which produces the most rows, was the cheapest by that measure.

Mark stop 9 done