A per-user cap is not a spend ceiling

I run a small bedtime-story app for my kid. A parent fills in a form, the backend queues a job, a worker calls an LLM to write a story, and if the parent wants it read aloud, a second worker calls a text-to-speech model to narrate it. Two paid calls per story, from two separate code paths, on my personal credit card.

Here is the thing nobody tells you until it’s your card: OpenAI doesn’t cap your spend for you. It just keeps billing. If a bug, a runaway retry loop, or (in my more optimistic moments) a wave of signups ever pushes usage up 100x, the first I’d hear about it would be the invoice.

So I built what I think of as bankruptcy insurance. And the first version, the one I was quietly proud of, wouldn’t have protected me at all.

The cap I thought was a ceiling

My first cut was one rule: per user, per day, one dollar. A reasonable-sounding fair-use limit. No single account can burn more than a dollar a day. I felt protected.

I wasn’t. Here’s the arithmetic that ruins the illusion:

total daily spend  ≤  (per-user daily cap)  ×  (number of active users)

The per-user cap is a fair-use rule, not a budget. With 10 users my worst case is $10 a day. With 1,000 it’s $1,000 a day. With a bug that mints a fresh user per request, it’s unbounded. The thing I built to protect me actually grows my exposure as the app succeeds.

A cap that scales with signups isn’t insurance. It’s a limit that grows with every new account, which is the opposite of a ceiling.

The fix is conceptually simple: put a hard number on the whole app, regardless of how the users distribute underneath it.

Three ceilings, three questions

Think of it less like a single lock on the front door and more like the breaker panel in a house. One breaker per room stops any single room from drawing too much. But there’s also a main breaker for the whole house, and that’s the one that saves you when something you didn’t predict pulls current from three rooms at once.

I ended up with three ceilings, checked in order, each answering a different question:

The per-user cap is the fair-share allocator. The two global caps are the actual ceilings: they bound total loss independently of how many users exist. Setting either global cap to 0 disables it, which is handy in tests and for the local dev database. Note that the two global defaults aren’t meant to line up: 30 days at the $10 daily cap would be $300, but the $100 monthly cap sits deliberately below that, a tighter backstop so a run of merely-abnormal days can’t quietly add up to a real bill.

All three are just sums over the same ledger, sliced differently. Every successful paid call writes a row to an llm_calls table: who made it, what task, which model, how many tokens, and the dollar cost. That ledger is the single source of truth for “how much have I spent.” Every cap in this post is a SUM(cost_usd) over a slice of that table, compared against a number.

Here’s the whole service, abbreviated but real:

export class CostService {
  // sum(cost_usd) over an arbitrary WHERE, as a number
  private async sumUsd(where: SQL): Promise<number> {
    const [{ sum }] = await getDb()
      .select({ sum: sql`coalesce(sum(cost_usd::numeric), 0)::text` })
      .from(llmCalls)
      .where(where);
    return Number(sum);
  }

  // this user, since the start of the current UTC day
  todayUsd(userId: number) {
    const startOfDay = sql`date_trunc('day', now() at time zone 'utc')`;
    return this.sumUsd(and(eq(llmCalls.userId, userId), gte(llmCalls.createdAt, startOfDay)));
  }

  // everyone, since the start of the UTC day / UTC month
  globalTodayUsd() { /* same, no userId filter, day  */ }
  globalMonthUsd() { /* same, no userId filter, month */ }

  async assertUnderCap(userId: number) {
    if ((await this.todayUsd(userId)) >= this.caps.perUserDailyUsd)
      throw new CostCapReached();                         // 'daily cost cap reached'

    if (this.caps.globalDailyUsd > 0 && (await this.globalTodayUsd()) >= this.caps.globalDailyUsd)
      throw new CostCapReached('service daily cost cap reached');

    if (this.caps.globalMonthlyUsd > 0 && (await this.globalMonthUsd()) >= this.caps.globalMonthlyUsd)
      throw new CostCapReached('service monthly cost cap reached');
  }
}

The cost_usd::numeric ... ::text then Number(sum) dance in sumUsd is deliberate: Drizzle returns numeric aggregates as strings to avoid precision loss, so I cast the sum to text in SQL and parse it to a JS float on the way out.

Three cheap indexed sums, one thrown error type (CostCapReached, which the global error filter turns into an HTTP 402 Payment Required). Notice all the windows are anchored to date_trunc(... 'utc'): a UTC day and a UTC month, not a rolling window and not the server’s local timezone. So “today” resets at the same instant for everyone, and the reset is deterministic.

Check before the call, and fail closed

Two properties do the actual work here, and both are easy to get subtly wrong.

The first: the check runs before every paid call, not after. assertUnderCap throws before the provider is contacted, so a call that would exceed a cap simply never goes out. This is the whole point. A spend guard that runs after the call has already paid for the thing it was supposed to prevent. It’s not a post-hoc audit; it’s a gate.

The second: it’s called from both money-spending paths. This is the part I could easily have botched, protecting the obvious code path and forgetting the second one. Story generation runs through the provider wrapper:

// provider.factory.ts: chatWithFallback(), inside the retry loop
for (let attempt = 0; attempt <= maxRetries; attempt++) {
  await opts.cost.assertUnderCap(opts.userId);  // throws CostCapReached (never caught here)
  const r = await provider.chat(opts.messages, chatOpts);
  const cost = provider.costUsd(r.inputTokens, r.outputTokens);
  await opts.cost.record({ userId: opts.userId, task: opts.task, /* … */ costUsd: cost });
  return r;
}

Narration runs through a completely separate worker, which checks the same cap object before each chunk of audio it synthesizes:

// audio-worker.ts: a long story is narrated in chunks; each chunk is a paid call
for (const chunk of chunkForTts(story.body)) {
  await this.cost.assertUnderCap(row.userId);     // re-check before every chunk
  const { bytes } = await this.tts.synthesize(chunk, { voice: row.voice, /* … */ });
  await this.cost.record({ userId: row.userId, task: 'tts', /* … */ costUsd: this.tts.costUsd(chunk.length) });
  parts.push(bytes);
}

One shortcut worth flagging: chunk.length counts UTF-16 code units, not characters or bytes, and I use it as a rough cost proxy for the TTS call. That’s fine here because the stories are almost entirely ASCII, where code units and characters line up; a story heavy in emoji or non-BMP text would mis-price slightly.

The re-check inside the chunk loop matters. A 3,000-word story is several TTS calls, and the cap can trip partway through. When it does, the chunks already synthesized are already recorded in the ledger (no silent undercount), and the job fails cleanly. A single cap object, checked in two files, guards both faucets.

One more detail, in the generation path: that throw is deliberately not caught by the retry and fallback logic. A transient network error retries; a CostCapReached propagates straight up and fails the job. You do not want your retry loop treating “out of money” as “try again in 500ms.”

The three sharp edges

A design like this looks airtight in a diagram and has a few sharp edges in reality. I’d rather name them than let you find them the hard way.

The per-user cap is checked first on purpose. When several caps trip at once, whichever throws first wins, and its message is what the user sees. I check per-user first because it’s the only message a single user can act on. “You’ve hit your daily limit, try tomorrow” is actionable; “the service is over its monthly budget” is not something they can do anything about. So the ordering is a UX decision disguised as control flow. The trade-off: if a user is over both their own cap and the global one, they never learn the global cap was also tripped. Fine, because the per-user message is the more useful one.

The global cap is a circuit breaker, not a fair-share allocator. This is the caveat I most want to land. When the global daily cap trips, it blocks everyone until the UTC day rolls over. It does not politely hand each user a proportional slice of what’s left. User #500 who has spent nothing today gets the same “service cap reached” wall as the user who caused it. That’s acceptable for me, because this is a one-household app where “everyone” is basically my family, and a global breaker that occasionally trips is a fine failure mode. For a real multi-tenant product it is not; there you’d want per-user quotas that partition the global budget, so one tenant can’t starve the rest. Know which one you’re building.

A small overshoot is possible, so set the caps with headroom. The sum is over recorded, completed calls. A call that’s in flight, dispatched but not yet returned and written to the ledger, isn’t in the sum yet. But because each worker re-checks the cap before every call, a single worker can only ever have one uncounted call in flight, so its overshoot is bounded to one call’s cost. With N concurrent workers the total overshoot is therefore at most N calls’ worth of spend before the next round of assertUnderCap checks sees the new total. My worker runs single-concurrency for generation, so the overshoot is tiny, but it isn’t zero. The honest framing: these are soft ceilings with a bounded overshoot, not hard transactional limits. If you needed a truly hard limit you’d reserve budget before the call and reconcile after, which is more machinery than a personal app warrants. I set the numbers with headroom instead and sleep fine.

What I’d tell you to take from this

• A per-user cap scales with signups; it is not a ceiling on your total spend. If the only thing between you and an unbounded invoice is a per-user limit, your worst case is per-user cap × users. Add a global circuit breaker that’s independent of user count.
• Layer the caps by the question they answer. Fair use, abnormal day, monthly budget. Three sums over one ledger, cheap to add, and each catches a failure the others miss.
• Check before the call, fail closed. Throw before contacting the provider, and don’t let your retry loop swallow the “out of money” error as if it were a network blip.
• Guard every faucet. I have two paid code paths, generation and narration, and it would have been easy to protect one and forget the other. One shared cost object, checked in both, is the whole trick.
• Know whether you built a circuit breaker or an allocator. A global cap that blocks everyone is right for a family app and wrong for a real product. Be honest about which you have before someone’s blank-slate account hits your wall.

That said, I don’t want to leave you thinking the per-user cap is a mistake to delete. It isn’t. It’s a genuinely useful fair-use rule, and it earns its place as the first check. The point isn’t “per-user caps are wrong.” It’s that a fair-use rule and a spend ceiling are two different tools, and the first version of this failed because I’d asked one of them to do the other’s job.

Three sums over one ledger now stand between me and a surprise invoice, and I sleep fine.

← all writing