The 4096-character wall between my kid and a story

My bedtime-story app can read a generated story aloud in a warm neural voice. It’s one of my favorite parts: my kid picks the characters, the app writes the story, and then it narrates it back in OpenAI’s fable voice while he lies there listening. It worked beautifully in testing. Every short story I threw at it narrated perfectly.

Then one night I hit play on a longer story and got nothing.

No audio. No obvious error in the UI. Just a narration that spun and spun and never became ready. The story was right there on screen, fully generated. It simply refused to speak.

The cause turned out to be mundane, and the kind of thing you only learn by tripping over it: OpenAI’s text-to-speech endpoint caps its input field at 4096 characters. A five-minute bedtime story is easily 5,000 to 9,000 characters. My short test stories fit under the cap. My real stories didn’t.

How narration works

The app is a one-shot LLM story generator. Narration is a separate, optional step: tap listen, and a background worker sends the story text to OpenAI’s /v1/audio/speech, gets back an mp3, writes it to disk, and marks the audio row ready. The client polls, then plays the file. It’s deliberately async and race-guarded, the same pattern as story generation, so a killed app or a concurrent retry never double-synthesizes (or double-bills) the same narration.

The provider looked, in its first incarnation, like the obvious thing:

async synthesize(text, { voice, instruction }) {
  const resp = await fetch('https://api.openai.com/v1/audio/speech', {
    method: 'POST',
    headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json' },
    body: JSON.stringify({ model, voice, input: text, instructions: instruction, response_format: 'mp3' }),
  });
  return { bytes: Buffer.from(await resp.arrayBuffer()), format: 'mp3' };
}

One call, one story, one mp3. Clean. And it works, right up to 4096 characters.

The silent failure

The failure mode is nasty precisely because it’s length-dependent. During development you write short stories to iterate fast. A “quick fox and the moon” test story is a few hundred characters and narrates perfectly. Every demo works. You ship.

Then a real generated bedtime story, the kind the app actually produces, tuned to land near the upper end of a word band, comes in at around 7,000 characters, sails past the cap, and the API rejects the input. The narration job fails. Depending on how you surface it, the user sees a spinner that never resolves, or a generic “couldn’t narrate,” because nobody writes the message “your story was 3,000 characters too long for one call” until they’ve hit this wall themselves.

The tell was correlation: short stories narrate, long stories don’t. Once you see that, the character limit is the first thing to suspect, and OpenAI documents it plainly. The input field maxes out at 4096 characters.

There’s nothing clever to say about the limit itself. It’s a hard API constraint:

/** OpenAI's /v1/audio/speech caps the `input` field at 4096 characters. */
export const OPENAI_TTS_MAX_INPUT_CHARS = 4096;

The interesting part is what you do about it. You can’t ask the model to “summarize to fit,” because that would mangle the story. You have to narrate the whole thing, which means splitting the text into pieces that each fit, narrating each, and stitching the audio back together. And the splitting has to be lossless: every character of the original must survive, in order, or the narration will drop or duplicate words at the seams.

Chunking without losing a word

Here’s the whole thing. It’s a pure function, text in and ordered chunks out, with an advancing cursor that guarantees the chunks concatenate back to the exact original.

export function chunkForTts(text: string, max = OPENAI_TTS_MAX_INPUT_CHARS): string[] {
  if (text.length <= max) return [text];
  const chunks: string[] = [];
  let pos = 0;
  while (text.length - pos > max) {
    const window = text.slice(pos, pos + max);
    let cut = window.lastIndexOf('\n\n');            // 1. paragraph break
    if (cut > 0) cut += 2;
    else {
      const sentence = window.match(/^[\s\S]*[.!?]['")\]]?\s/); // 2. sentence end
      if (sentence) cut = sentence[0].length;
      else {
        const space = window.lastIndexOf(' ');       // 3. word boundary
        cut = space > 0 ? space + 1 : window.length; // 4. hard cut (no boundary)
      }
    }
    chunks.push(text.slice(pos, pos + cut));
    pos += cut;
  }
  chunks.push(text.slice(pos));                      // 5. the tail
  return chunks;
}

The cursor (pos) only ever advances by exactly cut, and every chunk is text.slice(pos, pos + cut), so no character is added, dropped, or reordered. That’s the property that matters most, and I’ll come back to why.

The seam is the whole point

The order of those fallbacks, paragraph then sentence then word, isn’t cosmetic. It’s about where the seam between two audio clips falls, and how audible that seam is.

When you narrate chunk A, stop, then narrate chunk B as a separate synthesis call, there’s a tiny discontinuity at the join: two independent renders, each with its own start and stop prosody. Think of it like splicing two pieces of recorded tape. If you cut and rejoin at a moment of silence, nobody hears the splice. If you cut mid-word, everybody does.

If that seam lands between paragraphs, it’s invisible, because a natural pause was going to be there anyway. If it lands mid-sentence (“the fox looked up | at the silver moon”), the two halves get different intonation and the break is jarring. If it lands mid-word (“the sil | ver moon”), it’s a glitch.

So the algorithm prefers, within each 4096-character window:

1. Paragraph break (\n\n), the best seam. lastIndexOf finds the latest paragraph break that still fits, so chunks stay as full as possible (fewer chunks means fewer paid calls means fewer seams). Losslessness doesn’t come from the += 2: it comes from slicing the chunk with the same cut we then advance pos by. The += 2 just decides where the seam falls, keeping the newline pair at the end of the chunk we’re closing rather than at the start of the next one.
2. Sentence end, if no paragraph break fits. Fall back to the latest sentence-ending punctuation followed by whitespace (., !, or ?, optionally a closing quote or bracket, then a space). Seams at sentence boundaries are still very natural.
3. Word boundary, if a single “sentence” is somehow longer than 4096 characters (rare, but think of a run-on with no terminal punctuation). Break at the last space so we never split a word.
4. Hard cut, only if there’s no space at all in 4096 characters. This is pathological and effectively never happens with real prose; we cut at the window edge rather than loop forever.

Proving nothing got lost

That losslessness property is easy to assert and easy to accidentally break. So the unit tests pin down exactly the two things that matter: every chunk fits, and nothing is lost.

const text = 'The quiet fox watched the silver moon rise. '.repeat(200); // ~8800 chars
const r = await p.synthesize(text, { voice: 'fable' });

expect(calls.length).toBeGreaterThan(1);                    // it actually chunked
for (const b of calls) expect(b.input.length).toBeLessThanOrEqual(4096); // each fits
expect(calls.map(b => b.input).join('')).toBe(text);        // LOSSLESS: rejoins exactly

That middle assertion, every input posted to OpenAI is at most 4096 characters, is the bug I set out to fix. The last one, the joined inputs equal the original text character for character, is the invariant that keeps the fix from introducing a worse bug: dropped or duplicated words. If someone later “optimizes” the splitter and breaks losslessness, this test goes red before a kid ever hears a skipped sentence.

Joining the resulting mp3 buffers is almost anticlimactic:

const parts: Buffer[] = [];
for (const chunk of chunkForTts(text)) {
  const bytes = await synthesizeOneChunk(chunk); // one /v1/audio/speech call
  parts.push(bytes);
}
return { bytes: Buffer.concat(parts), format: 'mp3' };

Buffer.concat on the mp3 buffers works for sequential playback: as a pragmatic shortcut for these TTS-generated files, mp3 is close enough to a stream of independent frames that concatenated buffers play back in order. There’s a minor seam at each boundary, which is exactly why we push those seams to the quietest possible spot, but for a bedtime narration meant to be listened to rather than scrubbed, it’s imperceptible. Accurate seeking, correct reported duration, and truly gapless output would need proper remuxing (rebuilding the container and frame headers) rather than raw byte concatenation. For this product, byte concatenation is the right amount of engineering.

The fix uncovered more

This is the honest bit: shipping the chunking fix and stopping there would have left two real problems live. They were only visible because narration finally got far enough to hit them.

The first was a spend under-count. Every LLM and TTS call in this app is recorded to a llm_calls ledger, and that ledger is what the cost caps sum over to decide whether you’re allowed another paid call. Narration cost is charged per character. The original single-call provider recorded cost once, for the whole story. When I moved to chunking, synthesis got split into N calls, but the cost recording was still sitting outside the loop, recording one row per story.

I could have recorded costUsd(story.length) once and been arithmetically fine. Instead I moved cost recording inside the chunk loop, for a reason that matters:

for (const chunk of chunkForTts(story.body)) {
  await this.cost.assertUnderCap(row.userId);   // re-check the cap before each paid call
  const { bytes } = await this.tts.synthesize(chunk, { voice, instruction });
  await this.cost.record({
    userId: row.userId, task: 'tts',
    provider: this.tts.name, model: this.tts.model,
    costUsd: this.tts.costUsd(chunk.length),     // bill THIS chunk
    /* ... */
  });
  parts.push(bytes);
}

Two things this buys. The cap gets re-checked before every paid call, so a long, multi-chunk narration stops partway once earlier chunks have eaten the budget, rather than sailing to the end on a single check it passed at chunk 1. It’s a soft cap: assertUnderCap doesn’t reserve the next chunk’s cost, so a narration can still overshoot by one chunk before it trips, but it can’t run the whole story past the ceiling. And if a later chunk fails, the earlier chunks are already billed. With the old “record once at the end” approach, a narration that got through four expensive chunks and died on the fifth would record zero cost, a silent under-count. The provider genuinely charged me for those four chunks; my ledger pretended it didn’t. Over time that erodes the whole point of a spend cap.

The rule that fell out: record cost at the same granularity you incur it. One paid call, one ledger row. If synthesis is N calls, cost recording is N rows.

The second problem showed up at the very last step. With chunking working and cost recorded, narration synthesized fine, and then failed writing the mp3 to disk:

EACCES: permission denied, open '/data/audio/…/123.mp3'

A classic Docker volume-permissions gotcha. The audio directory is a mounted volume. It had been created and owned by root (the default when a volume is first materialized by a root process), but the app container runs as a non-root user, as it should, because you don’t run a web service as root. Non-root user, root-owned directory, write denied. Nothing to do with the TTS code at all. It only surfaced now because this was the first feature that wrote files to that volume rather than just reading or talking to a database.

The fix was at the ops layer, not the app: make sure the volume is owned by (or writable by) the runtime user. A chown baked into an image layer won’t help here, because the volume is mounted over that path at runtime and hides whatever the image had there. You have to chown (or pre-create with the right ownership) the mounted volume path itself, or do it in the entrypoint before dropping to the non-root user, and then the service can write. No application code changed for this one. The lesson generalizes past TTS: the first feature that writes to a persistent volume will find your volume-permission bug, and it’ll bite whatever feature happens to be first.

What I’d tell you to take from this

• API input limits are length-dependent landmines. They pass every short test and every demo, then fail on real-world-sized input in production. If an endpoint documents a max, assume your real data exceeds it and handle it before a user finds it.
• When you must chunk, chunk losslessly and prove it with a test. The one assertion that earns its keep is “the chunks rejoin to exactly the original.” Everything else is optimization; that one is correctness.
• Choose split points for the seam, not just the size. Paragraph, sentence, word ordering puts the audible discontinuity where a pause belonged anyway. The same idea applies to any chunking where the pieces get recombined and the join is observable: audio, streamed text, paginated rendering.
• Record cost at the granularity you incur it. If one operation became N paid calls, one cost row became N. Recording once at the end silently under-counts the moment a later call fails, and a spend cap that under-counts isn’t a cap.

That said, the point isn’t that chunking is hard. The chunkForTts function is thirty lines and the mp3 join is one. The point is that fixing the visible bug is where the investigation starts, not where it ends. Chunking was the headline. The under-count and the EACCES were only visible because narration finally got far enough to trip over them.

The whole thing was a backend fix. The next long story my kid picked narrated all the way through, in one warm voice, seams and all.

← all writing