Reading a novel chapter by chapter: how Authrot's scanner evolved

TL;DR

Authrot is a writing studio for serial fiction. When a writer asks for it, an AI "reading" scans a chapter and updates the story bible: who appears, where they are, how they are related, and what changed.

We started with one big AI call per chapter. Now each chapter gets 60–85 small calls that mostly share one cached prompt. For most steps, code does the reasoning and the model only picks from short lists that code prepares.

• More right: on our Monte Cristo gold chapters, the relationships it finds are correct 90% of the time, up from about 30%.
• Fewer missed: it finds 72% of them, up from about 25%.
• Cheap: about 5–8¢ a chapter on a six-chapter test run, with about 95% of the prompt served from cache.
• Small models: fast models with thinking off: flash for the reading, GPT-6 Luna to check relationships. Bigger models with thinking cost far more and did no better.

What a reading produces

Here is The Count of Monte Cristo, read through chapter 40. Apart from the generated pictures, everything below came out of the reading, with identities confirmed by the writer. Writers see it in the studio; readers of a published story get it as a spoiler-safe wiki.

Edmond Dantès at chapter 8, just imprisoned: 14 ties.

At chapter 40, in Paris as the Count: 61. Green is friends and family, red enemies, blue places and ships. Every tie is backed by a quote and starts at the chapter that shows it.

It tracks how people change, too: his appearance, chapter by chapter, each line quoted from the chapter that describes it.

The portrait shows him now, disguised as Sinbad the Sailor. Sinbad keeps his own entry, and Dantès's page lists him under Identities, revealed in chapter 40. The reading suggests the reveal and the writer confirms it.

This is what the reader knows, not what each character knows. The bible tracks secrets and who is in on them, but ordinary facts aren't yet scoped to who knows them. That's what we're working on now.

Read Monte Cristo on Authrot and watch its wiki fill in as you go →

The problem

A serialized novel grows one chapter at a time, and the writer edits old chapters too. Each scan has to update a structured bible:

• Entities: people, places, aliases, and reveals ("the abbé is Dantès").
• Ties: a fixed vocabulary of relationships, each backed by a quote.
• Ledger: standings (what is true now), changes (what this chapter changed), setups (what it plants), and payoffs against earlier facts.
• Scenes, quotes, secrets, traits, contradictions.

Three constraints shape everything that follows:

1. Cost. A writer may scan every chapter, and scan again after each revision. On 2026-10-16, readings were 88% of all credits spent on the platform.
2. Trust. A wrong tie or an invented person in the bible is worse than a missing one, because the writer then has to clean it up.
3. Any language. The manuscript's language is a project setting, and code must not assume English.

Version 1: one call, eight sections

The first version sent the whole chapter, the bible, and one large schema with eight sections, and asked for everything at once. It was cheap (3.8–3.9¢ a chapter on flash) and it failed in a revealing way.

• Sections starve each other. On one test chapter, flash wrote 23 facts, 38 mentions and 0 ties. Given ties as a call of its own, the same model found 27.
• Thinking eats the budget. With thinking on, one reading produced 41k output tokens. On a pro-tier model at low thinking, 35–70% of output was reasoning, and split into four parts it cost 24–27¢ a chapter.
• Mentions crowd out judgment. Even in later readings, mentions were 55% of output, and half or more of them were plain name matches that code can find for free.

The one-call path is still behind a flag (READ_IN_ONE_CALL=1) so we can compare against it.

Version 2: narrow calls over one cached prompt

We split the reading into parts and gave every part the same opening: the same system prompt, the same schema, and the same first message (chapter plus bible). That message ends with a cache breakpoint. Only the second message, the part's task, changes.

[system: role + rules]                   ┐
[schema: one shared JSON schema]          ├ cached, written once per chapter
[user 1: chapter text + bible handles]    ┘ ← breakpoint
[user 2: "List the ties of kind FAMILY"]    fresh, a few hundred tokens

What made it work:

• One schema for every call. A different schema per part breaks the shared prefix, and the cache goes with it.
• Short handles for ids. The model sees @1, @2, not UUIDs. Fewer tokens, and an invented handle is easy to catch and drop.
• Rules enforced at save, not in the prompt. A tie with no supporting quote, or a name not in the text, is dropped by code. A later audit (2026-10-16) added ten such save rules, and on two Monte Cristo runs the reading went from 4.87¢ and 5.38¢ to 4.26¢ and 3.83¢ a chapter.
• An explicit cache breakpoint, which OpenAI needs. Without one, OpenAI's automatic caching missed and a reading cost 5.4¢ instead of 2.1¢. With one, 7,212 of 7,228 tokens came from cache. On OpenAI, cache writes cost 1.25× input and cache reads 0.1×.

A narrow call costs 0.02–0.1¢ on flash, so ten of them cost less than one big call that tries to do everything.

Version 3: code reasons, the model looks

Narrow calls fixed cost, but not precision: ties sat near 50% against gold, and the ledger wrote many changes that weren't real. Two things moved them. Ties got one narrow call per kind of relationship, plus a check (see Results). The ledger, and the steps around it, stopped asking open questions. In such a step, code builds a numbered list of candidates and a closed list of choices. The model answers one question about them. Code then checks the answer and decides what runs next.

Every narrow step returns the same shape, which is part of the reading's single schema, so steps share the cached prefix:

const stepItemSchema = z.object({
  n: z.string(),     // the item answered: "C3", "P2"
  pick: z.string(),  // one of the question's choices, or "none"
  to: z.string(),    // an entry handle ("@12") when asked whose/which
  block: z.string(), // the block id that shows it (the evidence)
  words: z.string(), // capped free text
});
// answer = { items: StepItem[], more: StepItem[] }  // more = what code didn't list

The ledger's steps run on a small DAG scheduler. Each step declares needs. Every step whose needs are done starts at once, a wave at a time. On a cold cache the first step that asks runs alone, so it writes the prefix and the rest of its wave reads it. The runner is pure, because the asking is passed in, so every rule is unit-tested without a model.

Models copy block ids badly, so code repairs what it safely can. Blocks are shown tagged ⟦id⟧. A model's answer matches if it is the exact id, or a prefix of at least 6 characters that matches exactly one block. Anything else names no block, and the item is dropped.

The production reading today, roughly in dependency order:

The writer's rule for this phase was explicit: no thinking and no stronger model. The pipeline does the reasoning.

On the ledger, code flags doubtful facts and a narrow call confirms them. That took precision on "what this chapter changed" from 47% to 89%. The checks cost recall, which dipped to 65%; asking the story question a second way brought it back to 75%.

How we measured

Every change was scored before it shipped. Most of what we tried did not ship.

• Gold sets. Hand-built answers for chapters of The Count of Monte Cristo and The Time Machine, plus held-out chapters written before any run was scored. Ties are scored on recall against gold. Precision is reported two ways: against gold, and as judged (a found tie that is true but missing from gold counts as right).
• Replay recordings. A reading's answers are recorded once, so a change to one step can be replayed over the same chapters, with everything else held fixed, and diffed run against run.
• Per-call metering. Every call writes an llm_usage row: fresh, cache-read and cache-write input, output, reasoning tokens, and cost. "Measure before and after" is a project rule, and every cost in this post comes from those rows.
• A frontier ceiling. We ran the strongest model we had, at high thinking, as an upper bound. It came in below the pipeline (story-call recall: standings 24%, changes 55%, setups 48%) at 13–25¢ a chapter for that one call.
• Guards. Measurement runs stop at 9¢ a chapter (MEASURE_MAX_CREDITS). Production runs stop when spend passes 4× the estimate (RUN_OVERRUN_FACTOR).

Results

Ties: precision and recall

Each change below was measured on two runs before the next one started. Giving every kind of relationship its own narrow call raised recall from 38% to 56%. Adding a check call pushed judged precision to 97% (64% against gold) but cut recall to 40%. The check then had to learn to keep ties that the chapter, not just the quote, supports. The biggest recall gain came last: twelve parts run per person, with the check asked twice, took recall from 49% to 69%.

On a model of its own, the check runs over its own compact prompt (the chapter, plus who the ends are) and is asked twice. A list tie is dropped only when both asks reject it. A focused second look then re-asks only the ties the two asks split on, plus the kinds most often wrong. On replayed answers, that took precision from 85% to 92% for about 1.1¢ a chapter.

Cost per chapter

Costs are what writers pay, in US cents (1 credit = 1¢). The full pipeline cost 5.2–8.4¢ a chapter on the first six chapters of Monte Cristo. If that rate held, the whole book's 119 chapters would come to roughly $6–10, though later chapters, with a bigger bible, cost more.

On the same five chapters, the v3 pipeline cost about 55% more than the reading before it, making about six times the calls and checking each answer. That buys the precision gains above, and it still costs less than a pro model with thinking off (9–15¢ across its 11 parts).

Output size

Replacing each part's bespoke schema with the generic step answer cut output about sevenfold, from about 220 tokens per fact to about 35 per item. The old schema paid for every field the task didn't use.

The catch was empty answers: a call that returns no items at all for a chapter that has things to report. The model skips the task instead of answering it. With a free list ("list the standings that changed"), that happened in 6 of 11 calls. Giving the model a checklist to answer item by item fixed it: either the eight kinds of standing, or the people the chapter names most. Each item has to be answered, even if only with "none". In production, an empty or all-"none" answer is asked again.

Rewording the story part's task did the same for the ledger: from 4,700–6,400 output tokens to 1,700–2,500, and from 0.49–0.67¢ a chapter to 0.18–0.27¢.

Cache shape of one chapter

On one Monte Cristo chapter of 3,891 words, read by the full pipeline, input was 1,386,060 tokens read from cache, 62,236 fresh and 15,672 written to cache, with 51,629 tokens of output, for 7.71¢. That is about 95% of input served from cache.

What writers get from it

The pipeline only matters if writers can trust it and afford it. Four properties came out of building it this way:

• It never gets in the way. A scan runs in the background once the writer starts it, so writing never waits on it. One chapter asked for jumps ahead of a whole book's scan.
• Nothing is lost when something fails. Each narrow step is saved as it finishes, so a retry asks only for what's missing, and an answer cut off mid-stream keeps what it already had.
• No surprise bills. Writers see an estimate before a run. A run stops itself at 4× its estimate, or before it would take their balance below zero.
• Savings reach the writer. Prices are set per token from provider cost, so every saving in this post reached what writers pay.

Each chapter also gets a map of what happens in it: where each part takes place, who is there, and the events and promises in order. The beats step draws it.

And the whole book, read to the end. The Connections view shows everyone tied to Dantès, grouped by family and place:

For us, the payoff was measurement. Every call is metered on its own, so each change in this post was priced before it shipped, not guessed.

What didn't work, and what's next

Most of what we tried was measured, then switched off. A selection:

Three lessons carried across all of it:

1. Precision comes from closing the question. In our runs, a model picking from a short numbered list was right far more often than one writing open-ended JSON.
2. Keep the prefix identical. One schema, one first message, one breakpoint. A per-part schema or first message breaks the cache for every call after it.
3. Measure the miss, not just the hit. Gold-set recall exposed that our check was rejecting true ties: 14 times over eight runs on one quote, "my betrothed—Mademoiselle Eugénie Danglars".

Next. The ties goal is recall of at least 75% at 90% precision. On the ledger, we are adding per-thread trigger calls. And the bible's who-knows-what needs to reach past secrets: what each character knows, and when they learned it.