On 5 August we started six readers at once and gave each of them a different slice of the same book. It was an English academic title, 307 pages in the edition we worked from, built out of forty-odd short essays spread across six chapters.
What we wanted at the end was not a summary that reads nicely and disappears. We wanted notes we could cite in real work, with page numbers, so that when someone asks where a claim came from we can open the page and point at it.
So the question that decides whether this kind of work is usable is not whether the summary sounds good. It is how do you know it did not make any of it up.
After several books, the answer we keep arriving at is that credibility does not live in the model. It lives in the sequence of steps built around it. This post walks through all five stages, including the place we got each one wrong first.
Part 1The actual job: one book, six readers
All six ran in parallel, one slice each, so the book was read exactly once. No page read twice, no page skipped.
What came back was six sets of raw notes with page citations throughout. From those we wrote five thematic documents and one index page that points at everything.
The hardest part of this job is not the reading. Current models handle academic English comfortably. The hard part is making what they write back traceable line by line.
The dividing line is a single question: can you get back to the source? If you cannot, the whole set of notes is just an opinion that happens to be well written.
Part 2How do you split a book for AI without losing anything
Split along the structure the author built, not by page count, and read the table of contents and the preface before deciding where the cuts go. Those two things are the cheapest text in the book, and they determine where the holes in your notes will be.
Adler and Van Doren, who wrote How to Read a Book, call the survey-the-whole-book stage inspectional reading, and they insist it comes first every time. When you read for yourself you can get away with skipping it, because your eye sweeps the contents page automatically. Hand the work to readers that only ever see their own slice and skipping it becomes a hole you will not notice until you read the output.
On an earlier book we started the split at file page 11, reasoning that the first pages are cover and copyright. The preface and the contents page went with them. So a reader wrote "not found in this slice" under the heading asking what the book was written for, when the answer was sitting on page iv. We had to go back and fill it in ourselves.
Book page numbers and file page numbers are not the same number
In the book described here, book page N is file page N+13. We did not take that on one sample. We checked it at all six chapter boundaries, and only used it once they all agreed.
Checking several points matters because the offset is not always constant. On another book we read before that, page 235 lines up with file page 246, but by page 310 it lines up with file page 319. Pages were inserted along the way and nothing announced it.
Notes with wrong page numbers are worse than notes with no page numbers. The second kind admits you have to go looking. The first kind invites you to trust it, then sends you to the wrong place.
When we hand a slice to a reader we embed page markers directly in the text of the file, so the reader cites the page it can see in front of it instead of computing or guessing one.
Part 3The brief that killed four readers mid-run
On the book we read three days earlier, we told the readers to extract the important passages verbatim. Five went out. Four died.
They all failed with the same line: API Error: 400 Output blocked by content filtering policy. The provider blocks long stretches of copyrighted text on the way out.
The five books before that used the same brief and all went through, so our first read was bad luck. We relaunched. They died again. We were on the fourth casualty before we went back and reread our own brief.
The rule out of that round is short. A reader that dies twice with the same message is telling you the brief is wrong, not that you were unlucky. Stop relaunching and go read what you asked it to do.
What we use instead is: summarise in your own words, and quote only where the exact wording carries the point. Six new readers, all of them finished, both while reading and while writing.
The part we did not expect is that the output got better. Reading to understand rather than to copy made the readers stumble over things that contradict each other inside the book. On the most recent round they surfaced nine of them: a threshold written as 10% in one place and 5% in another, surveys called weak on one page and recommended on the next, a 90% figure the author himself brackets as really being somewhere between 65 and 95, and a worked return-on-investment example whose starting numbers disagree between the setup and the arithmetic.
A brief that asks for a copy gets you a photocopier. A brief that asks for understanding gets you a reader.
Things that genuinely have to be exact, like code, configuration or a list of names, can still be requested verbatim. Just scope the request narrowly instead of pointing it at the whole book.
Part 4How do you know the AI did not invent a quote
Take every quoted passage and compare it against the corpus mechanically, rather than reading past it and feeling that it looks about right. Our threshold is that any quote of 25 characters or more has to be an exact substring of the text.
Both sides get normalised before the comparison: strip all whitespace and punctuation. Justified typesetting inserts extra spacing into some lines, and without normalisation a correct quote fails for a reason that has nothing to do with correctness.
The result on this book was 95 of 95.
Every quote gets its own label rather than being rolled into one percentage for the book. There are three labels: passed with a page number, did not match plus where it diverges, and not found in the text at all. Each of the three needs a different response, which is why they cannot be collapsed.
93% similar is not a pass. Make a similarity score the passing threshold and a quote with one extra word sails through, and one word is exactly what flips a meaning. The score is only useful for ranking the failures once you have them.
Part 5The checker does not tell you the whole truth either
A checker that is green because nothing is wrong and a checker that is green because it is dead look identical. Both of them print the word pass.
So before trusting its output we test the checker itself with two things. The first is a passage lifted straight out of the text, which must be found. The second is a sentence we invented that cannot possibly be in the book, which must not be found. If the second one is found, the ruler was loose from the start and nothing else it reports means anything.
There are five ways a checker stops telling the whole truth, and we hit all five in a single day.
- A broken corpus accuses an innocent reader. One round we checked against text pulled by OCR, which was dropping words here and there. The result said the reader was inventing quotes at an unusual rate. Re-extracting the text with a more accurate tool showed the reader had been right all along. The lesson: when the failure rate looks abnormally high, suspect the checker before you suspect the thing being checked.
- Running headers land in the middle of a sentence. When a sentence straddles two pages, the chapter title printed at the top of the page ends up inside an otherwise correct quote. Strip running headers out of the corpus before you use it.
- Edited quotes fail by construction. Anything with an ellipsis or square brackets in it was never a continuous passage. Split it into pieces and check each piece.
- One search is not a verdict. We could not find
RULE 12anywhere in the book and nearly concluded the reader had invented a rule. In the summary list at the end of the chapter the author writes plain12.with no word in front of it. - Report what you actually measured. The sentence to write is "does not match the corpus we compared against", not "the AI made it up". The difference is that the first one leaves room for the corpus to be the thing that is wrong.
Part 6Reading one book is five stages, and each one uses something different
"Get an AI to read a book" sounds like one job. In practice it breaks into five stages, and they want completely different things. Two of them do not involve a model at all. Here is what actually ran on the two books behind this post.
| Stage | What runs it | Why that one |
|---|---|---|
| 1. Map the structure and calibrate page numbers | No model. pdftotext on the contents and preface only, then a human decides where the cuts go |
It is a judgement call that affects the whole book, and it is cheap enough that saving money here makes no sense |
| 2. Prepare the pages that are images | No model. pdftoppm at 110 dpi, downscaled to roughly 1,400 px on the long edge |
Image cost scales with area, roughly width times height over 750, so resolution is the price lever |
| 3. Read a slice and summarise it in its own words | The August 2 book: six Claude Haiku 4.5 readers. The August 5 book: six qwen3.8-max-preview readers through Alibaba's gateway |
It is endurance work repeated six times in parallel. It needs consistency and a price that makes six concurrent readers viable, not the sharpest judgement available |
| 4. Text that has to be exact to the character | Kimi K3, around 0.017 USD per page | This text is the ruler for stage 5. A bent ruler makes everything measured with it wrong |
| 5. Synthesise, run the verbatim check, decide whether to adopt | The main Claude Opus of this working session | The output goes straight to another person, and this is the stage that has to be willing to say the work is not usable yet |
One question decides every row: what happens to this stage's output next? If it becomes raw material for a later stage to refine, a cheaper model is fine. If it reaches a person, or gets used as a measuring stick, it has to be an accurate one.
What the results looked like
- Stage 1 gave a page offset of +13, confirmed at all six chapter boundaries, so every page number in the notes opens to the right page.
- Stage 3 read both books in a single pass with no reader failing, and the readers on the most recent one volunteered nine internal contradictions nobody asked for.
- Stage 5 checked 95 quotes against the corpus and passed all 95, with the ruler itself tested beforehand against something that had to be found and something that had to not be.
And what happens when you skip a stage
Answered from the record, because every row below is a stage we skipped ourselves and paid for. None of it is speculation. Some stages have two rows because we got them wrong in two different ways, and stage 2 has no row at all because we have never skipped it.
| Stage skipped | What actually happened |
|---|---|
| 1. Not mapping the structure first | Split started at file page 11, taking the preface and contents with it, so a reader wrote "not found in this slice" under what the book was written for, with the answer on page iv |
| 1. Trusting the page offset from one sample | On another book page 235 maps to file page 246 but page 310 maps to file page 319. One fixed offset means every page number after the shift is wrong |
| 3. Briefing for verbatim extraction | Five readers out, four dead on the same content-filter error, and we relaunched into it until the fourth casualty before rereading our own brief |
| 3. Using a cheap reader for work that must be exact | One document reached publication with 14 errors in it, including a font name and an escape character that break the moment someone copies them. The shape was consistent: accurate early pages, drift towards the end |
| 4. Using OCR text as the ruler | The corpus was dropping words, the check reported an unusual number of invented quotes, and re-extracting with a better tool showed the reader had been correct throughout |
| 5. Not checking quotes, or passing on a similarity score | A quote with one word added sails through, and that one word is the one most likely to change the meaning |
Every row is the same shape. Saving money in the wrong place does not break the work. It makes the work come out looking normal and wrong, which costs more than breaking, because broken work gets fixed on the spot and normal-looking work gets forwarded.
The rule we wrote down only after both books were finished: start with the cheap model every time, bounce to Claude Haiku when it dies or runs out of quota, send anything that has to be exact to K3, and keep synthesis and checking on the main model always. Before that we picked per job, in the moment. It worked just as well, and it was impossible to hand to anyone else.
One last note for this section: the model that reads the book and the model that checks whether it read correctly should not be the same one. Not out of distrust, but because whoever just finished writing an answer is the worst-placed person in the room to spot what is wrong with it.
Part 7The last question before you adopt anything, plus five takeaways
There is one more stage people skip, because it is not technical.
Adler writes that a practical book is judged on whether you accept where it wants to take you, not only on whether its reasoning holds together.
We use that directly. Before taking a working practice out of a book we just read and changing something real with it, we have to be able to answer whether we actually want the thing it improves. If we cannot answer, it stops at the notes and changes nothing.
Wrong notes cost you one bad read. A working practice adopted without wanting it stays with you for a long time.
Five things worth taking to your own reading work.
- Map before you split. The contents page and the preface are the cheapest text in the book and the only thing that tells you where a cut breaks the sense.
- Calibrate page numbers at more than one point. The offset between book pages and file pages can shift mid-book, and nothing warns you when it does.
- Ask for understanding, not for a copy. You get work that survives the content filter, and a reader that brings back the contradictions it found.
- Check quotes mechanically, one at a time. Not by overall impression, and never with a similarity percentage as the passing threshold.
- Test the checker before you believe it. One thing that must be found, one that must not. If you have never seen it go red, you do not know that it can.
Where to start
Take a PDF you already have, cut thirty pages out of it, and ask an AI to summarise them in its own words with five quoted passages. Then do the one step that matters most in the whole process: search for all five in the original file.
Before believing any of them, invent a fake sentence and search for it the same way. If the fake one turns up, your search method is broken and none of the other results mean anything yet. It takes a couple of minutes, and it is the step people skip most.
On checkers that go green without checking anything, we wrote that up in every test passes and the system is still broken. On document reading that has to know when it misread, see OCR accuracy is not about the model reading well. And if what you are building is a knowledge base rather than a one-off read, turning speech into notes without the AI making things up picks up where this leaves off.
What changed after several books is not a belief that AI can read on our behalf. It is being comfortable showing someone else the notes it produced, because every line in them has a source you can walk back to.
Sources and references
- Mortimer J. Adler & Charles Van Doren, How to Read a Book, Simon and Schuster 1972 (ISBN 978-0-671-21209-4) the source of surveying before reading closely, and of the question about accepting where a practical book takes you.
- The titles of the books we ran through this pipeline are deliberately not listed. This post is about the method, not a book review, and what has to be checkable is the numbers and the procedure, both of which are in the post.
- Every figure here, the 95 of 95, the nine internal contradictions, the four readers lost out of five, the 14 errors that reached a published document, and both page offsets, was measured in our own work between 1 and 5 August 2026. None of them are borrowed.