Turn Speech into Notes
You Can Trust, Without Letting AI Make Things Up

This started with a long lecture series I wanted to keep as knowledge notes. Usually I just drop the file into NotebookLM and I'm done: ask it anything and it answers, with a pointer to where it got it. But a few clips it refused to take, so I had to transcribe the audio into text myself first, then hand that to an AI to summarize.

The summary read beautifully, until one line stopped me. It said the lecturer talked about "the Pythagorean theorem," formula and all. But when I went back to the actual recording, he never mentioned it. Not once.

The scary part wasn't that it was wrong. It was wrong in a way that looked perfectly reasonable. What the AI added could plausibly have belonged in a lecture like that. Skim past it and you would never catch it. Leave it in, and it becomes "knowledge" that nobody ever actually said.

That is what forced a rethink: how do you turn speech into notes you can actually trust, without re-listening to the whole recording line by line? The story runs in order. First, get a usable transcript out of the audio. Then, set up a checkpoint that stops the AI from making things up. Then, store it as notes you can find and that won't go stale. And finally, how to put it to work on real tasks.

Part 1Get notes you can actually trust

Why AI finds it so easy to make things up here

A tool that answers with its sources attached is easy to keep honest: if it summarizes something off, you see it, because the citation is right there holding it up. But a transcript straight off the audio is just plain text with nothing behind it. Hand that to an AI and it will happily fill in whatever "should" be there.

This is the key point. AI doesn't make things up at random. It makes up "the thing that probably fits." So guarding against it randomly won't work. You need one rule: every claim (every point the AI writes) has to point back to where it was actually said in the audio. If it can't, it gets cut.

Three things to understand first

1. "Can't find it" is not the same as "AI made it up." Sometimes you can't find it because the transcription garbled the spelling, or the idea was said without the exact term. We cut it either way (until we can trace it) but don't rush to brand it a lie.
2. Complete matters less than correct. A short note that's true on every line beats a long one with a single made-up thing mixed in.
3. Finding the word doesn't mean it means what you think. You still have to read around that spot to confirm it's actually talking about the same thing.

How to do it

1. Transcribe it well. For clips NotebookLM accepts, let it transcribe and summarize directly. For the ones it refuses, transcribe them yourself with an open-source speech-to-text (ASR) tool, like faster-whisper large-v3 or SCB10X's Typhoon ASR, both on GitHub. A few things worth knowing before you start:

• Cut the audio into short chunks, around 30 seconds, and transcribe one at a time. Don't feed the whole file at once, long files eat RAM until the program chokes.
• Cut on the quiet parts. These tools ship with VAD (Voice Activity Detection) that finds the silence for you, otherwise words sitting on the cut line get lost.
• Load the model into memory once, then feed every chunk through it. The common mistake is a script that reloads the model on every chunk, a big model takes several seconds to load, so a clip cut into a hundred chunks crawls to a halt.
• Set the language to Thai, but English words mixed in still come out garbled, that's exactly why the clean-up step (step 4) exists.

2. Have the AI summarize into points. Tell it to summarize from this one file only, no outside knowledge mixed in. Keep the gist, don't copy word for word (raw transcripts are full of garbled words, copying it all just adds noise). But what you have now is still only a "draft." It has to clear the checkpoint before it counts as knowledge.

3. Set up the grounding gate. This is the heart of it. This is what separates the method from plain "transcribe then summarize." The idea: every claim the AI produced has to have evidence in the actual audio before it's kept. Do it in steps:

1. Break the note into individual claims. Every number, name, and conclusion gets checked separately.
2. Find where each claim was said in the actual audio (search the transcript you made).
3. Read around that spot with your own eyes. Confirm it's genuinely the same thing, not just a coincidental word match.
4. Any claim with no evidence gets cut. No "it's a little off but let's keep it for now."
5. Record which evidence let each claim through, so it's auditable later. For the important ones (numbers, names), check again with a second pass.

The principle is simple enough that anyone can do it by hand. Where we put in the work is the "tooling" that makes these steps fast and repeatable at real scale. That part is what's being built.

4. Clean up. English words the transcription mangled into broken Thai, write them back correctly in English. Numbers or events tied to a specific moment (a price that day, say) keep separate, don't fold them into permanent knowledge, because they go stale fast.

My favourite lesson: "can't find it" ≠ "fake"

There was a name in the lecture, Edward Thorp. Searching the transcript for "Thorp" returned nothing. Looked like the AI made it up. But reading around the spot, it turned out the transcription had written it as "เอ็ดเวิร์ดท็อป." It was real. Trust the search alone and you'd have cut something true. That's exactly why a human has to read with their own eyes, instead of letting the machine decide.

Part 2Store notes so they stay findable and don't go stale

Once you have notes that have been checked, the next question is how to store them so they become knowledge you can reach for, not a pile of files you can't find. We keep ours in Obsidian (a notes app where notes link to each other) on a few simple rules.

Store it so it stays trustworthy

• The note file is the real thing. Every other search tool is just a shortcut into the files. If anything disagrees, the note file wins.
• Keep the source material separate from the cleaned-up. The raw transcript lives elsewhere and never gets deleted, because it's the evidence the notes point back to. We keep ours in a sources/ folder, with the cleaned-up notes living separately.
• Label what's "fact" and what's "opinion." Set it as a property in the note's frontmatter (the header at the top of the file). An unlabelled opinion reads like fact when you come back to it months later.
• Date anything that can age. Some knowledge is true forever, some goes stale fast. A date tells you which ones to come back and re-check.
• One raw note = one source, never blended. Keep conflicting opinions on both sides, don't cut one of them away.
• A human signs off. AI can draft the labels, but a person has to confirm them. You don't let it decide on its own what counts as fact.

Link it so you can find it, not just pile it up

Linking in Obsidian uses wikilinks, type [[note name]] and it links itself. We link on three levels: notes on the same topic link to each other, notes link back to the source material they cite (that's a citation), and terms link to the page that defines them. Everything gathers in a central index note, an MOC (Map of Content). When you go looking for something, start with the cheapest way first, open the Index or search a word, then move to the more expensive ones. And the rule that matters most: if it isn't written into a file, it doesn't exist yet.

Part 3Putting it to work

The one rule to remember

If you remember one thing from this, make it this: don't let what the AI summarizes become knowledge until it can point back to evidence in the source. Everything else is just detail about how to make that one rule real and auditable.

Where this helps

• Meeting notes with no action item appearing out of nowhere that no one raised.
• Lectures and courses where every point traces back to the source.
• Customer interviews that are the customer's actual words, not the AI guessing on their behalf.
• Anything that has to be auditable like medical, legal, finance work, where a single wrong line carries a real cost.

How to start

Don't build automation first. Try it by hand on an old note, starting from the gate (Part 1, step 3):

1. Take one note an AI summarized for you, and put the source (the audio file or transcript) next to it.
2. Highlight each sentence that's a claim, every number, name, and conclusion.
3. Search each one in the source, and when you find it, read around it to confirm it's the same thing.
4. For anything you can't find, mark it but don't cut it yet, the transcription may have garbled it (remember Edward Thorp).
5. Count how many points in that one note can't be traced back to the source.

One pass is enough to see for yourself how quietly this stuff slips in.

Doing it by hand like that is fine for one clip, but it stops scaling once you have dozens. How to turn it into a system, from the transcription step all the way to a web of linked notes, and how to make the gate run on its own, I'll save that for the next article.