Talk Now, Sort Later: How My Voice Memos File Themselves

At ten to midnight on October 9, I recorded a memo about what to do with ConnecZen, an app I'd built and then parked. The idea was to turn it into a shared knowledge base for one family. My parents would send a quick voice note, and the app would sort it into a recipe, a ritual, a medication note or whatever else it was, so anyone in the family could find it later. The next day I started rebuilding ConnecZen around that idea, from a fresh codebase. It's the product I work on now.

I didn't type any of that, and I didn't sort it either. I said it into a phone I never unlocked, and by the time I sat down at the Mac it was a note in my Obsidian vault, already read by a model that decides whether anything in it is a task and which project's list it belongs on.

This post is about that pipeline. It's partly a how-to, but mostly it's about what changed once capturing a thought got cheap, and what's still broken.

The phone was the problem

For years I had plenty of note apps. The trouble was that every one of them lives on the phone, and unlocking the phone is a trap. I open it to save one thought, see a WhatsApp badge, and twenty minutes later the thought is gone. I also can't touch it while driving, which is where half my ideas show up.

I spent a couple of weeks in September looking at hardware to get around this: a pen recorder, an e-paper gadget with a record button, a few ESP32 boards. Then I realised the phone I already had could do it. Most phones come with a voice recorder app, and on mine I can start it from a widget without unlocking anything. Tap to record, tap to stop, and I never see a notification.

That's more of a behaviour fix than a tech one. If the first step of capturing a thought lands me in my notifications, I'll stop capturing, so the fix was making sure that step never happens.

The pipe

Here's the whole route, end to end:

1. The phone's recorder app saves an audio file to a folder.
2. A sync app copies that folder to my Mac in the background. I use Syncthing because it's free and goes straight from device to device with no account in the middle, but anything that mirrors a phone folder to a computer would do.
3. A small Python script on the Mac spots the new file and transcribes it locally with Whisper, OpenAI's open-source speech model. I run the whisper.cpp version.
4. The transcript lands in my vault's +Inbox/ as a markdown note with the time it was captured.
5. The watcher hands that note to Claude Code in headless mode (claude -p) with one fixed prompt.
6. The audio file gets moved to a done/ folder on the Mac, and because the sync runs both ways, it disappears from the phone too.

On the first real test, it took about 45 seconds from tap to note. The Claude step adds roughly 25 seconds per memo. I don't wait for any of it because I'm not watching.

Step 6 nearly cost me something on day one. When I first connected the phone, the sync pulled in four family recordings from November 2025 along with the test memos. Because the Mac deletes from the phone once it's done with a file, letting the watcher run on them would have wiped the only copies. I caught it and moved them out by hand first. Two-way sync is what makes the cleanup free, and it's also what makes it dangerous.

Step 5 is where the privacy line sits. The audio never leaves my Mac, because transcription is local. The transcript does go to Anthropic when Claude routes it. For errands and product ideas I'm fine with that. For anything I'd say to a therapist, I wouldn't be, and right now the pipeline treats both the same way.

What the model actually does

This part sounds the most like magic and is actually the dullest. The prompt tells Claude to read one memo and do three things:

• Decide whether it holds any tasks at all. The prompt says outright that reflections and ideas without an action aren't tasks.
• Put each task in the right file. There are eight task files: one per project (Profit Driven, ConnecZen, my novel, the YouTube channel, coaching, this website), one for the voice pipeline itself, and a root file for errands and anything it's unsure about.
• Stamp the memo with how many tasks it pulled out.

It's only allowed to read and edit files. It can't delete anything or reorder my lists, and it can't make up due dates the memo didn't mention.

Between September 19 and October 31 I recorded 39 memos. The 37 that went through the router produced 51 tasks. Nine produced none, and that's right: one was me saying I like the morning kriya because a fixed time gives my day a shape, and another was a "thought of the day" about motorcycles and solitude. Those get archived as notes, not to-dos.

The best thing the model does is fix whisper's mistakes. Whisper heard "Asana" as "a sauna" and "Claude" as "Cloud". It heard my company as "Prophet driven" once and "Profit Drivel" another time, which I'm choosing not to take personally. The prompt tells Claude to write the likely word and keep the misheard one in brackets, so the task reads "Asana (heard: a sauna)". Fixing that with a regex would be a pain. A language model does it without fuss, because it knows what I work on.

It only fixes what whisper gets close to, though. Whisper started on the base.en model, which turned ConnecZen into "Canadian project" and Echo Nests into "Econest". I moved to small.en. It's better on everyday words and still gets ConnecZen wrong. And one memo from a workshop on October 10 came out as a string of real words that don't add up to anything. The model can fix a misheard word. It can't rebuild a thought that never made it into the transcript.

Retrieval is where it pays off

Capture is only half of it. A pile of transcripts nobody reads is just a nicer-looking junk drawer.

Three things read these memos after the fact.

The first is planning. My /plan-day command reads every open task file plus the last 48 hours of memos before it builds tomorrow's plan. Something I said at a red light on Tuesday ends up in Wednesday's time blocks without me copying it anywhere.

The second is spotting an idea that keeps coming back. On September 25, 26 and 27 I recorded three separate memos about the same Profit Driven problem: how to pick blog topics by mapping a client's site into a hub-and-spoke graph. Each one on its own was half-formed. Read together, they were a spec. I wouldn't have connected them if they'd been three sticky notes in three jacket pockets.

The same thing happened with ConnecZen, except the router missed it. The October 9 memo from the top of this post is one of the nine that produced no tasks. So is one from October 6, three days earlier, where I said almost the same thing: turn ConnecZen into a family knowledge base, the original Echo Nests idea. By the router's rules both scores are correct. Neither memo says "I need to" anything. They're ideas, and ideas get archived. The memo that changed the most about my work this autumn was filed as having nothing to do. It didn't get lost, because the router isn't the whole system. The note was still there, in my own words and with a timestamp, and the second time I said it, I listened.

The third is turning talk into writing. On October 10, just after midnight, I recorded a memo asking for exactly this: let me end a memo with a prompt like "this could be a LinkedIn post", have it drafted the next morning, and have it fact-check me against the web. This post is a rough first version of that. The draft was put together from the vault notes and memos above, and every reference at the bottom was looked up and opened before it went in.

Why I think this works

Some of this is research and some is my opinion, so I'll keep them apart.

The research: a Stanford study found speech input was about three times faster than typing on a phone keyboard, with fewer errors. It's a 2016 study on one phone and one keyboard, so take the exact number loosely. But nobody who has thumb-typed a paragraph will argue with the direction.

Psychologists call putting memory and tracking work into the world, like a reminder or a knot in a handkerchief, cognitive offloading. Risko and Gilbert's review is careful to say it has costs as well as benefits, and that the long-term effects aren't well studied. And Masicampo and Baumeister found that making a specific plan for an unfinished goal cut down how much that goal kept intruding on unrelated tasks.

The opinion: I think that last finding is why this feels lighter than it should. Saying "call the phone company tomorrow about the voicemail password" out loud, and knowing it'll show up somewhere I'll actually look, is enough to make it stop nagging me. A memo isn't a plan. It's more like an IOU for one, and my head seems happy to accept it.

What it costs

Cheap capture makes more work. I recorded 22 memos in the first twelve days. Fifty-one tasks I didn't have to type is also 51 tasks I now owe. The system made it easy to say "I should" out loud, and it can't make me do any of them.

Next on the list is passing whisper a short list of my project names. In a test, that fixed Echo Nests, and it should stop ConnecZen coming back as a Canadian project. It's a one-line change in the watcher, and it still isn't done.