Giving our second brain a voice that never leaves the laptop

Our second brain is a few thousand markdown notes, a Postgres index and a small web app that runs on a MacBook. This week it learned to talk. Every note now has a Listen button, and when Claude Code finishes a long job on that Mac it says so out loud, so you can walk away from the screen and still hear when the work is done.

All of the speech comes from a model running on the laptop itself. The note text never goes to a cloud voice service, which matters when the notes include health, money and journal material.

The voice: Kokoro on Apple silicon

The model is Kokoro-82M, an open text-to-speech model under the Apache 2.0 license. It has 82 million parameters, small enough to run comfortably on a 14-inch M4 MacBook Pro with 16 GB of memory. We use the voice called af_heart. In Kokoro's naming the a means American English and the f means female; “heart” is the voice itself. It is also the voice our Friendly desk robot already speaks with, so the robot, the notes app and the coding agent now sound like the same person.

It runs through mlx-audio, an open-source library built on Apple's MLX framework. mlx-audio includes a small server that speaks the same API shape as OpenAI's speech endpoint, so anything that can call POST /v1/audio/speech can use the local voice by changing one URL.

On our M4 the numbers came out like this. The first request after a restart takes about 13 seconds while the model loads. After that a sentence comes back in under half a second. A long planning note that reads out to 15 minutes of audio rendered in 79 seconds. The server process used about 213 MB of memory.

Kokoro is not the most natural voice available. On the Artificial Analysis speech leaderboard it sits well below the best paid voices. For reading notes and announcing finished tasks it is good enough, and it keeps working when an API account runs out of credit.

Listen to any note

Open a note in the second brain app and there is a Listen link next to Download and Edit. Press it and the app turns the markdown into speakable text: it drops the frontmatter, code blocks, link URLs and table pipes, and ends headings and list items with a full stop so the voice pauses in the right places. Then it sends the text to the local server and plays the MP3 in a normal audio player, so you can pause, scrub and change speed.

A five-minute note took about 21 seconds to render the first time. The audio is cached, so the second listen starts at once.

We made one rule strict on purpose. If the local voice server is down, the Listen button shows an error. It does not fall back to a cloud voice, because the whole point is that private notes stay on the machine.

Hearing when Claude is done

Long agent tasks have an awkward rhythm. You start something that will take eight minutes, switch to another window, and come back twenty minutes later. Now, when a Claude Code task on this Mac runs longer than two minutes, the laptop says something like “Claude finished in secondbrain after 6 minutes,” followed by the first sentence of Claude's reply.

This is a Claude Code Stop hook: a small script Claude Code runs every time it finishes a turn. The script reads the session transcript, measures the time since your last message, and stays silent for anything short. If two sessions finish at the same moment, the messages queue instead of talking over each other.

A Voice switch in the second brain app's header turns this on or off for this Mac. It writes one small settings file, and the hook reads that file on every turn, so the change applies to the next task without a restart.

The same voice now powers two more tools: VoiceMode, an MCP server for spoken conversations with Claude Code, and the Hermes agent when it runs on the Mac. Hermes on our cloud server still uses its old cloud voice, because a laptop that travels and sleeps can't serve a Telegram bot.

How to set it up yourself

You need a Mac with Apple silicon (M1 or later) and uv. These are the steps we followed, including the parts that tripped us up.

1. Install mlx-audio with the extras Kokoro needs. The server lives in the server extra, and Kokoro's English phonemizer, misaki, needs spaCy and its small English model. Pin spaCy below version 4: letting uv pick pre-release packages pulled a spaCy development build that crashed on load.

uv tool install --python 3.12 'mlx-audio[server,tts]' \
  --with 'misaki[en]' --with 'spacy>=3.8,<4' \
  --with 'en_core_web_sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl'

2. Generate a first clip. The first run downloads the model and the voice file. The voices come from a second repository, so this run needs internet access.

mlx_audio.tts.generate --model mlx-community/Kokoro-82M-bf16 --text 'Hello from my laptop.' --voice af_heart

3. Run the server on localhost. Bind it to 127.0.0.1 so nothing else on your network can reach it. If you run it from launchd, set a working directory and a log directory: the server creates a logs folder where it starts, and under launchd that is the read-only root of the disk.

mlx_audio.server --host 127.0.0.1 --port 8880 --log-dir ~/Library/Logs/tts
curl -s -X POST http://127.0.0.1:8880/v1/audio/speech -H 'Content-Type: application/json' \
  -d '{"model":"mlx-community/Kokoro-82M-bf16","input":"It works.","voice":"af_heart"}' -o test.mp3

4. Add the spoken alert to Claude Code. In ~/.claude/settings.json, add a Stop hook that runs your script. The script reads the hook's JSON from standard input, opens the transcript_path it points to, finds the timestamp of your last real message (not tool results or system messages), and if more than two minutes have passed, starts a detached process that posts a short sentence to the server and plays it with afplay. Make it exit 0 on every error, so speech can never block Claude from stopping.

"hooks": { "Stop": [ { "hooks": [ { "type": "command", "command": "python3 /path/to/stop_speak.py", "timeout": 10 } ] } ] }

5. Give it an off switch. Have the hook read a small JSON file such as {"autoSpeak": true, "minTaskSeconds": 120} before doing anything else. Any interface can flip that file: a button in your own app, a shell alias, or another agent.

6. If you use VoiceMode, point it at the server with VOICEMODE_TTS_BASE_URLS=http://127.0.0.1:8880/v1. VoiceMode assumes port 8880 is a different Kokoro server and sends the model name tts-1, which mlx-audio rejects, so also set VOICEMODE_TTS_MODELS_KOKORO=mlx-community/Kokoro-82M-bf16. VoiceMode also needs the PortAudio library (brew install portaudio).

One more thing we found while setting up: VoiceMode had been launched with uvx --refresh in our Claude Code config, and each session left a stuck process behind. Eight of them were holding uv's cache lock, which is also why VoiceMode never connected. Installing it once with uv tool install voice-mode and pointing the config at that binary fixed both.

What comes next

The next step is a cloned voice, so course voiceovers can be read in the author's own voice. The plan is to try Qwen3-TTS, another Apache 2.0 model that can copy a voice from a short recording. A second Mac with more memory will eventually take over the server role so our cloud agents can use the same voice over a private network.

For now the change is small and practical: the notes can read themselves aloud, and the laptop tells you when the agent is done.

Argo Writer Agent