Voice cloning on a Mac with Voicebox and open models
Voice cloning on a Mac, all local. My Voicebox setup, which open models I use and why, and working API examples in curl, Python and TypeScript.
On this page
- Why I run voice cloning locally
- What I run
- Cloning a voice
- Your voice sample is the whole game
- Why I prefer Qwen3-TTS 1.7B
- Tutorial: call Voicebox from code
- 1. Check that the server is up
- 2. List models and profiles
- 3. Generate, poll, download
- The same flow as a script
- Transcribing: a gotcha
- Using it from an AI agent over MCP
- What went wrong, and what I’d do differently
- Whose voice you should clone
- Takeaway
- Further reading
TL;DR: I clone voices on my Mac with Voicebox and open-weight models, because a cloud service was too expensive for my whole Sprache course library. This post covers the setup, the models I tried, what went wrong, and how to call Voicebox’s local API from curl, Python and TypeScript.
This is for developers with an Apple Silicon Mac who are comfortable in a terminal. Everything below was run on an M2 Pro with 16 GB, macOS 26.3.1 and Voicebox 0.5.0, in November 2026. Voicebox 0.6.0 has been released since. I have not retested on it, so check the release notes if something differs.
Why I run voice cloning locally
Sprache has a feature where you hear yourself speak the language you are learning. I wanted it across the whole course library.
I tried ElevenLabs first. It was too expensive to cover everything. So I moved to open-weight models on my own Mac. I have since generated more than 20 episodes of voice and dubbed more than 2,000 pieces of material in German and French this way. An episode comes from Sprache’s story generator. It writes a story where you are the character, walking through the city you want to live in, in the language you are learning. In Sprache the voice is either the learner’s own or one of the presets.
It fits how I work. When a paid tool is almost right, I look at what it would take to build the missing part. I wrote more about that in Why I build my own tools.
What I run
Voicebox is a desktop app. Inside it are three programs:
| Program | What it does |
|---|---|
voicebox |
The desktop app. It starts the server. |
voicebox-server |
The local REST API. |
voicebox-mcp |
An MCP server over stdio. It is a thin client of the REST API. |
My machine:
| Chip | Apple M2 Pro |
| RAM | 16 GB |
| macOS | 26.3.1 |
| Voicebox | 0.5.0 (MIT licence) |
| Backend | MLX on Apple Silicon |
The models I have downloaded. Sizes are what the app reports.
| Model | Size | What I use it for |
|---|---|---|
| Qwen3-TTS 1.7B Base (MLX bf16 build: mlx-community) | 4.3 GB | Cloning a voice from a reference sample |
| Qwen3-TTS 1.7B CustomVoice | 4.3 GB | Built-in preset voices, with style instructions |
| Kokoro-82M | 312 MB | Small, very fast preset voices. No cloning. |
| Chatterbox | 3.1 GB | Multilingual cloning. The slowest one here. |
| Whisper base | 281 MB | Speech to text |
Qwen3-TTS and Kokoro are Apache-2.0 on their model cards. Chatterbox is MIT.
Cloning a voice
The flow has three parts, whatever the engine:
- Create a profile. A profile is a named voice.
- Give a cloned profile a reference recording of that voice.
- Generate speech from text with that profile and an engine that can clone: Qwen3-TTS Base or Chatterbox.
Preset voices work differently. They ship with the model (CustomVoice and Kokoro), so there is no sample to give. Voicebox marks each profile as cloned, preset or designed. A preset profile carries its own default engine. A cloned one has none, and you pick qwen or chatterbox when you generate.
Your voice sample is the whole game
With too little sample, the result is bad. Your data is gold. Get the recording right before you blame the model.
Why I prefer Qwen3-TTS 1.7B
Of the models I tried, Qwen3-TTS 1.7B gives me the best results. That is my ear, not a benchmark. I use both Qwen3-TTS 1.7B models: Base for cloning and CustomVoice for presets.
Speed is a separate question. I timed one short sentence per model (3.4 to 4.7 seconds of audio), from the generate call to the finished clip. Each number is a single run on a 16 GB machine with other apps open. Treat them as rough.
| Model | First run (includes model load) | Next run | Audio length |
|---|---|---|---|
| Kokoro 82M | 38.9 s | 1.0 s | 4.25 s |
| Qwen3-TTS 1.7B Base (cloned German voice) | 31.3 s | 9.8 s | 3.6 s |
| Qwen3-TTS 1.7B CustomVoice (preset) | 16.8 s | 21.3 s* | 3.8 to 4.2 s |
| Chatterbox (cloned German voice) | 68.2 s | 32.9 s | 3.8 s |
* One CustomVoice run took 49.1 s right after Chatterbox had loaded. With 16 GB, loading a 3 GB model pushes the others out of memory, so the next call pays to reload.
How I read this:
- Kokoro for speed. About a second for a short line. It cannot clone.
- Qwen3-TTS Base for cloning when quality matters. About ten seconds warm.
- Chatterbox when I need its multilingual cloning and can wait. About 33 seconds warm.
I do not have a trustworthy memory number for Qwen. The server process looked small in ps during Qwen runs, but that tool does not see MLX’s GPU memory. If you need a number, use Activity Monitor.
Tutorial: call Voicebox from code
Voicebox has a local REST API. It lets you script generation from your own code. Four facts first:
- The port is 17493, not 8000. Some write-ups say 8000. For the app build it is 17493.
- It listens on
127.0.0.1only. Other machines cannot reach it by default. - There is no auth. Anything on your Mac can call it. Do not forward the port.
- Generation is asynchronous.
POST /generatereturns at once withstatus: "generating". You poll, then download the audio.
The app has to be running, because it starts the server. The API description is at http://127.0.0.1:17493/openapi.json.
1. Check that the server is up
curl -s http://127.0.0.1:17493/health{"status":"healthy","model_loaded":true,"model_downloaded":true,"model_size":"1.7B","gpu_available":true,"gpu_type":"MPS (Apple Silicon)","vram_used_mb":null,"backend_type":"mlx","backend_variant":"cpu","gpu_compatibility_warning":null}2. List models and profiles
curl -s http://127.0.0.1:17493/models/status | jq '.models[] | select(.downloaded) | {model_name, hf_repo_id, size_mb, loaded}'
curl -s http://127.0.0.1:17493/profiles | jq -r '.[] | [.name, .voice_type, .default_engine] | @tsv'Each profile has an id, a name, a language, a voice_type and a default_engine. You need the id to generate.
3. Generate, poll, download
curl -s -X POST http://127.0.0.1:17493/generate -H 'content-type: application/json' \
-d '{"profile_id":"<id of your profile>","text":"Hello from a local voice server.","language":"en","engine":"qwen_custom_voice","model_size":"1.7B"}'{"id":"39f3abfe-4074-485e-8dec-071e1b0725cb","text":"Hello from a local voice server.","language":"en","audio_path":"","duration":0.0,"engine":"qwen_custom_voice","model_size":"1.7B","status":"generating","error":null,"source":"manual", …}Then poll GET /history/{id} until status is completed (or failed), and fetch the audio from GET /audio/{id}. The file is a 16-bit mono WAV at 24,000 Hz.
Use /history/{id} for polling. /generate/{id}/status also exists, but it is a server-sent events stream, not plain JSON. That is fine in a browser and awkward in a script.
Pass the right engine for the profile. A preset profile has its own default_engine. A cloned profile has none, so use qwen or chatterbox. If you leave engine out, it defaults to qwen, which needs a cloned profile. All three scripts below handle this: they use the profile’s default engine and fall back to qwen. I did not test which error comes back for a wrong pairing.
Other request fields I would look at: seed, instruct (style text, up to 500 characters), normalize, max_chunk_chars (long text is split) and crossfade_ms. Text can be up to 50,000 characters.
The same flow as a script
Each script checks health, finds a profile by name, generates, polls and saves out.wav. Change hs-en-Ryan to a profile name in your own app. I ran all three on the Mac above, and each one produced a playable WAV.
curl and jq (speak.sh; needs bash, curl and jq):
#!/usr/bin/env bash
# Voicebox 0.5.0 local API: text -> WAV. Needs curl and jq.
set -euo pipefail
B=${VOICEBOX_URL:-http://127.0.0.1:17493}
PROFILE_NAME=${1:-hs-en-Ryan}
TEXT=${2:-"Hello from a local voice server."}
OUT=${3:-out.wav}
curl -sf "$B/health" | jq -r '"health: \(.status), backend: \(.backend_type)"' \
|| { echo "Voicebox not reachable at $B. Is the app running?" >&2; exit 1; }
PID=$(curl -sf "$B/profiles" | jq -r --arg n "$PROFILE_NAME" '.[] | select(.name==$n) | .id')
[ -n "$PID" ] || { echo "profile not found: $PROFILE_NAME" >&2; exit 1; }
ENGINE=$(curl -sf "$B/profiles/$PID" | jq -r '.default_engine // "qwen"')
GID=$(curl -sf -X POST "$B/generate" -H 'content-type: application/json' \
-d "$(jq -n --arg p "$PID" --arg t "$TEXT" --arg e "$ENGINE" '{profile_id:$p,text:$t,engine:$e,language:"en"}')" | jq -r .id)
for _ in $(seq 1 120); do
STATUS=$(curl -sf "$B/history/$GID" | jq -r .status)
[ "$STATUS" = completed ] && break
[ "$STATUS" = failed ] && { echo "generation failed" >&2; exit 1; }
sleep 1
done
curl -sf "$B/audio/$GID" -o "$OUT"
echo "saved $OUT ($(wc -c < "$OUT") bytes)"health: healthy, backend: mlx
saved out.wav (153644 bytes)Python (speak.py; standard library only, tested on Python 3.13):
"""Voicebox 0.5.0 local API: text -> WAV. Tested on Python 3.13, standard library only."""
import json, sys, time, urllib.request
BASE = "http://127.0.0.1:17493"
def api(path, body=None):
req = urllib.request.Request(
BASE + path,
data=json.dumps(body).encode() if body is not None else None,
headers={"content-type": "application/json"},
)
with urllib.request.urlopen(req, timeout=300) as r:
return json.load(r)
def speak(text, profile_name="hs-en-Ryan", out="out.wav", language="en"):
profile = next((p for p in api("/profiles") if p["name"] == profile_name), None)
if profile is None:
raise SystemExit(f"profile not found: {profile_name}")
gen = api("/generate", {
"profile_id": profile["id"],
"text": text,
"language": language,
"engine": profile["default_engine"] or "qwen",
})
while True:
status = api(f"/history/{gen['id']}")
if status["status"] == "completed":
break
if status["status"] == "failed":
raise SystemExit(f"generation failed: {status['error']}")
time.sleep(0.5)
with urllib.request.urlopen(f"{BASE}/audio/{gen['id']}") as r, open(out, "wb") as f:
f.write(r.read())
return status["duration"]
if __name__ == "__main__":
print(api("/health")["status"])
seconds = speak(sys.argv[1] if len(sys.argv) > 1 else "Hello from a local voice server.")
print(f"saved out.wav, {seconds:.2f}s of audio")healthy
saved out.wav, 2.32s of audioTypeScript (speak.ts; I ran it on Node 23.9, which runs .ts files directly and prints an ExperimentalWarning. Bun and Deno should also work. I did not test Node 22 or older, which may need tsx or a build step):
// Voicebox 0.5.0 local API: text -> WAV. Tested on Node 23.9 (runs .ts natively
// with `node speak.ts`); Bun and Deno should also work.
import { writeFile } from "node:fs/promises";
const BASE = "http://127.0.0.1:17493";
type Profile = { id: string; name: string; default_engine: string | null };
type Generation = { id: string; status: string; duration: number; error: string | null };
async function api<T>(path: string, body?: unknown): Promise<T> {
const res = await fetch(BASE + path, {
method: body === undefined ? "GET" : "POST",
headers: { "content-type": "application/json" },
body: body === undefined ? undefined : JSON.stringify(body),
});
if (!res.ok) throw new Error(`${path} -> ${res.status} ${await res.text()}`);
return res.json() as Promise<T>;
}
async function speak(text: string, profileName = "hs-en-Ryan", out = "out.wav") {
const profiles = await api<Profile[]>("/profiles");
const profile = profiles.find((p) => p.name === profileName);
if (!profile) throw new Error(`profile not found: ${profileName}`);
const gen = await api<Generation>("/generate", {
profile_id: profile.id,
text,
language: "en",
engine: profile.default_engine ?? "qwen",
});
let status = gen;
while (status.status !== "completed") {
if (status.status === "failed") throw new Error(`generation failed: ${status.error}`);
await new Promise((r) => setTimeout(r, 500));
status = await api<Generation>(`/history/${gen.id}`);
}
const audio = await fetch(`${BASE}/audio/${gen.id}`);
await writeFile(out, Buffer.from(await audio.arrayBuffer()));
return status.duration;
}
const health = await api<{ status: string }>("/health");
console.log(health.status);
const seconds = await speak(process.argv[2] ?? "Hello from a local voice server.");
console.log(`saved out.wav, ${seconds.toFixed(2)}s of audio`);healthy
saved out.wav, 3.12s of audioThe audio length and file size differ between runs, so your numbers will not match mine. The first call after launch can take much longer while the model loads (16 to 68 seconds in my tests), so keep the timeouts generous.
Transcribing: a gotcha
Voicebox also exposes POST /transcribe, which uses Whisper base:
curl -s -X POST http://127.0.0.1:17493/transcribe -F [email protected]Once, on a 2 second clip, it returned the right words and then looped: Hello, from a local voice server! today today today today …. A second test of the same sentence on a fresh clip came back clean, so it does not always happen. Whisper models can repeat themselves on very short audio. If you see it, use longer audio, a larger Whisper model, or trim the repeats.
Using it from an AI agent over MCP
voicebox-mcp is an MCP server over stdio. It exposes four tools:
voicebox.speak(text,profile,engine,personality,language)voicebox.transcribe(audio_base64oraudio_path,language,model)voicebox.list_capturesvoicebox.list_profiles
I first checked it with a raw JSON-RPC handshake over stdio. initialize reported {"name":"voicebox","version":"3.2.4"}, which is the MCP server’s own version, not the app’s. list_profiles returned my profiles. This call started a generation:
{"name": "voicebox.speak", "arguments": {"text": "Hello from MCP.", "profile": "hs-en-Ryan"}}{"generation_id":"f5827029-…","status":"generating","profile":"hs-en-Ryan","source":"mcp","poll_url":"/generate/f5827029-…/status"}speak returns an id, not audio. The caller still has to poll /history/{id} and fetch /audio/{id}, as above.
To register it with Claude Code, the command is:
claude mcp add voicebox -- /Applications/Voicebox.app/Contents/MacOS/voicebox-mcpThis worked in Claude Code. claude mcp list showed voicebox as connected, and a call to list_profiles returned the profiles. It was tested with a temporary config, so my own Claude settings were not touched. The tool names Claude Code shows may differ slightly from the server’s names above.
What went wrong, and what I’d do differently
- Thin samples gave bad output. With too little voice sample, the result is bad.
- Cold starts are long. First use of a model took 16 to 68 seconds in my tests. Keep one engine loaded for a batch job.
- 16 GB is tight. Switching between Qwen and Chatterbox made later calls slow, because each load pushed the other out of memory. If you batch work, finish one model before you switch.
- The port. Some write-ups say 8000, but the app build listens on 17493. If calls fail, check the port first, and that the app is running.
Whose voice you should clone
My rule is simple: only my own voice, or a voice I have clear permission to use.
A cloned voice can say things the real person never said. Permission is what keeps that honest. Voicebox has no auth on its API, so keep it on localhost, and keep reference recordings private.
Takeaway
Local voice cloning on a Mac is practical. A 16 GB machine runs it, the API is easy to script, and a good voice sample matters a lot.
Next step: copy one of the scripts above, change the profile name and run it. To hear what I use it for, try Sprache and tell me what is missing.