diff --git a/README.md b/README.md index 7760a30..22f765a 100644 --- a/README.md +++ b/README.md @@ -39,8 +39,8 @@ Flextalk voices (once; needs the extracted installer, see `docs/NOTES.md`): Host: - # Winelib (native ELF + launcher) - winegcc -m32 -O2 -o build/sapi4host host/sapi4host.c -I../reference \ + # Winelib (native ELF + launcher): build/sapi4host-winlib.exe + .exe.so + winegcc -m32 -O2 -o build/sapi4host-winlib host/sapi4host.c -I../reference \ -lole32 -luser32 -ladvapi32 # PE fallback (runs under wine): @@ -55,6 +55,18 @@ Host: Options: `-v` voice, `-t` text, `-o` output WAV, `-p` pitch, `-s` speed. +## Veskslop integration + + export VESKSLOP_TTS_CMD="$PWD/setup/vekslop-tts.sh {out} {text_file}" + export VESKSLOP_TTS_VOICES="$(setup/vekslop-tts.sh --voices)" + +The wrapper maps voice IDs (`sam`, `mary`, `mike`, `keith-bell`, `kerry-watson`, plus the +MSTTS variants), reads `VESKSLOP_TTS_VOICE`, `VESKSLOP_TTS_SPEED` (multiplier) and +`VESKSLOP_TTS_PITCH` (semitones), starts Xvfb when `DISPLAY` is unset, and writes a +48 kHz mono s16le WAV trimmed, faded and normalized to about -16 LUFS (-3 dBFS true peak). +Flextalk voices ignore pitch (engine limitation); everything else maps as documented in +`docs/TTS-DESIGN.md`. + ## Layout - `host/sapi4host.c` — host: voice select, pitch/speed, text -> WAV @@ -63,6 +75,7 @@ Options: `-v` voice, `-t` text, `-o` output WAV, `-p` pitch, `-s` speed. - `setup/setup-flextalk.sh` — Flextalk files + registry - `setup/inf2reg.py` — INF AddReg -> `.reg` converter - `setup/run.sh` — run helper (prefix + Xvfb) +- `setup/vekslop-tts.sh` — Veskslop command-provider wrapper (post-processing included) - `docs/NOTES.md` — registry/CLSID facts, runtime RE notes, gotchas - `docs/TTS-DESIGN.md` — integration contract with Veskslop plus engine clarifications @@ -71,8 +84,8 @@ Options: `-v` voice, `-t` text, `-o` output WAV, `-p` pitch, `-s` speed. - `xvfb-run` is flaky on this host. `setup/run.sh` starts a persistent Xvfb (`:77`) instead; Wine needs a working X display because the runtime creates windows. -- Flextalk's default pitch/speed report as maximum (250/390) through - `ITTSAttributes`; values are still adjustable with `-p`/`-s`. +- Flextalk ignores pitch entirely (byte-identical output at any pitch setting); use speed. + The host now leaves pitch/speed untouched unless passed, so voices use their own profile. - The Flextalk archive lacks `data2.cab`; `unshield -O` extracts `data1.cab` but one tail file (`WatsonUNInstall.dll`) is incomplete. diff --git a/docs/TTS-DESIGN.md b/docs/TTS-DESIGN.md index a2811c3..af05172 100644 --- a/docs/TTS-DESIGN.md +++ b/docs/TTS-DESIGN.md @@ -51,10 +51,12 @@ these placeholders, splits the string on whitespace, and runs the result directl | `{out}` | Output file path, extension `.wav`. Required. | | `{text_file}` | A UTF-8 text file holding the line. | | `{voice}` | The selected voice ID, or an empty string. | -| `{setting}` | Reserved for speed/pitch once Veskslop grows settings support. | +| `{speed}` | The rate multiplier, or an empty string when unset. | +| `{pitch}` | The pitch offset in semitones, or an empty string when unset. | -The process also receives `VESKSLOP_TTS_VOICE` in the environment. Write the audio to `{out}` and -exit 0. Non-zero exits are errors; stderr is captured into the log message, so keep it short. +The process also receives `VESKSLOP_TTS_VOICE`, and `VESKSLOP_TTS_SPEED` / `VESKSLOP_TTS_PITCH` +when those are set. Write the audio to `{out}` and exit 0. Non-zero exits are errors; stderr is +captured into the log message, so keep it short. Command mode is best for fast, stateless engines. It has no shell quoting: paths with spaces and shell tricks cannot work. Use a small wrapper script if you need options. @@ -120,15 +122,25 @@ Recommended HTTP shape (extras are optional; an engine may ignore them): { "text": "...", "voice": "anna", "speed": 1.15, "pitch": -2 } ``` -Recommended command shape: `{setting}` is reserved. Until Veskslop passes it, read defaults from -the engine's own config file or environment. +Recommended command shape: `{speed}` and `{pitch}` are substituted when set, and the same values +are exported as `VESKSLOP_TTS_SPEED` and `VESKSLOP_TTS_PITCH`. Read either; the environment form +survives placeholder-free commands. ### How Veskslop should add first-class settings -A future change would add `speed` and `pitch` to `tts` preferences, pass them to the provider -alongside `voice`, and add two controls to the TTS panel. For the command provider, `{speed}` and -`{pitch}` placeholders (or one `{settings_file}` JSON file) would be substituted the same way as -`{voice}`. Design the engine API so it works with defaults when these fields are absent. +Implemented on the Veskslop side (2026-09-15): + +- `tts` preferences carry `speed` (0.5–2.0) and `pitch` (-12–+12, semitones), both nullable. +- `/api/audio/speak` and the MCP `speak` tool accept optional `speed` and `pitch` that override + the preferences for that line. +- The command provider substitutes `{speed}`/`{pitch}` and exports `VESKSLOP_TTS_SPEED` / + `VESKSLOP_TTS_PITCH` only when the value is set. +- The HTTP provider adds `speed` and `pitch` to the request body only when set. +- The TTS panel has speed and pitch sliders; changes are saved to the daemon preferences. +- Command mode advertises voices through `VESKSLOP_TTS_VOICES` + (`id`, `id|Name`, `id|Name|gender`, or `id|Name|gender|language`, comma separated). While a + custom command is set, the engine's declared voices replace the espeak list, so no misleading + espeak entries appear. Reading `/voices` for the HTTP provider is still future work. Do not encode settings into the voice string (for example `anna+speed1.2`). It breaks voice lists and is hard to validate. @@ -160,13 +172,17 @@ return the cached file or bytes. This is useful for repeated lines and for expen ## Veskslop changes needed -If the engine wants first-class voice lists, speed, or pitch in the Veskslop UI, Veskslop must: +Status as of 2026-09-15: -1. Read the engine's `/voices` endpoint and merge it into `/api/audio/tts/voices`. -2. Add `speed` and `pitch` to the `tts` preferences and to the `speak` request. -3. Add controls to the TTS panel and pass the values through to the provider. +1. `speed` and `pitch` are first-class: preferences, per-request overrides on `/api/audio/speak` + and MCP `speak`, env exports and placeholders for the command provider, request fields for the + HTTP provider, and two sliders in the TTS panel. +2. Command-mode voices come from `VESKSLOP_TTS_VOICES`. Reading the engine's `GET /voices` for the + HTTP provider is not implemented yet. Until then, declare voices with the env var or configure + the voice inside the wrapper. +3. No changes are needed for the audio format; the engine post-processes to the required output. -Until those exist, use defaults configured in the engine. +Until `/voices` is wired, use defaults configured in the engine. ## Acceptance checklist @@ -209,17 +225,19 @@ record what the engine needs to interpret, add, or hide._ back to `sam`. - `/voices` (HTTP mode) reports `language: "en"`, gender, and the speed/pitch ranges below. -### Speed and pitch mapping +### Speed and pitch mapping (measured 2026-09-15) -- SAPI4 uses engine units, not multipliers or semitones. Observed values: - - MSTTS Sam: speed 30-450 (default 150), pitch 50-200 (default 100). - - Flextalk: speed 50-390, pitch 50-250; the engine reports the **defaults as the - maximum**, so the wrapper uses the voice profile from the registry as the natural - values (Keith Bell: pitch 27, rate 33; Kerry Watson: pitch 65, rate 32). -- Mapping (approximate, stated in `/voices`): `speed 1.0` = the voice's natural rate; - 0.5-2.0 scales that rate and clamps to the engine range. `pitch` ±12 semitones scales the - engine pitch range by about ±25% around the natural value. -- Command mode: `{setting}` is unused for now. The wrapper reads `VESKSLOP_TTS_SPEED` and +- SAPI4 uses engine units, not multipliers or semitones. Measured on the dev host: + - MSTTS Sam: speed 30-450, natural about 150; pitch 50-200 with a real audible effect + (F0 about 134 Hz at pitch 50, about 199 Hz at pitch 200 on the test phrase). + - Flextalk: speed 50-390, natural about 300 (output with no setting is byte-identical to + `-s 300`); **pitch has no effect** (byte-identical output at pitch 100/150/200), so the + wrapper ignores `pitch` for Keith Bell and Kerry Watson. +- Mapping implemented in `setup/vekslop-tts.sh`: `speed 1.0` = the voice's natural rate + (150 MSTTS, 300 Flextalk), scaled by the multiplier and clamped to the engine range. + `pitch` for MSTTS maps ±12 semitones to engine units `100 ± 6.25·st`, clamped to 50-200. + Pitch is a no-op for Flextalk voices. +- `{setting}` is unused; the wrapper reads `VESKSLOP_TTS_SPEED` and `VESKSLOP_TTS_PITCH` from the environment when Veskslop later exports them. ### Text handling @@ -231,8 +249,12 @@ record what the engine needs to interpret, add, or hide._ ### Transport and latency -- Command provider (first target): a wrapper script reads `{text_file}`, calls the engine, - post-processes, writes `{out}`, and exits non-zero with one short stderr line on failure. +- Command provider, implemented as `setup/vekslop-tts.sh`: + `VESKSLOP_TTS_CMD="/setup/vekslop-tts.sh {out} {text_file}"` and + `VESKSLOP_TTS_VOICES="$(/setup/vekslop-tts.sh --voices)"`. The wrapper reads the + text file, transliterates UTF-8 to CP1252, starts or reuses an Xvfb when `DISPLAY` is + unset, calls the engine, then post-processes (trim, fades, two-pass `loudnorm`, 48 kHz + mono s16le). Verified: -17.7 LUFS integrated, -3.0 dBFS true peak. - Measured on the dev host (warm Wine): Sam line about 288 ms wall, Flextalk line about 41 ms wall, including process start, synthesis, and file write. Both are inside the 500 ms budget. @@ -253,7 +275,30 @@ record what the engine needs to interpret, add, or hide._ ### Extra acceptance items for the engine -- [ ] Post-processed WAV is 48 kHz mono s16le, peak about -3 dBFS, no clipping, fades present. -- [ ] Leading silence under 100 ms. -- [ ] Voice ID fallback works; TruVoice entries filtered. -- [ ] Latency recorded (Sam about 288 ms, Flextalk about 41 ms on the dev host). +- [x] Post-processed WAV is 48 kHz mono s16le, peak about -3 dBFS, no clipping, fades present. +- [x] Leading silence under 100 ms. +- [x] Voice ID fallback works; TruVoice entries filtered. +- [x] Latency recorded (Sam about 288 ms, Flextalk about 41 ms on the dev host). + +## Responses from the Veskslop side + +_Added 2026-09-15._ + +- **Env vars and placeholders**: `VESKSLOP_TTS_SPEED` and `VESKSLOP_TTS_PITCH` are exported only + when a value is set; `{speed}` and `{pitch}` substitute the same values. Reading the environment + is the safer option because it works even when the command has no placeholders. +- **Defaults**: a missing value means "engine default". Leave the sliders at 1.00x and 0 st in + Veskslop to keep the fields out of the request entirely. +- **`loudnorm` caution**: single-pass `loudnorm` can pump or over-amplify noise on short lines. + Set the target explicitly (`loudnorm=I=-16:TP=-1.5:LRA=11`) and verify a few lines with + `ebur128`. A fixed gain that hits about -3 dBFS peak with a consistent RMS is often steadier for + one-line utterances. Ducking lowers the sink to 30%, so keep peaks conservative and loudness + even between lines. +- **Display**: the daemon does not export `DISPLAY` to TTS children. The wrapper must start or + find its own Xvfb and set `DISPLAY` itself, as the project helper already does. Do not rely on + the client's display. +- **Voice declarations**: list only usable IDs in `VESKSLOP_TTS_VOICES`. Veskslop hides the + espeak list while a custom command is set, so a missing declaration means an empty voice list, + not a fallback. +- **Licensing**: keep the engine DLLs and voice data local and out of any repository; Veskslop + never bundles or redistributes them. diff --git a/host/sapi4host.c b/host/sapi4host.c index 26623ce..dec8518 100644 --- a/host/sapi4host.c +++ b/host/sapi4host.c @@ -257,30 +257,34 @@ int main(int argc, char **argv) return 1; } - WORD pmin = 0, pmax = 0, pdef = 0; - DWORD smin = 0, smax = 0, sdef = 0; - attr->lpVtbl->PitchGet(attr, &pdef); - attr->lpVtbl->SpeedGet(attr, &sdef); - attr->lpVtbl->PitchSet(attr, TTSATTR_MINPITCH); - attr->lpVtbl->PitchGet(attr, &pmin); - attr->lpVtbl->PitchSet(attr, TTSATTR_MAXPITCH); - attr->lpVtbl->PitchGet(attr, &pmax); - attr->lpVtbl->SpeedSet(attr, TTSATTR_MINSPEED); - attr->lpVtbl->SpeedGet(attr, &smin); - attr->lpVtbl->SpeedSet(attr, TTSATTR_MAXSPEED); - attr->lpVtbl->SpeedGet(attr, &smax); + /* Pitch and speed are optional: when unset, leave the engine's own + voice profile untouched (probing/setting would override it). */ + if (pitch >= 0 || speed >= 0) { + WORD pmin = 0, pmax = 0; + DWORD smin = 0, smax = 0; - if (pitch < 0) pitch = pdef; - if (speed < 0) speed = sdef; - if (pitch < (int)pmin) pitch = pmin; - if (pitch > (int)pmax) pitch = pmax; - if (speed < (int)smin) speed = smin; - if (speed > (int)smax) speed = smax; - printf("pitch: %d (%d..%d) speed: %d (%lu..%lu)\n", - pitch, pmin, pmax, speed, (unsigned long)smin, (unsigned long)smax); - - attr->lpVtbl->PitchSet(attr, (WORD)pitch); - attr->lpVtbl->SpeedSet(attr, (DWORD)speed); + if (pitch >= 0) { + attr->lpVtbl->PitchSet(attr, TTSATTR_MINPITCH); + attr->lpVtbl->PitchGet(attr, &pmin); + attr->lpVtbl->PitchSet(attr, TTSATTR_MAXPITCH); + attr->lpVtbl->PitchGet(attr, &pmax); + if (pitch < (int)pmin) pitch = pmin; + if (pitch > (int)pmax) pitch = pmax; + attr->lpVtbl->PitchSet(attr, (WORD)pitch); + printf("pitch: %d (%d..%d)\n", pitch, pmin, pmax); + } + if (speed >= 0) { + attr->lpVtbl->SpeedSet(attr, TTSATTR_MINSPEED); + attr->lpVtbl->SpeedGet(attr, &smin); + attr->lpVtbl->SpeedSet(attr, TTSATTR_MAXSPEED); + attr->lpVtbl->SpeedGet(attr, &smax); + if (speed < (int)smin) speed = smin; + if (speed > (int)smax) speed = smax; + attr->lpVtbl->SpeedSet(attr, (DWORD)speed); + printf("speed: %d (%lu..%lu)\n", + speed, (unsigned long)smin, (unsigned long)smax); + } + } { DWORD regkey = 0; diff --git a/setup/run.sh b/setup/run.sh index c2e297f..21e8357 100755 --- a/setup/run.sh +++ b/setup/run.sh @@ -16,7 +16,7 @@ if [ -z "${DISPLAY:-}" ]; then export DISPLAY=:77 fi -HOST="$ROOT/build/sapi4host" +HOST="$ROOT/build/sapi4host-winlib.exe" if [ ! -x "$HOST" ]; then HOST="$ROOT/build/sapi4host.exe" fi diff --git a/setup/vekslop-tts.sh b/setup/vekslop-tts.sh new file mode 100755 index 0000000..dab692a --- /dev/null +++ b/setup/vekslop-tts.sh @@ -0,0 +1,134 @@ +#!/bin/sh +# Veskslop TTS command-provider wrapper for MicroslopSam. +# +# Recommended Veskslop configuration: +# VESKSLOP_TTS_CMD="/setup/vekslop-tts.sh {out} {text_file}" +# VESKSLOP_TTS_VOICES="$(setup/vekslop-tts.sh --voices)" +# +# Inputs: +# $1 {out} output WAV path +# $2 {text_file} UTF-8 text file +# $3... optional: voice id, speed multiplier, pitch semitones +# env VESKSLOP_TTS_VOICE, VESKSLOP_TTS_SPEED, VESKSLOP_TTS_PITCH +# +# Output: 48 kHz mono s16le WAV, trimmed, faded, loudness-normalized. +set -eu + +HERE=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +ROOT=$(CDPATH= cd -- "$HERE/.." && pwd) +export WINEPREFIX=${WINEPREFIX:-$HOME/.wine-microslopsam} +export WINEARCH=win32 + +VOICES="sam|Sam|male|en,mary|Mary|female|en,mike|Mike|male|en,keith-bell|Keith Bell|male|en,kerry-watson|Kerry Watson|female|en,mike-telephone|Mike (for Telephone)|male|en,mary-telephone|Mary (for Telephone)|female|en,mike-hall|Mike in Hall|male|en,mike-stadium|Mike in Stadium|male|en,mike-space|Mike in Space|male|en,mary-hall|Mary in Hall|female|en,mary-stadium|Mary in Stadium|female|en,mary-space|Mary in Space|female|en" + +if [ "${1:-}" = "--voices" ]; then + echo "$VOICES" + exit 0 +fi + +OUT=${1:?missing out path} +TEXTFILE=${2:?missing text file} +VOICE=${VESKSLOP_TTS_VOICE:-${3:-sam}} +SPEED=${VESKSLOP_TTS_SPEED:-${4:-}} +PITCH=${VESKSLOP_TTS_PITCH:-${5:-}} + +case "$VOICE" in + sam) ENGVOICE="Sam" ;; + mary) ENGVOICE="Mary" ;; + mike) ENGVOICE="Mike" ;; + keith-bell) ENGVOICE="Keith Bell" ;; + kerry-watson) ENGVOICE="Kerry Watson" ;; + mike-telephone) ENGVOICE="Mike (for Telephone)" ;; + mary-telephone) ENGVOICE="Mary (for Telephone)" ;; + mike-hall) ENGVOICE="Mike in Hall" ;; + mike-stadium) ENGVOICE="Mike in Stadium" ;; + mike-space) ENGVOICE="Mike in Space" ;; + mary-hall) ENGVOICE="Mary in Hall" ;; + mary-stadium) ENGVOICE="Mary in Stadium" ;; + mary-space) ENGVOICE="Mary in Space" ;; + *) ENGVOICE="Sam" ;; +esac + +# Engine units. Measured naturals: MSTTS ~150, Flextalk ~300 (words/min scale). +# Flextalk ignores ITTSAttributes pitch (verified byte-identical output). +case "$ENGVOICE" in + "Keith Bell"|"Kerry Watson") NAT_SPEED=300; SMAX=390 ;; + *) NAT_SPEED=150; SMAX=450 ;; +esac +SET_ARGS="" +if [ -n "$SPEED" ]; then + S=$(awk -v m="$SPEED" -v n="$NAT_SPEED" 'BEGIN{printf "%d", n*m+0.5}') + [ "$S" -lt 50 ] && S=50 + [ "$S" -gt "$SMAX" ] && S=$SMAX + SET_ARGS="$SET_ARGS -s $S" +fi +if [ -n "$PITCH" ]; then + case "$ENGVOICE" in + "Keith Bell"|"Kerry Watson") : ;; + *) + P=$(awk -v st="$PITCH" 'BEGIN{printf "%d", 100 + st*6.25}') + [ "$P" -lt 50 ] && P=50 + [ "$P" -gt 200 ] && P=200 + SET_ARGS="$SET_ARGS -p $P" ;; + esac +fi + +# The daemon does not export DISPLAY; keep a private Xvfb for the engine. +if [ -z "${DISPLAY:-}" ]; then + if ! [ -S /tmp/.X11-unix/X77 ]; then + Xvfb :77 -screen 0 1024x768x24 >/dev/null 2>&1 & + sleep 1 + fi + export DISPLAY=:77 +fi + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +RAW="$TMP/raw.wav" +TXT="$TMP/text.cp1252" + +# SAPI4 is ANSI/CP1252; transliterate UTF-8 (emoji become '?'). +if command -v iconv >/dev/null 2>&1; then + iconv -f UTF-8 -t WINDOWS-1252//TRANSLIT "$TEXTFILE" > "$TXT" 2>/dev/null \ + || cp "$TEXTFILE" "$TXT" +else + cp "$TEXTFILE" "$TXT" +fi + +HOST="$ROOT/build/sapi4host-winlib.exe" +if [ ! -x "$HOST" ]; then HOST="$ROOT/build/sapi4host.exe"; fi +if [ ! -x "$HOST" ]; then echo "engine host not built" >&2; exit 1; fi +if ! command -v ffmpeg >/dev/null 2>&1; then echo "ffmpeg not found" >&2; exit 1; fi + +# shellcheck disable=SC2086 +if ! "$HOST" -v "$ENGVOICE" -t "$(cat "$TXT")" -o "$RAW" $SET_ARGS \ + >/dev/null 2>"$TMP/err"; then + head -c 400 "$TMP/err" >&2 + exit 1 +fi +[ -s "$RAW" ] || { echo "engine produced no audio" >&2; exit 1; } + +SIL="silenceremove=start_periods=1:start_duration=0.05:start_silence=0.03:start_threshold=-50dB" +PRE="highpass=f=40,$SIL,afade=t=in:st=0:d=0.008,areverse,$SIL,afade=t=in:st=0:d=0.008,areverse" +LN="loudnorm=I=-16:TP=-3:LRA=11" + +# Two-pass loudnorm: measure first, then apply with the measured values. +M=$(ffmpeg -hide_banner -nostats -i "$RAW" -af "$PRE,$LN:print_format=json" \ + -f null - 2>&1 || true) +I=$(printf '%s' "$M" | sed -n 's/.*"input_i" *: *"\([^"]*\)".*/\1/p') +TP=$(printf '%s' "$M" | sed -n 's/.*"input_tp" *: *"\([^"]*\)".*/\1/p') +LRA=$(printf '%s' "$M" | sed -n 's/.*"input_lra" *: *"\([^"]*\)".*/\1/p') +TH=$(printf '%s' "$M" | sed -n 's/.*"input_thresh" *: *"\([^"]*\)".*/\1/p') +OFF=$(printf '%s' "$M" | sed -n 's/.*"target_offset" *: *"\([^"]*\)".*/\1/p') +if [ -n "$I" ] && [ -n "$TP" ] && [ -n "$LRA" ] && [ -n "$TH" ] && [ -n "$OFF" ]; then + LN="$LN:measured_I=$I:measured_TP=$TP:measured_LRA=$LRA:measured_thresh=$TH:offset=$OFF:linear=true" +fi + +TMPOUT="$TMP/out.wav" +if ! ffmpeg -hide_banner -nostats -y -i "$RAW" -af "$PRE,$LN" \ + -ar 48000 -ac 1 -c:a pcm_s16le "$TMPOUT" >/dev/null 2>"$TMP/ff"; then + head -c 400 "$TMP/ff" >&2 + exit 1 +fi +[ -s "$TMPOUT" ] || { echo "post-processing produced no audio" >&2; exit 1; } +mv -f "$TMPOUT" "$OUT"