diff --git a/README.md b/README.md index cb4ad66..97342cc 100644 --- a/README.md +++ b/README.md @@ -64,8 +64,8 @@ The wrapper maps voice IDs (`sam`, `mary`, `mike`, `keith-bell`, `kerry-watson`, MSTTS variants), reads `VESKSLOP_TTS_VOICE`, `VESKSLOP_TTS_SPEED` (multiplier) and `VESKSLOP_TTS_PITCH` (semitones), starts Xvfb when `DISPLAY` is unset, and writes a 48 kHz mono s16le WAV trimmed, faded and normalized to about -16 LUFS (-3 dBFS true peak). -Flextalk voices ignore pitch (engine limitation) and are declared with the `no-pitch` -flag; everything else maps as documented in `docs/TTS-DESIGN.md`. +Speed and pitch apply to every voice (Flextalk's numeric settings must be `REG_DWORD`, +which the setup script writes). Mapping details are in `docs/TTS-DESIGN.md`. ## Layout @@ -84,8 +84,8 @@ flag; everything else maps as documented in `docs/TTS-DESIGN.md`. - `xvfb-run` is flaky on this host. `setup/run.sh` starts a persistent Xvfb (`:77`) instead; Wine needs a working X display because the runtime creates windows. -- Flextalk ignores pitch entirely (byte-identical output at any pitch setting); use speed. - The host now leaves pitch/speed untouched unless passed, so voices use their own profile. +- Flextalk numeric speaker settings must be `REG_DWORD` (the installer script handles + this); `REG_SZ` values cause a monotone buzz. Pitch and speed work on all voices. - The Flextalk archive lacks `data2.cab`; `unshield -O` extracts `data1.cab` but one tail file (`WatsonUNInstall.dll`) is incomplete. diff --git a/docs/NOTES.md b/docs/NOTES.md index a3ad793..618d3c8 100644 --- a/docs/NOTES.md +++ b/docs/NOTES.md @@ -33,7 +33,9 @@ From `msttsl/mstts.inf` (`[TTSReg]`): `HomographDictionary="1 ft_hgrap.dll"`, `ProperNameDictionary="3 ft_namsa.dll"`, `LetterToSoundDictionary="4 ft_l2s.dll"`. - Options: `...\FlexTalk\2.0\Options` `Gender=Male`, `SampleRate=11025`. -- Speakers (per-voice settings; names come from these values): +- Speakers (per-voice settings; names come from these values). **All numeric + values must be REG_DWORD**: with REG_SZ the engine reads garbage and + synthesizes a monotone buzz (pitch fixed at max, pitch control dead). - `{11260610-C84A-11ce-AEC4-0000C0E9CB87}`: Mode Name / Speaker Name "Keith Bell", Pitch 27, Speaking Rate 33, Head Size 46, Expression 59, Lowering 40, Prominences 17, Age 30, Volume 100. @@ -71,6 +73,16 @@ final file `WatsonUNInstall.dll` is incomplete). - 32-bit engines require a 32-bit prefix (`WINEARCH=win32`). - `sudo` on this machine requires a password; installs are manual. +## Engine units (measured) + +- speed: MSTTS range 30-450, Flextalk 50-390; the natural rate is about 150 for every + voice (duration at 100/200/300 fits a natural value near 140-150). +- pitch: engine units, roughly linear in F0 near the natural point (~1.1 Hz per unit). + Natural values: about 100 (MSTTS), 106 (Keith Bell), 186 (Kerry Watson); Flextalk max 250. + Verified with the corrected DWORD registry: Kerry 0 st = 209 Hz, -6 st = 148 Hz, + +6 st = 250 Hz (clamped at max); Keith 0 st = 119 Hz, +6 st = 170 Hz. +- The wrapper maps semitones as `natural * 2^(st/12)` and clamps to the range. + ## Decompiled corpora (Ghidra) - `/mnt/ghidra/decomp/msttssyn.dll/` (786 functions, engine) diff --git a/docs/TTS-DESIGN.md b/docs/TTS-DESIGN.md index 63f6736..e5e3de3 100644 --- a/docs/TTS-DESIGN.md +++ b/docs/TTS-DESIGN.md @@ -267,7 +267,8 @@ record what the engine needs to interpret, add, or hide._ ### Quirks the wrapper must hide -- Flextalk reports default pitch/speed as maximum through `ITTSAttributes`. +- Flextalk numeric speaker settings must be `REG_DWORD`; `REG_SZ` values make the engine + synthesize a buzz. `setup/setup-flextalk.sh` writes them correctly. - The mode list includes TruVoice names with no installed voice data; filter those out of `/voices` (or mark them `unavailable`). - Wine needs a working X display; the project helper starts a persistent Xvfb. @@ -301,6 +302,19 @@ _Added 2026-09-15._ - **Display**: the daemon does not export `DISPLAY` to TTS children. The wrapper must start or find its own Xvfb and set `DISPLAY` itself, as the project helper already does. Do not rely on the client's display. +- **Engine selection**: the Speech settings and the TTS panel list every configured engine. The + command engine appears under `VESKSLOP_TTS_NAME` (default "custom engine") and is selectable + directly; Auto uses the command engine first. Selecting it hides the language row and shows the + `VESKSLOP_TTS_VOICES` list. +- **Start engine**: the Speech settings have a **Start engine** button (`POST /api/audio/tts/warm`, + MCP `start_tts_engine`) that runs one throwaway synthesis to warm the command engine (Xvfb, Wine, + host). Keep the wrapper able to handle a short sentence; the warm-up uses "The quick brown fox + jumps over the lazy dog." and reports the elapsed milliseconds. Two-pass `loudnorm` must not fail + on that input. +- **Lead-in silence**: Veskslop prepends a short silence at playback time (`VESKSLOP_TTS_LEAD_MS`, + default 150 ms) so Discord's voice pipeline is ready before the first syllable. It is added with + `adelay` in the player, not in the audio file. The engine needs no change and can keep its + 5–10 ms edge fades. - **Voice declarations**: list only usable IDs in `VESKSLOP_TTS_VOICES`. Veskslop hides the espeak list while a custom command is set, so a missing declaration means an empty voice list, not a fallback. The list accepts commas or newlines, and the optional `no-pitch` flag tells the @@ -308,5 +322,8 @@ _Added 2026-09-15._ - **Flextalk pitch**: confirmed that pitch is a no-op for Keith Bell and Kerry Watson. Mark those entries `no-pitch` so users are not offered a control that does nothing. Speed applies to every voice and needs no flag. + _Engine correction (2026-09-15): the no-op was caused by the speaker registry values being + `REG_SZ`; with `REG_DWORD` pitch works and the voices are declared without the flag. See the + speed/pitch section._ - **Licensing**: keep the engine DLLs and voice data local and out of any repository; Veskslop never bundles or redistributes them. diff --git a/setup/setup-flextalk.sh b/setup/setup-flextalk.sh index f3d5ac9..16dc6bc 100755 --- a/setup/setup-flextalk.sh +++ b/setup/setup-flextalk.sh @@ -62,30 +62,30 @@ REGEDIT4 "SampleRate"="11025" [HKEY_LOCAL_MACHINE\Software\AT&T\Watson\2.0\FlexTalk\2.0\Speakers\{11260610-C84A-11ce-AEC4-0000C0E9CB87}] -"Age"="30" -"Expression"="59" +"Age"=dword:0000001e +"Expression"=dword:0000003b "Gender"="" -"Head Size"="46" -"Lowering"="40" +"Head Size"=dword:0000002e +"Lowering"=dword:00000028 "Mode Name"="Keith Bell" -"Pitch"="27" -"Prominences"="17" +"Pitch"=dword:0000001b +"Prominences"=dword:00000011 "Speaker Name"="Keith Bell" -"Speaking Rate"="33" -"Volume"="100" +"Speaking Rate"=dword:00000021 +"Volume"=dword:00000064 [HKEY_LOCAL_MACHINE\Software\AT&T\Watson\2.0\FlexTalk\2.0\Speakers\{000031CB-3699-551F-AED0-6B4893382D57}] -"Age"="30" -"Expression"="100" +"Age"=dword:0000001e +"Expression"=dword:00000064 "Gender"="" -"Head Size"="45" -"Lowering"="40" +"Head Size"=dword:0000002d +"Lowering"=dword:00000028 "Mode Name"="Kerry Watson" -"Pitch"="65" -"Prominences"="17" +"Pitch"=dword:00000041 +"Prominences"=dword:00000011 "Speaker Name"="Kerry Watson" -"Speaking Rate"="32" -"Volume"="100" +"Speaking Rate"=dword:00000020 +"Volume"=dword:00000064 EOF echo "== registry" diff --git a/setup/vekslop-tts.sh b/setup/vekslop-tts.sh index e115199..c0b9b73 100755 --- a/setup/vekslop-tts.sh +++ b/setup/vekslop-tts.sh @@ -20,8 +20,7 @@ export WINEPREFIX=${WINEPREFIX:-$HOME/.wine-microslopsam} export WINEARCH=win32 # VESKSLOP_TTS_VOICES entries: id|Name|gender|language[|flags] -# `no-pitch` marks voices whose engine ignores pitch (Flextalk). -VOICES="sam|Sam|male|en,mary|Mary|female|en,mike|Mike|male|en,keith-bell|Keith Bell|male|en|no-pitch,kerry-watson|Kerry Watson|female|en|no-pitch,mike-telephone|Mike (for Telephone)|male|en,mary-telephone|Mary (for Telephone)|female|en,mike-hall|Mike in Hall|male|en,mike-stadium|Mike in Stadium|male|en,mike-space|Mike in Space|male|en,mary-hall|Mary in Hall|female|en,mary-stadium|Mary in Stadium|female|en,mary-space|Mary in Space|female|en" +VOICES="sam|Sam|male|en,mary|Mary|female|en,mike|Mike|male|en,keith-bell|Keith Bell|male|en,kerry-watson|Kerry Watson|female|en,mike-telephone|Mike (for Telephone)|male|en,mary-telephone|Mary (for Telephone)|female|en,mike-hall|Mike in Hall|male|en,mike-stadium|Mike in Stadium|male|en,mike-space|Mike in Space|male|en,mary-hall|Mary in Hall|female|en,mary-stadium|Mary in Stadium|female|en,mary-space|Mary in Space|female|en" if [ "${1:-}" = "--voices" ]; then echo "$VOICES" @@ -51,36 +50,38 @@ case "$VOICE" in *) ENGVOICE="Sam" ;; esac -# Engine units. Measured naturals: MSTTS ~150, Flextalk ~300 (words/min scale). -# Flextalk ignores ITTSAttributes pitch (verified byte-identical output). +# Engine units (measured). Speed is words/min, natural rate near 150 for every +# voice. Pitch units differ per voice: the engine's natural pitch corresponds to +# about 100 (MSTTS), 106 (Keith Bell) and 186 (Kerry Watson). Semitones scale +# that natural value by 2^(st/12) and clamp to the engine range. case "$ENGVOICE" in - "Keith Bell"|"Kerry Watson") NAT_SPEED=300; SMAX=390 ;; - *) NAT_SPEED=150; SMAX=450 ;; + "Keith Bell") SMAX=390; PMAX=250; NAT_PITCH=106 ;; + "Kerry Watson") SMAX=390; PMAX=250; NAT_PITCH=186 ;; + *) SMAX=450; PMAX=200; NAT_PITCH=100 ;; esac SET_ARGS="" if [ -n "$SPEED" ]; then - S=$(awk -v m="$SPEED" -v n="$NAT_SPEED" 'BEGIN{printf "%d", n*m+0.5}') + S=$(awk -v m="$SPEED" 'BEGIN{printf "%d", 150*m+0.5}') [ "$S" -lt 50 ] && S=50 [ "$S" -gt "$SMAX" ] && S=$SMAX SET_ARGS="$SET_ARGS -s $S" fi if [ -n "$PITCH" ]; then - case "$ENGVOICE" in - "Keith Bell"|"Kerry Watson") : ;; - *) - P=$(awk -v st="$PITCH" 'BEGIN{printf "%d", 100 + st*6.25}') - [ "$P" -lt 50 ] && P=50 - [ "$P" -gt 200 ] && P=200 - SET_ARGS="$SET_ARGS -p $P" ;; - esac + P=$(awk -v st="$PITCH" -v nat="$NAT_PITCH" 'BEGIN{printf "%d", nat*exp(log(2)*st/12)+0.5}') + [ "$P" -lt 50 ] && P=50 + [ "$P" -gt "$PMAX" ] && P=$PMAX + SET_ARGS="$SET_ARGS -p $P" fi # The daemon does not export DISPLAY; keep a private Xvfb for the engine. -if [ -z "${DISPLAY:-}" ]; then +start_xvfb() { if ! [ -S /tmp/.X11-unix/X77 ]; then Xvfb :77 -screen 0 1024x768x24 >/dev/null 2>&1 & sleep 1 fi +} +if [ -z "${DISPLAY:-}" ]; then + start_xvfb export DISPLAY=:77 fi @@ -102,32 +103,53 @@ if [ ! -x "$HOST" ]; then HOST="$ROOT/build/sapi4host.exe"; fi if [ ! -x "$HOST" ]; then echo "engine host not built" >&2; exit 1; fi if ! command -v ffmpeg >/dev/null 2>&1; then echo "ffmpeg not found" >&2; exit 1; fi -# shellcheck disable=SC2086 -if ! "$HOST" -v "$ENGVOICE" -t "$(cat "$TXT")" -o "$RAW" $SET_ARGS \ - >/dev/null 2>"$TMP/err"; then - head -c 400 "$TMP/err" >&2 - exit 1 +run_engine() { + # shellcheck disable=SC2086 + "$HOST" -v "$ENGVOICE" -t "$(cat "$TXT")" -o "$RAW" $SET_ARGS \ + >/dev/null 2>"$TMP/err" +} +if ! run_engine; then + # A stale private Xvfb can break audio-destination creation; retry once. + if [ "${DISPLAY:-}" = ":77" ]; then + pkill -f "Xvfb :77" 2>/dev/null || true + rm -f /tmp/.X11-unix/X77 + start_xvfb + if ! run_engine; then + head -c 400 "$TMP/err" >&2 + exit 1 + fi + else + head -c 400 "$TMP/err" >&2 + exit 1 + fi fi [ -s "$RAW" ] || { echo "engine produced no audio" >&2; exit 1; } SIL="silenceremove=start_periods=1:start_duration=0.05:start_silence=0.03:start_threshold=-50dB" PRE="highpass=f=40,$SIL,afade=t=in:st=0:d=0.008,areverse,$SIL,afade=t=in:st=0:d=0.008,areverse" -LN="loudnorm=I=-16:TP=-3:LRA=11" -# Two-pass loudnorm: measure first, then apply with the measured values. -M=$(ffmpeg -hide_banner -nostats -i "$RAW" -af "$PRE,$LN:print_format=json" \ +# Fixed-gain normalization: single-pass loudnorm misbehaves on short lines +# (LRA=0 causes heavy attenuation), so measure EBU R128 integrated loudness and +# true peak, then apply one gain clamped by the -3 dBFS peak target. +M=$(ffmpeg -hide_banner -nostats -i "$RAW" -af "$PRE,ebur128=peak=true" \ -f null - 2>&1 || true) -I=$(printf '%s' "$M" | sed -n 's/.*"input_i" *: *"\([^"]*\)".*/\1/p') -TP=$(printf '%s' "$M" | sed -n 's/.*"input_tp" *: *"\([^"]*\)".*/\1/p') -LRA=$(printf '%s' "$M" | sed -n 's/.*"input_lra" *: *"\([^"]*\)".*/\1/p') -TH=$(printf '%s' "$M" | sed -n 's/.*"input_thresh" *: *"\([^"]*\)".*/\1/p') -OFF=$(printf '%s' "$M" | sed -n 's/.*"target_offset" *: *"\([^"]*\)".*/\1/p') -if [ -n "$I" ] && [ -n "$TP" ] && [ -n "$LRA" ] && [ -n "$TH" ] && [ -n "$OFF" ]; then - LN="$LN:measured_I=$I:measured_TP=$TP:measured_LRA=$LRA:measured_thresh=$TH:offset=$OFF:linear=true" +I=$(printf '%s' "$M" | sed -n 's/ *I: *\(-\{0,1\}[0-9.]*\) *LUFS.*/\1/p' | tail -1) +TP=$(printf '%s' "$M" | sed -n 's/ *Peak: *\(-\{0,1\}[0-9.]*\) *dBFS.*/\1/p' | tail -1) +if [ -n "$I" ] && [ -n "$TP" ]; then + GAIN=$(awk -v i="$I" -v tp="$TP" 'BEGIN{ + g = -16 - i; + lim = -3 - tp; + if (lim < g) g = lim; + if (g > 30) g = 30; + if (g < -20) g = -20; + printf "%.2f", g + }') +else + GAIN=0 fi TMPOUT="$TMP/out.wav" -if ! ffmpeg -hide_banner -nostats -y -i "$RAW" -af "$PRE,$LN" \ +if ! ffmpeg -hide_banner -nostats -y -i "$RAW" -af "$PRE,volume=${GAIN}dB" \ -ar 48000 -ac 1 -c:a pcm_s16le "$TMPOUT" >/dev/null 2>"$TMP/ff"; then head -c 400 "$TMP/ff" >&2 exit 1