Fix Flextalk buzz: REG_DWORD speaker settings; measured pitch/speed mapping; fixed-gain normalization
This commit is contained in:
Fisto
@@ -64,8 +64,8 @@ The wrapper maps voice IDs (`sam`, `mary`, `mike`, `keith-bell`, `kerry-watson`,
|
||||
MSTTS variants), reads `VESKSLOP_TTS_VOICE`, `VESKSLOP_TTS_SPEED` (multiplier) and
|
||||
`VESKSLOP_TTS_PITCH` (semitones), starts Xvfb when `DISPLAY` is unset, and writes a
|
||||
48 kHz mono s16le WAV trimmed, faded and normalized to about -16 LUFS (-3 dBFS true peak).
|
||||
Flextalk voices ignore pitch (engine limitation) and are declared with the `no-pitch`
|
||||
flag; everything else maps as documented in `docs/TTS-DESIGN.md`.
|
||||
Speed and pitch apply to every voice (Flextalk's numeric settings must be `REG_DWORD`,
|
||||
which the setup script writes). Mapping details are in `docs/TTS-DESIGN.md`.
|
||||
|
||||
## Layout
|
||||
|
||||
@@ -84,8 +84,8 @@ flag; everything else maps as documented in `docs/TTS-DESIGN.md`.
|
||||
- `xvfb-run` is flaky on this host. `setup/run.sh` starts a persistent Xvfb
|
||||
(`:77`) instead; Wine needs a working X display because the runtime creates
|
||||
windows.
|
||||
- Flextalk ignores pitch entirely (byte-identical output at any pitch setting); use speed.
|
||||
The host now leaves pitch/speed untouched unless passed, so voices use their own profile.
|
||||
- Flextalk numeric speaker settings must be `REG_DWORD` (the installer script handles
|
||||
this); `REG_SZ` values cause a monotone buzz. Pitch and speed work on all voices.
|
||||
- The Flextalk archive lacks `data2.cab`; `unshield -O` extracts `data1.cab`
|
||||
but one tail file (`WatsonUNInstall.dll`) is incomplete.
|
||||
|
||||
|
||||
+13
-1
@@ -33,7 +33,9 @@ From `msttsl/mstts.inf` (`[TTSReg]`):
|
||||
`HomographDictionary="1 ft_hgrap.dll"`, `ProperNameDictionary="3 ft_namsa.dll"`,
|
||||
`LetterToSoundDictionary="4 ft_l2s.dll"`.
|
||||
- Options: `...\FlexTalk\2.0\Options` `Gender=Male`, `SampleRate=11025`.
|
||||
- Speakers (per-voice settings; names come from these values):
|
||||
- Speakers (per-voice settings; names come from these values). **All numeric
|
||||
values must be REG_DWORD**: with REG_SZ the engine reads garbage and
|
||||
synthesizes a monotone buzz (pitch fixed at max, pitch control dead).
|
||||
- `{11260610-C84A-11ce-AEC4-0000C0E9CB87}`: Mode Name / Speaker Name
|
||||
"Keith Bell", Pitch 27, Speaking Rate 33, Head Size 46, Expression 59,
|
||||
Lowering 40, Prominences 17, Age 30, Volume 100.
|
||||
@@ -71,6 +73,16 @@ final file `WatsonUNInstall.dll` is incomplete).
|
||||
- 32-bit engines require a 32-bit prefix (`WINEARCH=win32`).
|
||||
- `sudo` on this machine requires a password; installs are manual.
|
||||
|
||||
## Engine units (measured)
|
||||
|
||||
- speed: MSTTS range 30-450, Flextalk 50-390; the natural rate is about 150 for every
|
||||
voice (duration at 100/200/300 fits a natural value near 140-150).
|
||||
- pitch: engine units, roughly linear in F0 near the natural point (~1.1 Hz per unit).
|
||||
Natural values: about 100 (MSTTS), 106 (Keith Bell), 186 (Kerry Watson); Flextalk max 250.
|
||||
Verified with the corrected DWORD registry: Kerry 0 st = 209 Hz, -6 st = 148 Hz,
|
||||
+6 st = 250 Hz (clamped at max); Keith 0 st = 119 Hz, +6 st = 170 Hz.
|
||||
- The wrapper maps semitones as `natural * 2^(st/12)` and clamps to the range.
|
||||
|
||||
## Decompiled corpora (Ghidra)
|
||||
|
||||
- `/mnt/ghidra/decomp/msttssyn.dll/` (786 functions, engine)
|
||||
|
||||
+18
-1
@@ -267,7 +267,8 @@ record what the engine needs to interpret, add, or hide._
|
||||
|
||||
### Quirks the wrapper must hide
|
||||
|
||||
- Flextalk reports default pitch/speed as maximum through `ITTSAttributes`.
|
||||
- Flextalk numeric speaker settings must be `REG_DWORD`; `REG_SZ` values make the engine
|
||||
synthesize a buzz. `setup/setup-flextalk.sh` writes them correctly.
|
||||
- The mode list includes TruVoice names with no installed voice data; filter those out of
|
||||
`/voices` (or mark them `unavailable`).
|
||||
- Wine needs a working X display; the project helper starts a persistent Xvfb.
|
||||
@@ -301,6 +302,19 @@ _Added 2026-09-15._
|
||||
- **Display**: the daemon does not export `DISPLAY` to TTS children. The wrapper must start or
|
||||
find its own Xvfb and set `DISPLAY` itself, as the project helper already does. Do not rely on
|
||||
the client's display.
|
||||
- **Engine selection**: the Speech settings and the TTS panel list every configured engine. The
|
||||
command engine appears under `VESKSLOP_TTS_NAME` (default "custom engine") and is selectable
|
||||
directly; Auto uses the command engine first. Selecting it hides the language row and shows the
|
||||
`VESKSLOP_TTS_VOICES` list.
|
||||
- **Start engine**: the Speech settings have a **Start engine** button (`POST /api/audio/tts/warm`,
|
||||
MCP `start_tts_engine`) that runs one throwaway synthesis to warm the command engine (Xvfb, Wine,
|
||||
host). Keep the wrapper able to handle a short sentence; the warm-up uses "The quick brown fox
|
||||
jumps over the lazy dog." and reports the elapsed milliseconds. Two-pass `loudnorm` must not fail
|
||||
on that input.
|
||||
- **Lead-in silence**: Veskslop prepends a short silence at playback time (`VESKSLOP_TTS_LEAD_MS`,
|
||||
default 150 ms) so Discord's voice pipeline is ready before the first syllable. It is added with
|
||||
`adelay` in the player, not in the audio file. The engine needs no change and can keep its
|
||||
5–10 ms edge fades.
|
||||
- **Voice declarations**: list only usable IDs in `VESKSLOP_TTS_VOICES`. Veskslop hides the
|
||||
espeak list while a custom command is set, so a missing declaration means an empty voice list,
|
||||
not a fallback. The list accepts commas or newlines, and the optional `no-pitch` flag tells the
|
||||
@@ -308,5 +322,8 @@ _Added 2026-09-15._
|
||||
- **Flextalk pitch**: confirmed that pitch is a no-op for Keith Bell and Kerry Watson. Mark those
|
||||
entries `no-pitch` so users are not offered a control that does nothing. Speed applies to every
|
||||
voice and needs no flag.
|
||||
_Engine correction (2026-09-15): the no-op was caused by the speaker registry values being
|
||||
`REG_SZ`; with `REG_DWORD` pitch works and the voices are declared without the flag. See the
|
||||
speed/pitch section._
|
||||
- **Licensing**: keep the engine DLLs and voice data local and out of any repository; Veskslop
|
||||
never bundles or redistributes them.
|
||||
|
||||
+16
-16
@@ -62,30 +62,30 @@ REGEDIT4
|
||||
"SampleRate"="11025"
|
||||
|
||||
[HKEY_LOCAL_MACHINE\Software\AT&T\Watson\2.0\FlexTalk\2.0\Speakers\{11260610-C84A-11ce-AEC4-0000C0E9CB87}]
|
||||
"Age"="30"
|
||||
"Expression"="59"
|
||||
"Age"=dword:0000001e
|
||||
"Expression"=dword:0000003b
|
||||
"Gender"=""
|
||||
"Head Size"="46"
|
||||
"Lowering"="40"
|
||||
"Head Size"=dword:0000002e
|
||||
"Lowering"=dword:00000028
|
||||
"Mode Name"="Keith Bell"
|
||||
"Pitch"="27"
|
||||
"Prominences"="17"
|
||||
"Pitch"=dword:0000001b
|
||||
"Prominences"=dword:00000011
|
||||
"Speaker Name"="Keith Bell"
|
||||
"Speaking Rate"="33"
|
||||
"Volume"="100"
|
||||
"Speaking Rate"=dword:00000021
|
||||
"Volume"=dword:00000064
|
||||
|
||||
[HKEY_LOCAL_MACHINE\Software\AT&T\Watson\2.0\FlexTalk\2.0\Speakers\{000031CB-3699-551F-AED0-6B4893382D57}]
|
||||
"Age"="30"
|
||||
"Expression"="100"
|
||||
"Age"=dword:0000001e
|
||||
"Expression"=dword:00000064
|
||||
"Gender"=""
|
||||
"Head Size"="45"
|
||||
"Lowering"="40"
|
||||
"Head Size"=dword:0000002d
|
||||
"Lowering"=dword:00000028
|
||||
"Mode Name"="Kerry Watson"
|
||||
"Pitch"="65"
|
||||
"Prominences"="17"
|
||||
"Pitch"=dword:00000041
|
||||
"Prominences"=dword:00000011
|
||||
"Speaker Name"="Kerry Watson"
|
||||
"Speaking Rate"="32"
|
||||
"Volume"="100"
|
||||
"Speaking Rate"=dword:00000020
|
||||
"Volume"=dword:00000064
|
||||
EOF
|
||||
|
||||
echo "== registry"
|
||||
|
||||
+50
-28
@@ -20,8 +20,7 @@ export WINEPREFIX=${WINEPREFIX:-$HOME/.wine-microslopsam}
|
||||
export WINEARCH=win32
|
||||
|
||||
# VESKSLOP_TTS_VOICES entries: id|Name|gender|language[|flags]
|
||||
# `no-pitch` marks voices whose engine ignores pitch (Flextalk).
|
||||
VOICES="sam|Sam|male|en,mary|Mary|female|en,mike|Mike|male|en,keith-bell|Keith Bell|male|en|no-pitch,kerry-watson|Kerry Watson|female|en|no-pitch,mike-telephone|Mike (for Telephone)|male|en,mary-telephone|Mary (for Telephone)|female|en,mike-hall|Mike in Hall|male|en,mike-stadium|Mike in Stadium|male|en,mike-space|Mike in Space|male|en,mary-hall|Mary in Hall|female|en,mary-stadium|Mary in Stadium|female|en,mary-space|Mary in Space|female|en"
|
||||
VOICES="sam|Sam|male|en,mary|Mary|female|en,mike|Mike|male|en,keith-bell|Keith Bell|male|en,kerry-watson|Kerry Watson|female|en,mike-telephone|Mike (for Telephone)|male|en,mary-telephone|Mary (for Telephone)|female|en,mike-hall|Mike in Hall|male|en,mike-stadium|Mike in Stadium|male|en,mike-space|Mike in Space|male|en,mary-hall|Mary in Hall|female|en,mary-stadium|Mary in Stadium|female|en,mary-space|Mary in Space|female|en"
|
||||
|
||||
if [ "${1:-}" = "--voices" ]; then
|
||||
echo "$VOICES"
|
||||
@@ -51,36 +50,38 @@ case "$VOICE" in
|
||||
*) ENGVOICE="Sam" ;;
|
||||
esac
|
||||
|
||||
# Engine units. Measured naturals: MSTTS ~150, Flextalk ~300 (words/min scale).
|
||||
# Flextalk ignores ITTSAttributes pitch (verified byte-identical output).
|
||||
# Engine units (measured). Speed is words/min, natural rate near 150 for every
|
||||
# voice. Pitch units differ per voice: the engine's natural pitch corresponds to
|
||||
# about 100 (MSTTS), 106 (Keith Bell) and 186 (Kerry Watson). Semitones scale
|
||||
# that natural value by 2^(st/12) and clamp to the engine range.
|
||||
case "$ENGVOICE" in
|
||||
"Keith Bell"|"Kerry Watson") NAT_SPEED=300; SMAX=390 ;;
|
||||
*) NAT_SPEED=150; SMAX=450 ;;
|
||||
"Keith Bell") SMAX=390; PMAX=250; NAT_PITCH=106 ;;
|
||||
"Kerry Watson") SMAX=390; PMAX=250; NAT_PITCH=186 ;;
|
||||
*) SMAX=450; PMAX=200; NAT_PITCH=100 ;;
|
||||
esac
|
||||
SET_ARGS=""
|
||||
if [ -n "$SPEED" ]; then
|
||||
S=$(awk -v m="$SPEED" -v n="$NAT_SPEED" 'BEGIN{printf "%d", n*m+0.5}')
|
||||
S=$(awk -v m="$SPEED" 'BEGIN{printf "%d", 150*m+0.5}')
|
||||
[ "$S" -lt 50 ] && S=50
|
||||
[ "$S" -gt "$SMAX" ] && S=$SMAX
|
||||
SET_ARGS="$SET_ARGS -s $S"
|
||||
fi
|
||||
if [ -n "$PITCH" ]; then
|
||||
case "$ENGVOICE" in
|
||||
"Keith Bell"|"Kerry Watson") : ;;
|
||||
*)
|
||||
P=$(awk -v st="$PITCH" 'BEGIN{printf "%d", 100 + st*6.25}')
|
||||
P=$(awk -v st="$PITCH" -v nat="$NAT_PITCH" 'BEGIN{printf "%d", nat*exp(log(2)*st/12)+0.5}')
|
||||
[ "$P" -lt 50 ] && P=50
|
||||
[ "$P" -gt 200 ] && P=200
|
||||
SET_ARGS="$SET_ARGS -p $P" ;;
|
||||
esac
|
||||
[ "$P" -gt "$PMAX" ] && P=$PMAX
|
||||
SET_ARGS="$SET_ARGS -p $P"
|
||||
fi
|
||||
|
||||
# The daemon does not export DISPLAY; keep a private Xvfb for the engine.
|
||||
if [ -z "${DISPLAY:-}" ]; then
|
||||
start_xvfb() {
|
||||
if ! [ -S /tmp/.X11-unix/X77 ]; then
|
||||
Xvfb :77 -screen 0 1024x768x24 >/dev/null 2>&1 &
|
||||
sleep 1
|
||||
fi
|
||||
}
|
||||
if [ -z "${DISPLAY:-}" ]; then
|
||||
start_xvfb
|
||||
export DISPLAY=:77
|
||||
fi
|
||||
|
||||
@@ -102,32 +103,53 @@ if [ ! -x "$HOST" ]; then HOST="$ROOT/build/sapi4host.exe"; fi
|
||||
if [ ! -x "$HOST" ]; then echo "engine host not built" >&2; exit 1; fi
|
||||
if ! command -v ffmpeg >/dev/null 2>&1; then echo "ffmpeg not found" >&2; exit 1; fi
|
||||
|
||||
run_engine() {
|
||||
# shellcheck disable=SC2086
|
||||
if ! "$HOST" -v "$ENGVOICE" -t "$(cat "$TXT")" -o "$RAW" $SET_ARGS \
|
||||
>/dev/null 2>"$TMP/err"; then
|
||||
"$HOST" -v "$ENGVOICE" -t "$(cat "$TXT")" -o "$RAW" $SET_ARGS \
|
||||
>/dev/null 2>"$TMP/err"
|
||||
}
|
||||
if ! run_engine; then
|
||||
# A stale private Xvfb can break audio-destination creation; retry once.
|
||||
if [ "${DISPLAY:-}" = ":77" ]; then
|
||||
pkill -f "Xvfb :77" 2>/dev/null || true
|
||||
rm -f /tmp/.X11-unix/X77
|
||||
start_xvfb
|
||||
if ! run_engine; then
|
||||
head -c 400 "$TMP/err" >&2
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
head -c 400 "$TMP/err" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
[ -s "$RAW" ] || { echo "engine produced no audio" >&2; exit 1; }
|
||||
|
||||
SIL="silenceremove=start_periods=1:start_duration=0.05:start_silence=0.03:start_threshold=-50dB"
|
||||
PRE="highpass=f=40,$SIL,afade=t=in:st=0:d=0.008,areverse,$SIL,afade=t=in:st=0:d=0.008,areverse"
|
||||
LN="loudnorm=I=-16:TP=-3:LRA=11"
|
||||
|
||||
# Two-pass loudnorm: measure first, then apply with the measured values.
|
||||
M=$(ffmpeg -hide_banner -nostats -i "$RAW" -af "$PRE,$LN:print_format=json" \
|
||||
# Fixed-gain normalization: single-pass loudnorm misbehaves on short lines
|
||||
# (LRA=0 causes heavy attenuation), so measure EBU R128 integrated loudness and
|
||||
# true peak, then apply one gain clamped by the -3 dBFS peak target.
|
||||
M=$(ffmpeg -hide_banner -nostats -i "$RAW" -af "$PRE,ebur128=peak=true" \
|
||||
-f null - 2>&1 || true)
|
||||
I=$(printf '%s' "$M" | sed -n 's/.*"input_i" *: *"\([^"]*\)".*/\1/p')
|
||||
TP=$(printf '%s' "$M" | sed -n 's/.*"input_tp" *: *"\([^"]*\)".*/\1/p')
|
||||
LRA=$(printf '%s' "$M" | sed -n 's/.*"input_lra" *: *"\([^"]*\)".*/\1/p')
|
||||
TH=$(printf '%s' "$M" | sed -n 's/.*"input_thresh" *: *"\([^"]*\)".*/\1/p')
|
||||
OFF=$(printf '%s' "$M" | sed -n 's/.*"target_offset" *: *"\([^"]*\)".*/\1/p')
|
||||
if [ -n "$I" ] && [ -n "$TP" ] && [ -n "$LRA" ] && [ -n "$TH" ] && [ -n "$OFF" ]; then
|
||||
LN="$LN:measured_I=$I:measured_TP=$TP:measured_LRA=$LRA:measured_thresh=$TH:offset=$OFF:linear=true"
|
||||
I=$(printf '%s' "$M" | sed -n 's/ *I: *\(-\{0,1\}[0-9.]*\) *LUFS.*/\1/p' | tail -1)
|
||||
TP=$(printf '%s' "$M" | sed -n 's/ *Peak: *\(-\{0,1\}[0-9.]*\) *dBFS.*/\1/p' | tail -1)
|
||||
if [ -n "$I" ] && [ -n "$TP" ]; then
|
||||
GAIN=$(awk -v i="$I" -v tp="$TP" 'BEGIN{
|
||||
g = -16 - i;
|
||||
lim = -3 - tp;
|
||||
if (lim < g) g = lim;
|
||||
if (g > 30) g = 30;
|
||||
if (g < -20) g = -20;
|
||||
printf "%.2f", g
|
||||
}')
|
||||
else
|
||||
GAIN=0
|
||||
fi
|
||||
|
||||
TMPOUT="$TMP/out.wav"
|
||||
if ! ffmpeg -hide_banner -nostats -y -i "$RAW" -af "$PRE,$LN" \
|
||||
if ! ffmpeg -hide_banner -nostats -y -i "$RAW" -af "$PRE,volume=${GAIN}dB" \
|
||||
-ar 48000 -ac 1 -c:a pcm_s16le "$TMPOUT" >/dev/null 2>"$TMP/ff"; then
|
||||
head -c 400 "$TMP/ff" >&2
|
||||
exit 1
|
||||
|
||||
Reference in New Issue
Block a user