Veskslop command wrapper, optional pitch/speed, measured calibration

This commit is contained in:
Fisto
2026-09-15 11:49:13 +02:00
parent 2a00496cfe
commit 4bed006ea9
5 changed files with 254 additions and 58 deletions
+17 -4
View File
@@ -39,8 +39,8 @@ Flextalk voices (once; needs the extracted installer, see `docs/NOTES.md`):
Host:
# Winelib (native ELF + launcher)
winegcc -m32 -O2 -o build/sapi4host host/sapi4host.c -I../reference \
# Winelib (native ELF + launcher): build/sapi4host-winlib.exe + .exe.so
winegcc -m32 -O2 -o build/sapi4host-winlib host/sapi4host.c -I../reference \
-lole32 -luser32 -ladvapi32
# PE fallback (runs under wine):
@@ -55,6 +55,18 @@ Host:
Options: `-v` voice, `-t` text, `-o` output WAV, `-p` pitch, `-s` speed.
## Veskslop integration
export VESKSLOP_TTS_CMD="$PWD/setup/vekslop-tts.sh {out} {text_file}"
export VESKSLOP_TTS_VOICES="$(setup/vekslop-tts.sh --voices)"
The wrapper maps voice IDs (`sam`, `mary`, `mike`, `keith-bell`, `kerry-watson`, plus the
MSTTS variants), reads `VESKSLOP_TTS_VOICE`, `VESKSLOP_TTS_SPEED` (multiplier) and
`VESKSLOP_TTS_PITCH` (semitones), starts Xvfb when `DISPLAY` is unset, and writes a
48 kHz mono s16le WAV trimmed, faded and normalized to about -16 LUFS (-3 dBFS true peak).
Flextalk voices ignore pitch (engine limitation); everything else maps as documented in
`docs/TTS-DESIGN.md`.
## Layout
- `host/sapi4host.c` — host: voice select, pitch/speed, text -> WAV
@@ -63,6 +75,7 @@ Options: `-v` voice, `-t` text, `-o` output WAV, `-p` pitch, `-s` speed.
- `setup/setup-flextalk.sh` — Flextalk files + registry
- `setup/inf2reg.py` — INF AddReg -> `.reg` converter
- `setup/run.sh` — run helper (prefix + Xvfb)
- `setup/vekslop-tts.sh` — Veskslop command-provider wrapper (post-processing included)
- `docs/NOTES.md` — registry/CLSID facts, runtime RE notes, gotchas
- `docs/TTS-DESIGN.md` — integration contract with Veskslop plus engine clarifications
@@ -71,8 +84,8 @@ Options: `-v` voice, `-t` text, `-o` output WAV, `-p` pitch, `-s` speed.
- `xvfb-run` is flaky on this host. `setup/run.sh` starts a persistent Xvfb
(`:77`) instead; Wine needs a working X display because the runtime creates
windows.
- Flextalk's default pitch/speed report as maximum (250/390) through
`ITTSAttributes`; values are still adjustable with `-p`/`-s`.
- Flextalk ignores pitch entirely (byte-identical output at any pitch setting); use speed.
The host now leaves pitch/speed untouched unless passed, so voices use their own profile.
- The Flextalk archive lacks `data2.cab`; `unshield -O` extracts `data1.cab`
but one tail file (`WatsonUNInstall.dll`) is incomplete.
+75 -30
View File
@@ -51,10 +51,12 @@ these placeholders, splits the string on whitespace, and runs the result directl
| `{out}` | Output file path, extension `.wav`. Required. |
| `{text_file}` | A UTF-8 text file holding the line. |
| `{voice}` | The selected voice ID, or an empty string. |
| `{setting}` | Reserved for speed/pitch once Veskslop grows settings support. |
| `{speed}` | The rate multiplier, or an empty string when unset. |
| `{pitch}` | The pitch offset in semitones, or an empty string when unset. |
The process also receives `VESKSLOP_TTS_VOICE` in the environment. Write the audio to `{out}` and
exit 0. Non-zero exits are errors; stderr is captured into the log message, so keep it short.
The process also receives `VESKSLOP_TTS_VOICE`, and `VESKSLOP_TTS_SPEED` / `VESKSLOP_TTS_PITCH`
when those are set. Write the audio to `{out}` and exit 0. Non-zero exits are errors; stderr is
captured into the log message, so keep it short.
Command mode is best for fast, stateless engines. It has no shell quoting: paths with spaces and
shell tricks cannot work. Use a small wrapper script if you need options.
@@ -120,15 +122,25 @@ Recommended HTTP shape (extras are optional; an engine may ignore them):
{ "text": "...", "voice": "anna", "speed": 1.15, "pitch": -2 }
```
Recommended command shape: `{setting}` is reserved. Until Veskslop passes it, read defaults from
the engine's own config file or environment.
Recommended command shape: `{speed}` and `{pitch}` are substituted when set, and the same values
are exported as `VESKSLOP_TTS_SPEED` and `VESKSLOP_TTS_PITCH`. Read either; the environment form
survives placeholder-free commands.
### How Veskslop should add first-class settings
A future change would add `speed` and `pitch` to `tts` preferences, pass them to the provider
alongside `voice`, and add two controls to the TTS panel. For the command provider, `{speed}` and
`{pitch}` placeholders (or one `{settings_file}` JSON file) would be substituted the same way as
`{voice}`. Design the engine API so it works with defaults when these fields are absent.
Implemented on the Veskslop side (2026-09-15):
- `tts` preferences carry `speed` (0.5–2.0) and `pitch` (-12–+12, semitones), both nullable.
- `/api/audio/speak` and the MCP `speak` tool accept optional `speed` and `pitch` that override
the preferences for that line.
- The command provider substitutes `{speed}`/`{pitch}` and exports `VESKSLOP_TTS_SPEED` /
`VESKSLOP_TTS_PITCH` only when the value is set.
- The HTTP provider adds `speed` and `pitch` to the request body only when set.
- The TTS panel has speed and pitch sliders; changes are saved to the daemon preferences.
- Command mode advertises voices through `VESKSLOP_TTS_VOICES`
(`id`, `id|Name`, `id|Name|gender`, or `id|Name|gender|language`, comma separated). While a
custom command is set, the engine's declared voices replace the espeak list, so no misleading
espeak entries appear. Reading `/voices` for the HTTP provider is still future work.
Do not encode settings into the voice string (for example `anna+speed1.2`). It breaks voice lists
and is hard to validate.
@@ -160,13 +172,17 @@ return the cached file or bytes. This is useful for repeated lines and for expen
## Veskslop changes needed
If the engine wants first-class voice lists, speed, or pitch in the Veskslop UI, Veskslop must:
Status as of 2026-09-15:
1. Read the engine's `/voices` endpoint and merge it into `/api/audio/tts/voices`.
2. Add `speed` and `pitch` to the `tts` preferences and to the `speak` request.
3. Add controls to the TTS panel and pass the values through to the provider.
1. `speed` and `pitch` are first-class: preferences, per-request overrides on `/api/audio/speak`
and MCP `speak`, env exports and placeholders for the command provider, request fields for the
HTTP provider, and two sliders in the TTS panel.
2. Command-mode voices come from `VESKSLOP_TTS_VOICES`. Reading the engine's `GET /voices` for the
HTTP provider is not implemented yet. Until then, declare voices with the env var or configure
the voice inside the wrapper.
3. No changes are needed for the audio format; the engine post-processes to the required output.
Until those exist, use defaults configured in the engine.
Until `/voices` is wired, use defaults configured in the engine.
## Acceptance checklist
@@ -209,17 +225,19 @@ record what the engine needs to interpret, add, or hide._
back to `sam`.
- `/voices` (HTTP mode) reports `language: "en"`, gender, and the speed/pitch ranges below.
### Speed and pitch mapping
### Speed and pitch mapping (measured 2026-09-15)
- SAPI4 uses engine units, not multipliers or semitones. Observed values:
- MSTTS Sam: speed 30-450 (default 150), pitch 50-200 (default 100).
- Flextalk: speed 50-390, pitch 50-250; the engine reports the **defaults as the
maximum**, so the wrapper uses the voice profile from the registry as the natural
values (Keith Bell: pitch 27, rate 33; Kerry Watson: pitch 65, rate 32).
- Mapping (approximate, stated in `/voices`): `speed 1.0` = the voice's natural rate;
0.5-2.0 scales that rate and clamps to the engine range. `pitch` ±12 semitones scales the
engine pitch range by about ±25% around the natural value.
- Command mode: `{setting}` is unused for now. The wrapper reads `VESKSLOP_TTS_SPEED` and
- SAPI4 uses engine units, not multipliers or semitones. Measured on the dev host:
- MSTTS Sam: speed 30-450, natural about 150; pitch 50-200 with a real audible effect
(F0 about 134 Hz at pitch 50, about 199 Hz at pitch 200 on the test phrase).
- Flextalk: speed 50-390, natural about 300 (output with no setting is byte-identical to
`-s 300`); **pitch has no effect** (byte-identical output at pitch 100/150/200), so the
wrapper ignores `pitch` for Keith Bell and Kerry Watson.
- Mapping implemented in `setup/vekslop-tts.sh`: `speed 1.0` = the voice's natural rate
(150 MSTTS, 300 Flextalk), scaled by the multiplier and clamped to the engine range.
`pitch` for MSTTS maps ±12 semitones to engine units `100 ± 6.25·st`, clamped to 50-200.
Pitch is a no-op for Flextalk voices.
- `{setting}` is unused; the wrapper reads `VESKSLOP_TTS_SPEED` and
`VESKSLOP_TTS_PITCH` from the environment when Veskslop later exports them.
### Text handling
@@ -231,8 +249,12 @@ record what the engine needs to interpret, add, or hide._
### Transport and latency
- Command provider (first target): a wrapper script reads `{text_file}`, calls the engine,
post-processes, writes `{out}`, and exits non-zero with one short stderr line on failure.
- Command provider, implemented as `setup/vekslop-tts.sh`:
`VESKSLOP_TTS_CMD="<repo>/setup/vekslop-tts.sh {out} {text_file}"` and
`VESKSLOP_TTS_VOICES="$(<repo>/setup/vekslop-tts.sh --voices)"`. The wrapper reads the
text file, transliterates UTF-8 to CP1252, starts or reuses an Xvfb when `DISPLAY` is
unset, calls the engine, then post-processes (trim, fades, two-pass `loudnorm`, 48 kHz
mono s16le). Verified: -17.7 LUFS integrated, -3.0 dBFS true peak.
- Measured on the dev host (warm Wine): Sam line about 288 ms wall, Flextalk line about
41 ms wall, including process start, synthesis, and file write. Both are inside the
500 ms budget.
@@ -253,7 +275,30 @@ record what the engine needs to interpret, add, or hide._
### Extra acceptance items for the engine
- [ ] Post-processed WAV is 48 kHz mono s16le, peak about -3 dBFS, no clipping, fades present.
- [ ] Leading silence under 100 ms.
- [ ] Voice ID fallback works; TruVoice entries filtered.
- [ ] Latency recorded (Sam about 288 ms, Flextalk about 41 ms on the dev host).
- [x] Post-processed WAV is 48 kHz mono s16le, peak about -3 dBFS, no clipping, fades present.
- [x] Leading silence under 100 ms.
- [x] Voice ID fallback works; TruVoice entries filtered.
- [x] Latency recorded (Sam about 288 ms, Flextalk about 41 ms on the dev host).
## Responses from the Veskslop side
_Added 2026-09-15._
- **Env vars and placeholders**: `VESKSLOP_TTS_SPEED` and `VESKSLOP_TTS_PITCH` are exported only
when a value is set; `{speed}` and `{pitch}` substitute the same values. Reading the environment
is the safer option because it works even when the command has no placeholders.
- **Defaults**: a missing value means "engine default". Leave the sliders at 1.00x and 0 st in
Veskslop to keep the fields out of the request entirely.
- **`loudnorm` caution**: single-pass `loudnorm` can pump or over-amplify noise on short lines.
Set the target explicitly (`loudnorm=I=-16:TP=-1.5:LRA=11`) and verify a few lines with
`ebur128`. A fixed gain that hits about -3 dBFS peak with a consistent RMS is often steadier for
one-line utterances. Ducking lowers the sink to 30%, so keep peaks conservative and loudness
even between lines.
- **Display**: the daemon does not export `DISPLAY` to TTS children. The wrapper must start or
find its own Xvfb and set `DISPLAY` itself, as the project helper already does. Do not rely on
the client's display.
- **Voice declarations**: list only usable IDs in `VESKSLOP_TTS_VOICES`. Veskslop hides the
espeak list while a custom command is set, so a missing declaration means an empty voice list,
not a fallback.
- **Licensing**: keep the engine DLLs and voice data local and out of any repository; Veskslop
never bundles or redistributes them.
+27 -23
View File
@@ -257,30 +257,34 @@ int main(int argc, char **argv)
return 1;
}
WORD pmin = 0, pmax = 0, pdef = 0;
DWORD smin = 0, smax = 0, sdef = 0;
attr->lpVtbl->PitchGet(attr, &pdef);
attr->lpVtbl->SpeedGet(attr, &sdef);
attr->lpVtbl->PitchSet(attr, TTSATTR_MINPITCH);
attr->lpVtbl->PitchGet(attr, &pmin);
attr->lpVtbl->PitchSet(attr, TTSATTR_MAXPITCH);
attr->lpVtbl->PitchGet(attr, &pmax);
attr->lpVtbl->SpeedSet(attr, TTSATTR_MINSPEED);
attr->lpVtbl->SpeedGet(attr, &smin);
attr->lpVtbl->SpeedSet(attr, TTSATTR_MAXSPEED);
attr->lpVtbl->SpeedGet(attr, &smax);
/* Pitch and speed are optional: when unset, leave the engine's own
voice profile untouched (probing/setting would override it). */
if (pitch >= 0 || speed >= 0) {
WORD pmin = 0, pmax = 0;
DWORD smin = 0, smax = 0;
if (pitch < 0) pitch = pdef;
if (speed < 0) speed = sdef;
if (pitch < (int)pmin) pitch = pmin;
if (pitch > (int)pmax) pitch = pmax;
if (speed < (int)smin) speed = smin;
if (speed > (int)smax) speed = smax;
printf("pitch: %d (%d..%d) speed: %d (%lu..%lu)\n",
pitch, pmin, pmax, speed, (unsigned long)smin, (unsigned long)smax);
attr->lpVtbl->PitchSet(attr, (WORD)pitch);
attr->lpVtbl->SpeedSet(attr, (DWORD)speed);
if (pitch >= 0) {
attr->lpVtbl->PitchSet(attr, TTSATTR_MINPITCH);
attr->lpVtbl->PitchGet(attr, &pmin);
attr->lpVtbl->PitchSet(attr, TTSATTR_MAXPITCH);
attr->lpVtbl->PitchGet(attr, &pmax);
if (pitch < (int)pmin) pitch = pmin;
if (pitch > (int)pmax) pitch = pmax;
attr->lpVtbl->PitchSet(attr, (WORD)pitch);
printf("pitch: %d (%d..%d)\n", pitch, pmin, pmax);
}
if (speed >= 0) {
attr->lpVtbl->SpeedSet(attr, TTSATTR_MINSPEED);
attr->lpVtbl->SpeedGet(attr, &smin);
attr->lpVtbl->SpeedSet(attr, TTSATTR_MAXSPEED);
attr->lpVtbl->SpeedGet(attr, &smax);
if (speed < (int)smin) speed = smin;
if (speed > (int)smax) speed = smax;
attr->lpVtbl->SpeedSet(attr, (DWORD)speed);
printf("speed: %d (%lu..%lu)\n",
speed, (unsigned long)smin, (unsigned long)smax);
}
}
{
DWORD regkey = 0;
+1 -1
View File
@@ -16,7 +16,7 @@ if [ -z "${DISPLAY:-}" ]; then
export DISPLAY=:77
fi
HOST="$ROOT/build/sapi4host"
HOST="$ROOT/build/sapi4host-winlib.exe"
if [ ! -x "$HOST" ]; then
HOST="$ROOT/build/sapi4host.exe"
fi
+134
View File
@@ -0,0 +1,134 @@
#!/bin/sh
# Veskslop TTS command-provider wrapper for MicroslopSam.
#
# Recommended Veskslop configuration:
# VESKSLOP_TTS_CMD="<repo>/setup/vekslop-tts.sh {out} {text_file}"
# VESKSLOP_TTS_VOICES="$(setup/vekslop-tts.sh --voices)"
#
# Inputs:
# $1 {out} output WAV path
# $2 {text_file} UTF-8 text file
# $3... optional: voice id, speed multiplier, pitch semitones
# env VESKSLOP_TTS_VOICE, VESKSLOP_TTS_SPEED, VESKSLOP_TTS_PITCH
#
# Output: 48 kHz mono s16le WAV, trimmed, faded, loudness-normalized.
set -eu
HERE=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
ROOT=$(CDPATH= cd -- "$HERE/.." && pwd)
export WINEPREFIX=${WINEPREFIX:-$HOME/.wine-microslopsam}
export WINEARCH=win32
VOICES="sam|Sam|male|en,mary|Mary|female|en,mike|Mike|male|en,keith-bell|Keith Bell|male|en,kerry-watson|Kerry Watson|female|en,mike-telephone|Mike (for Telephone)|male|en,mary-telephone|Mary (for Telephone)|female|en,mike-hall|Mike in Hall|male|en,mike-stadium|Mike in Stadium|male|en,mike-space|Mike in Space|male|en,mary-hall|Mary in Hall|female|en,mary-stadium|Mary in Stadium|female|en,mary-space|Mary in Space|female|en"
if [ "${1:-}" = "--voices" ]; then
echo "$VOICES"
exit 0
fi
OUT=${1:?missing out path}
TEXTFILE=${2:?missing text file}
VOICE=${VESKSLOP_TTS_VOICE:-${3:-sam}}
SPEED=${VESKSLOP_TTS_SPEED:-${4:-}}
PITCH=${VESKSLOP_TTS_PITCH:-${5:-}}
case "$VOICE" in
sam) ENGVOICE="Sam" ;;
mary) ENGVOICE="Mary" ;;
mike) ENGVOICE="Mike" ;;
keith-bell) ENGVOICE="Keith Bell" ;;
kerry-watson) ENGVOICE="Kerry Watson" ;;
mike-telephone) ENGVOICE="Mike (for Telephone)" ;;
mary-telephone) ENGVOICE="Mary (for Telephone)" ;;
mike-hall) ENGVOICE="Mike in Hall" ;;
mike-stadium) ENGVOICE="Mike in Stadium" ;;
mike-space) ENGVOICE="Mike in Space" ;;
mary-hall) ENGVOICE="Mary in Hall" ;;
mary-stadium) ENGVOICE="Mary in Stadium" ;;
mary-space) ENGVOICE="Mary in Space" ;;
*) ENGVOICE="Sam" ;;
esac
# Engine units. Measured naturals: MSTTS ~150, Flextalk ~300 (words/min scale).
# Flextalk ignores ITTSAttributes pitch (verified byte-identical output).
case "$ENGVOICE" in
"Keith Bell"|"Kerry Watson") NAT_SPEED=300; SMAX=390 ;;
*) NAT_SPEED=150; SMAX=450 ;;
esac
SET_ARGS=""
if [ -n "$SPEED" ]; then
S=$(awk -v m="$SPEED" -v n="$NAT_SPEED" 'BEGIN{printf "%d", n*m+0.5}')
[ "$S" -lt 50 ] && S=50
[ "$S" -gt "$SMAX" ] && S=$SMAX
SET_ARGS="$SET_ARGS -s $S"
fi
if [ -n "$PITCH" ]; then
case "$ENGVOICE" in
"Keith Bell"|"Kerry Watson") : ;;
*)
P=$(awk -v st="$PITCH" 'BEGIN{printf "%d", 100 + st*6.25}')
[ "$P" -lt 50 ] && P=50
[ "$P" -gt 200 ] && P=200
SET_ARGS="$SET_ARGS -p $P" ;;
esac
fi
# The daemon does not export DISPLAY; keep a private Xvfb for the engine.
if [ -z "${DISPLAY:-}" ]; then
if ! [ -S /tmp/.X11-unix/X77 ]; then
Xvfb :77 -screen 0 1024x768x24 >/dev/null 2>&1 &
sleep 1
fi
export DISPLAY=:77
fi
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
RAW="$TMP/raw.wav"
TXT="$TMP/text.cp1252"
# SAPI4 is ANSI/CP1252; transliterate UTF-8 (emoji become '?').
if command -v iconv >/dev/null 2>&1; then
iconv -f UTF-8 -t WINDOWS-1252//TRANSLIT "$TEXTFILE" > "$TXT" 2>/dev/null \
|| cp "$TEXTFILE" "$TXT"
else
cp "$TEXTFILE" "$TXT"
fi
HOST="$ROOT/build/sapi4host-winlib.exe"
if [ ! -x "$HOST" ]; then HOST="$ROOT/build/sapi4host.exe"; fi
if [ ! -x "$HOST" ]; then echo "engine host not built" >&2; exit 1; fi
if ! command -v ffmpeg >/dev/null 2>&1; then echo "ffmpeg not found" >&2; exit 1; fi
# shellcheck disable=SC2086
if ! "$HOST" -v "$ENGVOICE" -t "$(cat "$TXT")" -o "$RAW" $SET_ARGS \
>/dev/null 2>"$TMP/err"; then
head -c 400 "$TMP/err" >&2
exit 1
fi
[ -s "$RAW" ] || { echo "engine produced no audio" >&2; exit 1; }
SIL="silenceremove=start_periods=1:start_duration=0.05:start_silence=0.03:start_threshold=-50dB"
PRE="highpass=f=40,$SIL,afade=t=in:st=0:d=0.008,areverse,$SIL,afade=t=in:st=0:d=0.008,areverse"
LN="loudnorm=I=-16:TP=-3:LRA=11"
# Two-pass loudnorm: measure first, then apply with the measured values.
M=$(ffmpeg -hide_banner -nostats -i "$RAW" -af "$PRE,$LN:print_format=json" \
-f null - 2>&1 || true)
I=$(printf '%s' "$M" | sed -n 's/.*"input_i" *: *"\([^"]*\)".*/\1/p')
TP=$(printf '%s' "$M" | sed -n 's/.*"input_tp" *: *"\([^"]*\)".*/\1/p')
LRA=$(printf '%s' "$M" | sed -n 's/.*"input_lra" *: *"\([^"]*\)".*/\1/p')
TH=$(printf '%s' "$M" | sed -n 's/.*"input_thresh" *: *"\([^"]*\)".*/\1/p')
OFF=$(printf '%s' "$M" | sed -n 's/.*"target_offset" *: *"\([^"]*\)".*/\1/p')
if [ -n "$I" ] && [ -n "$TP" ] && [ -n "$LRA" ] && [ -n "$TH" ] && [ -n "$OFF" ]; then
LN="$LN:measured_I=$I:measured_TP=$TP:measured_LRA=$LRA:measured_thresh=$TH:offset=$OFF:linear=true"
fi
TMPOUT="$TMP/out.wav"
if ! ffmpeg -hide_banner -nostats -y -i "$RAW" -af "$PRE,$LN" \
-ar 48000 -ac 1 -c:a pcm_s16le "$TMPOUT" >/dev/null 2>"$TMP/ff"; then
head -c 400 "$TMP/ff" >&2
exit 1
fi
[ -s "$TMPOUT" ] || { echo "post-processing produced no audio" >&2; exit 1; }
mv -f "$TMPOUT" "$OUT"