VOX prosody: NetTTS parser port, wrapper VOX mode, docs
This commit is contained in:
@@ -93,6 +93,19 @@ final file `WatsonUNInstall.dll` is incomplete).
|
||||
verified audible (0/59/100 produce different waveforms at equal duration). Changing
|
||||
them requires a registry edit and re-selecting the voice.
|
||||
|
||||
## VOX parser
|
||||
|
||||
- `setup/vox_parser.py` reimplements the NetTTS VOX prosody encoder (vendor tags
|
||||
`\!wH1/\!wH0`, `\!br`, `\!sfNN`); the host already passes `TTSDATAFLAG_TAGGED`, so
|
||||
the engine honors them.
|
||||
- Validation: the reference `vox_parser.cpp` from a local clone of
|
||||
`h4rm0n1c/NetTTS` was compiled with g++ in a throwaway harness and diffed against
|
||||
the Python port; outputs are identical on the 31-line corpus. Re-check after any
|
||||
parser change while the clone is available.
|
||||
- VOX mode is Flextalk-only. The wrapper enables it with `VESKSLOP_TTS_VOX=1`
|
||||
(`VESKSLOP_TTS_VOX_CLEAN=1` drops the stutter wrapper) and it takes precedence
|
||||
over `VESKSLOP_TTS_WORDS`.
|
||||
|
||||
## Decompiled corpora (Ghidra)
|
||||
|
||||
- `/mnt/ghidra/decomp/msttssyn.dll/` (786 functions, engine)
|
||||
|
||||
+41
-5
@@ -218,7 +218,7 @@ record what the engine needs to interpret, add, or hide._
|
||||
so one shared level target is required. The engine post-processes every line:
|
||||
resample to 48000 Hz, trim leading silence (<100 ms), 5-10 ms fades, DC removal, and
|
||||
level normalization (peak about -3 dBFS, integrated loudness -16 to -20 LUFS).
|
||||
- Implemented in `setup/vekslop-tts.sh` with ffmpeg: high-pass at 40 Hz, silence trim at
|
||||
- Implemented in `setup/veskslop-tts.sh` with ffmpeg: high-pass at 40 Hz, silence trim at
|
||||
both ends (trim-start, reverse, trim-start, reverse), 8 ms fades, then a **fixed gain**
|
||||
computed from an EBU R128 measurement (`ebur128=peak=true`):
|
||||
`gain = min(-16 LUFS target, -3 dBFS true-peak clamp)`, then 48 kHz mono s16le.
|
||||
@@ -253,7 +253,7 @@ record what the engine needs to interpret, add, or hide._
|
||||
engine requires `REG_DWORD` for numeric speaker values; with strings it synthesizes a
|
||||
monotone buzz at maximum pitch. Fixed in `setup/setup-flextalk.sh`; no `no-pitch` flag
|
||||
is needed.
|
||||
- Mapping implemented in `setup/vekslop-tts.sh`: `speed 1.0` = natural rate 150 for every
|
||||
- Mapping implemented in `setup/veskslop-tts.sh`: `speed 1.0` = natural rate 150 for every
|
||||
voice, scaled by the multiplier and clamped to the engine range. `pitch` semitones scale
|
||||
the voice's natural pitch unit by `2^(st/12)` and clamp to the range. Natural units are
|
||||
about 100 (MSTTS), 106 (Keith Bell) and 186 (Kerry Watson). The engines' F0 response is
|
||||
@@ -335,14 +335,50 @@ How to apply a change:
|
||||
5. Runtime `ITTSAttributes` pitch/speed/volume layer on top of the profile: the profile
|
||||
defines the natural values, the attributes shift them per request.
|
||||
|
||||
### VOX mode (NetTTS-inspired, Flextalk only)
|
||||
|
||||
`VESKSLOP_TTS_VOX=1` routes the text through a rule-based prosody encoder before
|
||||
synthesis. `setup/vox_parser.py` is an independent reimplementation of the VOX parser
|
||||
from [h4rm0n1c/NetTTS](https://github.com/h4rm0n1c/NetTTS), which models Half-Life's
|
||||
Black Mesa announcement delivery. It was verified byte-identical to the NetTTS parser
|
||||
on a 31-line corpus (the reference was built locally with g++ for the comparison).
|
||||
|
||||
What it does: sentence splitting; `the` -> `thee` before prepositions, proper nouns,
|
||||
single letters or numbers; lead-in breaks ("Now...", "Please...", "A reminder...");
|
||||
time/number/degree normalization; token weighting (light/medium/heavy) with break
|
||||
insertion at linkers, title-case runs, heavy gates and a 3-beat limiter; letter
|
||||
tokens (`A` -> `Ay:`, `C` -> `C:`); `\!sf500 \!br` at sentence ends; and the
|
||||
`\!wH1 ... \!wH0` staccato wrapper (disable with `VESKSLOP_TTS_VOX_CLEAN=1`).
|
||||
|
||||
Requirements and scope:
|
||||
|
||||
- Flextalk voices only: `\!br`, `\!sfNN` and `\!wH1/\!wH0` are FlexTalk vendor tags.
|
||||
The host passes `TTSDATAFLAG_TAGGED`, so the engine honors them. Other voices ignore
|
||||
`VESKSLOP_TTS_VOX`.
|
||||
- Needs `python3` at runtime; the wrapper fails loudly when it is missing.
|
||||
- Mutually exclusive with `VESKSLOP_TTS_WORDS` (VOX wins).
|
||||
- Measured on the dev host for "Now approaching Sector C test lab. Please stand clear
|
||||
of the test chamber.": plain 4.70 s with 1 pause, VOX 7.81 s with 8 pauses.
|
||||
|
||||
Other NetTTS features and their relevance:
|
||||
|
||||
- Inline `/rate N`, `/pitch N`, `[[pause N]]`: not ported. Veskslop has speed/pitch
|
||||
controls and the profile window; `[[pause N]]` could be added as a wrapper-level
|
||||
text splitter later (NetTTS supports it only outside VOX mode).
|
||||
- TCP server with `START`/`STOP` status sockets: not needed; Veskslop owns playback
|
||||
and knows when a line ends.
|
||||
- Winetricks prefix automation (`winxp`, `vcrun6`, `mfc42`, `riched20`): not needed;
|
||||
our prefix runs the engines with builtin Wine libraries.
|
||||
- Win32 GUI and preset sliders: Veskslop provides the UI.
|
||||
|
||||
### Transport and latency
|
||||
|
||||
- Installation is one command: `setup/install.sh` (system dependencies with sudo, wine
|
||||
prefix + SAPI4 runtime + MSTTS voices, Flextalk extraction/registration, host build).
|
||||
Re-running is safe; a fresh machine needs nothing else.
|
||||
- Command provider, implemented as `setup/vekslop-tts.sh`:
|
||||
`VESKSLOP_TTS_CMD="<repo>/setup/vekslop-tts.sh {out} {text_file}"` and
|
||||
`VESKSLOP_TTS_VOICES="$(<repo>/setup/vekslop-tts.sh --voices)"`. The wrapper reads the
|
||||
- Command provider, implemented as `setup/veskslop-tts.sh`:
|
||||
`VESKSLOP_TTS_CMD="<repo>/setup/veskslop-tts.sh {out} {text_file}"` and
|
||||
`VESKSLOP_TTS_VOICES="$(<repo>/setup/veskslop-tts.sh --voices)"`. The wrapper reads the
|
||||
text file, transliterates UTF-8 to CP1252, starts or reuses an Xvfb when `DISPLAY` is
|
||||
unset, calls the engine, then post-processes (trim, fades, measured fixed gain, 48 kHz
|
||||
mono s16le). A stale private Xvfb is retried once.
|
||||
|
||||
Reference in New Issue
Block a user