zib/Veskslop-t
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Sync MicroslopSam VOX mode and path configuration

Browse Source
This commit is contained in:
Fisto
2026-09-16 08:46:39 +02:00
parent 49fd28756f
commit 6b961ddcef
12 changed files with 1033 additions and 95 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TTS-DESIGN.md
+105
View File
@@ -335,6 +335,111 @@ How to apply a change:
5. Runtime `ITTSAttributes` pitch/speed/volume layer on top of the profile: the profile
defines the natural values, the attributes shift them per request.
### VOX mode (NetTTS-inspired, Flextalk only)
`VESKSLOP_TTS_VOX=1` routes the text through a rule-based prosody encoder before
synthesis. `setup/vox_parser.py` is an independent reimplementation of the VOX parser
from [h4rm0n1c/NetTTS](https://github.com/h4rm0n1c/NetTTS), which models Half-Life's
Black Mesa announcement delivery. It was verified byte-identical to the NetTTS parser
on a 31-line corpus (the reference was built locally with g++ for the comparison).
What it does: sentence splitting; `the` -> `thee` before prepositions, proper nouns,
single letters or numbers; lead-in breaks ("Now...", "Please...", "A reminder...");
time/number/degree normalization; token weighting (light/medium/heavy) with break
insertion at linkers, title-case runs, heavy gates and a 3-beat limiter; letter
tokens (`A` -> `Ay:`, `C` -> `C:`); `\!sf500 \!br` at sentence ends; and the
`\!wH1 ... \!wH0` staccato wrapper (disable with `VESKSLOP_TTS_VOX_CLEAN=1`).
Requirements and scope:
- Flextalk voices only: `\!br`, `\!sfNN` and `\!wH1/\!wH0` are FlexTalk vendor tags.
The host passes `TTSDATAFLAG_TAGGED`, so the engine honors them. Other voices ignore
`VESKSLOP_TTS_VOX`.
- Needs `python3` at runtime; the wrapper fails loudly when it is missing.
- Mutually exclusive with `VESKSLOP_TTS_WORDS` (VOX wins).
- Measured on the dev host for "Now approaching Sector C test lab. Please stand clear
of the test chamber.": plain 4.70 s with 1 pause, VOX 7.81 s with 8 pauses.
Other NetTTS features and their relevance:
- Inline `/rate N`, `/pitch N`, `[[pause N]]`: not ported. Veskslop has speed/pitch
controls and the profile window; `[[pause N]]` could be added as a wrapper-level
text splitter later (NetTTS supports it only outside VOX mode).
- TCP server with `START`/`STOP` status sockets: not needed; Veskslop owns playback
and knows when a line ends.
- Winetricks prefix automation (`winxp`, `vcrun6`, `mfc42`, `riched20`): not needed;
our prefix runs the engines with builtin Wine libraries.
- Win32 GUI and preset sliders: Veskslop provides the UI.
### Inline rate/pitch controls (engine-side)
The wrapper consumes NetTTS-style, line-leading controls, so Veskslop and the WebUI do
not need to parse or implement them:
/rate 150 Attention all personnel. # 150% rate, then the line
/pitch 90 Attention all personnel. # 90% pitch
/rate 120 /pitch 110 Inbound tram. # both, in any order
- `N` is a percent (100 = normal, clamped to 0-200); values `<= 2.0` are multipliers
(`/rate 1.5` = 150%).
- They override `VESKSLOP_TTS_SPEED` / `VESKSLOP_TTS_PITCH` for that line and are not
spoken.
- A line that contains only controls yields 100 ms of silence, so the daemon always
receives a valid WAV.
- Rate maps through the normal speed scale; pitch percent maps to semitones
(`st = 12*log2(N/100)`) and then through the per-voice natural pitch.
- Verified on the dev host: `/rate 50` doubles the line duration; `/pitch 150` raises
the median F0 from 122 Hz to 177 Hz.
### Paths and configuration (standalone)
The engine project is standalone and archival: it has no path dependency on Veskslop,
and every non-installable dependency is vendored under `third_party/`, so nothing is
fetched from the network at build or run time:
- `third_party/speech.h` — SAPI4 SDK header used by the build.
- `third_party/sapi4/` — SAPI4 runtime (`SPEECH.DLL` + INF), MSTTS engine + voices,
and the original installers.
- `third_party/flextalk/data1.cab` — AT&T Flextalk installer (Keith Bell / Kerry Watson).
- `third_party/nettss-reference/` — NetTTS VOX parser source, kept to validate
`setup/vox_parser.py`.
Only toolchain packages are installed by `setup/install-deps.sh` (Wine/Winelib, ffmpeg,
python3, unshield, mingw). Everything else lives in the repository.
`setup/install.sh` writes `setup/microslopsam.env` with absolute paths; environment
variables take precedence over that file, so anything can be relocated without editing
scripts:
| Variable | Default | Meaning |
| --- | --- | --- |
| `MICROSLOP_ROOT` | repo root | project root |
| `MICROSLOP_BUILD` | `$MICROSLOP_ROOT/build` | host binaries, samples |
| `MICROSLOP_PREFIX` | `~/.wine-microslopsam` | wine prefix (exported as `WINEPREFIX`) |
| `MICROSLOP_ASSETS` | `third_party/sapi4` | SAPI4/MSTTS payloads (vendored) |
| `MICROSLOP_FLEXTALK_DIR` | `third_party/flextalk` | Flextalk cabinet/extraction (vendored) |
| `MICROSLOP_SPEECH_H_DIR` | `$MICROSLOP_ROOT/third_party` | `speech.h` include dir (vendored) |
| `MICROSLOP_DISPLAY` | `:77` | Xvfb display used for Wine |
| `MICROSLOP_WINE` / `_XVFB` / `_PYTHON` / `_FFMPEG` | tool names | tool overrides |
The wrapper needs only `VESKSLOP_TTS_CMD` to point at its absolute path; it derives
everything else from its own location plus the env file.
### Merging into Veskslop
This tree is written to survive a copy into the Veskslop repository next to its daemon:
1. Copy `microslopsam/` (or replace the existing copy). The vendored assets under
`third_party/` come along, so the merged tree stays self-contained. Then refresh the
configuration: `MICROSLOP_UPDATE_ENV=1 microslopsam/setup/install.sh`. Re-running
install is idempotent; it rewrites `setup/microslopsam.env` for the new absolute paths.
2. No daemon changes are required. The env/placeholder contract is unchanged
(`VESKSLOP_TTS_*`, `{out}`, `{text_file}`, voice/speed/pitch/expression/profile/
highpass/echo/words/word_gap_ms). New features are additive and default off.
3. Optional Veskslop-side follow-up: a `tts.vox` preference that exports
`VESKSLOP_TTS_VOX` (like `words`) and a line in the TTS panel help. The inline
`/rate` and `/pitch` controls need no UI work at all.
### Transport and latency
- Installation is one command: `setup/install.sh` (system dependencies with sudo, wine
microslopsam/.gitignore
+4
View File
@@ -6,3 +6,7 @@ build/
*.dll
setup/out-*.reg
*.wav
# Generated per-checkout configuration.
setup/microslopsam.env
# Engine DLLs and voice data are proprietary; never commit them.
third_party/
microslopsam/README.md
+38 -8
View File
@@ -35,10 +35,35 @@ Manual steps, if needed:
setup/setup-prefix.sh # prefix + SAPI4 runtime + MSTTS voices
setup/setup-flextalk.sh # Flextalk extraction + registry
# host build:
winegcc -m32 -O2 -o build/sapi4host-winlib host/sapi4host.c -I../reference \
winegcc -m32 -O2 -o build/sapi4host-winlib host/sapi4host.c -Ithird_party \
-lole32 -luser32 -ladvapi32
i686-w64-mingw32-gcc -O2 -static -o build/sapi4host.exe host/sapi4host.c \
-I../reference -lole32 -luser32 -ladvapi32 -luuid
-Ithird_party -lole32 -luser32 -ladvapi32 -luuid
## Configuration
`setup/install.sh` writes `setup/microslopsam.env` (generated, gitignored) with absolute
paths. Environment variables take precedence over the file, so anything can be relocated
without editing scripts:
| Variable | Default | Meaning |
| --- | --- | --- |
| `MICROSLOP_ROOT` | repo root | project root |
| `MICROSLOP_BUILD` | `$MICROSLOP_ROOT/build` | host binaries, samples |
| `MICROSLOP_PREFIX` | `~/.wine-microslopsam` | wine prefix (exported as `WINEPREFIX`) |
| `MICROSLOP_ASSETS` | `third_party/sapi4` | SAPI4/MSTTS payloads (vendored) |
| `MICROSLOP_FLEXTALK_DIR` | `third_party/flextalk` | Flextalk cabinet/extraction (vendored) |
| `MICROSLOP_SPEECH_H_DIR` | `$MICROSLOP_ROOT/third_party` | `speech.h` include dir (vendored) |
| `MICROSLOP_DISPLAY` | `:77` | Xvfb display used for Wine |
| `MICROSLOP_WINE`, `MICROSLOP_XVFB`, `MICROSLOP_PYTHON`, `MICROSLOP_FFMPEG` | tool names | tool overrides |
Refresh the file after moving the tree: `MICROSLOP_UPDATE_ENV=1 setup/install.sh`.
In the Veskslop tree the `third_party/` assets are **not committed** (licensing; see
`docs/TTS-DESIGN.md`). This checkout has a local `setup/microslopsam.env` (gitignored) that
points `MICROSLOP_ASSETS`, `MICROSLOP_FLEXTALK_DIR`, and `MICROSLOP_SPEECH_H_DIR` at the
MicroslopSam development checkout. A manual host build needs the same override, or a local
`third_party/` with `speech.h`.
## Usage
@@ -65,6 +90,11 @@ load: the wrapper writes it to the Wine registry before synthesis. `--voices` ma
voices with `expression=NN`, where NN is the registry default. Mapping details are in
`docs/TTS-DESIGN.md`.
**VOX prosody** (NetTTS-style, Flextalk voices): set `VESKSLOP_TTS_VOX=1` in the
command environment (`VESKSLOP_TTS_VOX_CLEAN=1` omits the `\!wH1/\!wH0` stutter
wrapper). Line-leading `/rate N` and `/pitch N` controls are also handled by the
wrapper, so the WebUI does not need to implement them. See `docs/TTS-DESIGN.md`.
## Layout
- `host/sapi4host.c` — host: voice select, pitch/speed, text -> WAV
@@ -75,6 +105,7 @@ voices with `expression=NN`, where NN is the registry default. Mapping details a
- `setup/inf2reg.py` — INF AddReg -> `.reg` converter
- `setup/run.sh` — run helper (prefix + Xvfb)
- `setup/veskslop-tts.sh` — Veskslop command-provider wrapper (post-processing included)
- `setup/vox_parser.py` — NetTTS-style VOX prosody encoder (Flextalk voices)
- `docs/NOTES.md` — registry/CLSID facts, runtime RE notes, gotchas
- `docs/TTS-DESIGN.md` — integration contract with Veskslop plus engine clarifications
@@ -88,10 +119,9 @@ voices with `expression=NN`, where NN is the registry default. Mapping details a
- The Flextalk archive lacks `data2.cab`; `unshield -O` extracts `data1.cab`
but one tail file (`WatsonUNInstall.dll`) is incomplete.
## Reference material (not in this repo)
## Reference material
- `../reference/speech.h` — SAPI4 SDK header
- `../reference/SAPI4-CLI`, `../reference/TETYYS-SAPI4` — host examples
- `../Microsoft-Sam-Mary-Mike-TruVoice-WSAPI4` — installers/payloads
- `../flextalk` — Flextalk installer archive + extraction
- `/mnt/ghidra/decomp/{msttssyn.dll,speech.dll}/` — decompiled corpora
- The MicroslopSam development checkout (`third_party/`): SAPI4 runtime, MSTTS voices,
Flextalk cabinet, `speech.h`, NetTTS parser source. Referenced through
`setup/microslopsam.env`; not committed here.
- `/mnt/ghidra/decomp/{msttssyn.dll,speech.dll}/` — decompiled corpora (external, optional)
microslopsam/docs/NOTES.md
+16
View File
@@ -93,6 +93,22 @@ final file `WatsonUNInstall.dll` is incomplete).
verified audible (0/59/100 produce different waveforms at equal duration). Changing
them requires a registry edit and re-selecting the voice.
## VOX parser
- `setup/vox_parser.py` reimplements the NetTTS VOX prosody encoder (vendor tags
`\!wH1/\!wH0`, `\!br`, `\!sfNN`); the host already passes `TTSDATAFLAG_TAGGED`, so
the engine honors them.
- Validation: the reference `vox_parser.cpp` from a local clone of
`h4rm0n1c/NetTTS` was compiled with g++ in a throwaway harness and diffed against
the Python port; outputs are identical on the 31-line corpus. Re-check after any
parser change while the clone is available.
- VOX mode is Flextalk-only. The wrapper enables it with `VESKSLOP_TTS_VOX=1`
(`VESKSLOP_TTS_VOX_CLEAN=1` drops the stutter wrapper) and it takes precedence
over `VESKSLOP_TTS_WORDS`.
- Line-leading `/rate N` and `/pitch N` controls are consumed by the wrapper
(percent, 100 = normal; `<= 2.0` means multiplier). They override the env
speed/pitch and are not spoken; a controls-only line yields 100 ms of silence.
## Decompiled corpora (Ghidra)
- `/mnt/ghidra/decomp/msttssyn.dll/` (786 functions, engine)
microslopsam/docs/TTS-DESIGN.md
+105
View File
@@ -335,6 +335,111 @@ How to apply a change:
5. Runtime `ITTSAttributes` pitch/speed/volume layer on top of the profile: the profile
defines the natural values, the attributes shift them per request.
### VOX mode (NetTTS-inspired, Flextalk only)
`VESKSLOP_TTS_VOX=1` routes the text through a rule-based prosody encoder before
synthesis. `setup/vox_parser.py` is an independent reimplementation of the VOX parser
from [h4rm0n1c/NetTTS](https://github.com/h4rm0n1c/NetTTS), which models Half-Life's
Black Mesa announcement delivery. It was verified byte-identical to the NetTTS parser
on a 31-line corpus (the reference was built locally with g++ for the comparison).
What it does: sentence splitting; `the` -> `thee` before prepositions, proper nouns,
single letters or numbers; lead-in breaks ("Now...", "Please...", "A reminder...");
time/number/degree normalization; token weighting (light/medium/heavy) with break
insertion at linkers, title-case runs, heavy gates and a 3-beat limiter; letter
tokens (`A` -> `Ay:`, `C` -> `C:`); `\!sf500 \!br` at sentence ends; and the
`\!wH1 ... \!wH0` staccato wrapper (disable with `VESKSLOP_TTS_VOX_CLEAN=1`).
Requirements and scope:
- Flextalk voices only: `\!br`, `\!sfNN` and `\!wH1/\!wH0` are FlexTalk vendor tags.
The host passes `TTSDATAFLAG_TAGGED`, so the engine honors them. Other voices ignore
`VESKSLOP_TTS_VOX`.
- Needs `python3` at runtime; the wrapper fails loudly when it is missing.
- Mutually exclusive with `VESKSLOP_TTS_WORDS` (VOX wins).
- Measured on the dev host for "Now approaching Sector C test lab. Please stand clear
of the test chamber.": plain 4.70 s with 1 pause, VOX 7.81 s with 8 pauses.
Other NetTTS features and their relevance:
- Inline `/rate N`, `/pitch N`, `[[pause N]]`: not ported. Veskslop has speed/pitch
controls and the profile window; `[[pause N]]` could be added as a wrapper-level
text splitter later (NetTTS supports it only outside VOX mode).
- TCP server with `START`/`STOP` status sockets: not needed; Veskslop owns playback
and knows when a line ends.
- Winetricks prefix automation (`winxp`, `vcrun6`, `mfc42`, `riched20`): not needed;
our prefix runs the engines with builtin Wine libraries.
- Win32 GUI and preset sliders: Veskslop provides the UI.
### Inline rate/pitch controls (engine-side)
The wrapper consumes NetTTS-style, line-leading controls, so Veskslop and the WebUI do
not need to parse or implement them:
/rate 150 Attention all personnel. # 150% rate, then the line
/pitch 90 Attention all personnel. # 90% pitch
/rate 120 /pitch 110 Inbound tram. # both, in any order
- `N` is a percent (100 = normal, clamped to 0-200); values `<= 2.0` are multipliers
(`/rate 1.5` = 150%).
- They override `VESKSLOP_TTS_SPEED` / `VESKSLOP_TTS_PITCH` for that line and are not
spoken.
- A line that contains only controls yields 100 ms of silence, so the daemon always
receives a valid WAV.
- Rate maps through the normal speed scale; pitch percent maps to semitones
(`st = 12*log2(N/100)`) and then through the per-voice natural pitch.
- Verified on the dev host: `/rate 50` doubles the line duration; `/pitch 150` raises
the median F0 from 122 Hz to 177 Hz.
### Paths and configuration (standalone)
The engine project is standalone and archival: it has no path dependency on Veskslop,
and every non-installable dependency is vendored under `third_party/`, so nothing is
fetched from the network at build or run time:
- `third_party/speech.h` — SAPI4 SDK header used by the build.
- `third_party/sapi4/` — SAPI4 runtime (`SPEECH.DLL` + INF), MSTTS engine + voices,
and the original installers.
- `third_party/flextalk/data1.cab` — AT&T Flextalk installer (Keith Bell / Kerry Watson).
- `third_party/nettss-reference/` — NetTTS VOX parser source, kept to validate
`setup/vox_parser.py`.
Only toolchain packages are installed by `setup/install-deps.sh` (Wine/Winelib, ffmpeg,
python3, unshield, mingw). Everything else lives in the repository.
`setup/install.sh` writes `setup/microslopsam.env` with absolute paths; environment
variables take precedence over that file, so anything can be relocated without editing
scripts:
| Variable | Default | Meaning |
| --- | --- | --- |
| `MICROSLOP_ROOT` | repo root | project root |
| `MICROSLOP_BUILD` | `$MICROSLOP_ROOT/build` | host binaries, samples |
| `MICROSLOP_PREFIX` | `~/.wine-microslopsam` | wine prefix (exported as `WINEPREFIX`) |
| `MICROSLOP_ASSETS` | `third_party/sapi4` | SAPI4/MSTTS payloads (vendored) |
| `MICROSLOP_FLEXTALK_DIR` | `third_party/flextalk` | Flextalk cabinet/extraction (vendored) |
| `MICROSLOP_SPEECH_H_DIR` | `$MICROSLOP_ROOT/third_party` | `speech.h` include dir (vendored) |
| `MICROSLOP_DISPLAY` | `:77` | Xvfb display used for Wine |
| `MICROSLOP_WINE` / `_XVFB` / `_PYTHON` / `_FFMPEG` | tool names | tool overrides |
The wrapper needs only `VESKSLOP_TTS_CMD` to point at its absolute path; it derives
everything else from its own location plus the env file.
### Merging into Veskslop
This tree is written to survive a copy into the Veskslop repository next to its daemon:
1. Copy `microslopsam/` (or replace the existing copy). The vendored assets under
`third_party/` come along, so the merged tree stays self-contained. Then refresh the
configuration: `MICROSLOP_UPDATE_ENV=1 microslopsam/setup/install.sh`. Re-running
install is idempotent; it rewrites `setup/microslopsam.env` for the new absolute paths.
2. No daemon changes are required. The env/placeholder contract is unchanged
(`VESKSLOP_TTS_*`, `{out}`, `{text_file}`, voice/speed/pitch/expression/profile/
highpass/echo/words/word_gap_ms). New features are additive and default off.
3. Optional Veskslop-side follow-up: a `tts.vox` preference that exports
`VESKSLOP_TTS_VOX` (like `words`) and a line in the TTS panel help. The inline
`/rate` and `/pitch` controls need no UI work at all.
### Transport and latency
- Installation is one command: `setup/install.sh` (system dependencies with sudo, wine
microslopsam/setup/config.sh
+50
View File
@@ -0,0 +1,50 @@
# shellcheck shell=sh
# Shared configuration for MicroslopSam.
#
# Precedence: environment variables > setup/microslopsam.env (generated by
# setup/install.sh) > built-in defaults.
#
# Source this file near the top of every setup script:
# . "$HERE/config.sh"
#
# All values are absolute paths or tool names. Nothing here depends on the
# project living next to other checkouts, with one exception: the asset repo
# and the Flextalk archive default to sibling directories, which is how the
# development tree is laid out. Set MICROSLOP_ASSETS / MICROSLOP_FLEXTALK_DIR
# to point anywhere else.
: "${HERE:?config.sh: set HERE to the setup directory before sourcing}"
if [ -f "$HERE/microslopsam.env" ]; then
# The generated file uses ': ${VAR:=default}' lines, so values already
# present in the environment win over the file.
. "$HERE/microslopsam.env"
fi
MICROSLOP_ROOT=${MICROSLOP_ROOT:-$(CDPATH= cd -- "$HERE/.." && pwd)}
MICROSLOP_BUILD=${MICROSLOP_BUILD:-$MICROSLOP_ROOT/build}
MICROSLOP_PREFIX=${MICROSLOP_PREFIX:-$HOME/.wine-microslopsam}
MICROSLOP_ASSETS=${MICROSLOP_ASSETS:-$MICROSLOP_ROOT/third_party/sapi4}
MICROSLOP_FLEXTALK_DIR=${MICROSLOP_FLEXTALK_DIR:-$MICROSLOP_ROOT/third_party/flextalk}
MICROSLOP_SPEECH_H_DIR=${MICROSLOP_SPEECH_H_DIR:-$MICROSLOP_ROOT/third_party}
MICROSLOP_DISPLAY=${MICROSLOP_DISPLAY:-:77}
MICROSLOP_XVFB=${MICROSLOP_XVFB:-Xvfb}
MICROSLOP_WINE=${MICROSLOP_WINE:-wine}
MICROSLOP_PYTHON=${MICROSLOP_PYTHON:-python3}
MICROSLOP_FFMPEG=${MICROSLOP_FFMPEG:-ffmpeg}
export MICROSLOP_ROOT MICROSLOP_BUILD MICROSLOP_PREFIX MICROSLOP_ASSETS
export MICROSLOP_FLEXTALK_DIR MICROSLOP_SPEECH_H_DIR MICROSLOP_DISPLAY
export MICROSLOP_XVFB MICROSLOP_WINE MICROSLOP_PYTHON MICROSLOP_FFMPEG
# The wine prefix and arch are what the scripts actually need; keep the
# standard names so callers can override them directly too.
export WINEPREFIX=${WINEPREFIX:-$MICROSLOP_PREFIX}
export WINEARCH=${WINEARCH:-win32}
# Convenience alias for scripts that use $ROOT for repo-relative files.
ROOT=$MICROSLOP_ROOT
# Derived helpers.
MICROSLOP_XSOCK="/tmp/.X11-unix/X${MICROSLOP_DISPLAY#:}"
export MICROSLOP_XSOCK
microslopsam/setup/install.sh
+52 -22
View File
@@ -1,22 +1,41 @@
#!/bin/sh
# One-shot installer for MicroslopSam. Runs all setup steps, idempotently:
# 1. system dependencies (Wine, Winelib, helpers; needs sudo)
# 2. wine prefix + SAPI4 runtime + MSTTS voices + registration
# 3. Flextalk extraction + voices + registration
# 4. host build (Winelib and PE)
# 5. quick voice check
# 1. record resolved paths in setup/microslopsam.env
# 2. system dependencies (Wine, Winelib, helpers; needs sudo)
# 3. wine prefix + SAPI4 runtime + MSTTS voices + registration
# 4. Flextalk extraction + voices + registration
# 5. host build (Winelib and PE)
# 6. quick voice check
#
# Usage: setup/install.sh
# Re-running is safe.
# Paths and tools come from setup/config.sh. Override anything with environment
# variables (they win over setup/microslopsam.env), for example:
# MICROSLOP_ASSETS=/srv/assets MICROSLOP_PREFIX=~/.wine-sam setup/install.sh
# Set MICROSLOP_UPDATE_ENV=1 to rewrite an existing microslopsam.env.
set -eu
HERE=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
ROOT=$(CDPATH= cd -- "$HERE/.." && pwd)
ASSETS=${ASSETS:-$ROOT/../Microsoft-Sam-Mary-Mike-TruVoice-WSAPI4}
FLEXTALK_DIR=${FLEXTALK_DIR:-$ROOT/../flextalk}
. "$HERE/config.sh"
log() { printf '\n== %s\n' "$*"; }
# 0) record the resolved configuration ----------------------------------------
ENVFILE="$HERE/microslopsam.env"
if [ ! -f "$ENVFILE" ] || [ "${MICROSLOP_UPDATE_ENV:-0}" = "1" ]; then
{
echo "# Generated by setup/install.sh -- safe to edit or delete."
echo "# Values are defaults; environment variables take precedence."
printf ': "${MICROSLOP_ROOT:=%s}"\n' "$MICROSLOP_ROOT"
printf ': "${MICROSLOP_BUILD:=%s}"\n' "$MICROSLOP_BUILD"
printf ': "${MICROSLOP_PREFIX:=%s}"\n' "$MICROSLOP_PREFIX"
printf ': "${MICROSLOP_ASSETS:=%s}"\n' "$MICROSLOP_ASSETS"
printf ': "${MICROSLOP_FLEXTALK_DIR:=%s}"\n' "$MICROSLOP_FLEXTALK_DIR"
printf ': "${MICROSLOP_SPEECH_H_DIR:=%s}"\n' "$MICROSLOP_SPEECH_H_DIR"
printf ': "${MICROSLOP_DISPLAY:=%s}"\n' "$MICROSLOP_DISPLAY"
} > "$ENVFILE"
chmod 644 "$ENVFILE" 2>/dev/null || true
log "configuration written to $ENVFILE"
fi
# 1) system dependencies -----------------------------------------------------
if [ "$(id -u)" = "0" ]; then
log "installing system dependencies"
@@ -28,18 +47,19 @@ fi
# 2) SAPI4 runtime + MSTTS voices --------------------------------------------
log "setting up wine prefix, SAPI4 runtime and MSTTS voices"
"$HERE/setup-prefix.sh" "$ASSETS"
"$HERE/setup-prefix.sh" "$MICROSLOP_ASSETS"
# 3) Flextalk voices ----------------------------------------------------------
EXTRACTED="$FLEXTALK_DIR/extracted/TTS_Executables"
EXTRACTED="$MICROSLOP_FLEXTALK_DIR/extracted/TTS_Executables"
if [ ! -d "$EXTRACTED/EngineDLLs" ]; then
CAB=""
for c in "$FLEXTALK_DIR/installer/flextalk/data1.cab" \
"$ASSETS/flextalk/data1.cab"; do
for c in "$MICROSLOP_FLEXTALK_DIR/data1.cab" \
"$MICROSLOP_FLEXTALK_DIR/installer/flextalk/data1.cab" \
"$MICROSLOP_ASSETS/flextalk/data1.cab"; do
if [ -f "$c" ]; then CAB="$c"; break; fi
done
if [ -z "$CAB" ]; then
echo "Flextalk data1.cab not found; set FLEXTALK_DIR to the archive dir" >&2
echo "Flextalk data1.cab not found; set MICROSLOP_FLEXTALK_DIR" >&2
exit 1
fi
command -v unshield >/dev/null 2>&1 || {
@@ -47,20 +67,24 @@ if [ ! -d "$EXTRACTED/EngineDLLs" ]; then
exit 1
}
log "extracting Flextalk installer ($CAB)"
rm -rf "$FLEXTALK_DIR/extracted"
unshield -O -d "$FLEXTALK_DIR/extracted" x "$CAB" >/dev/null
rm -rf "$MICROSLOP_FLEXTALK_DIR/extracted"
unshield -O -d "$MICROSLOP_FLEXTALK_DIR/extracted" x "$CAB" >/dev/null
fi
log "installing Flextalk voices"
"$HERE/setup-flextalk.sh" "$EXTRACTED"
# 4) host build ---------------------------------------------------------------
log "building the host"
mkdir -p "$ROOT/build"
winegcc -m32 -O2 -o "$ROOT/build/sapi4host-winlib" "$ROOT/host/sapi4host.c" \
-I"$ROOT/../reference" -lole32 -luser32 -ladvapi32
[ -f "$MICROSLOP_SPEECH_H_DIR/speech.h" ] || {
echo "speech.h not found in $MICROSLOP_SPEECH_H_DIR" >&2
exit 1
}
mkdir -p "$MICROSLOP_BUILD"
winegcc -m32 -O2 -o "$MICROSLOP_BUILD/sapi4host-winlib" "$MICROSLOP_ROOT/host/sapi4host.c" \
-I"$MICROSLOP_SPEECH_H_DIR" -lole32 -luser32 -ladvapi32
if command -v i686-w64-mingw32-gcc >/dev/null 2>&1; then
i686-w64-mingw32-gcc -O2 -static -o "$ROOT/build/sapi4host.exe" \
"$ROOT/host/sapi4host.c" -I"$ROOT/../reference" \
i686-w64-mingw32-gcc -O2 -static -o "$MICROSLOP_BUILD/sapi4host.exe" \
"$MICROSLOP_ROOT/host/sapi4host.c" -I"$MICROSLOP_SPEECH_H_DIR" \
-lole32 -luser32 -ladvapi32 -luuid
fi
@@ -72,6 +96,12 @@ cat <<EOF
Done.
Configuration: $ENVFILE
MICROSLOP_ROOT=$MICROSLOP_ROOT
MICROSLOP_PREFIX=$MICROSLOP_PREFIX
MICROSLOP_ASSETS=$MICROSLOP_ASSETS
MICROSLOP_FLEXTALK_DIR=$MICROSLOP_FLEXTALK_DIR
Veskslop configuration:
VESKSLOP_TTS_CMD="$HERE/veskslop-tts.sh {out} {text_file}"
VESKSLOP_TTS_VOICES="\$($HERE/veskslop-tts.sh --voices)"
microslopsam/setup/run.sh
+8 -10
View File
@@ -1,27 +1,25 @@
#!/bin/sh
# Run the SAPI4 host with the project's wine prefix and a working X display.
# Run the SAPI4 host with the configured wine prefix and X display.
# Usage: setup/run.sh [sapi4host args...]
set -eu
HERE=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
ROOT=$(CDPATH= cd -- "$HERE/.." && pwd)
export WINEPREFIX=${WINEPREFIX:-$HOME/.wine-microslopsam}
export WINEARCH=win32
. "$HERE/config.sh"
if [ -z "${DISPLAY:-}" ]; then
if ! [ -S /tmp/.X11-unix/X77 ]; then
Xvfb :77 -screen 0 1024x768x24 >/dev/null 2>&1 &
if ! [ -S "$MICROSLOP_XSOCK" ]; then
"$MICROSLOP_XVFB" "$MICROSLOP_DISPLAY" -screen 0 1024x768x24 >/dev/null 2>&1 &
sleep 1
fi
export DISPLAY=:77
export DISPLAY=$MICROSLOP_DISPLAY
fi
HOST="$ROOT/build/sapi4host-winlib.exe"
HOST="$MICROSLOP_BUILD/sapi4host-winlib.exe"
if [ ! -x "$HOST" ]; then
HOST="$ROOT/build/sapi4host.exe"
HOST="$MICROSLOP_BUILD/sapi4host.exe"
fi
if [ ! -x "$HOST" ]; then
echo "no host binary; build first (see README)" >&2
echo "no host binary; run setup/install.sh first" >&2
exit 1
fi
microslopsam/setup/setup-flextalk.sh
+5 -8
View File
@@ -7,18 +7,15 @@
set -eu
HERE=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
ROOT=$(CDPATH= cd -- "$HERE/.." && pwd)
SRC=${1:-$ROOT/../flextalk/extracted/TTS_Executables}
. "$HERE/config.sh"
SRC=${1:-$MICROSLOP_FLEXTALK_DIR/extracted/TTS_Executables}
[ -d "$SRC/EngineDLLs" ] || { echo "source not found: $SRC" >&2; exit 1; }
export WINEPREFIX=${WINEPREFIX:-$HOME/.wine-microslopsam}
export WINEARCH=win32
if [ -z "${DISPLAY:-}" ]; then
WRUN="xvfb-run -a wine"
WRUN="xvfb-run -a $MICROSLOP_WINE"
else
WRUN="wine"
WRUN="$MICROSLOP_WINE"
fi
DEST="$WINEPREFIX/drive_c/FlexTalk"
@@ -92,4 +89,4 @@ echo "== registry"
$WRUN regedit /S "$REG" 2>/dev/null || true
echo "== done; list voices:"
echo " $WRUN $ROOT/host/sapi4host.exe -l"
echo " $WRUN $MICROSLOP_BUILD/sapi4host.exe -l"
microslopsam/setup/setup-prefix.sh
+5 -8
View File
@@ -7,18 +7,15 @@
set -eu
HERE=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
ROOT=$(CDPATH= cd -- "$HERE/.." && pwd)
ASSETS=${1:-$ROOT/../Microsoft-Sam-Mary-Mike-TruVoice-WSAPI4}
. "$HERE/config.sh"
ASSETS=${1:-$MICROSLOP_ASSETS}
[ -d "$ASSETS/spchapi" ] || { echo "asset dir not found: $ASSETS" >&2; exit 1; }
export WINEPREFIX=${WINEPREFIX:-$HOME/.wine-microslopsam}
export WINEARCH=win32
if [ -z "${DISPLAY:-}" ]; then
WRUN="xvfb-run -a wine"
WRUN="xvfb-run -a $MICROSLOP_WINE"
else
WRUN="wine"
WRUN="$MICROSLOP_WINE"
fi
echo "== prefix: $WINEPREFIX"
@@ -48,4 +45,4 @@ $WRUN regedit /S "$HERE/out-spchapi.reg"
$WRUN regedit /S "$HERE/out-mstts.reg"
echo "== done"
echo "list voices: $WRUN $ROOT/host/sapi4host.exe -l"
echo "list voices: $WRUN $MICROSLOP_BUILD/sapi4host.exe -l"
microslopsam/setup/veskslop-tts.sh
+104 -39
View File
@@ -18,14 +18,15 @@
# VESKSLOP_TTS_ECHO mono echo amount 0-100 (default 0)
# VESKSLOP_TTS_WORDS 1 = synthesize each word, merge (Flextalk)
# VESKSLOP_TTS_WORD_GAP_MS silence between words in ms (0-500)
# VESKSLOP_TTS_VOX 1 = VOX prosody, NetTTS-style (Flextalk)
# VESKSLOP_TTS_VOX_CLEAN 1 = VOX without the \!wH1/\!wH0 stutter
#
# Output: 48 kHz mono s16le WAV, trimmed, faded, loudness-normalized.
set -eu
HERE=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
ROOT=$(CDPATH= cd -- "$HERE/.." && pwd)
export WINEPREFIX=${WINEPREFIX:-$HOME/.wine-microslopsam}
export WINEARCH=win32
. "$HERE/config.sh"
ROOT=$MICROSLOP_ROOT
# VESKSLOP_TTS_VOICES entries: id|Name|gender|language[|flags]
# `expression=NN` marks a Flextalk voice with an Expression profile parameter and
@@ -49,6 +50,8 @@ HIGHPASS=${VESKSLOP_TTS_HIGHPASS:-40}
ECHO=${VESKSLOP_TTS_ECHO:-0}
WORDS=${VESKSLOP_TTS_WORDS:-0}
WORD_GAP=${VESKSLOP_TTS_WORD_GAP_MS:-120}
VOX=${VESKSLOP_TTS_VOX:-0}
VOX_CLEAN=${VESKSLOP_TTS_VOX_CLEAN:-0}
case "$VOICE" in
sam) ENGVOICE="Sam" ;;
@@ -83,6 +86,66 @@ case "$ENGVOICE" in
"Kerry Watson") SMAX=390; PMAX=250; NAT_PITCH=186 ;;
*) SMAX=450; PMAX=200; NAT_PITCH=100 ;;
esac
# The daemon does not export DISPLAY; keep a private Xvfb for the engine.
start_xvfb() {
if ! [ -S "$MICROSLOP_XSOCK" ]; then
"$MICROSLOP_XVFB" "$MICROSLOP_DISPLAY" -screen 0 1024x768x24 >/dev/null 2>&1 &
sleep 1
fi
}
if [ -z "${DISPLAY:-}" ]; then
start_xvfb
export DISPLAY=$MICROSLOP_DISPLAY
fi
# NetTTS-style line controls, handled here so no caller/WebUI work is needed:
# /rate N and/or /pitch N at the start of the line.
# N is a percent (100 = normal); values <= 2.0 are multipliers (/rate 1.5 = 150%).
# They override the env speed/pitch for this line and are not spoken. A line that
# contains only commands yields 100 ms of silence, so the daemon still gets a WAV.
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
RAW="$TMP/raw.wav"
TXT="$TMP/text.cp1252"
TEXT_IN="$TEXTFILE"
if [ -f "$TEXTFILE" ]; then
set -f
# shellcheck disable=SC2046
set -- $(cat "$TEXTFILE")
set +f
CMD_SPEED=""
CMD_PITCH=""
while [ "$#" -ge 2 ]; do
case "$1" in
/rate|/pitch)
case "$2" in
''|*[!0-9.]*) break ;;
esac
if [ "$1" = "/rate" ]; then CMD_SPEED=$2; else CMD_PITCH=$2; fi
shift 2
;;
*) break ;;
esac
done
if [ -n "$CMD_SPEED" ] || [ -n "$CMD_PITCH" ]; then
if [ -n "$CMD_SPEED" ]; then
SPEED=$(awk -v v="$CMD_SPEED" 'BEGIN{ if (v<=2.0) v*=100; if (v<0) v=0; if (v>200) v=200; printf "%.4f", v/100 }')
fi
if [ -n "$CMD_PITCH" ]; then
PITCH=$(awk -v v="$CMD_PITCH" 'BEGIN{ if (v<=2.0) v*=100; if (v<0) v=0; if (v>200) v=200; printf "%.4f", 12*log(v/100)/log(2) }')
fi
if [ -z "$*" ]; then
"$MICROSLOP_FFMPEG" -hide_banner -loglevel error -f lavfi -i anullsrc=r=48000:cl=mono \
-t 0.1 -c:a pcm_s16le -y "$OUT" >/dev/null 2>&1 \
|| { echo "could not write silence" >&2; exit 1; }
exit 0
fi
printf '%s\n' "$*" > "$TMP/input.txt"
TEXT_IN="$TMP/input.txt"
fi
fi
# Build the engine arguments from the (possibly command-overridden) speed/pitch.
SET_ARGS=""
if [ -n "$SPEED" ]; then
S=$(awk -v m="$SPEED" 'BEGIN{printf "%d", 150*m+0.5}')
@@ -97,23 +160,6 @@ if [ -n "$PITCH" ]; then
SET_ARGS="$SET_ARGS -p $P"
fi
# The daemon does not export DISPLAY; keep a private Xvfb for the engine.
start_xvfb() {
if ! [ -S /tmp/.X11-unix/X77 ]; then
Xvfb :77 -screen 0 1024x768x24 >/dev/null 2>&1 &
sleep 1
fi
}
if [ -z "${DISPLAY:-}" ]; then
start_xvfb
export DISPLAY=:77
fi
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
RAW="$TMP/raw.wav"
TXT="$TMP/text.cp1252"
# Apply the Flextalk profile before synthesis. Profiles are read at voice load and
# the host loads the engine per request, so the values are written every call: the
# engine defaults plus any VESKSLOP_TTS_PROFILE overrides. A missing value therefore
@@ -187,28 +233,46 @@ if [ -n "$SPEAKER_GUID" ]; then
exit 1
fi
if ! command -v wine >/dev/null 2>&1; then
if ! command -v "$MICROSLOP_WINE" >/dev/null 2>&1; then
echo "wine is required to set the Flextalk profile" >&2
exit 1
fi
if ! wine reg import "$TMP/profile.reg" >/dev/null 2>&1; then
if ! "$MICROSLOP_WINE" reg import "$TMP/profile.reg" >/dev/null 2>&1; then
echo "could not write the Flextalk profile" >&2
exit 1
fi
fi
# SAPI4 is ANSI/CP1252; transliterate UTF-8 (emoji become '?').
if command -v iconv >/dev/null 2>&1; then
iconv -f UTF-8 -t WINDOWS-1252//TRANSLIT "$TEXTFILE" > "$TXT" 2>/dev/null \
|| cp "$TEXTFILE" "$TXT"
else
cp "$TEXTFILE" "$TXT"
# SAPI4 is ANSI/CP1252. VOX mode (Flextalk only) first transforms the text into
# vendor tags (\!wH1/\!wH0, \!br, \!sfNN) with setup/vox_parser.py; the parser
# works on UTF-8, so it runs before the CP1252 transliteration.
TEXT_SRC="$TEXT_IN"
if [ "$VOX" = "1" ] && [ -n "$SPEAKER_GUID" ]; then
if command -v "$MICROSLOP_PYTHON" >/dev/null 2>&1 && [ -f "$MICROSLOP_ROOT/setup/vox_parser.py" ]; then
if [ "$VOX_CLEAN" = "1" ]; then
"$MICROSLOP_PYTHON" "$MICROSLOP_ROOT/setup/vox_parser.py" --clean < "$TEXT_IN" > "$TMP/vox.txt"
else
"$MICROSLOP_PYTHON" "$MICROSLOP_ROOT/setup/vox_parser.py" < "$TEXT_IN" > "$TMP/vox.txt"
fi
TEXT_SRC="$TMP/vox.txt"
else
echo "vox mode needs python3 and setup/vox_parser.py" >&2
exit 1
fi
fi
HOST="$ROOT/build/sapi4host-winlib.exe"
if [ ! -x "$HOST" ]; then HOST="$ROOT/build/sapi4host.exe"; fi
# transliterate UTF-8 (emoji become '?')
if command -v iconv >/dev/null 2>&1; then
iconv -f UTF-8 -t WINDOWS-1252//TRANSLIT "$TEXT_SRC" > "$TXT" 2>/dev/null \
|| cp "$TEXT_SRC" "$TXT"
else
cp "$TEXT_SRC" "$TXT"
fi
HOST="$MICROSLOP_BUILD/sapi4host-winlib.exe"
if [ ! -x "$HOST" ]; then HOST="$MICROSLOP_BUILD/sapi4host.exe"; fi
if [ ! -x "$HOST" ]; then echo "engine host not built" >&2; exit 1; fi
if ! command -v ffmpeg >/dev/null 2>&1; then echo "ffmpeg not found" >&2; exit 1; fi
if ! command -v "$MICROSLOP_FFMPEG" >/dev/null 2>&1; then echo "ffmpeg not found" >&2; exit 1; fi
run_host() {
# shellcheck disable=SC2086
@@ -219,9 +283,9 @@ run_host() {
try_run_host() {
if run_host "$1" "$2"; then return 0; fi
if [ "${DISPLAY:-}" = ":77" ]; then
pkill -f "Xvfb :77" 2>/dev/null || true
rm -f /tmp/.X11-unix/X77
if [ "${DISPLAY:-}" = "$MICROSLOP_DISPLAY" ]; then
pkill -f "Xvfb $MICROSLOP_DISPLAY" 2>/dev/null || true
rm -f "$MICROSLOP_XSOCK"
start_xvfb
if run_host "$1" "$2"; then return 0; fi
fi
@@ -232,7 +296,8 @@ try_run_host() {
# Word mode (Flextalk): synthesize each word and merge with silence gaps, which
# gives the staccato radio cadence the Half-Life VOX/HEV voices are known for.
if [ "$WORDS" = "1" ] && [ -n "$SPEAKER_GUID" ]; then
# VOX mode takes precedence: it produces its own cadence with vendor tags.
if [ "$WORDS" = "1" ] && [ -n "$SPEAKER_GUID" ] && [ "$VOX" != "1" ]; then
GAP=$WORD_GAP
case "$GAP" in *[!0-9]*|"") GAP=120 ;; esac
[ "$GAP" -gt 500 ] && GAP=500
@@ -245,7 +310,7 @@ if [ "$WORDS" = "1" ] && [ -n "$SPEAKER_GUID" ]; then
echo "file '$TMP/w$INDEX.wav'" >> "$TMP/list.txt"
if [ "$GAP" -gt 0 ]; then
ffmpeg -hide_banner -loglevel error -f lavfi -i "anullsrc=r=11025:cl=mono" \
"$MICROSLOP_FFMPEG" -hide_banner -loglevel error -f lavfi -i "anullsrc=r=11025:cl=mono" \
-t "$(awk -v g="$GAP" 'BEGIN{printf "%.3f", g/1000}')" -c:a pcm_s16le -y "$TMP/sil.wav" \
>/dev/null 2>&1
echo "file '$TMP/sil.wav'" >> "$TMP/list.txt"
@@ -253,7 +318,7 @@ if [ "$WORDS" = "1" ] && [ -n "$SPEAKER_GUID" ]; then
done
[ "$INDEX" -gt 0 ] || { echo "no words to speak" >&2; exit 1; }
if ! ffmpeg -hide_banner -loglevel error -f concat -safe 0 -i "$TMP/list.txt" -c copy -y "$RAW" \
if ! "$MICROSLOP_FFMPEG" -hide_banner -loglevel error -f concat -safe 0 -i "$TMP/list.txt" -c copy -y "$RAW" \
>/dev/null 2>&1; then
echo "could not merge the words" >&2
exit 1
@@ -281,7 +346,7 @@ fi
# Fixed-gain normalization: single-pass loudnorm misbehaves on short lines
# (LRA=0 causes heavy attenuation), so measure EBU R128 integrated loudness and
# true peak, then apply one gain clamped by the -3 dBFS peak target.
M=$(ffmpeg -hide_banner -nostats -i "$RAW" -af "$PRE,ebur128=peak=true" \
M=$("$MICROSLOP_FFMPEG" -hide_banner -nostats -i "$RAW" -af "$PRE,ebur128=peak=true" \
-f null - 2>&1 || true)
I=$(printf '%s' "$M" | sed -n 's/ *I: *\(-\{0,1\}[0-9.]*\) *LUFS.*/\1/p' | tail -1)
TP=$(printf '%s' "$M" | sed -n 's/ *Peak: *\(-\{0,1\}[0-9.]*\) *dBFS.*/\1/p' | tail -1)
@@ -299,7 +364,7 @@ else
fi
TMPOUT="$TMP/out.wav"
if ! ffmpeg -hide_banner -nostats -y -i "$RAW" -af "$PRE,volume=${GAIN}dB" \
if ! "$MICROSLOP_FFMPEG" -hide_banner -nostats -y -i "$RAW" -af "$PRE,volume=${GAIN}dB" \
-ar 48000 -ac 1 -c:a pcm_s16le "$TMPOUT" >/dev/null 2>"$TMP/ff"; then
head -c 400 "$TMP/ff" >&2
exit 1
microslopsam/setup/vox_parser.py
+541
View File
@@ -0,0 +1,541 @@
#!/usr/bin/env python3
"""VOX prosody encoder for FlexTalk, ported from h4rm0n1c/NetTTS.
Transforms plain text into FlexTalk vendor-tagged text:
\\!wH1 ... \\!wH0 staccato word-cadence wrapper
\\!br beat boundary
\\!sfNN final pause (centiseconds)
Our host passes TTSDATAFLAG_TAGGED, so the engine honors these tags.
Rule set studied from https://github.com/h4rm0n1c/NetTTS (vox_parser.cpp,
no license declared; this is an independent reimplementation).
Usage:
vox_parser.py [--clean] [text]
echo "text" | vox_parser.py
--clean omits the \\!wH1/\\!wH0 wrapper (plain cadence, no word stutter).
"""
import re
import sys
BR = r'\!br'
WH1 = r'\!wH1'
WH0 = r'\!wH0'
TAGS = (BR, WH1, WH0)
RE_LEADIN = re.compile(
r'^\s*((?:Now|Please|A\s+reminder|On\s+behalf\s+of|Good\s+(?:morning|evening)'
r'|This\s+(?:automated\s+train|tram)))\b', re.IGNORECASE | re.ASCII)
RE_TIME12 = re.compile(
r'\b(\d{1,2})[:.](\d{2})\s*((?:A\.?M\.?)|(?:P\.?M\.?))(?:\.)?\b',
re.IGNORECASE | re.ASCII)
RE_TIME_IS = re.compile(r'(\b[Tt]he\s+time\s+is)\s+(\d+)', re.ASCII)
RE_24H = re.compile(r'\b(\d{2})00\s+hours\b', re.IGNORECASE | re.ASCII)
RE_N3 = re.compile(r'\b([1-9])(\d)(\d)\b', re.ASCII)
RE_DEGREES = re.compile(r'\b(\d+)\s+degrees\b', re.IGNORECASE | re.ASCII)
LIGHT, MEDIUM, HEAVY = 0, 1, 2
STOP_WORDS = {
'the', 'thee', 'a', 'an', 'to', 'for', 'of', 'in', 'on', 'at', 'by', 'with',
'from', 'as', 'is', 'are', 'was', 'were', 'this', 'that',
}
LIGHT_VERBS = {
'welcome', 'wait', 'stand', 'provide', 'provided', 'return', 'remain',
'maintain', 'maintained', 'verify', 'contact', 'board', 'arriving',
'inbound', 'bound', 'commence', 'commences',
}
UNITS = {'degrees', 'hours', 'percent', '%'}
LINKERS = {'at', 'in', 'on', 'to', 'of', 'for', 'from', 'by'}
DETERMINERS = {'the', 'thee', 'a', 'an', 'this', 'that', 'these', 'those',
'some', 'any', 'each', 'every', 'no'}
PUNCT = '.,;:!?\'"'
def trim(s):
return s.strip()
def lower(s):
return s.lower()
def bare(tok):
return strip_trailing_punct(tok)
def is_all_digits(t):
return bool(t) and all('0' <= c <= '9' for c in t)
def is_titlecase_word(t):
return bool(t) and t[0].isalpha() and t[0].isupper()
def strip_trailing_punct(t):
while t and t[-1] in PUNCT:
t = t[:-1]
return t
def split_tail(tok):
"""Return (core, trailing punctuation)."""
core, tail = tok, ''
while core and core[-1] in PUNCT:
tail = core[-1] + tail
core = core[:-1]
return core, tail
def ends_with_br(s):
return s.endswith(' ' + BR)
def is_tag(tok):
return tok in TAGS
def split_sentences(text):
out, cur = [], ''
i = 0
n = len(text)
while i < n:
c = text[i]
cur += c
if c in '.!?':
j = i + 1
while j < n and text[j] in '"\'':
cur += text[j]
j += 1
if trim(cur):
out.append(trim(cur))
cur = ''
i = j - 1
i += 1
tail = trim(cur)
if tail:
out.append(tail)
return out
def apply_leadin_break(s):
m = RE_LEADIN.search(s)
if not m:
return s
head = trim(m.group(0))
rest = trim(s[m.end():])
return head + ' ' + BR + ' ' + rest
def is_single_letter(tok):
b = bare(tok)
if len(b) == 1 and b[0].isalpha() and b[0].isupper():
return True
return len(b) == 2 and b[0].isalpha() and b[0].isupper() and b[1] == ':'
def is_number(tok):
b = bare(tok)
return is_all_digits(b)
def apply_thee_rule(s):
tok = s.split()
if not tok:
return s
def prev_word(i, nth):
for k in range(i - 1, -1, -1):
if is_tag(tok[k]):
continue
nth -= 1
if nth == 0:
return lower(split_tail(tok[k])[0])
return ''
def next_word(i, nth):
for k in range(i + 1, len(tok)):
if is_tag(tok[k]):
continue
nth -= 1
if nth == 0:
return tok[k]
return ''
for i in range(len(tok)):
if is_tag(tok[i]):
continue
core, tail = split_tail(tok[i])
if lower(core) != 'the':
tok[i] = core + tail
continue
make_thee = False
p1 = prev_word(i, 1)
if p1:
if p1 in LINKERS:
make_thee = True
elif p1 == 'of':
p2, p3 = prev_word(i, 2), prev_word(i, 3)
if p2 == 'behalf' and p3 == 'on':
make_thee = True
if not make_thee:
n1 = next_word(i, 1)
if n1 and (is_titlecase_word(bare(n1)) or is_single_letter(n1)
or is_number(n1)):
make_thee = True
tok[i] = ('thee' if make_thee else core) + tail
return ' '.join(tok)
def apply_time_numbers_degrees(s):
def time12(m):
hh, mm, ap = m.group(1), m.group(2), lower(m.group(3))
ap1 = 'P:' if ap.startswith('p') else 'Ay:'
return hh + ' ' + BR + ' ' + mm + ' ' + BR + ' ' + ap1 + ' ' + BR + ' M:'
s = RE_TIME12.sub(time12, s)
s = RE_TIME_IS.sub(r'\1 ' + BR + r' \2', s)
s = RE_24H.sub(r'\1 ' + BR + ' hundred ' + BR + ' hours', s)
def n3(m):
yz = m.group(2) + m.group(3)
if yz == '00':
return m.group(0)
rem = yz.lstrip('0') or yz
return m.group(1) + '00 ' + BR + ' and ' + BR + ' ' + rem
s = RE_N3.sub(n3, s)
s = RE_DEGREES.sub(r'\1 ' + BR + ' degrees ' + BR, s)
s = re.sub(r'M:\s*\.', 'M:', s)
return s
def syllables(word):
if not word:
return 0
w = lower(word)
vowels = set('aeiouy')
count = 0
in_v = False
for c in w:
if c in vowels:
if not in_v:
count += 1
in_v = True
else:
in_v = False
if count > 1 and len(w) > 1 and w[-1] == 'e' and not (len(w) > 2 and w[-2] == 'l'):
count -= 1
if len(w) > 2 and w[-1] == 'e' and w[-2] == 'l':
count += 1
return max(1, count)
def weight(tok):
if not tok:
return LIGHT
pfree = strip_trailing_punct(tok)
tl = lower(pfree)
if tl in STOP_WORDS or tl in LIGHT_VERBS:
return LIGHT
if is_all_digits(pfree):
return HEAVY
if '-' in pfree:
return HEAVY
if is_titlecase_word(pfree):
return HEAVY
if tl in UNITS:
return HEAVY
if syllables(pfree) >= 3 or len(pfree) >= 8:
return HEAVY
return MEDIUM
def is_and(tok):
return lower(strip_trailing_punct(tok)) == 'and'
def starts_with_punct(tok):
return bool(tok) and tok[0] in '.,;:!?)'
def is_open_punct(tok):
return bool(tok) and tok[0] in '("\''
def emit_titlecase_run(v, i, j, out):
if out and not ends_with_br(out):
out += ' ' + BR
for k in range(i, j):
out += ' ' + v[k] + ' ' + BR
return out
def build_beats(sentence):
v = sentence.split()
if not v:
return sentence
out = ['']
state = {'since_break_all_light': True, 'prev_w': LIGHT, 'content_run': 0}
def reset():
state['since_break_all_light'] = True
state['content_run'] = 0
state['prev_w'] = LIGHT
def ensure_break():
if not ends_with_br(out[0]):
out[0] += ' ' + BR
def append_token(tok):
if out[0] and not starts_with_punct(tok) and not is_open_punct(tok):
out[0] += ' '
out[0] += tok
def emit_dash_split(tok):
core, tail = split_tail(tok)
cuts = []
for p, c in enumerate(core):
if c in ('-', '\u2013', '\u2014'):
if 0 < p < len(core) - 1 and core[p - 1].isalpha() and core[p + 1].isalpha():
cuts.append(p)
else:
return False
if not cuts:
return False
start = 0
for idx in range(len(cuts) + 1):
end = cuts[idx] if idx < len(cuts) else len(core)
part = core[start:end]
if out[0] and not ends_with_br(out[0]) and not starts_with_punct(part) and not is_open_punct(part):
out[0] += ' '
if idx == len(cuts):
part += tail
out[0] += part
ensure_break()
reset()
start = end + (1 if idx < len(cuts) else 0)
return True
i = 0
while i < len(v):
# Proper-noun blocks
if bare(v[i]) in ('Area', 'Level', 'Sector'):
j = i + 1
while j < len(v):
wj = bare(v[j])
if is_all_digits(wj) or is_titlecase_word(wj):
j += 1
else:
break
if j > i + 1:
out[0] = emit_titlecase_run(v, i, j, out[0])
i = j
reset()
continue
if is_titlecase_word(bare(v[i])):
j = i
while j < len(v) and is_titlecase_word(bare(v[j])):
j += 1
if j >= i + 2:
out[0] = emit_titlecase_run(v, i, j, out[0])
i = j
reset()
continue
# "and" between heavier items
if is_and(v[i]) and 0 < i < len(v) - 1:
if weight(v[i - 1]) >= MEDIUM and weight(v[i + 1]) >= MEDIUM:
ensure_break()
out[0] += ' and ' + BR
i += 1
reset()
continue
# Standalone dash
if v[i] in ('-', '\u2013', '\u2014'):
ensure_break()
reset()
i += 1
continue
# Symmetric linker isolation
tl_tok = lower(bare(v[i]))
if i >= 2 and tl_tok in LINKERS:
ensure_break()
reset()
append_token(v[i])
ensure_break()
reset()
i += 1
continue
# Pre-copula break
if i >= 2 and lower(bare(v[i])) == 'is':
k = i + 1
while k < len(v) and is_tag(v[k]):
k += 1
trigger = False
if k < len(v):
n1 = v[k]
if (lower(strip_trailing_punct(n1)) in DETERMINERS
or is_titlecase_word(strip_trailing_punct(n1))
or is_all_digits(strip_trailing_punct(n1))
or is_single_letter(n1)
or weight(n1) == HEAVY):
trigger = True
if trigger and not ends_with_br(out[0]):
out[0] += ' ' + BR
reset()
# Hyphenated compounds
if emit_dash_split(v[i]):
i += 1
continue
# Generic token logic
tok = v[i]
w_eff = weight(tok)
in_head_span = i < 2
if i == 0 and is_titlecase_word(bare(tok)):
next_is_title = (i + 1 < len(v)) and is_titlecase_word(bare(v[i + 1]))
if not next_is_title and w_eff == HEAVY:
w_eff = MEDIUM
if w_eff == HEAVY and state['content_run'] >= 1:
k = i - 1
while k >= 0 and is_tag(v[k]):
k -= 1
exempt = False
if k >= 0:
prev = v[k]
if (is_titlecase_word(bare(prev)) or is_all_digits(bare(prev))
or is_single_letter(prev)):
exempt = True
if not exempt:
ensure_break()
reset()
if w_eff == HEAVY and state['content_run'] >= 2:
ensure_break()
reset()
if w_eff == HEAVY and state['since_break_all_light'] and not in_head_span:
if out[0] and not ends_with_br(out[0]):
out[0] += ' ' + BR
reset()
if i > 0 and state['prev_w'] == HEAVY and w_eff == HEAVY and not in_head_span:
ensure_break()
reset()
append_token(tok)
if tok and tok[-1] in ',;' and not ends_with_br(out[0]):
out[0] += ' ' + BR
reset()
i += 1
continue
if w_eff == LIGHT:
pass
else:
state['content_run'] += 1
state['since_break_all_light'] = False
state['prev_w'] = w_eff
i += 1
return trim(out[0])
def normalize_letter_tokens(s):
tok = s.split()
if not tok:
return s
def is_boundary(idx):
return idx < 0 or idx >= len(tok) or is_tag(tok[idx])
for i in range(len(tok)):
if is_tag(tok[i]):
continue
core, tail = tok[i], ''
had_colon = False
while core:
c = core[-1]
if c == ':' and not had_colon:
had_colon = True
core = core[:-1]
continue
if c in '.,;!?\'"':
tail = c + tail
core = core[:-1]
continue
break
near_bound = is_boundary(i - 1) or is_boundary(i + 1)
if core in ('A', 'a') and (near_bound or had_colon):
core = 'Ay'
had_colon = True
elif len(core) == 1 and core.isalpha() and core.isupper() and near_bound:
had_colon = True
tok[i] = core + (':' if had_colon else '') + tail
return ' '.join(tok)
def tidy(s):
t = re.sub(r'\s+', ' ', s)
t = re.sub(r'\s*' + re.escape(BR) + r'\s*', ' ' + BR + ' ', t)
t = re.sub(r'(\s*' + re.escape(BR) + r'\s*){2,}', ' ' + BR + ' ', t)
t = re.sub(r'\s+([.,!?;:])', r'\1', t)
return t.strip()
def vox_process(text, wrap_vox_tags=True):
out = ''
for sent in split_sentences(text):
if not sent:
continue
sent = apply_thee_rule(sent)
sent = apply_leadin_break(sent)
sent = apply_time_numbers_degrees(sent)
beats = build_beats(sent)
beats = normalize_letter_tokens(beats)
beats = tidy(beats)
if beats and not beats.endswith(' '):
beats += ' '
beats += r'\!sf500 ' + BR
if out:
out += ' '
out += beats
out = trim(out)
out = re.sub(r'(\s*' + re.escape(BR) + r'\s*){2,}', ' ' + BR + ' ', out)
out = re.sub(re.escape(BR) + r'\s*([,;:])', r'\1 ' + BR, out)
if out:
if wrap_vox_tags:
out = WH1 + ' ' + out + ' ' + WH0 + ' '
else:
out = out + ' '
return out
def main():
args = sys.argv[1:]
clean = False
if '--clean' in args:
clean = True
args.remove('--clean')
if args:
lines = [' '.join(args)]
else:
lines = sys.stdin.read().splitlines()
out_lines = [vox_process(line, wrap_vox_tags=not clean) for line in lines if line.strip()]
sys.stdout.write('\n'.join(out_lines))
if out_lines:
sys.stdout.write('\n')
if __name__ == '__main__':
main()