Merge the SAM VOX preset tuning and comparison tool
This commit is contained in:
Fisto
@@ -40,6 +40,15 @@ mobile draft at 390px.
|
||||
- [x] C11 preview caching: Discord-hosted image/GIF previews (WebP 320px, hash-addressed, 5 MiB
|
||||
cap, one attempt) run after every walk in both modes. Live run `...-20260916090722`:
|
||||
`previewed: 2`; thumbs served over `/api/media/<scrape>/<item>/thumb` (200).
|
||||
- [x] C83 Merge the SAM updates. `~/Projects/Sam/MicroslopSam` moved ahead (VOX preset tuned
|
||||
against the game word assets, `tools/vox_compare.py`, docs, wrapper/test sync). The updates
|
||||
matched our tree except the local `microslopsam.env` paths, so the merge is: the retuned VOX
|
||||
profile (pitch 0, rate 15, head 34, lowering 50), the matching test expectations, the README
|
||||
and design-doc updates, and the new comparison tool. **Also regenerated `poc/.env`'s
|
||||
`VESKSLOP_TTS_VOICES`** (gitignored) from `setup/veskslop-tts.sh --voices` so the UI profile
|
||||
defaults match what the wrapper writes — without it the panel would have offered the old snap
|
||||
points. Wrapper tests pass; a live vox preview renders. A stored profile override keeps the old
|
||||
tuning until it is double-clicked back to the profile default.
|
||||
- [x] C82 Double-click resets in the voice settings. Every slider in the TTS panel resets on
|
||||
double-click: the voice-character profile sliders (to the voice's **profile default**),
|
||||
expression (the profile default, and the override is truly cleared so the label reads
|
||||
|
||||
+16
-5
@@ -73,8 +73,8 @@ Options: `-v` voice, `-t` text, `-o` output WAV, `-p` pitch, `-s` speed, `-V` vo
|
||||
export VESKSLOP_TTS_CMD="$PWD/setup/veskslop-tts.sh {out} {text_file}"
|
||||
export VESKSLOP_TTS_VOICES="$(setup/veskslop-tts.sh --voices)"
|
||||
|
||||
The wrapper maps voice IDs (`sam`, `mary`, `mike`, `keith-bell`, `kerry-watson`, plus the
|
||||
MSTTS variants), reads `VESKSLOP_TTS_VOICE`, `VESKSLOP_TTS_SPEED` (multiplier),
|
||||
The wrapper maps voice IDs (`sam`, `mary`, `mike`, `keith-bell`, `kerry-watson`, `vox`,
|
||||
`hev`, plus the MSTTS variants), reads `VESKSLOP_TTS_VOICE`, `VESKSLOP_TTS_SPEED` (multiplier),
|
||||
`VESKSLOP_TTS_PITCH` (semitones) and `VESKSLOP_TTS_EXPRESSION` (0-100, Flextalk voices
|
||||
only), starts Xvfb when `DISPLAY` is unset, and writes a 48 kHz mono s16le WAV trimmed,
|
||||
faded and normalized to about -16 LUFS (-3 dBFS true peak).
|
||||
@@ -93,9 +93,18 @@ A short low-level leading hum (80 ms, 75 Hz, -48 dBFS) primes Discord's audio pi
|
||||
so the first word is not gated; tune with `VESKSLOP_TTS_HUM_MS` (0 = off),
|
||||
`VESKSLOP_TTS_HUM_HZ`, and `VESKSLOP_TTS_HUM_DB`.
|
||||
|
||||
**Half-Life presets** (Keith Bell): `VESKSLOP_TTS_PRESET=vox` applies the approximated
|
||||
Valve VOX profile and enables VOX prosody; `VESKSLOP_TTS_PRESET=hev` is a starting point
|
||||
for the HEV suit voice. Values are in `docs/TTS-DESIGN.md`.
|
||||
**Half-Life presets** (Keith Bell): `VESKSLOP_TTS_PRESET=vox` applies the Valve VOX
|
||||
profile (pitch 0, rate 15, head size 34, lowering 50) and enables VOX prosody;
|
||||
`VESKSLOP_TTS_PRESET=hev` is a starting point for the HEV suit voice. The `vox` and `hev`
|
||||
voice IDs select these directly. The VOX values were
|
||||
tuned against the game's word assets with `tools/vox_compare.py`; details in
|
||||
`docs/TTS-DESIGN.md`.
|
||||
|
||||
`VESKSLOP_TTS_CRUNCH` (`hl90`, `hl90hq`, `telephone`, `adpcm`, `crush`) adds 1990s lossy
|
||||
character to the output; `VESKSLOP_TTS_CRUNCH_BITS`/`_RATE` override its bit depth and
|
||||
sample rate. `VESKSLOP_TTS_STREAM=1` makes word mode write the WAV as it goes (no
|
||||
measurement pass; `VESKSLOP_TTS_STREAM_GAIN_DB` sets the gain). Echo timing and level are
|
||||
`VESKSLOP_TTS_ECHO_DELAY_MS` and `VESKSLOP_TTS_ECHO_LEVEL`.
|
||||
|
||||
## Layout
|
||||
|
||||
@@ -107,7 +116,9 @@ for the HEV suit voice. Values are in `docs/TTS-DESIGN.md`.
|
||||
- `setup/inf2reg.py` — INF AddReg -> `.reg` converter
|
||||
- `setup/run.sh` — run helper (prefix + Xvfb)
|
||||
- `setup/veskslop-tts.sh` — Veskslop command-provider wrapper (post-processing included)
|
||||
- `setup/test-wrapper.sh` — wrapper regression test (stubbed Wine/host, no prefix needed)
|
||||
- `setup/vox_parser.py` — NetTTS-style VOX prosody encoder (Flextalk voices)
|
||||
- `tools/vox_compare.py` — reference-vs-synthesis metrics for preset tuning
|
||||
- `docs/NOTES.md` — registry/CLSID facts, runtime RE notes, gotchas
|
||||
- `docs/TTS-DESIGN.md` — integration contract with Veskslop plus engine clarifications
|
||||
|
||||
|
||||
@@ -127,12 +127,16 @@ without them.
|
||||
the Python port; outputs are identical on the 31-line corpus. Re-check after any
|
||||
parser change while the clone is available.
|
||||
- VOX mode is Flextalk-only. The wrapper enables it with `VESKSLOP_TTS_VOX=1`
|
||||
(`VESKSLOP_TTS_VOX_CLEAN=1` drops the stutter wrapper) and it takes precedence
|
||||
over `VESKSLOP_TTS_WORDS`.
|
||||
- Half-Life presets (`VESKSLOP_TTS_PRESET=vox|hev`) target Keith Bell only. VOX values
|
||||
come from NetTTS `nettts_flextalk_vox_profile.reg`: pitch 3, rate 17, prominences 22,
|
||||
head size 52, volume 70 (factory: 27/33/17/46/100); `vox` also implies VOX prosody.
|
||||
`hev` (pitch 15, rate 24) is an unverified starting point - HL1 credits the suit
|
||||
(`VESKSLOP_TTS_VOX_CLEAN=1` drops the stutter wrapper); word mode
|
||||
(`VESKSLOP_TTS_WORDS=1`) wins and skips VOX prosody.
|
||||
- Half-Life presets (`VESKSLOP_TTS_PRESET=vox|hev`) target Keith Bell only. VOX starts
|
||||
from NetTTS `nettts_flextalk_vox_profile.reg` (pitch 3, rate 17, head 52, lowering 40)
|
||||
and is tuned against the game's word assets from `suchipi/half-life-vox-console`:
|
||||
shipped values are pitch 0, rate 15, prominences 22, head size 34, lowering 50,
|
||||
volume 70. Measured on 38 words vs the game assets: F0 82.3 vs 82.3 Hz, duration
|
||||
22.99 vs 23.36 s, >2 kHz share 0.0172 vs 0.0186 (old approximation: 87.5 Hz /
|
||||
22.28 s / 0.0137). `tools/vox_compare.py` reproduces the comparison. `hev`
|
||||
(pitch 15, rate 24) remains an unverified starting point - HL1 credits the suit
|
||||
sounds to Kelly Bailey with no published settings.
|
||||
- Flextalk rate/pitch are applied as **vendor tags** (`\!R` = duration factor,
|
||||
`\!%` = pitch multiplier), never `-s`/`-p`, because ITTSAttributes override the
|
||||
|
||||
@@ -254,6 +254,9 @@ record what the engine needs to interpret, add, or hide._
|
||||
- Assigned stable IDs: `sam`, `mary`, `mike`, `keith-bell`, `kerry-watson`; variants get
|
||||
`mike-telephone`, `mike-hall`, `mary-stadium`, `mike-space`, and so on. Unknown IDs fall
|
||||
back to `sam`.
|
||||
- `vox` (VOX (Half-Life)) and `hev` (HEV suit) are Keith Bell with a fixed preset profile;
|
||||
selecting one implies `VESKSLOP_TTS_PRESET` (`vox` also enables VOX prosody). Their
|
||||
`defaults=` metadata follows the active preset so the UI snap points match.
|
||||
- `/voices` (HTTP mode) reports `language: "en"`, gender, and the speed/pitch ranges below.
|
||||
|
||||
### Speed and pitch mapping (measured 2026-09-15)
|
||||
@@ -372,15 +375,16 @@ Requirements and scope:
|
||||
The host passes `TTSDATAFLAG_TAGGED`, so the engine honors them. Other voices ignore
|
||||
`VESKSLOP_TTS_VOX`.
|
||||
- Needs `python3` at runtime; the wrapper fails loudly when it is missing.
|
||||
- Mutually exclusive with `VESKSLOP_TTS_WORDS` (VOX wins).
|
||||
- Combines with `VESKSLOP_TTS_WORDS`: word mode wins and VOX prosody is skipped.
|
||||
- Measured on the dev host for "Now approaching Sector C test lab. Please stand clear
|
||||
of the test chamber.": plain 4.70 s with 1 pause, VOX 7.81 s with 8 pauses.
|
||||
|
||||
Other NetTTS features and their relevance:
|
||||
|
||||
- Inline `/rate N`, `/pitch N`, `[[pause N]]`: not ported. Veskslop has speed/pitch
|
||||
controls and the profile window; `[[pause N]]` could be added as a wrapper-level
|
||||
text splitter later (NetTTS supports it only outside VOX mode).
|
||||
- Inline `/rate N`, `/pitch N` are handled by the wrapper at the start of a line
|
||||
(percent, or a multiplier at or below 2.0); they override the env sliders for that
|
||||
line and are not spoken. `[[pause N]]` is not ported; it could be added as a
|
||||
wrapper-level text splitter later.
|
||||
- TCP server with `START`/`STOP` status sockets: not needed; Veskslop owns playback
|
||||
and knows when a line ends.
|
||||
- Winetricks prefix automation (`winxp`, `vcrun6`, `mfc42`, `riched20`): not needed;
|
||||
@@ -398,16 +402,20 @@ Kerry Watson at factory values. The wrapper implements it as a preset:
|
||||
VESKSLOP_TTS_PRESET=vox # implies VOX prosody unless VESKSLOP_TTS_VOX is set
|
||||
VESKSLOP_TTS_PRESET=hev # HEV suit starting point (see note below)
|
||||
|
||||
| Profile value | Factory (Keith) | VOX preset | HEV preset (approx.) |
|
||||
| Profile value | Factory (Keith) | VOX preset (tuned) | HEV preset (approx.) |
|
||||
| --- | --- | --- | --- |
|
||||
| Pitch | 27 | 3 | 15 |
|
||||
| Speaking Rate | 33 | 17 | 24 |
|
||||
| Pitch | 27 | 0 | 15 |
|
||||
| Speaking Rate | 33 | 15 | 24 |
|
||||
| Prominences | 17 | 22 | 22 |
|
||||
| Head Size | 46 | 52 | 52 |
|
||||
| Lowering | 40 | 40 | 40 |
|
||||
| Head Size | 46 | 34 | 52 |
|
||||
| Lowering | 40 | 50 | 40 |
|
||||
| Volume | 100 | 70 | 70 |
|
||||
| Expression | 59 | 59 | 59 |
|
||||
|
||||
The VOX column starts from NetTTS's approximation (pitch 3, rate 17, head 52, lowering
|
||||
40) and is refined against the actual game word assets; see
|
||||
"Tuning against the actual game assets" below.
|
||||
|
||||
- Presets apply only to `keith-bell`; every other voice keeps its defaults.
|
||||
- Precedence: preset < `VESKSLOP_TTS_PROFILE` < `VESKSLOP_TTS_EXPRESSION`.
|
||||
- Equivalent explicit override:
|
||||
@@ -465,16 +473,49 @@ Verified matrix (same line, Keith Bell):
|
||||
| Run | Duration | F0 median |
|
||||
| --- | --- | --- |
|
||||
| factory, no adjustments | 3.35 s | 123 Hz |
|
||||
| VOX preset, no adjustments | 5.83 s | 87 Hz |
|
||||
| VOX preset + `speed=1.0 pitch=0` | 5.83 s, byte-identical | 87 Hz |
|
||||
| VOX preset + `speed=1.3` | 4.02 s | 84 Hz |
|
||||
| VOX preset + `pitch=+3` | 5.83 s | 88 Hz |
|
||||
| HEV preset, no adjustments | 3.79 s | - |
|
||||
| VOX preset, no adjustments | 6.06 s | 82 Hz |
|
||||
| VOX preset + `speed=1.0 pitch=0` | 6.06 s, byte-identical | 82 Hz |
|
||||
| VOX preset + `speed=1.3` | 4.02 s | - |
|
||||
| VOX preset + `pitch=+3` | 6.06 s | 96 Hz |
|
||||
|
||||
Registry inspection after a preset run: 26 values in the Keith speaker key in both
|
||||
hives, `Pitch=0x3`, `Speaking Rate=0x11`, `Gender=0x2`, `Line Mode=0x1`,
|
||||
hives, `Pitch=0x0`, `Speaking Rate=0xf`, `Head Size=0x22`, `Lowering=0x32`,
|
||||
`Gender=0x2`, `Line Mode=0x1`,
|
||||
`Default Speaker={11260610-C84A-11CE-AEC4-0000C0E9CB87}`.
|
||||
|
||||
#### Tuning against the actual game assets (2026-09-22)
|
||||
|
||||
Reference: the word WAVs from `suchipi/half-life-vox-console`
|
||||
(`public/vox/*.wav`, Valve assets - used locally for analysis only, never committed).
|
||||
38 words at 11025 Hz 16-bit, the engine's native rate. `tools/vox_compare.py`
|
||||
reproduces the comparison; it reports the cepstrum F0 median, total audible duration,
|
||||
energy-weighted spectral centroid and the >2 kHz energy share per word set.
|
||||
|
||||
| Metric (38 words) | Game assets | NetTTS approximation | Shipped VOX preset |
|
||||
| --- | --- | --- | --- |
|
||||
| F0 median | 82.3 Hz | 87.5 Hz | 82.3 Hz |
|
||||
| Duration | 23.36 s | 22.28 s | 22.99 s |
|
||||
| Spectral centroid | 515 Hz | 479 Hz | 453 Hz |
|
||||
| >2 kHz energy share | 0.0186 | 0.0137 | 0.0172 |
|
||||
|
||||
Findings:
|
||||
|
||||
- Registry `Pitch 0` is the engine minimum and matches the game's F0; the NetTTS
|
||||
approximation (3) was about 5 Hz high.
|
||||
- `Speaking Rate 15` matches the word durations within about 1.6%; NetTTS's 17 was
|
||||
about 4.6% fast on this corpus.
|
||||
- The game voice is brighter than the approximation. Smaller `Head Size` raises
|
||||
brightness (34 is the engine's effective minimum here; 30 gives identical output) and
|
||||
`Lowering 50` adds a little more, together closing most of the >2 kHz gap
|
||||
(0.0137 -> 0.0172 against 0.0186).
|
||||
- The residual centroid difference is mostly the reference's mid/high-band noise floor
|
||||
(game assets), not voice timbre.
|
||||
- HEV stays a starting point: no published settings and no word-level reference exists.
|
||||
|
||||
Re-run recipe: write the profile to both hives, synthesize the reference word list with
|
||||
`build/sapi4host-winlib.exe -v "Keith Bell" -t <word> -o <dir>/<word>.wav`, then
|
||||
`tools/vox_compare.py <game-assets-dir> <synth-dir>`.
|
||||
|
||||
### Inline rate/pitch controls (engine-side)
|
||||
|
||||
The wrapper consumes NetTTS-style, line-leading controls, so Veskslop and the WebUI do
|
||||
@@ -666,9 +707,15 @@ Inputs: `{out}`, `{text_file}`, optional positional voice/speed/pitch/expression
|
||||
| `VESKSLOP_TTS_PROFILE` | `pitch:27:...`/`key=value,...` overrides; the wrapper writes the full profile (defaults + overrides) to the Wine registry with `wine reg import` before every synthesis. |
|
||||
| `VESKSLOP_TTS_HIGHPASS` | High-pass Hz, 40-400 (default 40). |
|
||||
| `VESKSLOP_TTS_ECHO` | Mono echo 0-100 (`aecho`), default 0. The echo is mixed under the dry voice, so the voice level does not change. |
|
||||
| `VESKSLOP_TTS_ECHO_DELAY_MS` / `VESKSLOP_TTS_ECHO_LEVEL` | Echo delay 10-400 ms (default 60) and repeat level 0-0.95 relative to the dry voice (default 0.85). |
|
||||
| `VESKSLOP_TTS_CRUNCH` | `off` (default) or `hl90`, `hl90hq`, `telephone`, `adpcm`, `crush`: 1990s lossy character, applied last. |
|
||||
| `VESKSLOP_TTS_CRUNCH_BITS` / `VESKSLOP_TTS_CRUNCH_RATE` | Bit depth 1-16 and sample rate 4000-48000 overrides for the crunch stage. |
|
||||
| `VESKSLOP_TTS_STREAM` / `VESKSLOP_TTS_STREAM_GAIN_DB` | `1` = word mode writes the WAV as it goes (no measurement pass); fixed gain in dB for that path. |
|
||||
| `VESKSLOP_TTS_WORDS` / `VESKSLOP_TTS_WORD_GAP_MS` | Word-by-word synthesis with silence gaps (Flextalk only). |
|
||||
|
||||
Placeholders `{profile}`, `{highpass}`, `{echo}`, `{words}`, `{word_gap_ms}` mirror the env vars.
|
||||
Placeholders `{profile}`, `{highpass}`, `{echo}`, `{echo_delay_ms}`, `{echo_level}`,
|
||||
`{words}`, `{word_gap_ms}`, `{crunch}`, `{crunch_bits}`, `{crunch_rate}`, `{stream}`,
|
||||
`{vox}`, `{vox_clean}`, `{preset}` mirror the env vars.
|
||||
`wine regedit` fails headlessly but returns 0: use `wine reg import`. A missing profile value
|
||||
must always write the engine default so no stale registry state survives.
|
||||
|
||||
|
||||
@@ -94,8 +94,8 @@ check "vox preset implies VOX parser" "1" "$(grep -c 'wH1' "$STUB_LOG")"
|
||||
# The preset writes the full audible profile to both hives.
|
||||
VESKSLOP_TTS_VOICE=keith-bell VESKSLOP_TTS_PRESET=vox run_wrapper
|
||||
check "preset writes both hives" "2" "$(grep -c 'Speakers\\{11260610' "$STUB_REG")"
|
||||
check "preset pitch (both hives)" "2" "$(grep -c '"Pitch"=dword:00000003' "$STUB_REG")"
|
||||
check "preset rate (both hives)" "2" "$(grep -c '"Speaking Rate"=dword:00000011' "$STUB_REG")"
|
||||
check "preset pitch (both hives)" "2" "$(grep -c '"Pitch"=dword:00000000' "$STUB_REG")"
|
||||
check "preset rate (both hives)" "2" "$(grep -c '"Speaking Rate"=dword:0000000f' "$STUB_REG")"
|
||||
check "preset gender and line mode" "4" "$(grep -cE '"Gender"=dword:00000002|"Line Mode"=dword:00000001' "$STUB_REG")"
|
||||
check "preset default speaker" "2" "$(grep -c '"Default Speaker"' "$STUB_REG")"
|
||||
|
||||
@@ -111,7 +111,7 @@ check "word mode with preset synthesizes every word" "3" "$(grep -c '^ARGV:' "$S
|
||||
check "word mode skips the VOX parser" "0" "$(grep -c 'wH1' "$STUB_LOG")"
|
||||
check "word mode tags every word" "3" \
|
||||
"$(sed -n 's/.*\[-t\] \[\(.*\)\] \[-o\].*/\1/p' "$STUB_LOG" | grep -c '^\\!R0.80 ')"
|
||||
check "word mode keeps the preset profile" "2" "$(grep -c '"Pitch"=dword:00000003' "$STUB_REG")"
|
||||
check "word mode keeps the preset profile" "2" "$(grep -c '"Pitch"=dword:00000000' "$STUB_REG")"
|
||||
check "word mode emits no attributes" "0" "$(attr_count)"
|
||||
|
||||
# The vox preset implies VOX when the variable is unset, but word mode still wins.
|
||||
@@ -139,7 +139,7 @@ check "stream has a gap per word break" "2" \
|
||||
|
||||
# The Half-Life voices carry their profile preset.
|
||||
VESKSLOP_TTS_VOICE=vox VESKSLOP_TTS_VOX=1 run_wrapper
|
||||
check "vox voice writes the VOX profile (both hives)" "2" "$(grep -c '"Pitch"=dword:00000003' "$STUB_REG")"
|
||||
check "vox voice writes the VOX profile (both hives)" "2" "$(grep -c '"Pitch"=dword:00000000' "$STUB_REG")"
|
||||
check "vox voice runs VOX prosody" "1" "$(grep -c 'wH1' "$STUB_LOG")"
|
||||
VESKSLOP_TTS_VOICE=hev run_wrapper
|
||||
check "hev voice writes the HEV profile (both hives)" "2" "$(grep -c '"Pitch"=dword:0000000f' "$STUB_REG")"
|
||||
|
||||
@@ -46,7 +46,7 @@ ROOT=$MICROSLOP_ROOT
|
||||
|
||||
# The Keith Bell defaults follow the active preset so the UI's "default" snap points
|
||||
# match the values the preset actually writes.
|
||||
VOX_DEFAULTS="pitch:3:rate:17:prominences:22:head_size:52:lowering:40:expression:59"
|
||||
VOX_DEFAULTS="pitch:0:rate:15:prominences:22:head_size:34:lowering:50:expression:59"
|
||||
HEV_DEFAULTS="pitch:15:rate:24:prominences:22:head_size:52:lowering:40:expression:59"
|
||||
case "${VESKSLOP_TTS_PRESET:-}" in
|
||||
vox) KB_DEFAULTS="$VOX_DEFAULTS" ;;
|
||||
@@ -243,15 +243,17 @@ if [ -n "$SPEAKER_GUID" ]; then
|
||||
P_LOWER=40
|
||||
P_EXPR=""
|
||||
|
||||
# Half-Life presets for Keith Bell (the game's announcement voice).
|
||||
# "vox" is the approximated Valve VOX profile published by NetTTS
|
||||
# (scripts/winetricks/nettts_flextalk_vox_profile.reg); "hev" is a starting
|
||||
# point for the HEV suit voice, which has no published profile.
|
||||
# Half-Life presets for Keith Bell (the game's announcement voice). "vox"
|
||||
# starts from the approximated Valve VOX profile published by NetTTS
|
||||
# (scripts/winetricks/nettts_flextalk_vox_profile.reg), then refines pitch,
|
||||
# rate, head size and lowering against the actual game word assets
|
||||
# (see docs/TTS-DESIGN.md). "hev" is a starting point for the HEV suit
|
||||
# voice, which has no published profile.
|
||||
case "$PRESET" in
|
||||
vox)
|
||||
if [ "$ENGVOICE" = "Keith Bell" ]; then
|
||||
P_PITCH=3; P_RATE=17; P_VOLUME=70
|
||||
P_PROM=22; P_HEAD=52; P_LOWER=40
|
||||
P_PITCH=0; P_RATE=15; P_VOLUME=70
|
||||
P_PROM=22; P_HEAD=34; P_LOWER=50
|
||||
fi
|
||||
;;
|
||||
hev)
|
||||
|
||||
Executable
+134
@@ -0,0 +1,134 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Compare synthesized words against reference WAV directories.
|
||||
|
||||
Used to tune the Half-Life VOX preset against the game's word assets
|
||||
(`suchipi/half-life-vox-console`, `public/vox/*.wav` - Valve assets, keep them
|
||||
outside this repository).
|
||||
|
||||
Usage:
|
||||
vox_compare.py REFERENCE_DIR SYNTH_DIR [SYNTH_DIR ...]
|
||||
|
||||
Directories must hold the same word file names (one WAV per word). Prints, per
|
||||
directory: median F0 (cepstrum), total audible duration, energy-weighted
|
||||
spectral centroid and the >2 kHz energy ratio. These are the metrics used to
|
||||
pick the VOX preset values in setup/veskslop-tts.sh.
|
||||
|
||||
Requires numpy.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
import numpy as np
|
||||
import wave
|
||||
|
||||
|
||||
def frames(path):
|
||||
with wave.open(path) as w:
|
||||
data = w.readframes(w.getnframes())
|
||||
rate = w.getframerate()
|
||||
samples = np.frombuffer(data, dtype="<i2").astype(float)
|
||||
loud = np.nonzero(np.abs(samples) > 200)[0]
|
||||
if len(loud) == 0:
|
||||
return None, rate
|
||||
return samples[loud[0]:loud[-1] + 1], rate
|
||||
|
||||
|
||||
def f0_cepstrum(seg, rate):
|
||||
win = min(2048, (len(seg) // 2) * 2)
|
||||
if win < 256:
|
||||
return 0.0
|
||||
values = []
|
||||
for i in range(0, max(1, len(seg) - win), win // 2):
|
||||
fr = seg[i:i + win]
|
||||
if (fr ** 2).sum() < 1e6:
|
||||
continue
|
||||
fr = fr - fr.mean()
|
||||
spectrum = np.abs(np.fft.rfft(fr * np.hanning(win))) + 1e-9
|
||||
cepstrum = np.fft.irfft(np.log(spectrum))
|
||||
lo, hi = int(rate / 400), int(rate / 60)
|
||||
if hi <= lo:
|
||||
continue
|
||||
lag = lo + int(np.argmax(cepstrum[lo:hi]))
|
||||
values.append(rate / lag)
|
||||
values = sorted(v for v in values if 50 < v < 400)
|
||||
return values[len(values) // 2] if values else 0.0
|
||||
|
||||
|
||||
def brightness(seg, rate):
|
||||
win = min(2048, (len(seg) // 2) * 2)
|
||||
cent, high, energy = [], [], []
|
||||
for i in range(0, max(1, len(seg) - win), win // 2):
|
||||
fr = seg[i:i + win]
|
||||
if len(fr) < win:
|
||||
break
|
||||
e = (fr ** 2).sum() / win
|
||||
if e < 1e5:
|
||||
continue
|
||||
fr = fr - fr.mean()
|
||||
power = np.abs(np.fft.rfft(fr * np.hanning(win))) ** 2
|
||||
freq = np.fft.rfftfreq(win, 1 / rate)
|
||||
total = power.sum() + 1e-9
|
||||
cent.append((power * freq).sum() / total)
|
||||
high.append(power[(freq >= 2000) & (freq < 5000)].sum() / total)
|
||||
energy.append(e)
|
||||
if not cent:
|
||||
return 0.0, 0.0
|
||||
cent = np.array(cent)
|
||||
high = np.array(high)
|
||||
energy = np.array(energy)
|
||||
keep = energy >= np.percentile(energy, 70)
|
||||
return float(np.median(cent[keep])), float(np.median(high[keep]))
|
||||
|
||||
|
||||
def summarize(directory, words):
|
||||
f0s, duration, cents, highs = [], 0.0, [], []
|
||||
for word in words:
|
||||
path = os.path.join(directory, word + ".wav")
|
||||
if not os.path.exists(path):
|
||||
continue
|
||||
seg, rate = frames(path)
|
||||
if seg is None or len(seg) == 0:
|
||||
continue
|
||||
duration += len(seg) / rate
|
||||
value = f0_cepstrum(seg, rate)
|
||||
if value > 0:
|
||||
f0s.append(value)
|
||||
c, h = brightness(seg, rate)
|
||||
cents.append(c)
|
||||
highs.append(h)
|
||||
f0s.sort()
|
||||
return {
|
||||
"n": len(f0s),
|
||||
"f0": f0s[len(f0s) // 2] if f0s else 0.0,
|
||||
"dur": duration,
|
||||
"cent": float(np.median(cents)) if cents else 0.0,
|
||||
"high": float(np.median(highs)) if highs else 0.0,
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 3:
|
||||
print(__doc__)
|
||||
return 2
|
||||
reference = sys.argv[1]
|
||||
candidates = sorted(
|
||||
name[:-4] for name in os.listdir(reference)
|
||||
if name.endswith(".wav") and os.path.isfile(os.path.join(reference, name))
|
||||
)
|
||||
# Keep only words present in every directory so the summaries compare.
|
||||
words = [
|
||||
word for word in candidates
|
||||
if all(os.path.exists(os.path.join(d, word + ".wav")) for d in sys.argv[1:])
|
||||
]
|
||||
for directory in sys.argv[1:]:
|
||||
s = summarize(directory, words)
|
||||
print(
|
||||
"%-28s words=%3d f0=%6.1fHz dur=%6.2fs cent=%5.0fHz high2k=%.4f"
|
||||
% (os.path.basename(directory.rstrip("/")), s["n"], s["f0"], s["dur"],
|
||||
s["cent"], s["high"])
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user