VOX preset tuned against game word assets; add vox_compare tool
This commit is contained in:
Fisto
@@ -93,9 +93,11 @@ A short low-level leading hum (80 ms, 75 Hz, -48 dBFS) primes Discord's audio pi
|
||||
so the first word is not gated; tune with `VESKSLOP_TTS_HUM_MS` (0 = off),
|
||||
`VESKSLOP_TTS_HUM_HZ`, and `VESKSLOP_TTS_HUM_DB`.
|
||||
|
||||
**Half-Life presets** (Keith Bell): `VESKSLOP_TTS_PRESET=vox` applies the approximated
|
||||
Valve VOX profile and enables VOX prosody; `VESKSLOP_TTS_PRESET=hev` is a starting point
|
||||
for the HEV suit voice. Values are in `docs/TTS-DESIGN.md`.
|
||||
**Half-Life presets** (Keith Bell): `VESKSLOP_TTS_PRESET=vox` applies the Valve VOX
|
||||
profile (pitch 0, rate 15, head size 34, lowering 50) and enables VOX prosody;
|
||||
`VESKSLOP_TTS_PRESET=hev` is a starting point for the HEV suit voice. The VOX values were
|
||||
tuned against the game's word assets with `tools/vox_compare.py`; details in
|
||||
`docs/TTS-DESIGN.md`.
|
||||
|
||||
## Layout
|
||||
|
||||
@@ -108,6 +110,7 @@ for the HEV suit voice. Values are in `docs/TTS-DESIGN.md`.
|
||||
- `setup/run.sh` — run helper (prefix + Xvfb)
|
||||
- `setup/veskslop-tts.sh` — Veskslop command-provider wrapper (post-processing included)
|
||||
- `setup/vox_parser.py` — NetTTS-style VOX prosody encoder (Flextalk voices)
|
||||
- `tools/vox_compare.py` — reference-vs-synthesis metrics for preset tuning
|
||||
- `docs/NOTES.md` — registry/CLSID facts, runtime RE notes, gotchas
|
||||
- `docs/TTS-DESIGN.md` — integration contract with Veskslop plus engine clarifications
|
||||
|
||||
|
||||
+8
-4
@@ -129,10 +129,14 @@ without them.
|
||||
- VOX mode is Flextalk-only. The wrapper enables it with `VESKSLOP_TTS_VOX=1`
|
||||
(`VESKSLOP_TTS_VOX_CLEAN=1` drops the stutter wrapper) and it takes precedence
|
||||
over `VESKSLOP_TTS_WORDS`.
|
||||
- Half-Life presets (`VESKSLOP_TTS_PRESET=vox|hev`) target Keith Bell only. VOX values
|
||||
come from NetTTS `nettts_flextalk_vox_profile.reg`: pitch 3, rate 17, prominences 22,
|
||||
head size 52, volume 70 (factory: 27/33/17/46/100); `vox` also implies VOX prosody.
|
||||
`hev` (pitch 15, rate 24) is an unverified starting point - HL1 credits the suit
|
||||
- Half-Life presets (`VESKSLOP_TTS_PRESET=vox|hev`) target Keith Bell only. VOX starts
|
||||
from NetTTS `nettts_flextalk_vox_profile.reg` (pitch 3, rate 17, head 52, lowering 40)
|
||||
and is tuned against the game's word assets from `suchipi/half-life-vox-console`:
|
||||
shipped values are pitch 0, rate 15, prominences 22, head size 34, lowering 50,
|
||||
volume 70. Measured on 38 words vs the game assets: F0 82.3 vs 82.3 Hz, duration
|
||||
22.99 vs 23.36 s, >2 kHz share 0.0172 vs 0.0186 (old approximation: 87.5 Hz /
|
||||
22.28 s / 0.0137). `tools/vox_compare.py` reproduces the comparison. `hev`
|
||||
(pitch 15, rate 24) remains an unverified starting point - HL1 credits the suit
|
||||
sounds to Kelly Bailey with no published settings.
|
||||
- Flextalk rate/pitch are applied as **vendor tags** (`\!R` = duration factor,
|
||||
`\!%` = pitch multiplier), never `-s`/`-p`, because ITTSAttributes override the
|
||||
|
||||
+48
-11
@@ -398,16 +398,20 @@ Kerry Watson at factory values. The wrapper implements it as a preset:
|
||||
VESKSLOP_TTS_PRESET=vox # implies VOX prosody unless VESKSLOP_TTS_VOX is set
|
||||
VESKSLOP_TTS_PRESET=hev # HEV suit starting point (see note below)
|
||||
|
||||
| Profile value | Factory (Keith) | VOX preset | HEV preset (approx.) |
|
||||
| Profile value | Factory (Keith) | VOX preset (tuned) | HEV preset (approx.) |
|
||||
| --- | --- | --- | --- |
|
||||
| Pitch | 27 | 3 | 15 |
|
||||
| Speaking Rate | 33 | 17 | 24 |
|
||||
| Pitch | 27 | 0 | 15 |
|
||||
| Speaking Rate | 33 | 15 | 24 |
|
||||
| Prominences | 17 | 22 | 22 |
|
||||
| Head Size | 46 | 52 | 52 |
|
||||
| Lowering | 40 | 40 | 40 |
|
||||
| Head Size | 46 | 34 | 52 |
|
||||
| Lowering | 40 | 50 | 40 |
|
||||
| Volume | 100 | 70 | 70 |
|
||||
| Expression | 59 | 59 | 59 |
|
||||
|
||||
The VOX column starts from NetTTS's approximation (pitch 3, rate 17, head 52, lowering
|
||||
40) and is refined against the actual game word assets; see
|
||||
"Tuning against the actual game assets" below.
|
||||
|
||||
- Presets apply only to `keith-bell`; every other voice keeps its defaults.
|
||||
- Precedence: preset < `VESKSLOP_TTS_PROFILE` < `VESKSLOP_TTS_EXPRESSION`.
|
||||
- Equivalent explicit override:
|
||||
@@ -465,16 +469,49 @@ Verified matrix (same line, Keith Bell):
|
||||
| Run | Duration | F0 median |
|
||||
| --- | --- | --- |
|
||||
| factory, no adjustments | 3.35 s | 123 Hz |
|
||||
| VOX preset, no adjustments | 5.83 s | 87 Hz |
|
||||
| VOX preset + `speed=1.0 pitch=0` | 5.83 s, byte-identical | 87 Hz |
|
||||
| VOX preset + `speed=1.3` | 4.02 s | 84 Hz |
|
||||
| VOX preset + `pitch=+3` | 5.83 s | 88 Hz |
|
||||
| HEV preset, no adjustments | 3.79 s | - |
|
||||
| VOX preset, no adjustments | 6.06 s | 82 Hz |
|
||||
| VOX preset + `speed=1.0 pitch=0` | 6.06 s, byte-identical | 82 Hz |
|
||||
| VOX preset + `speed=1.3` | 4.02 s | - |
|
||||
| VOX preset + `pitch=+3` | 6.06 s | 96 Hz |
|
||||
|
||||
Registry inspection after a preset run: 26 values in the Keith speaker key in both
|
||||
hives, `Pitch=0x3`, `Speaking Rate=0x11`, `Gender=0x2`, `Line Mode=0x1`,
|
||||
hives, `Pitch=0x0`, `Speaking Rate=0xf`, `Head Size=0x22`, `Lowering=0x32`,
|
||||
`Gender=0x2`, `Line Mode=0x1`,
|
||||
`Default Speaker={11260610-C84A-11CE-AEC4-0000C0E9CB87}`.
|
||||
|
||||
#### Tuning against the actual game assets (2026-09-22)
|
||||
|
||||
Reference: the word WAVs from `suchipi/half-life-vox-console`
|
||||
(`public/vox/*.wav`, Valve assets - used locally for analysis only, never committed).
|
||||
38 words at 11025 Hz 16-bit, the engine's native rate. `tools/vox_compare.py`
|
||||
reproduces the comparison; it reports the cepstrum F0 median, total audible duration,
|
||||
energy-weighted spectral centroid and the >2 kHz energy share per word set.
|
||||
|
||||
| Metric (38 words) | Game assets | NetTTS approximation | Shipped VOX preset |
|
||||
| --- | --- | --- | --- |
|
||||
| F0 median | 82.3 Hz | 87.5 Hz | 82.3 Hz |
|
||||
| Duration | 23.36 s | 22.28 s | 22.99 s |
|
||||
| Spectral centroid | 515 Hz | 479 Hz | 453 Hz |
|
||||
| >2 kHz energy share | 0.0186 | 0.0137 | 0.0172 |
|
||||
|
||||
Findings:
|
||||
|
||||
- Registry `Pitch 0` is the engine minimum and matches the game's F0; the NetTTS
|
||||
approximation (3) was about 5 Hz high.
|
||||
- `Speaking Rate 15` matches the word durations within about 1.6%; NetTTS's 17 was
|
||||
about 4.6% fast on this corpus.
|
||||
- The game voice is brighter than the approximation. Smaller `Head Size` raises
|
||||
brightness (34 is the engine's effective minimum here; 30 gives identical output) and
|
||||
`Lowering 50` adds a little more, together closing most of the >2 kHz gap
|
||||
(0.0137 -> 0.0172 against 0.0186).
|
||||
- The residual centroid difference is mostly the reference's mid/high-band noise floor
|
||||
(game assets), not voice timbre.
|
||||
- HEV stays a starting point: no published settings and no word-level reference exists.
|
||||
|
||||
Re-run recipe: write the profile to both hives, synthesize the reference word list with
|
||||
`build/sapi4host-winlib.exe -v "Keith Bell" -t <word> -o <dir>/<word>.wav`, then
|
||||
`tools/vox_compare.py <game-assets-dir> <synth-dir>`.
|
||||
|
||||
### Inline rate/pitch controls (engine-side)
|
||||
|
||||
The wrapper consumes NetTTS-style, line-leading controls, so Veskslop and the WebUI do
|
||||
|
||||
Executable
+152
@@ -0,0 +1,152 @@
|
||||
#!/bin/sh
|
||||
# Regression test for the Veskslop TTS wrapper contract. No Wine needed: the wine
|
||||
# and engine-host binaries are stubbed, so this checks the arguments the wrapper
|
||||
# builds, the registry import it generates, and the voice metadata it publishes.
|
||||
#
|
||||
# The bug this guards against: rate/pitch sent as ITTSAttributes (-s/-p) override
|
||||
# the Flextalk speaker profile, so the VOX/HEV presets sounded too fast and not
|
||||
# deep enough. Flextalk must use the sticky vendor tags (\!R / \!%), and neutral
|
||||
# sliders must emit neither.
|
||||
set -eu
|
||||
|
||||
HERE=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
|
||||
WRAP="$HERE/veskslop-tts.sh"
|
||||
WORK=$(mktemp -d)
|
||||
trap 'rm -rf "$WORK"' EXIT
|
||||
|
||||
mkdir -p "$WORK/build" "$WORK/bin" "$WORK/out"
|
||||
"${MICROSLOP_FFMPEG:-ffmpeg}" -hide_banner -loglevel error -f lavfi \
|
||||
-i "sine=frequency=200:duration=0.3" -ar 11025 -ac 1 -c:a pcm_s16le -y "$WORK/sine.wav"
|
||||
|
||||
cat > "$WORK/build/sapi4host.exe" <<'STUB'
|
||||
#!/bin/sh
|
||||
{ printf 'ARGV:'; for a in "$@"; do printf ' [%s]' "$a"; done; printf '\n'; } >> "$STUB_LOG"
|
||||
OUT=""
|
||||
while [ "$#" -ge 2 ]; do
|
||||
[ "$1" = "-o" ] && OUT=$2
|
||||
shift
|
||||
done
|
||||
cp "$STUB_SINE" "$OUT"
|
||||
STUB
|
||||
|
||||
cat > "$WORK/bin/wine" <<'STUB'
|
||||
#!/bin/sh
|
||||
[ "${1:-}" = "reg" ] && [ "${2:-}" = "import" ] && cp "$3" "$STUB_REG"
|
||||
exit 0
|
||||
STUB
|
||||
chmod +x "$WORK/build/sapi4host.exe" "$WORK/bin/wine"
|
||||
|
||||
printf 'Attention all personnel.\n' > "$WORK/text.txt"
|
||||
|
||||
export MICROSLOP_BUILD="$WORK/build"
|
||||
export MICROSLOP_WINE="$WORK/bin/wine"
|
||||
export MICROSLOP_FFMPEG="${MICROSLOP_FFMPEG:-ffmpeg}"
|
||||
export DISPLAY=:99
|
||||
export WINEPREFIX="$WORK/prefix"
|
||||
export STUB_LOG="$WORK/host.log"
|
||||
export STUB_REG="$WORK/profile.reg"
|
||||
export STUB_SINE="$WORK/sine.wav"
|
||||
|
||||
FAIL=0
|
||||
check() {
|
||||
if [ "$2" = "$3" ]; then
|
||||
echo "ok - $1"
|
||||
else
|
||||
echo "FAIL - $1: expected [$2], got [$3]"
|
||||
FAIL=1
|
||||
fi
|
||||
}
|
||||
|
||||
run_wrapper() {
|
||||
: > "$STUB_LOG"
|
||||
sh "$WRAP" "$WORK/out.wav" "$WORK/text.txt" >/dev/null
|
||||
}
|
||||
|
||||
# The spoken text is the -t argument; the engine host writes to the wrapper's temp file.
|
||||
text_for() {
|
||||
sed -n '1s/.*\[-t\] \[\(.*\)\] \[-o\].*/\1/p' "$STUB_LOG"
|
||||
}
|
||||
|
||||
attr_count() {
|
||||
grep -oE '\[-s\]|\[-p\]' "$STUB_LOG" | wc -l | tr -d ' '
|
||||
}
|
||||
|
||||
# Neutral Flextalk sliders emit neither attributes nor tags.
|
||||
VESKSLOP_TTS_VOICE=keith-bell run_wrapper
|
||||
check "neutral flextalk text" "Attention all personnel." "$(text_for)"
|
||||
check "neutral flextalk has no attributes" "0" "$(attr_count)"
|
||||
|
||||
# Adjustments ride as tags; attributes stay out even at non-neutral values.
|
||||
VESKSLOP_TTS_VOICE=keith-bell VESKSLOP_TTS_VOX=0 \
|
||||
VESKSLOP_TTS_SPEED=1.3 VESKSLOP_TTS_PITCH=3 run_wrapper
|
||||
check "flextalk rate tag" "\!R0.77 \!%1.19 Attention all personnel." "$(text_for)"
|
||||
check "flextalk has no attributes" "0" "$(attr_count)"
|
||||
|
||||
# MSTTS has no vendor tags and keeps the attributes.
|
||||
VESKSLOP_TTS_VOICE=sam VESKSLOP_TTS_SPEED=1.3 VESKSLOP_TTS_PITCH=3 run_wrapper
|
||||
check "mstts text has no tags" "Attention all personnel." "$(text_for)"
|
||||
check "mstts keeps attributes" "2" "$(attr_count)"
|
||||
|
||||
# The vox preset implies VOX prosody when VESKSLOP_TTS_VOX is unset.
|
||||
VESKSLOP_TTS_VOICE=keith-bell VESKSLOP_TTS_PRESET=vox run_wrapper
|
||||
check "vox preset implies VOX parser" "1" "$(grep -c 'wH1' "$STUB_LOG")"
|
||||
|
||||
# The preset writes the full audible profile to both hives.
|
||||
VESKSLOP_TTS_VOICE=keith-bell VESKSLOP_TTS_PRESET=vox run_wrapper
|
||||
check "preset writes both hives" "2" "$(grep -c 'Speakers\\{11260610' "$STUB_REG")"
|
||||
check "preset pitch (both hives)" "2" "$(grep -c '"Pitch"=dword:00000003' "$STUB_REG")"
|
||||
check "preset rate (both hives)" "2" "$(grep -c '"Speaking Rate"=dword:00000011' "$STUB_REG")"
|
||||
check "preset gender and line mode" "4" "$(grep -cE '"Gender"=dword:00000002|"Line Mode"=dword:00000001' "$STUB_REG")"
|
||||
check "preset default speaker" "2" "$(grep -c '"Default Speaker"' "$STUB_REG")"
|
||||
|
||||
# A plain Flextalk run keeps the defaults and writes no preset extras.
|
||||
VESKSLOP_TTS_VOICE=keith-bell run_wrapper
|
||||
check "plain run writes both hives" "2" "$(grep -c 'Speakers\\{11260610' "$STUB_REG")"
|
||||
check "plain run has no preset extras" "0" "$(grep -cE 'Gender|Line Mode|Default Speaker' "$STUB_REG")"
|
||||
|
||||
# Word mode combines with a preset and skips the VOX parser, with tags per word.
|
||||
VESKSLOP_TTS_VOICE=keith-bell VESKSLOP_TTS_PRESET=vox VESKSLOP_TTS_VOX=1 \
|
||||
VESKSLOP_TTS_WORDS=1 VESKSLOP_TTS_WORD_GAP_MS=0 VESKSLOP_TTS_SPEED=1.25 run_wrapper
|
||||
check "word mode with preset synthesizes every word" "3" "$(grep -c '^ARGV:' "$STUB_LOG")"
|
||||
check "word mode skips the VOX parser" "0" "$(grep -c 'wH1' "$STUB_LOG")"
|
||||
check "word mode tags every word" "3" \
|
||||
"$(sed -n 's/.*\[-t\] \[\(.*\)\] \[-o\].*/\1/p' "$STUB_LOG" | grep -c '^\\!R0.80 ')"
|
||||
check "word mode keeps the preset profile" "2" "$(grep -c '"Pitch"=dword:00000003' "$STUB_REG")"
|
||||
check "word mode emits no attributes" "0" "$(attr_count)"
|
||||
|
||||
# The vox preset implies VOX when the variable is unset, but word mode still wins.
|
||||
VESKSLOP_TTS_VOICE=keith-bell VESKSLOP_TTS_PRESET=vox VESKSLOP_TTS_WORDS=1 \
|
||||
VESKSLOP_TTS_WORD_GAP_MS=0 run_wrapper
|
||||
check "implied VOX yields to word mode" "3" "$(grep -c '^ARGV:' "$STUB_LOG")"
|
||||
check "implied VOX skipped for words" "0" "$(grep -c 'wH1' "$STUB_LOG")"
|
||||
|
||||
# Crunch keeps the output contract and changes the sound.
|
||||
VESKSLOP_TTS_VOICE=keith-bell run_wrapper
|
||||
cp "$WORK/out.wav" "$WORK/off.wav"
|
||||
VESKSLOP_TTS_VOICE=keith-bell VESKSLOP_TTS_CRUNCH=telephone run_wrapper
|
||||
check "crunch keeps 48k s16le" "pcm_s16le,48000,1" \
|
||||
"$("${MICROSLOP_FFMPEG:-ffmpeg}" -hide_banner -loglevel error -i "$WORK/out.wav" -f null - 2>&1 >/dev/null; ffprobe -v error -show_entries stream=codec_name,sample_rate,channels -of csv=p=0 "$WORK/out.wav")"
|
||||
check "crunch changes the audio" "1" "$([ "$(md5sum < "$WORK/off.wav")" != "$(md5sum < "$WORK/out.wav")" ] && echo 1 || echo 0)"
|
||||
|
||||
# Streaming word mode writes a live WAV with a gap between each pair of words.
|
||||
VESKSLOP_TTS_VOICE=keith-bell VESKSLOP_TTS_WORDS=1 VESKSLOP_TTS_WORD_GAP_MS=200 \
|
||||
VESKSLOP_TTS_STREAM=1 run_wrapper
|
||||
check "stream header" "RIFF" "$(head -c 4 "$WORK/out.wav")"
|
||||
check "stream is decodable" "0" \
|
||||
"$("${MICROSLOP_FFMPEG:-ffmpeg}" -hide_banner -loglevel error -i "$WORK/out.wav" -f null - >/dev/null 2>&1; echo $?)"
|
||||
check "stream has a gap per word break" "2" \
|
||||
"$("${MICROSLOP_FFMPEG:-ffmpeg}" -hide_banner -i "$WORK/out.wav" -af silencedetect=noise=-45dB:d=0.15 -f null - 2>&1 | grep -c silence_start)"
|
||||
|
||||
# The Half-Life voices carry their profile preset.
|
||||
VESKSLOP_TTS_VOICE=vox VESKSLOP_TTS_VOX=1 run_wrapper
|
||||
check "vox voice writes the VOX profile (both hives)" "2" "$(grep -c '"Pitch"=dword:00000003' "$STUB_REG")"
|
||||
check "vox voice runs VOX prosody" "1" "$(grep -c 'wH1' "$STUB_LOG")"
|
||||
VESKSLOP_TTS_VOICE=hev run_wrapper
|
||||
check "hev voice writes the HEV profile (both hives)" "2" "$(grep -c '"Pitch"=dword:0000000f' "$STUB_REG")"
|
||||
check "hev voice has no prosody" "0" "$(grep -c 'wH1' "$STUB_LOG")"
|
||||
|
||||
# Voice metadata follows the active preset.
|
||||
check "voices reflect hev" "1" \
|
||||
"$(VESKSLOP_TTS_PRESET=hev sh "$WRAP" --voices | tr ',' '\n' | grep -c 'keith-bell|Keith Bell|male|en|expression=59+defaults=pitch:15:rate:24')"
|
||||
|
||||
[ "$FAIL" = "0" ] && echo "wrapper tests passed" || { echo "wrapper tests failed"; exit 1; }
|
||||
@@ -206,15 +206,17 @@ if [ -n "$SPEAKER_GUID" ]; then
|
||||
P_LOWER=40
|
||||
P_EXPR=""
|
||||
|
||||
# Half-Life presets for Keith Bell (the game's announcement voice).
|
||||
# "vox" is the approximated Valve VOX profile published by NetTTS
|
||||
# (scripts/winetricks/nettts_flextalk_vox_profile.reg); "hev" is a starting
|
||||
# point for the HEV suit voice, which has no published profile.
|
||||
# Half-Life presets for Keith Bell (the game's announcement voice). "vox"
|
||||
# starts from the approximated Valve VOX profile published by NetTTS
|
||||
# (scripts/winetricks/nettts_flextalk_vox_profile.reg), then refines pitch,
|
||||
# rate, head size and lowering against the actual game word assets
|
||||
# (see docs/TTS-DESIGN.md). "hev" is a starting point for the HEV suit
|
||||
# voice, which has no published profile.
|
||||
case "$PRESET" in
|
||||
vox)
|
||||
if [ "$ENGVOICE" = "Keith Bell" ]; then
|
||||
P_PITCH=3; P_RATE=17; P_VOLUME=70
|
||||
P_PROM=22; P_HEAD=52; P_LOWER=40
|
||||
P_PITCH=0; P_RATE=15; P_VOLUME=70
|
||||
P_PROM=22; P_HEAD=34; P_LOWER=50
|
||||
fi
|
||||
;;
|
||||
hev)
|
||||
|
||||
Executable
+134
@@ -0,0 +1,134 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Compare synthesized words against reference WAV directories.
|
||||
|
||||
Used to tune the Half-Life VOX preset against the game's word assets
|
||||
(`suchipi/half-life-vox-console`, `public/vox/*.wav` - Valve assets, keep them
|
||||
outside this repository).
|
||||
|
||||
Usage:
|
||||
vox_compare.py REFERENCE_DIR SYNTH_DIR [SYNTH_DIR ...]
|
||||
|
||||
Directories must hold the same word file names (one WAV per word). Prints, per
|
||||
directory: median F0 (cepstrum), total audible duration, energy-weighted
|
||||
spectral centroid and the >2 kHz energy ratio. These are the metrics used to
|
||||
pick the VOX preset values in setup/veskslop-tts.sh.
|
||||
|
||||
Requires numpy.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
import numpy as np
|
||||
import wave
|
||||
|
||||
|
||||
def frames(path):
|
||||
with wave.open(path) as w:
|
||||
data = w.readframes(w.getnframes())
|
||||
rate = w.getframerate()
|
||||
samples = np.frombuffer(data, dtype="<i2").astype(float)
|
||||
loud = np.nonzero(np.abs(samples) > 200)[0]
|
||||
if len(loud) == 0:
|
||||
return None, rate
|
||||
return samples[loud[0]:loud[-1] + 1], rate
|
||||
|
||||
|
||||
def f0_cepstrum(seg, rate):
|
||||
win = min(2048, (len(seg) // 2) * 2)
|
||||
if win < 256:
|
||||
return 0.0
|
||||
values = []
|
||||
for i in range(0, max(1, len(seg) - win), win // 2):
|
||||
fr = seg[i:i + win]
|
||||
if (fr ** 2).sum() < 1e6:
|
||||
continue
|
||||
fr = fr - fr.mean()
|
||||
spectrum = np.abs(np.fft.rfft(fr * np.hanning(win))) + 1e-9
|
||||
cepstrum = np.fft.irfft(np.log(spectrum))
|
||||
lo, hi = int(rate / 400), int(rate / 60)
|
||||
if hi <= lo:
|
||||
continue
|
||||
lag = lo + int(np.argmax(cepstrum[lo:hi]))
|
||||
values.append(rate / lag)
|
||||
values = sorted(v for v in values if 50 < v < 400)
|
||||
return values[len(values) // 2] if values else 0.0
|
||||
|
||||
|
||||
def brightness(seg, rate):
|
||||
win = min(2048, (len(seg) // 2) * 2)
|
||||
cent, high, energy = [], [], []
|
||||
for i in range(0, max(1, len(seg) - win), win // 2):
|
||||
fr = seg[i:i + win]
|
||||
if len(fr) < win:
|
||||
break
|
||||
e = (fr ** 2).sum() / win
|
||||
if e < 1e5:
|
||||
continue
|
||||
fr = fr - fr.mean()
|
||||
power = np.abs(np.fft.rfft(fr * np.hanning(win))) ** 2
|
||||
freq = np.fft.rfftfreq(win, 1 / rate)
|
||||
total = power.sum() + 1e-9
|
||||
cent.append((power * freq).sum() / total)
|
||||
high.append(power[(freq >= 2000) & (freq < 5000)].sum() / total)
|
||||
energy.append(e)
|
||||
if not cent:
|
||||
return 0.0, 0.0
|
||||
cent = np.array(cent)
|
||||
high = np.array(high)
|
||||
energy = np.array(energy)
|
||||
keep = energy >= np.percentile(energy, 70)
|
||||
return float(np.median(cent[keep])), float(np.median(high[keep]))
|
||||
|
||||
|
||||
def summarize(directory, words):
|
||||
f0s, duration, cents, highs = [], 0.0, [], []
|
||||
for word in words:
|
||||
path = os.path.join(directory, word + ".wav")
|
||||
if not os.path.exists(path):
|
||||
continue
|
||||
seg, rate = frames(path)
|
||||
if seg is None or len(seg) == 0:
|
||||
continue
|
||||
duration += len(seg) / rate
|
||||
value = f0_cepstrum(seg, rate)
|
||||
if value > 0:
|
||||
f0s.append(value)
|
||||
c, h = brightness(seg, rate)
|
||||
cents.append(c)
|
||||
highs.append(h)
|
||||
f0s.sort()
|
||||
return {
|
||||
"n": len(f0s),
|
||||
"f0": f0s[len(f0s) // 2] if f0s else 0.0,
|
||||
"dur": duration,
|
||||
"cent": float(np.median(cents)) if cents else 0.0,
|
||||
"high": float(np.median(highs)) if highs else 0.0,
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 3:
|
||||
print(__doc__)
|
||||
return 2
|
||||
reference = sys.argv[1]
|
||||
candidates = sorted(
|
||||
name[:-4] for name in os.listdir(reference)
|
||||
if name.endswith(".wav") and os.path.isfile(os.path.join(reference, name))
|
||||
)
|
||||
# Keep only words present in every directory so the summaries compare.
|
||||
words = [
|
||||
word for word in candidates
|
||||
if all(os.path.exists(os.path.join(d, word + ".wav")) for d in sys.argv[1:])
|
||||
]
|
||||
for directory in sys.argv[1:]:
|
||||
s = summarize(directory, words)
|
||||
print(
|
||||
"%-28s words=%3d f0=%6.1fHz dur=%6.2fs cent=%5.0fHz high2k=%.4f"
|
||||
% (os.path.basename(directory.rstrip("/")), s["n"], s["f0"], s["dur"],
|
||||
s["cent"], s["high"])
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user