MicroslopSam/docs/TTS-DESIGN.md

# TTS-DESIGN.md — building a TTS engine for Veskslop

This document is the integration contract for a text-to-speech engine that Veskslop will use. It is
written to be self-contained: an engine developer can work from this file alone.

## How Veskslop uses TTS

1. A line of text is queued (`POST /api/audio/speak` or the MCP `speak` tool).
2. The daemon asks the engine for an audio file (command mode) or audio bytes (HTTP mode).
3. `ffplay` decodes the file and plays it into the PulseAudio null sink `veskslop_tts`.
4. `veskslop_tts.monitor` is exposed as the source `veskslop_mic`.
5. The Vesktop client uses `veskslop_mic` as its microphone, so the call hears the speech.
6. While TTS plays and another person speaks, the daemon lowers the `veskslop_tts` sink volume to
   30% and restores it after. This is "ducking". The engine does not implement ducking.

Lines are serialized: one synthesis, then playback, then the next line. There is no streaming
playback path and no partial-audio protocol. The engine returns a complete audio object.

## Integration modes

### HTTP provider (recommended)

Set `VESKSLOP_TTS_API_URL`. Select the API provider with `VESKSLOP_TTS_PROVIDER=api` or the
**API TTS** choice in the TTS panel. The daemon sends:

```
POST <VESKSLOP_TTS_API_URL>
Content-Type: application/json
Authorization: Bearer <VESKSLOP_TTS_API_KEY>   (only when the key is set)

{ "text": "Hello there.", "voice": "anna" }
```

The response body must be the audio bytes. `Content-Type` selects the file extension: `audio/mpeg`
or `audio/mp3` → `.mp3`, `audio/ogg` → `.ogg`, `audio/wav` or `audio/x-wav` → `.wav`, anything else
is treated as `.mp3`. Prefer `audio/wav`.

A non-2xx response is an error. Return a short JSON or text body with the reason; the daemon logs
it and drops the line.

Use the HTTP mode for any engine with a model load time. It keeps the model warm and avoids the
per-line process start cost.

### Local command provider

Set `VESKSLOP_TTS_CMD`. It takes precedence over the built-in espeak-ng engine. Veskslop replaces
these placeholders, splits the string on whitespace, and runs the result directly (no shell):

| Placeholder | Replaced with |
| --- | --- |
| `{out}` | Output file path, extension `.wav`. Required. |
| `{text_file}` | A UTF-8 text file holding the line. |
| `{voice}` | The selected voice ID, or an empty string. |
| `{speed}` | The rate multiplier, or an empty string when unset. |
| `{pitch}` | The pitch offset in semitones, or an empty string when unset. |
| `{expression}` | The Flextalk Expression profile value, or an empty string when unset. |

The process also receives `VESKSLOP_TTS_VOICE`, `VESKSLOP_TTS_SPEED`,
`VESKSLOP_TTS_PITCH`, and `VESKSLOP_TTS_EXPRESSION` when those are set. Write the audio to
`{out}` and exit 0. Non-zero exits are errors; stderr is captured into the log message, so keep it
short.

Command mode is best for fast, stateless engines. It has no shell quoting: paths with spaces and
shell tricks cannot work. Use a small wrapper script if you need options.

## Audio output requirements

| Property | Recommendation | Notes |
| --- | --- | --- |
| Container | WAV (RIFF) | Any format ffmpeg decodes works, but WAV avoids decoder startup cost. |
| Codec | PCM signed 16-bit little-endian | `pcm_s16le`. |
| Sample rate | 48000 Hz | 44100 Hz is fine. Discord re-encodes to 48 kHz Opus; a 48 kHz source avoids one resample. |
| Channels | Mono | Discord voice is effectively mono. Stereo is accepted but adds no value. |
| Duration | One line of speech | No intro/outro music, no chimes. |
| Peak level | About -3 dBFS | Leave headroom; ducking and Discord gain must not clip. |
| Loudness | Consistent across lines | Aim for a stable integrated loudness (roughly -16 to -20 LUFS) so lines do not jump in volume. |
| Silence | Trim start to under 100 ms, keep roughly 20-50 ms of padding | Long leading silence delays the first audible word. |
| Edges | 5-10 ms fade in and out | Prevents clicks. |
| DC offset | None | Center the waveform. |

Veskslop does not normalize, limit, or re-encode the audio. Playback is `ffplay` at 100% sink
volume, so the engine's levels are what Discord hears.

## Voices

A voice ID is an opaque string to Veskslop. Prefer short, stable, lowercase ASCII identifiers such
as `anna`, `sv-anna`, or `en-male-2`. The ID can contain letters, digits, `-`, `_`, and `+`.

If the engine has multiple voices, expose them so a UI can list them:

```
GET /voices
[
  {
    "id": "anna",
    "name": "Anna",
    "language": "sv",
    "gender": "female",
    "speed":  { "min": 0.5, "max": 2.0, "default": 1.0 },
    "pitch":  { "min": -12,  "max": 12,  "default": 0, "unit": "semitones" }
  }
]
```

Note: Veskslop currently lists voices from espeak-ng only. A custom engine's voices appear after
Veskslop adds support for reading this endpoint (see "Veskslop changes needed"). Until then, set
the voice through the engine's own configuration and leave the Veskslop voice setting empty.

## Speed and pitch

Speed and pitch are engine settings, not Veskslop concepts today. Veskslop passes only `text`,
`voice`, and `provider`. No UI or preference exists for speed or pitch.

Recommended semantics for the engine:

- **speed**: a multiplier, `1.0` is normal, useful range `0.5` to `2.0`. Do not change pitch when
  changing speed unless the user asks for it.
- **pitch**: semitones, `0` is the voice's natural pitch, useful range `-12` to `+12`. State the
  unit in the `/voices` response so clients do not guess.

Recommended HTTP shape (extras are optional; an engine may ignore them):

```
{ "text": "...", "voice": "anna", "speed": 1.15, "pitch": -2 }
```

Recommended command shape: `{speed}` and `{pitch}` are substituted when set, and the same values
are exported as `VESKSLOP_TTS_SPEED` and `VESKSLOP_TTS_PITCH`. Read either; the environment form
survives placeholder-free commands.

### How Veskslop should add first-class settings

Implemented on the Veskslop side (2026-09-15):

- `tts` preferences carry `speed` (0.5–2.0) and `pitch` (-12–+12, semitones), both nullable.
- `/api/audio/speak` and the MCP `speak` tool accept optional `speed` and `pitch` that override
  the preferences for that line.
- The command provider substitutes `{speed}`/`{pitch}`/`{expression}` and exports
  `VESKSLOP_TTS_SPEED` / `VESKSLOP_TTS_PITCH` / `VESKSLOP_TTS_EXPRESSION` only when the value is set.
- The HTTP provider adds `speed`, `pitch`, and `expression` to the request body only when set.
- The TTS panel has speed presets, speed/pitch sliders, and an Expression slider that appears only
  for voices that declare it (`expression` / `expression=NN` flags). Changes save to preferences.
- Command mode advertises voices through `VESKSLOP_TTS_VOICES`. Entries are separated by commas or
  newlines and use `id`, `id|Name`, `id|Name|gender`, `id|Name|gender|language`, or
  `id|Name|gender|language|flags`, where flags are `+`-separated. Flags: `no-pitch` (engine ignores
  pitch) and `expression` / `expression=NN` (Flextalk profile parameter with its registry default).
  `/api/audio/tts/voices` reports `supportsPitch` and `supportsExpression` /
  `expressionDefault`; the TTS panel hides controls the voice does not support. While a custom
  command is set, the engine's declared voices replace the espeak list. Reading `/voices` for the
  HTTP provider is still future work.

Do not encode settings into the voice string (for example `anna+speed1.2`). It breaks voice lists
and is hard to validate.

## Text handling

- UTF-8 input. Lines arrive trimmed and are at most 1000 characters.
- Do not parse or require SSML. Plain text is the contract. SSML support can be additive.
- Emoji, URLs, markdown, and code blocks can appear. Veskslop does not sanitize them. The engine
  should not fail on an unknown character; skip or verbalize it as it prefers.
- Keep punctuation-based prosody. A period, comma, question mark, and newline are the only hints
  for pacing.

## Errors, limits, and behavior

- One request = one complete audio object. No chunked playback, no callbacks.
- Synthesis blocks the queue. Target under about 500 ms for a short line; state a realistic budget
  if the engine is slower. A warm HTTP server is the way to meet this.
- Requests are serialized by Veskslop, so the engine does not need a request queue, but it should
  tolerate repeated back-to-back requests.
- The daemon never retries a failed line. Make failures visible in the response and stderr.
- Do not write to the `veskslop_tts` sink or the `veskslop_mic` source directly. The daemon owns
  playback routing.

## Optional: caching

Veskslop does not cache audio. The engine may cache by `hash(text + voice + speed + pitch)` and
return the cached file or bytes. This is useful for repeated lines and for expensive models.

## Veskslop changes needed

Status as of 2026-09-15:

1. `speed` and `pitch` are first-class: preferences, per-request overrides on `/api/audio/speak`
   and MCP `speak`, env exports and placeholders for the command provider, request fields for the
   HTTP provider, and two sliders in the TTS panel.
2. Command-mode voices come from `VESKSLOP_TTS_VOICES`. Reading the engine's `GET /voices` for the
   HTTP provider is not implemented yet. Until then, declare voices with the env var or configure
   the voice inside the wrapper.
3. No changes are needed for the audio format; the engine post-processes to the required output.

Until `/voices` is wired, use defaults configured in the engine.

## Acceptance checklist

- [ ] Engine starts and answers on a stable URL or command.
- [ ] Output is mono PCM WAV at 48 kHz (or 44.1 kHz), peak near -3 dBFS.
- [ ] First audible word starts within about 500 ms of the request.
- [ ] Speech is clean: no clicks, no clipping, no long leading silence.
- [ ] Lines are consistent in loudness.
- [ ] `voice` accepts stable ASCII IDs; unknown IDs fall back to a default instead of failing.
- [ ] `speed` and `pitch` are optional and documented in `/voices`.
- [ ] Errors return a short reason and do not leave the engine wedged.
- [ ] No files are written outside the engine's own cache or the `{out}` path.

---

## Clarifications from the engine side (MicroslopSam)

_Added 2026-09-15 by the engine implementer. The contract above stands; these points
record what the engine needs to interpret, add, or hide._

### Output pipeline (WAV requirements)

- Raw SAPI4 output is mono PCM s16le WAV but not at the target rate: the MSTTS voices
  (Sam/Mary/Mike) render at 22050 Hz, the Flextalk voices at 11025 Hz.
- Raw Flextalk output **clips** (peak 32768, RMS about 22000) and MSTTS is much quieter,
  so one shared level target is required. The engine post-processes every line:
  resample to 48000 Hz, trim leading silence (<100 ms), 5-10 ms fades, DC removal, and
  level normalization (peak about -3 dBFS, integrated loudness -16 to -20 LUFS).
- Implemented in `setup/veskslop-tts.sh` with ffmpeg: high-pass at 40 Hz, silence trim at
  both ends (trim-start, reverse, trim-start, reverse), 8 ms fades, then a **fixed gain**
  computed from an EBU R128 measurement (`ebur128=peak=true`):
  `gain = min(-16 LUFS target, -3 dBFS true-peak clamp)`, then 48 kHz mono s16le.
- `loudnorm` was rejected: on short lines (under about 3 s) the loudness range is 0 and
  dynamic mode attenuated a 2.8 s line to -37.9 LUFS instead of about -18 LUFS; two-pass
  linear mode showed the same result. The fixed gain is steady for one-line utterances,
  as Veskslop suggested.
- Verified: Keith Bell -18.4 LUFS, Kerry Watson -17.9 LUFS, Sam -20.6 LUFS; every file is
  48 kHz mono s16le at -3.0 dBFS true peak. The WAV handed to Veskslop is compliant; the
  "Veskslop does not normalize" rule holds.
- Fades: the 8 ms fade-in/out is applied to the trimmed file, and the retained 25-30 ms
  padding silence sits at the edges, so the fade shapes silence, not the first phoneme.
  Measured on a Keith Bell line: first sample >30 at 22.6 ms, speech (>500) at 29.6 ms,
  attack rising naturally over about 15 ms, tail decaying over about 18 ms. No click and
  no attenuation of the first word; Veskslop's 150 ms playback lead-in sits on top.
- Leading hum (on by default): 80 ms of a 75 Hz sine at -48 dBFS peak is prepended
  before the engine padding, to keep Discord's audio pipeline open so the first word is
  not gated. Configurable: `VESKSLOP_TTS_HUM_MS` (0 disables), `_HUM_HZ` (30-200),
  `_HUM_DB` (-80..-6). It sits about 45 dB under the speech peak; raise `_HUM_DB` toward
  -36 if a client still gates the start, lower it for stealth. Generated in the same
  ffmpeg pass (no extra process) and not part of the EBU measurement, so the documented
  loudness/peak targets still hold. The hum is mixed on its own chain and bypasses the
  high-pass/trim chain, so raising `VESKSLOP_TTS_HIGHPASS` (40-400 Hz) never removes it
  (verified at 200 and 400 Hz: still 75.0 Hz / -48.0 dBFS). Measured: hum 75.0 Hz /
  -48.0 dBFS, speech onset at about 110 ms, final -18.6 LUFS / -3.0 dBFS; `HUM_MS=0`
  reproduces the previous output byte-for-byte.

### Voice IDs

- Engine voice names are "Sam", "Mary", "Mike", "Keith Bell", "Kerry Watson", plus MSTTS
  variants (telephone, hall, stadium, space) and TruVoice names that the engine lists but
  has no installed data for.
- Assigned stable IDs: `sam`, `mary`, `mike`, `keith-bell`, `kerry-watson`; variants get
  `mike-telephone`, `mike-hall`, `mary-stadium`, `mike-space`, and so on. Unknown IDs fall
  back to `sam`.
- `vox` (VOX (Half-Life)) and `hev` (HEV suit) are Keith Bell with a fixed preset profile;
  selecting one implies `VESKSLOP_TTS_PRESET` (`vox` also enables VOX prosody). Their
  `defaults=` metadata follows the active preset so the UI snap points match.
- `/voices` (HTTP mode) reports `language: "en"`, gender, and the speed/pitch ranges below.

### Speed and pitch mapping (measured 2026-09-15)

- SAPI4 uses engine units, not multipliers or semitones. Measured on the dev host with a
  correctly typed Flextalk install:
  - MSTTS Sam: speed 30-450, natural about 150; pitch 50-200 with a real audible effect.
  - Flextalk: speed 50-390, natural about 150 (durations at 100/200/300 fit a natural rate
    near 140-150); pitch 50-250 works. Measured F0: Kerry 0 st = 209 Hz, -6 st = 148 Hz,
    +6 st = 250 Hz (clamped); Keith 0 st = 119 Hz, +6 st = 170 Hz.
- **Correction to the earlier note**: the previous "Flextalk ignores pitch / natural 300"
  finding was an artifact of the speaker registry values being stored as `REG_SZ`. The
  engine requires `REG_DWORD` for numeric speaker values; with strings it synthesizes a
  monotone buzz at maximum pitch. Fixed in `setup/setup-flextalk.sh`; no `no-pitch` flag
  is needed.
- Mapping implemented in `setup/veskslop-tts.sh`: `speed 1.0` = natural rate 150 for every
  voice, scaled by the multiplier and clamped to the engine range. `pitch` semitones scale
  the voice's natural pitch unit by `2^(st/12)` and clamp to the range. Natural units are
  about 100 (MSTTS), 106 (Keith Bell) and 186 (Kerry Watson). The engines' F0 response is
  approximate, not semitone-calibrated.
- `{setting}` is unused; the wrapper reads `VESKSLOP_TTS_SPEED` and
  `VESKSLOP_TTS_PITCH` from the environment when Veskslop later exports them.

### Text handling

- SAPI4 is ANSI/CP1252. The wrapper converts UTF-8 to CP1252 and maps emoji and
  unmappable characters to spaces. The host passes `TTSDATAFLAG_TAGGED`, so SAPI4 control
  tags in the text are enabled at the engine level; tagged input is not part of this
  contract and is untested. Plain text (no SSML) is the contract.
- Lines up to 1000 characters are well within engine limits.

### Other voice adjustments (beyond speed)

All voices implement SAPI4 `ITTSAttributes`, so **pitch, speed and volume** exist at the
API level. The Veskslop contract exposes only speed and pitch; volume is available in the
host as `sapi4host -V <value>` and is not sent by Veskslop.

- **Volume** (measured 2026-09-15):
  - Flextalk: linear, range 0-65535, default 65535; half (32767) is about -6 dB, 0 is
    silence.
  - MSTTS: default is 0xFFFFFFFF; 0 is silence, but the response is not linear (for
    example 4096 gives about -24 dB while 65535 and larger are full scale). Treat volume
    as effectively on/off for MSTTS.
- **Flextalk per-speaker profile** (applied at voice load, not runtime-adjustable):
  values live in `HKLM\Software\AT&T\Watson\2.0\FlexTalk\2.0\Speakers\{GUID}` (see
  `docs/NOTES.md` for the full list and defaults). `Pitch`/`Speaking Rate` set the natural
  voice; **Expression** was verified to change the synthesis audibly (Keith Bell at
  Expression 0 / 59 / 100 yields different waveforms with the same duration). Changing a
  profile requires a registry edit and re-selecting the voice.
- **MSTTS voice character** is fixed by the `.vce`/`.cfg` data; the engine exposes no
  per-voice profile in the registry. Its effect variants (telephone, in Hall, in Stadium,
  in Space) are separate voice modes with separate data, exposed as distinct voice IDs
  (`mike-telephone`, `mary-hall`, ...), not runtime effects.
- Not exposed by this integration: SSML, prosody/style tags, or SAPI4 attribute
  extensions beyond pitch/speed/volume.

### Flextalk profile parameters (how to implement)

Each Flextalk speaker has a profile the engine reads when the voice is loaded. It lives in
the registry; **every numeric value must be `REG_DWORD`**:

    HKLM\Software\AT&T\Watson\2.0\FlexTalk\2.0\Speakers\{11260610-C84A-11ce-AEC4-0000C0E9CB87}   # Keith Bell
    HKLM\Software\AT&T\Watson\2.0\FlexTalk\2.0\Speakers\{000031CB-3699-551F-AED0-6B4893382D57}   # Kerry Watson

| Value (REG_DWORD unless noted) | Keith Bell | Kerry Watson | Measured effect |
| --- | --- | --- | --- |
| Pitch | 27 | 65 | Natural pitch (F0); the same scale `ITTSAttributes` shifts at runtime |
| Speaking Rate | 33 | 32 | Natural speech rate (about 150 wpm equivalent) |
| Volume | 100 | 100 | Output gain; 0 = silence |
| Expression | 59 | 100 | Audible; changes the waveform at equal duration |
| Prominences | 17 | 17 | Audible; strong prosody/emphasis effect (F0 median 122 -> 197 Hz at 100) |
| Head Size | 46 | 45 | Audible; brightness/level (peak 20001 -> 32205 at 100) |
| Lowering | 40 | 40 | Audible but subtle |
| Age | 30 | 30 | No measured effect on synthesis (informational) |
| Gender (REG_SZ) | "" | "" | No measured effect on synthesis (informational) |
| Mode Name / Speaker Name (REG_SZ) | "Keith Bell" | "Kerry Watson" | Mode name SAPI4 shows for the voice |

How to apply a change:

1. `setup/setup-flextalk.sh` writes these defaults, so re-running it restores them.
2. Profiles are read at voice load. The host loads the engine per request, so the next
   synthesis after a registry change uses the new profile; no engine restart is needed.
3. Example (lower and slow Keith Bell), with `WINEPREFIX` pointing at the project prefix:

       wine reg add 'HKLM\Software\AT&T\Watson\2.0\FlexTalk\2.0\Speakers\{11260610-C84A-11ce-AEC4-0000C0E9CB87}' \
           /v Pitch /t REG_DWORD /d 20 /f
       wine reg add '...' /v "Speaking Rate" /t REG_DWORD /d 28 /f

   Never use `REG_SZ` for numeric values: the engine reads garbage and synthesizes a
   monotone buzz at maximum pitch.
4. Not exposed by the Veskslop contract (which sends only text/voice/speed/pitch). If
   profile control is wanted later, the clean place is the wrapper: read an env var such
   as `VESKSLOP_TTS_PROFILE` (`key=value` list or JSON), write the registry values before
   synthesis, then restore. That keeps the Veskslop interface unchanged.
5. Runtime `ITTSAttributes` pitch/speed/volume layer on top of the profile: the profile
   defines the natural values, the attributes shift them per request.

### VOX mode (NetTTS-inspired, Flextalk only)

`VESKSLOP_TTS_VOX=1` routes the text through a rule-based prosody encoder before
synthesis. `setup/vox_parser.py` is an independent reimplementation of the VOX parser
from [h4rm0n1c/NetTTS](https://github.com/h4rm0n1c/NetTTS), which models Half-Life's
Black Mesa announcement delivery. It was verified byte-identical to the NetTTS parser
on a 31-line corpus (the reference was built locally with g++ for the comparison).

What it does: sentence splitting; `the` -> `thee` before prepositions, proper nouns,
single letters or numbers; lead-in breaks ("Now...", "Please...", "A reminder...");
time/number/degree normalization; token weighting (light/medium/heavy) with break
insertion at linkers, title-case runs, heavy gates and a 3-beat limiter; letter
tokens (`A` -> `Ay:`, `C` -> `C:`); `\!sf500 \!br` at sentence ends; and the
`\!wH1 ... \!wH0` staccato wrapper (disable with `VESKSLOP_TTS_VOX_CLEAN=1`).

Requirements and scope:

- Flextalk voices only: `\!br`, `\!sfNN` and `\!wH1/\!wH0` are FlexTalk vendor tags.
  The host passes `TTSDATAFLAG_TAGGED`, so the engine honors them. Other voices ignore
  `VESKSLOP_TTS_VOX`.
- Needs `python3` at runtime; the wrapper fails loudly when it is missing.
- Combines with `VESKSLOP_TTS_WORDS`: word mode wins and VOX prosody is skipped.
- Measured on the dev host for "Now approaching Sector C test lab. Please stand clear
  of the test chamber.": plain 4.70 s with 1 pause, VOX 7.81 s with 8 pauses.

Other NetTTS features and their relevance:

- Inline `/rate N`, `/pitch N` are handled by the wrapper at the start of a line
  (percent, or a multiplier at or below 2.0); they override the env sliders for that
  line and are not spoken. `[[pause N]]` is not ported; it could be added as a
  wrapper-level text splitter later.
- TCP server with `START`/`STOP` status sockets: not needed; Veskslop owns playback
  and knows when a line ends.
- Winetricks prefix automation (`winxp`, `vcrun6`, `mfc42`, `riched20`): not needed;
  our prefix runs the engines with builtin Wine libraries.
- Win32 GUI and preset sliders: Veskslop provides the UI.

### Half-Life VOX / HEV presets

The Half-Life announcement voice (VOX / Black Mesa Announcement System) is the Flextalk
engine's **Keith Bell** voice with an adjusted speaker profile. NetTTS publishes an
approximation of Valve's settings (`scripts/winetricks/nettts_flextalk_vox_profile.reg`,
see its issues #53/#54/#67): the profile targets Keith Bell's speaker key and leaves
Kerry Watson at factory values. The wrapper implements it as a preset:

    VESKSLOP_TTS_PRESET=vox   # implies VOX prosody unless VESKSLOP_TTS_VOX is set
    VESKSLOP_TTS_PRESET=hev   # HEV suit starting point (see note below)

| Profile value | Factory (Keith) | VOX preset (tuned) | HEV preset (approx.) |
| --- | --- | --- | --- |
| Pitch | 27 | 0 | 15 |
| Speaking Rate | 33 | 15 | 24 |
| Prominences | 17 | 22 | 22 |
| Head Size | 46 | 34 | 52 |
| Lowering | 40 | 50 | 40 |
| Volume | 100 | 70 | 70 |
| Expression | 59 | 59 | 59 |

The VOX column starts from NetTTS's approximation (pitch 3, rate 17, head 52, lowering
40) and is refined against the actual game word assets; see
"Tuning against the actual game assets" below.

- Presets apply only to `keith-bell`; every other voice keeps its defaults.
- Precedence: preset < `VESKSLOP_TTS_PROFILE` < `VESKSLOP_TTS_EXPRESSION`.
- Equivalent explicit override:
  `VESKSLOP_TTS_PROFILE='pitch:3,rate:17,volume:70,prominences:22,head_size:52,lowering:40'`.
- HEV suit: Half-Life 1 credits the suit sounds to Kelly Bailey; no settings were ever
  published. The `hev` preset is a starting point (less extreme than VOX); tune pitch and
  rate to taste. VOX prosody is optional there (`VESKSLOP_TTS_VOX=1`).

#### How NetTTS applies the settings (reference behaviour)

Checked against the NetTTS sources (`src/main.cpp`, `src/tts_engine.cpp`,
`scripts/winetricks/nettts_flextalk_vox_profile.reg`):

1. **Voice**: the imported profile points `Default Speaker` at Keith Bell's GUID, and
   NetTTS selects the default mode (empty `TTSMODEINFO`) rather than a voice name.
2. **Speaker profile**: the full key set is imported into **both** hives
   (`HKEY_CURRENT_USER` and `HKEY_LOCAL_MACHINE`) at prefix setup; every numeric value is
   `REG_DWORD`.
3. **Rate and pitch are never set through `ITTSAttributes`.** NetTTS keeps UI percentages
   and prepends sticky vendor tags to the utterance text:
   - `\!R<factor>` = duration factor. Smaller is faster: `0.30` very fast, `1.00` 1.0x,
     `1.60` slow. No tag at 100%.
   - `\!%<factor>` = pitch multiplier. `0.85` lower, `1.20` higher. No tag at 100%.
4. **Volume** goes through `ITTSAttributes::VolumeSet(percent * 65535 / 100)` (Flextalk
   scale; MSTTS uses 0..0xFFFFFFFF).
5. **Prosody**: the VOX parser output wrapped in `\!wH1 ... \!wH0`.
6. The tags are sticky inside an engine instance; NetTTS sends the neutral tag only to
   reset. Our wrapper starts a fresh engine per line, so stickiness cannot leak.

#### How to implement it correctly (and the bug this fixed)

Earlier wrapper versions mapped `VESKSLOP_TTS_SPEED`/`_PITCH` to `ITTSAttributes`
`-s`/`-p`. Measured with the VOX profile loaded, Veskslop's neutral slider values
(`speed=1.0, pitch=0`) sent `-s 150 -p 106`, which overrode the registry profile: F0
jumped from 87 Hz back to 120 Hz and the line shortened from 5.83 s to 5.10 s. That is
the "too fast, not deep enough" report.

The correct implementation for Flextalk:

1. Write the speaker profile first (both hives, `REG_DWORD`, full key set for presets;
   `Default Speaker` for presets).
2. Translate rate/pitch to vendor tags, **not** attributes:
   - speed multiplier `m` -> `\!R(1/m)` (clamp 0.25-4.0)
   - pitch semitones `st` -> `\!%(2^(st/12))` (clamp 0.5-2.0)
3. Emit **no tag when neutral** (`m == 1`, `st == 0`). `\!R1.00` is a literal 1.0x rate,
   not the profile rate, so emitting it defeats the preset.
4. Prepend the tags to the text (`\!R0.77 \!%1.19 Attention all personnel.`); for word
   mode, prepend to each word because every word is a separate engine instance.
5. Keep `-s`/`-p` only for MSTTS voices, which have no vendor tags.
6. Verify with F0 and duration, not by ear: factory Keith ~3.35 s / ~123 Hz; VOX preset
   ~5.83 s / ~87 Hz; neutral sliders must reproduce the preset byte-for-byte.

Verified matrix (same line, Keith Bell):

| Run | Duration | F0 median |
| --- | --- | --- |
| factory, no adjustments | 3.35 s | 123 Hz |
| VOX preset, no adjustments | 6.06 s | 82 Hz |
| VOX preset + `speed=1.0 pitch=0` | 6.06 s, byte-identical | 82 Hz |
| VOX preset + `speed=1.3` | 4.02 s | - |
| VOX preset + `pitch=+3` | 6.06 s | 96 Hz |

Registry inspection after a preset run: 26 values in the Keith speaker key in both
hives, `Pitch=0x0`, `Speaking Rate=0xf`, `Head Size=0x22`, `Lowering=0x32`,
`Gender=0x2`, `Line Mode=0x1`,
`Default Speaker={11260610-C84A-11CE-AEC4-0000C0E9CB87}`.

#### Tuning against the actual game assets (2026-09-22)

Reference: the word WAVs from `suchipi/half-life-vox-console`
(`public/vox/*.wav`, Valve assets - used locally for analysis only, never committed).
38 words at 11025 Hz 16-bit, the engine's native rate. `tools/vox_compare.py`
reproduces the comparison; it reports the cepstrum F0 median, total audible duration,
energy-weighted spectral centroid and the >2 kHz energy share per word set.

| Metric (38 words) | Game assets | NetTTS approximation | Shipped VOX preset |
| --- | --- | --- | --- |
| F0 median | 82.3 Hz | 87.5 Hz | 82.3 Hz |
| Duration | 23.36 s | 22.28 s | 22.99 s |
| Spectral centroid | 515 Hz | 479 Hz | 453 Hz |
| >2 kHz energy share | 0.0186 | 0.0137 | 0.0172 |

Findings:

- Registry `Pitch 0` is the engine minimum and matches the game's F0; the NetTTS
  approximation (3) was about 5 Hz high.
- `Speaking Rate 15` matches the word durations within about 1.6%; NetTTS's 17 was
  about 4.6% fast on this corpus.
- The game voice is brighter than the approximation. Smaller `Head Size` raises
  brightness (34 is the engine's effective minimum here; 30 gives identical output) and
  `Lowering 50` adds a little more, together closing most of the >2 kHz gap
  (0.0137 -> 0.0172 against 0.0186).
- The residual centroid difference is mostly the reference's mid/high-band noise floor
  (game assets), not voice timbre.
- HEV stays a starting point: no published settings and no word-level reference exists.

Re-run recipe: write the profile to both hives, synthesize the reference word list with
`build/sapi4host-winlib.exe -v "Keith Bell" -t <word> -o <dir>/<word>.wav`, then
`tools/vox_compare.py <game-assets-dir> <synth-dir>`.

### Inline rate/pitch controls (engine-side)

The wrapper consumes NetTTS-style, line-leading controls, so Veskslop and the WebUI do
not need to parse or implement them:

    /rate 150 Attention all personnel.      # 150% rate, then the line
    /pitch 90 Attention all personnel.      # 90% pitch
    /rate 120 /pitch 110 Inbound tram.      # both, in any order

- `N` is a percent (100 = normal, clamped to 0-200); values `<= 2.0` are multipliers
  (`/rate 1.5` = 150%).
- They override `VESKSLOP_TTS_SPEED` / `VESKSLOP_TTS_PITCH` for that line and are not
  spoken.
- A line that contains only controls yields 100 ms of silence, so the daemon always
  receives a valid WAV.
- Rate maps through the normal speed scale; pitch percent maps to semitones
  (`st = 12*log2(N/100)`) and then through the per-voice natural pitch.
- Verified on the dev host: `/rate 50` doubles the line duration; `/pitch 150` raises
  the median F0 from 122 Hz to 177 Hz.

### Paths and configuration (standalone)

The engine project is standalone and archival: it has no path dependency on Veskslop,
and every non-installable dependency is vendored under `third_party/`, so nothing is
fetched from the network at build or run time:

- `third_party/speech.h` — SAPI4 SDK header used by the build.
- `third_party/sapi4/` — SAPI4 runtime (`SPEECH.DLL` + INF), MSTTS engine + voices,
  and the original installers.
- `third_party/flextalk/data1.cab` — AT&T Flextalk installer (Keith Bell / Kerry Watson).
- `third_party/nettss-reference/` — NetTTS VOX parser source, kept to validate
  `setup/vox_parser.py`.

Only toolchain packages are installed by `setup/install-deps.sh` (Wine/Winelib, ffmpeg,
python3, unshield, mingw). Everything else lives in the repository.

`setup/install.sh` writes `setup/microslopsam.env` with absolute paths; environment
variables take precedence over that file, so anything can be relocated without editing
scripts:

| Variable | Default | Meaning |
| --- | --- | --- |
| `MICROSLOP_ROOT` | repo root | project root |
| `MICROSLOP_BUILD` | `$MICROSLOP_ROOT/build` | host binaries, samples |
| `MICROSLOP_PREFIX` | `~/.wine-microslopsam` | wine prefix (exported as `WINEPREFIX`) |
| `MICROSLOP_ASSETS` | `third_party/sapi4` | SAPI4/MSTTS payloads (vendored) |
| `MICROSLOP_FLEXTALK_DIR` | `third_party/flextalk` | Flextalk cabinet/extraction (vendored) |
| `MICROSLOP_SPEECH_H_DIR` | `$MICROSLOP_ROOT/third_party` | `speech.h` include dir (vendored) |
| `MICROSLOP_DISPLAY` | `:77` | Xvfb display used for Wine |
| `MICROSLOP_WINE` / `_XVFB` / `_PYTHON` / `_FFMPEG` | tool names | tool overrides |

The wrapper needs only `VESKSLOP_TTS_CMD` to point at its absolute path; it derives
everything else from its own location plus the env file.

### Merging into Veskslop

This tree is written to survive a copy into the Veskslop repository next to its daemon:

1. Copy `microslopsam/` (or replace the existing copy). The vendored assets under
   `third_party/` come along, so the merged tree stays self-contained. Then refresh the
   configuration: `MICROSLOP_UPDATE_ENV=1 microslopsam/setup/install.sh`. Re-running
   install is idempotent; it rewrites `setup/microslopsam.env` for the new absolute paths.
2. No daemon changes are required. The env/placeholder contract is unchanged
   (`VESKSLOP_TTS_*`, `{out}`, `{text_file}`, voice/speed/pitch/expression/profile/
   highpass/echo/words/word_gap_ms). New features are additive and default off.
3. Optional Veskslop-side follow-up: a `tts.vox` preference that exports
   `VESKSLOP_TTS_VOX` (like `words`) and a line in the TTS panel help. If presets are
   exposed, also export `VESKSLOP_TTS_PRESET` and have the voice metadata defaults reflect
   the preset values so the UI's "default" snap points match. The inline `/rate` and
   `/pitch` controls need no UI work at all.

### Transport and latency

- Installation is one command: `setup/install.sh` (system dependencies with sudo, wine
  prefix + SAPI4 runtime + MSTTS voices, Flextalk extraction/registration, host build).
  Re-running is safe; a fresh machine needs nothing else.
- Command provider, implemented as `setup/veskslop-tts.sh`:
  `VESKSLOP_TTS_CMD="<repo>/setup/veskslop-tts.sh {out} {text_file}"` and
  `VESKSLOP_TTS_VOICES="$(<repo>/setup/veskslop-tts.sh --voices)"`. The wrapper reads the
  text file, transliterates UTF-8 to CP1252, starts or reuses an Xvfb when `DISPLAY` is
  unset, calls the engine, then post-processes (trim, fades, measured fixed gain, 48 kHz
  mono s16le). A stale private Xvfb is retried once.
- Measured on the dev host (warm): engine call about 190 ms (Flextalk) to 220 ms (Sam)
  including Wine start, ffmpeg measure about 110 ms, ffmpeg apply about 90 ms, profile
  registry write about 57 ms when it runs. Full wrapper: about 410-425 ms per short line
  warm, about 730 ms for the first (cold) call. The 500 ms target is met warm; Veskslop's
  "Start engine" warm-up covers the cold case.
- Latency optimization implemented: the Flextalk profile write is cached per voice inside
  the Wine prefix (`.microslopsam-profile<GUID>.reg`); an unchanged profile skips the
  `wine reg import` round-trip (about 57 ms warm, 300 ms cold). Recreating the prefix
  invalidates the cache automatically.
- Manual checks: `setup/run.sh -l`, `setup/run.sh -v "Keith Bell" -t "Hello." -o out.wav`
  (raw engine WAV), and the wrapper above for normalized output.
- HTTP provider (later): a persistent process returning `audio/wav` bytes and `/voices`,
  with optional caching by `hash(text + voice + speed + pitch)`.

### Quirks the wrapper must hide

- Flextalk numeric speaker settings must be `REG_DWORD`; `REG_SZ` values make the engine
  synthesize a buzz. `setup/setup-flextalk.sh` writes them correctly.
- Flextalk cabinet is single-volume: `unshield -O` extracts all 450 entries with exact
  sizes. Without `-O`, the new decompressor reports a phantom "volume 2" and fails on
  `MSVCRT.DLL` / `WatsonUNInstall.dll`; both are complete with `-O` (verified against the
  original 1997 upload log, which lists `data1.cab` only).
- The mode list includes TruVoice names with no installed voice data; filter those out of
  `/voices` (or mark them `unavailable`).
- Wine needs a working X display; the project helper starts a persistent Xvfb.

### Licensing

- The engines and voice data are proprietary Microsoft/AT&T/L&H artifacts. Keep this a
  local integration; do not redistribute the DLLs or voice data with Veskslop.

### Extra acceptance items for the engine

- [x] Post-processed WAV is 48 kHz mono s16le, peak about -3 dBFS, no clipping, fades present.
- [x] Leading silence under 100 ms.
- [x] Voice ID fallback works; TruVoice entries filtered.
- [x] Latency recorded (Sam about 288 ms, Flextalk about 41 ms on the dev host).

## Responses from the Veskslop side

_Added 2026-09-15._

- **Env vars and placeholders**: `VESKSLOP_TTS_SPEED` and `VESKSLOP_TTS_PITCH` are exported only
  when a value is set; `{speed}` and `{pitch}` substitute the same values. Reading the environment
  is the safer option because it works even when the command has no placeholders.
- **Defaults**: a missing value means "engine default". Leave the sliders at 1.00x and 0 st in
  Veskslop to keep the fields out of the request entirely.
- **`loudnorm` caution**: single-pass `loudnorm` can pump or over-amplify noise on short lines.
  Set the target explicitly (`loudnorm=I=-16:TP=-1.5:LRA=11`) and verify a few lines with
  `ebur128`. A fixed gain that hits about -3 dBFS peak with a consistent RMS is often steadier for
  one-line utterances. Ducking lowers the sink to 30%, so keep peaks conservative and loudness
  even between lines.
- **Display**: the daemon does not export `DISPLAY` to TTS children. The wrapper must start or
  find its own Xvfb and set `DISPLAY` itself, as the project helper already does. Do not rely on
  the client's display.
- **Engine selection**: the Speech settings and the TTS panel list every configured engine. The
  command engine appears under `VESKSLOP_TTS_NAME` (default "custom engine") and is selectable
  directly; Auto uses the command engine first. Selecting it hides the language row and shows the
  `VESKSLOP_TTS_VOICES` list.
- **Start engine**: the Speech settings have a **Start engine** button (`POST /api/audio/tts/warm`,
  MCP `start_tts_engine`) that runs one throwaway synthesis to warm the command engine (Xvfb, Wine,
  host). Keep the wrapper able to handle a short sentence; the warm-up uses "The quick brown fox
  jumps over the lazy dog." and reports the elapsed milliseconds. Two-pass `loudnorm` must not fail
  on that input.
- **Lead-in silence**: Veskslop prepends a short silence at playback time (`VESKSLOP_TTS_LEAD_MS`,
  default 150 ms) so Discord's voice pipeline is ready before the first syllable. It is added with
  `adelay` in the player, not in the audio file. The engine needs no change and can keep its
  5–10 ms edge fades.
- **Voice declarations**: list only usable IDs in `VESKSLOP_TTS_VOICES`. Veskslop hides the
  espeak list while a custom command is set, so a missing declaration means an empty voice list,
  not a fallback. The list accepts commas or newlines. Optional flags: `no-pitch` (hide the pitch
  control) and `expression` / `expression=NN` (show the Expression slider and use NN as its
  default).
- **Flextalk pitch**: works now that the speaker registry values are `REG_DWORD`; no flag is
  needed and Veskslop shows pitch for Keith Bell and Kerry Watson.
- **Expression profile**: implemented for the Flextalk voices. Veskslop stores `tts.expression`
  (0-100, nullable), sends it per line (`{expression}` / `VESKSLOP_TTS_EXPRESSION` / the HTTP body),
  and shows the slider only for voices flagged `expression=NN`. A null value leaves the registry
  untouched, so the voice keeps its engine default.
- **Licensing**: keep the engine DLLs and voice data local and out of any repository; Veskslop
  never bundles or redistributes them.

## Current integration path (Veskslop side, 2026-09-15)

Everything below is implemented. The engine stays unchanged; the wrapper and the daemon do the work.

### Voice metadata

`VESKSLOP_TTS_VOICES` entries may carry flags after the language field:

    id|Name|gender|language|flags

Flags are `+`-separated:

- `no-pitch` - hide the pitch control for this voice.
- `expression=NN` - Flextalk profile voice; NN is the default Expression value.
- `defaults=pitch:27:rate:33:prominences:17:head_size:46:lowering:40:expression:59` -
  per-voice engine defaults. The UI shows "default" at these values and snaps to them
  within +/-2 units.

### Wrapper contract (`microslopsam/setup/veskslop-tts.sh`)

Inputs: `{out}`, `{text_file}`, optional positional voice/speed/pitch/expression, and:

| Env | Meaning |
| --- | --- |
| `VESKSLOP_TTS_PROFILE` | `pitch:27:...`/`key=value,...` overrides; the wrapper writes the full profile (defaults + overrides) to the Wine registry with `wine reg import` before every synthesis. |
| `VESKSLOP_TTS_HIGHPASS` | High-pass Hz, 40-400 (default 40). |
| `VESKSLOP_TTS_ECHO` | Mono echo 0-100 (`aecho`), default 0. The echo is mixed under the dry voice, so the voice level does not change. |
| `VESKSLOP_TTS_ECHO_DELAY_MS` / `VESKSLOP_TTS_ECHO_LEVEL` | Echo delay 10-400 ms (default 60) and repeat level 0-0.95 relative to the dry voice (default 0.85). |
| `VESKSLOP_TTS_CRUNCH` | `off` (default) or `hl90`, `hl90hq`, `telephone`, `adpcm`, `crush`: 1990s lossy character, applied last. |
| `VESKSLOP_TTS_CRUNCH_BITS` / `VESKSLOP_TTS_CRUNCH_RATE` | Bit depth 1-16 and sample rate 4000-48000 overrides for the crunch stage. |
| `VESKSLOP_TTS_STREAM` / `VESKSLOP_TTS_STREAM_GAIN_DB` | `1` = word mode writes the WAV as it goes (no measurement pass); fixed gain in dB for that path. |
| `VESKSLOP_TTS_WORDS` / `VESKSLOP_TTS_WORD_GAP_MS` | Word-by-word synthesis with silence gaps (Flextalk only). |

Placeholders `{profile}`, `{highpass}`, `{echo}`, `{echo_delay_ms}`, `{echo_level}`,
`{words}`, `{word_gap_ms}`, `{crunch}`, `{crunch_bits}`, `{crunch_rate}`, `{stream}`,
`{vox}`, `{vox_clean}`, `{preset}` mirror the env vars.
`wine regedit` fails headlessly but returns 0: use `wine reg import`. A missing profile value
must always write the engine default so no stale registry state survives.

### Daemon and UI path

1. Preferences (`tts`): `profile` (changed keys only), `profilePresets` (5 named slots),
   `highpass`, `echo`, `words`, `wordGapMs`, plus `speed`, `pitch`, `expression`, `voice`.
2. `/api/audio/speak` and MCP `speak` accept all of them per line; the request overrides the
   preferences and empty fields are omitted so the engine default applies.
3. The TTS panel opens the **Flextalk voice profile** window (movable, recentered on open) with
   profile sliders, named presets, radio-kit controls, and test lines (Attention, Laboratory,
   Critical, Health, Radiation, plus two persisted custom lines). Test buttons post the same
   options as `speak`, so they audition the current profile.

### Known pitfalls

- The daemon runs from `tsx`: restart it after protocol, preference, or metadata changes.
- Static web files can be cached; hard-reload the browser after UI changes.
- `Number(null)` is 0: preference normalizers must treat null/undefined/"" as unset.
- The renderer log spam on a dead client is throttled in the bridge; background polls fail with
  `CORE_UNAVAILABLE` and retry.
- Generate the Flextalk profile `.reg` with `printf`, never `echo`: dash's `echo` interprets
  `\2` in `...\Watson\2.0\...` as an escape (0x02), so the key is corrupted and
  `wine reg import` becomes a silent no-op (it still returns 0). The wrapper now rejects a
  generated profile that contains control characters. This was the 2026-09-15 profile bug.