Add MicroslopSam TTS, Flextalk voice profiles, effects, and engine controls

This commit is contained in:
Fisto
2026-09-15 15:09:23 +02:00
parent eab94132bb
commit 302436f95f
42 changed files with 4675 additions and 180 deletions
+27 -7
View File
@@ -64,9 +64,15 @@ With plain HTTP, `navigator.mediaDevices` only exists on localhost; remote brows
- Right column: **In voice** (favourited users in a call, grouped by call), **Favourites**, and
**Open DMs & groups** as quick access. The guild's own occupied channels live in the left rail.
- Speaking users show animated level bars and a bold name (design 2c: audio → video → share marks).
The bars scale up when speaking starts and stay on until an explicit stop, a mute, or a channel
change. A missing stop event leaves them on by design; they do not decay on their own. The
fade-out runs the equalizer inside a shrinking container, so the amplitude drops to nothing
instead of the block collapsing at full height. Muted members never show bars.
Nicknames come from the guild member store; DMs fall back to the username.
- Console: Tool calls, Events (live SSE), Transcript. The ticker shows the newest event. The
Transcript tab lists captured segments (time, speaker, text, WAV) and updates live.
Transcript tab lists captured segments (time, speaker, text, WAV) and updates live. It has
**Start**, **Stop**, and **Refresh** buttons; the status line shows `capturing` or `capture off`
plus the segment count.
- Opening a chat fetches the latest 50 messages from Discord into the client cache. The refresh
button by the channel header re-fetches. Live message events read the cache. Edited messages show
an `edited` marker and replies show `↳ replying to <name>`.
@@ -131,9 +137,19 @@ The Settings button opens a slide-over:
- **Client** — a collapsible group with:
- **Bot audio** — engine status, TTS provider state, a **Connect devices** button, and the
capture toggle. These are the daemon host's devices.
- **Speech** — the TTS engine (Auto / Local / API), language, and voice. Voices are grouped by
base language and filtered by the chosen language; English, Swedish, German, Danish,
Norwegian, Russian, and Cherokee are pinned on top. Stored in the daemon preferences.
- **Speech** — the TTS engine (Auto plus every configured engine: espeak-ng, the command engine
by its `VESKSLOP_TTS_NAME`, and the HTTP API), language, and voice. **Start engine** warms the
selected engine (starts Xvfb, Wine, and the host for the command engine). Selecting an engine
switches the voice list to that engine; the language row hides for single-language engines.
Voices are grouped by base language and filtered by the chosen language; English, Swedish,
German, Danish, Norwegian, Russian, and Cherokee are pinned on top. Stored in the daemon
preferences. Speed (0.5x–2.0x with 0.5/0.75/1/1.25/1.5 presets), pitch (-12 to +12
semitones), and an Expression slider (0–100, Flextalk voices only) live in the TTS panel and
save to the same preferences; controls the selected voice does not support stay hidden. For
Flextalk voices the TTS panel also opens the **Flextalk voice profile** window: a movable panel
that recenters on each open, with character sliders, five named presets, radio-kit effects
(high-pass, echo, word mode), and test lines (Attention, Laboratory, Critical, Health,
Radiation, plus two custom) that speak immediately.
- **Display** — the timestamp timezone. Defaults to the browser's zone
(`Intl.DateTimeFormat().resolvedOptions().timeZone`); pick any IANA zone to override. Stored in
`localStorage` and applied to messages, the transcript, the event log, the action log, and the
@@ -141,7 +157,9 @@ The Settings button opens a slide-over:
- **Browser audio** — puppeting controls: the browser output device for Client audio, a
placeholder mic select, and the call-audio state. **Enable browser audio** asks for the
microphone permission, which browsers require before they list output devices. Separate from
the bot's devices.
the bot's devices. Clicking **Client audio** in the header does the same: it asks for the
permission, unlocks the audio context in the same click, and then starts listening. A second
click stops it.
- **Server** — a collapsible group with:
- **Core process** — state, last exit, output, Start / Stop / Restart.
- **Account login** — token file state, QR login, captcha.
@@ -200,6 +218,8 @@ replaceable.
| GET | `/api/dms` | `limit?` | Cached DM and group DM channels with recipients |
| GET | `/api/voice` | `userId?`, `guildId?` | One user's voice state |
| GET | `/api/voice/overview` | `guildId` | Occupied voice channels and members |
| GET | `/api/voice/volume` | `userIds?` comma-separated | Per-user volumes plus output volume; defaults to the current call |
| POST | `/api/voice/volume` | `userId`, `volume` | Set a user's playback volume (0 to 200) |
| POST | `/api/voice/join` | `channelId`, `mute?` | Join a voice channel (muted by default) |
| POST | `/api/voice/leave` | none | Leave the current voice channel |
| POST | `/api/voice/move` | `channelId` | Move to another voice channel |
@@ -207,11 +227,11 @@ replaceable.
| POST | `/api/preferences` | `visibleGuilds?`, `tts?` | Update preferences. Stored in `<state dir>/preferences.json`. |
| GET | `/api/audio` | none | Audio engine, TTS, queue, ducking, capture state |
| GET | `/api/audio/devices` | none | The client's input and output devices |
| GET | `/api/audio/tts/voices` | none | Local TTS voices (espeak-ng) |
| GET | `/api/audio/tts/voices` | `engine?` | Voices for the selected engine (espeak-ng or `VESKSLOP_TTS_VOICES`) |
| POST | `/api/audio/connect` | none | Re-apply the virtual devices to the client |
| WS | `/audio` | `{"type":"listen","on":bool}` | Raw 48 kHz mono PCM of the call |
| POST | `/api/audio/devices` | `inputDeviceId?`, `outputDeviceId?` | Set the client's devices |
| POST | `/api/audio/speak` | `text`, `provider?`, `voice?` | Queue text to speech |
| POST | `/api/audio/speak` | `text`, `provider?`, `voice?`, `speed?`, `pitch?` | Queue text to speech |
| POST | `/api/audio/stop` | none | Stop speech and clear the queue |
| POST | `/api/audio/capture` | `start` | Start or stop capture |
| GET | `/api/transcript` | `limit?`, `sessionId?` | Sessions and captured segments |