# Voice

Voice state reads and the basic actions exist. Auto-leave, audio, and video do not. This document
records the state and the plan.

## Current

| Part | Status |
| --- | --- |
| `voice.get` | Implemented. One user's voice channel and its members. Without `guildId` it searches all guilds. |
| `voice.overview` | Implemented. Occupied voice channels and members for one guild. |
| `VOICE_STATE_UPDATE` events | Normalized and pushed. Mock-verified. Live check pending. |
| `voice.join` | Implemented. Joins muted. Internal lookups verified live. |
| `voice.leave` | Implemented. Disconnects the client. |
| `voice.move` | Implemented. Moves to another channel. |
| Auto-leave | Not started. |
| `stay @user` | Not started. |
| Capture and transcript | Implemented. VAD segments, WAV per utterance, JSONL store, live events. |
| Speech to text | Hook implemented (local command or HTTP API); provider not chosen yet. |
| Audio in and out | Output and capture work. Browser mic puppeting is not started. |
| Video | Not started. |

The core reads `VoiceStateStore` only. It never joins a call on its own.

## Actions

| Action | Parameters | Result |
| --- | --- | --- |
| `voice.join` | `channelId`, optional `mute` (default true) | `{ channelId, joined: true, muted }` |
| `voice.leave` | none | `{ left: true }` |
| `voice.move` | `channelId` | `{ channelId, joined: true }` |

Rules:

- The channel must be a voice channel. The core validates the snowflake and the channel type.
- Join joins the call muted. The bot does not capture or send audio.
- Voice actions are state changes, not send floods. They stay out of the message pacing queue. The
  daemon spaces them 250-750 ms (`pacedVoice`).
- The join goes through Discord's internal channel action, located at runtime.
  `selectVoiceChannel` and `setSelfMute` were verified live on 2026-09-13. The end-to-end join was
  not run.

The plan for later audio: the bot unmutes only while it plays an audio stream, then mutes again.

## Audio

Voice audio uses the system audio stack, not Discord internals. The daemon owns the wiring.

### Devices

| Device | Kind | Purpose |
| --- | --- | --- |
| `veskslop_tts` | null sink | Playback target for TTS |
| `veskslop_mic` | remap source of `veskslop_tts.monitor` | The client microphone |
| `veskslop_capture` | null sink | Client output, recorded by the daemon |

The host needs a sound server. On Debian and Ubuntu:

```sh
apt install pulseaudio pulseaudio-utils ffmpeg espeak-ng
```

The daemon starts `pulseaudio` automatically when `pactl` cannot reach a server, unless
`VESKSLOP_AUDIO_MANAGE_SERVER=0`. If `pactl` is missing, the status reports the install hint. The
Web UI Audio section has a **Connect devices** button that rechecks the client device list.

On startup the daemon creates missing devices with `pactl` and unloads the modules it created on
exit. When the client connects, the daemon finds the devices by label and sets them through
`voice.setDevices`, unless `VESKSLOP_AUDIO_AUTOCONNECT=0`. Names are configurable with
`VESKSLOP_AUDIO_SINK`, `VESKSLOP_AUDIO_SOURCE`, and `VESKSLOP_AUDIO_CAPTURE_SINK`.

If the stack is unavailable, playback/speak calls return the reason and the rest of the daemon
works as before.

### Playback

- TTS has two providers:
  - **local**: `VESKSLOP_TTS_CMD` with `{out}` and `{text_file}` placeholders. Example:
    `piper --model sv_SE-nst-medium.onnx --output_file {out} -f {text_file}`.
  - **api**: `VESKSLOP_TTS_API_URL` receives a JSON body `{ text, voice? }` and returns audio
    bytes. `VESKSLOP_TTS_API_KEY` adds a bearer token.
- `ffplay` decodes and plays into `veskslop_tts` (`PULSE_SINK`). `VESKSLOP_FFPLAY` overrides the
  binary. Lines play one at a time in order.

### Browser monitor (Client audio)

The header **Client audio** button streams the call to the browser:

- The daemon reads `veskslop_capture.monitor` with `ffmpeg` and sends raw 48 kHz mono PCM over
  the WebSocket at `/audio`.
- The browser plays the PCM with the Web Audio API and can send it to a chosen output device.
- This is separate from the bot's microphone and TTS. Nothing the browser plays affects the call.

Browser microphone puppeting (talking through the bot) is not implemented. It needs a mix sink so
the browser mic and TTS share `veskslop_mic`, a `pacat` input, and echo handling.

### Ducking

When another person speaks, the daemon lowers the `veskslop_tts` sink volume to
`VESKSLOP_AUDIO_DUCK` percent (default 30) and restores it when they stop. Speaking comes from the
`VOICE_SPEAKING` event. Staying quiet while someone else talks is a queue property, not a switch.
Yield (stop mid-line) is not implemented.

### Capture and transcript

The daemon records the client output and splits it into utterances.

1. `ffmpeg` reads `veskslop_capture.monitor` as 16 kHz mono PCM (`VESKSLOP_FFMPEG` overrides).
2. The VAD opens a segment after `VESKSLOP_VAD_MIN_SPEECH_MS` above `VESKSLOP_VAD_THRESHOLD_DB`,
   keeps `VESKSLOP_VAD_PRE_ROLL_MS` of audio from before, and closes after
   `VESKSLOP_VAD_SILENCE_MS` of quiet.
3. Each segment becomes a WAV under `<cache>/audio/captures/` and a transcript entry in
   `VESKSLOP_TRANSCRIPT` (default `<state dir>/transcripts.jsonl`).
4. When `VOICE_SPEAKING` covers the segment, the dominant speaker is attached.
5. When STT is configured, the segment is transcribed and the text is attached.

STT providers:

- **Vosk (default)**: CPU, small and light. Install it and drop a model in `poc/models/`
  (auto-detected), or set `VESKSLOP_VOSK_MODEL`:
  ```sh
  pip install vosk
  curl -LO https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip
  unzip vosk-model-small-en-us-0.15.zip -d poc/models
  ```
  The daemon runs `poc/scripts/stt-vosk.py` for each segment. Quality is robotic but light.
- **local command**: `VESKSLOP_STT_CMD` with `{wav}`; the command prints text on stdout.
- **api**: `VESKSLOP_STT_API_URL` receives WAV bytes and returns text or JSON `{ text }`.
  `VESKSLOP_STT_API_KEY` adds a bearer token.

TTS engine and voice are chosen in Settings → Client → Speech (stored in the daemon preferences).
The local engine defaults to `espeak-ng` when installed; the voice list comes from
`espeak-ng --voices`.

Start and stop with `POST /api/audio/capture`, the Audio settings button, or the MCP tools. Set
`VESKSLOP_CAPTURE_AUTO=1` to start capture when the bot joins a call and stop it on leave.

Transcript events go over the SSE stream: `TRANSCRIPT_SESSION`, `TRANSCRIPT_SEGMENT`,
`TRANSCRIPT_TEXT`, `TRANSCRIPT_SESSION_END`, and `TRANSCRIPT_ERROR`. The agent console's
Transcript tab shows the live list; `GET /api/transcript` and the `get_transcript` MCP tool read
the store. Segments without text still carry the WAV path, so an LLM can transcribe them later.

### Controls

- Chat: `speak <text>` and `speak stop` from a trusted user.
- MCP: `speak`, `stop_audio`, `get_audio_status`, `start_capture`, `stop_capture`,
  `list_audio_devices`, `set_audio_devices`, `get_transcript`.
- HTTP: `/api/audio`, `/api/audio/speak`, `/api/audio/stop`, `/api/audio/capture`,
  `/api/audio/devices`.
- Web UI: the TTS panel (provider, Speak, Stop) and the Audio settings section.


## Lifecycle

The bot leaves a call on its own in two cases.

1. **Owner leaves.** When the owner leaves the call, the bot waits `voiceLeaveDelaySeconds` (default
   30, `VESKSLOP_VOICE_LEAVE_DELAY` sets the default, `set_puppet_config` changes it) and then
   leaves. A rejoin inside the window cancels the leave. The owner is zzib for now.
2. **A stay target leaves.** The `stay @user` command pins the bot to a user. Not implemented yet;
   the owner rule is the only one active.

The bot never leaves while the owner is present. A manual `voice.leave` wins over both rules.

## Collector account

The collector account is a stripped-down client with read-only voice membership. The daemon uses it
when the puppet cannot see a user's voice channel or the channel members. It stays deferred. See
[TODO.md](../TODO.md).

## Verification list

A live run must confirm these points.

1. `voice.join` joins the given channel end to end. (The `selectVoiceChannel` lookup is verified.)
2. `voice.leave` disconnects the call.
3. `VOICE_STATE_UPDATE` carries the fields that the core normalizes.
4. The join is muted and Discord shows the mute state.
5. The auto-leave timer works across a client restart. The timer is daemon state, not core state.