Update docs and README for voice audio, capture, and the web UI
This commit is contained in:
Fisto
@@ -59,11 +59,14 @@ before you stop.
|
||||
| Response policy (channel or DM) | Implemented |
|
||||
| Chat commands (`whitelist`) | Implemented. `poll` is paused. |
|
||||
| Persistent action log | Implemented |
|
||||
| Web UI and JSON API | Nocturne 7a debug shell + roadmap placeholders. No auth; LAN or Tailscale only. |
|
||||
| Web UI and JSON API | Nocturne 7a debug shell + roadmap placeholders. Server-side prefs. No auth; LAN or Tailscale only. |
|
||||
| Core process control and web QR login | Implemented. Core start/stop verified live. |
|
||||
| User search (cache and guild members) | Implemented |
|
||||
| Voice state read and `VOICE_STATE_UPDATE` events | Implemented |
|
||||
| Voice actions, audio, video | Not started. Plan in [docs/voice.md](docs/voice.md). |
|
||||
| Voice actions (`voice.join`, `voice.leave`, `voice.move`) | Implemented. Auto-leave and `stay @user` pending. |
|
||||
| Voice audio (TTS, virtual mic, ducking, browser monitor) | Implemented. Microphone puppeting pending. |
|
||||
| Audio capture and transcript (VAD, WAV, STT hook, live events) | Implemented. STT provider not chosen. |
|
||||
| Voice video | Not started. Plan in [docs/voice.md](docs/voice.md). |
|
||||
| Notification parsing | Not started |
|
||||
| Auth persistence in the core | Implemented and verified live |
|
||||
| CDP transport | Kept as a development fallback |
|
||||
@@ -72,12 +75,11 @@ before you stop.
|
||||
|
||||
See [CURRENT.md](CURRENT.md) and [TODO.md](TODO.md) for the live list. The main items are:
|
||||
|
||||
1. Voice backend behind the Web UI Join/Leave buttons.
|
||||
2. Voice: `voice.join`, `voice.leave`, `voice.move`, auto-leave, and the `stay @user` command.
|
||||
3. Notification parsing: edits, deletes, reactions, and replies.
|
||||
4. Reply-to-puppet as a trigger.
|
||||
5. Per-user rate limit and command cooldowns.
|
||||
6. Collector account with read-only voice membership.
|
||||
1. Voice auto-leave and the `stay @user` command, then live-verify join/leave and audio.
|
||||
2. Notification parsing: edits, deletes, reactions, and replies.
|
||||
3. Reply-to-puppet as a trigger.
|
||||
4. Per-user rate limit and command cooldowns.
|
||||
5. Collector account with read-only voice membership.
|
||||
|
||||
Paused: the `poll` command and the setup command flow. Parked: MCP event delivery and the LLM
|
||||
attachment. Open decisions wait in [CURRENT.md](CURRENT.md) → Open discussions.
|
||||
|
||||
+59
-7
@@ -6,12 +6,39 @@ This file tracks the work state. Read it first. Update it before you stop.
|
||||
|
||||
## Current focus (broad)
|
||||
|
||||
The text layer is done. The web UI follows the Nocturne design handoff. The voice backend is the
|
||||
next feature; live verification and a full QR approval are pending. The LLM attachment and MCP
|
||||
event delivery are parked.
|
||||
The text layer is done. The web UI follows the Nocturne design handoff. Voice actions are in
|
||||
progress: join/leave/move work; auto-leave and `stay @user` are next. Live verification and a full
|
||||
QR approval are pending. The LLM attachment and MCP event delivery are parked.
|
||||
|
||||
## In progress
|
||||
|
||||
- Voice actions are started: `voice.join` (muted), `voice.leave`, and `voice.move` are implemented
|
||||
in the core, wired through the daemon, Web UI, and MCP, and mock-verified. The internal
|
||||
`selectVoiceChannel` and `setSelfMute` lookups were probed live on the real client.
|
||||
- Auto-leave and `stay @user` are the next voice step.
|
||||
- Voice audio is implemented: daemon-managed virtual devices (`veskslop_tts`, `veskslop_mic`,
|
||||
`veskslop_capture`), TTS providers (local command + HTTP API), queued playback through `ffplay`,
|
||||
ducking on `VOICE_SPEAKING`, and WAV capture. The daemon starts `pulseaudio` when `pactl` cannot
|
||||
reach a server and reports install hints. Verified live on 2026-09-13 after installing
|
||||
`pulseaudio`/`ffmpeg`: engine ok, devices created, PCM streamed.
|
||||
- The Web UI serves self-signed HTTPS (`openssl`, stored under the state dir) so browser media
|
||||
APIs have a secure context. `WEBUI_TLS=0` disables it.
|
||||
- Capture and transcript are implemented: ffmpeg reads the client output, a VAD cuts it into
|
||||
utterances, each becomes a WAV plus a transcript entry with the dominant speaker, and a
|
||||
pluggable STT hook (local command or HTTP API) fills the text. Live events feed the console
|
||||
Transcript tab; `get_transcript` and `/api/transcript` read the store. `VESKSLOP_CAPTURE_AUTO=1`
|
||||
records calls automatically. Verified end to end with real PulseAudio on 2026-09-13.
|
||||
- STT uses Vosk (CPU, light): `poc/scripts/stt-vosk.py` plus a small model in `poc/models/`
|
||||
(auto-detected). Verified with a TTS→capture→Vosk round trip. Other providers remain available
|
||||
via `VESKSLOP_STT_CMD`/`_API_URL`.
|
||||
- TTS defaults to `espeak-ng` when installed; engine and voice live in Settings → Client → Speech
|
||||
(daemon preferences, so all browsers share them).
|
||||
- Preferences moved to `<state dir>/preferences.json` and no longer depend on the core, so guild
|
||||
visibility saves even while the client is down. API responses send `Cache-Control: no-store`.
|
||||
- Browser audio has an **Enable browser audio** button that requests the microphone permission,
|
||||
which browsers require before listing output devices.
|
||||
- Browser microphone puppeting (talking through the bot) is next; it needs a mix sink, a `pacat`
|
||||
input, and echo handling.
|
||||
- The web UI follows the Nocturne design handoff (`poc/WebUIMockup.zip`). The 7a shell is
|
||||
implemented and rendered headless for review: header, call strip, three-column body, composer,
|
||||
and the agent console.
|
||||
@@ -20,7 +47,6 @@ event delivery are parked.
|
||||
auto-Xvfb, the bridge connects, and stop exits cleanly.
|
||||
- Web QR login is implemented. The session reached `pending` against the real gateway; a full
|
||||
phone approval is pending.
|
||||
- Voice actions are designed in [docs/voice.md](docs/voice.md). The backend is not started.
|
||||
|
||||
## Done
|
||||
|
||||
@@ -39,6 +65,24 @@ event delivery are parked.
|
||||
- `user.search`: cache and guild-member lookup. `voice.get` searches all guilds without a `guildId`.
|
||||
- `voice.overview`: occupied voice channels and members for a guild.
|
||||
- `VOICE_STATE_UPDATE` events: normalized and pushed. Mock-verified.
|
||||
- Voice actions: `voice.join` (muted), `voice.leave`, and `voice.move` in the core, with daemon
|
||||
routes (`/api/voice/*`), MCP tools, and Web UI wiring. Voice actions are paced 250-750 ms and
|
||||
written to the action log. The internal lookups were probed on the live client.
|
||||
- Web UI voice rail: every voice channel is clickable. Clicking joins (muted); clicking the active
|
||||
channel leaves. Verified with the stateful mock.
|
||||
- Web UI startup: a **Start server** button appears left of Takeover while the bridge is down and
|
||||
hides when it connects. State reloads retry while the client boots, so guilds appear without a
|
||||
manual refresh. Core requests now time out after `VESKSLOP_REQUEST_TIMEOUT_MS` instead of
|
||||
hanging forever on a missing reply.
|
||||
- Settings: "Visible guilds" stays top level; Audio sits under a collapsible "Client" group;
|
||||
Core process, Account login, and Action log sit under a collapsible "Server" group.
|
||||
- Web UI timestamps use the browser timezone by default; Settings → Client → Display can override
|
||||
it with any IANA zone. The choice is per browser (`localStorage`).
|
||||
- Web UI preferences: visible guilds persist server-side in the daemon state file
|
||||
(`/api/preferences`), so every browser shares them. The Settings panel has an explicit **Save**
|
||||
button; selections filter the UI live and persist only on Save.
|
||||
- DM list: recipients resolve from IDs through the user store, so names and avatars load.
|
||||
- Guild menu: fixed the light default button background.
|
||||
- Web UI: Nocturne design tokens in `poc/public/nocturne.css`, the 7a debug shell, live guild and
|
||||
channel rail, messages, DMs, voice membership, agent console with a live event tab, and a
|
||||
Settings slide-over for core process, login, visible guilds, and the action log. Roadmap
|
||||
@@ -59,11 +103,19 @@ event delivery are parked.
|
||||
is written on approval and a running managed core restarts with the new token.
|
||||
- Unit tests cover the pacing policy. `pnpm lint` and `pnpm testTypes` pass at the root.
|
||||
|
||||
- Web UI navigation: Guild/DM scope tabs, a DM rail with unread badges (purple/orange, grey while
|
||||
Takeover is off), last-message times, favourite stars, and a People list of users seen in voice.
|
||||
Favourited users get presence and call tracking, grouped by call in **In voice**. Speaking shows
|
||||
level bars and bold names (2c marks for mute/deaf/video/share). The body dims while the bridge
|
||||
is down; the Start button becomes Refresh when the daemon is unreachable.
|
||||
- TTS settings are split into language and voice, grouped by base language with the requested
|
||||
languages pinned on top.
|
||||
|
||||
## Next (in order)
|
||||
|
||||
1. Voice backend: `voice.join`, `voice.leave`, `voice.move`, auto-leave, and the `stay @user` command.
|
||||
2. Live verification: `user.search`, `voice.overview`, `VOICE_STATE_UPDATE`, and a full web QR login
|
||||
approval on the real client.
|
||||
1. Voice auto-leave (`voiceLeaveDelaySeconds`) and the `stay @user` command.
|
||||
2. Live verification: `voice.join`/`voice.leave` end to end, `user.search`, `voice.overview`,
|
||||
`VOICE_STATE_UPDATE`, and a full web QR login approval on the real client.
|
||||
3. Notification parsing: edits, deletes, reactions, and message references.
|
||||
4. Reply-to-puppet counts as a trigger.
|
||||
5. Per-user rate limit and command cooldowns.
|
||||
|
||||
@@ -36,7 +36,7 @@
|
||||
|
||||
## 🚀 What is Veskslop
|
||||
|
||||
Veskslop takes the beloved Vesktop base and deletes the part where a person has to click things. What remains is a **headless Discord client** driven through a **validated capability layer**, so an **AI agent can run a real Discord account over MCP**. Text automation works today. Audio pipes and video streams are the next descent into madness.
|
||||
Veskslop takes the beloved Vesktop base and deletes the part where a person has to click things. What remains is a **headless Discord client** driven through a **validated capability layer**, so an **AI agent can run a real Discord account over MCP**. Text automation and the first audio pipes work today. Video streams are the next descent into madness.
|
||||
|
||||
It is Discord, but as infrastructure. It is the account you already have, plus `curl`, minus dignity.
|
||||
|
||||
@@ -45,11 +45,11 @@ Think of it as:
|
||||
- 🖥️ **A Discord client with the client removed.** Runs on a server, in a container, on a Raspberry Pi, in your weird home lab, wherever.
|
||||
- 🧠 **An agent body (text tools live).** Your MCP-capable AI gets a real account with real permissions and real consequences.
|
||||
- 🔀 **A proxy (planned).** Messages, events, voice state, attachments — everything forwarded, interceptable, scriptable, and API-shaped.
|
||||
- 🔊 **An audio router (planned).** Play anything into voice. Capture anything out. TTS in, STT out, pipes for days.
|
||||
- 🔊 **An audio router (partly live).** TTS into the call, ducking, browser monitoring, and captured call audio with Vosk STT. Browser microphone puppeting is next.
|
||||
- 📹 **A streaming endpoint (planned).** Push files, cameras, generative visuals, or pure slop into a voice channel.
|
||||
- 🧪 **A science experiment that will absolutely not get your account banned** *(allegedly)*.
|
||||
|
||||
> **Status: text layer works.** The real Vesktop client runs headless and is driven through a validated capability layer. Triggers, trust tiers, conversation context, response policy, persistent logging, MCP tools, user search, voice state reads, and the Nocturne-style debug web UI are live. Reaction polls are paused. Voice actions, audio, and video are still the vision. The design docs live in [docs/INDEX.md](docs/INDEX.md).
|
||||
> **Status: text and voice basics work.** The real Vesktop client runs headless and is driven through a validated capability layer. Triggers, trust tiers, conversation context, response policy, persistent logging, MCP tools, user search, voice state, voice join/leave/move, a virtual-mic TTS pipeline with ducking, browser call monitoring, VAD-segmented call capture with Vosk STT, core process control, web QR login, and the Nocturne-style debug web UI over self-signed HTTPS are live. Reaction polls are paused. Auto-leave, browser microphone puppeting, and video are still ahead. The design docs live in [docs/INDEX.md](docs/INDEX.md).
|
||||
|
||||
## 🗿 Why though
|
||||
|
||||
@@ -74,7 +74,11 @@ Veskslop ships an **MCP server** so any MCP-capable agent (Claude, GPT-based age
|
||||
| `get_state` | User, guilds, and channels. Cache-only, zero queries. |
|
||||
| `get_voice_state` | Which voice channel a user is in and who else is suffering there. |
|
||||
| `get_voice_overview` | Occupied voice channels in a guild and their members. |
|
||||
| `voice_join` / `voice_leave` / `voice_move` | Join muted, leave, or move the account's call. |
|
||||
| `get_audio_status` / `speak` / `stop_audio` | Virtual-mic TTS state, queue text, stop playback. |
|
||||
| `start_capture` / `stop_capture` / `get_transcript` | Segment call audio, stop, read the transcript. |
|
||||
| `search_users` | Find a user ID from a username, nickname, or cached profile. |
|
||||
| `list_dms` | Cached DMs and group DMs with recipients, unread counts, and last-message time. |
|
||||
| `send_message` | Send a message as the account. Send-paced. |
|
||||
| `read_messages` | Read the client's message cache. |
|
||||
| `send_dm` | Slide into someone's DMs. |
|
||||
@@ -85,10 +89,11 @@ Veskslop ships an **MCP server** so any MCP-capable agent (Claude, GPT-based age
|
||||
| `whitelist_list` / `whitelist_add` / `whitelist_remove` | Trust tiers: user, trusted, admin. |
|
||||
| `run_commands` | Run several read-only tools in one call. |
|
||||
|
||||
There is also a small web control UI on `http://<host>:8789/` with a JSON API and an SSE event
|
||||
stream. See [docs/webui.md](docs/webui.md).
|
||||
There is also a debug web UI on `https://<host>:8789/` (self-signed certificate) with a JSON API,
|
||||
an SSE event stream, a DM/favourites tab, and browser call audio. See [docs/webui.md](docs/webui.md).
|
||||
|
||||
**Planned**: voice join/leave, audio play/capture, video streams, message edit/delete/upload, presence and voice state control.
|
||||
**Planned**: voice auto-leave and `stay @user`, browser microphone puppeting, video streams, message
|
||||
edit/delete/upload, notification parsing, presence control.
|
||||
|
||||
Every action is **safe by default**: strict input validation in the client, structured errors, a hard outbound rate limit, and paced sends. A buggy agent cannot crash the client or spam Discord.
|
||||
|
||||
@@ -210,17 +215,19 @@ The full setup, including the headless client and the mock core, is in
|
||||
- [x] Response policy, hard rate limits, persistent action log
|
||||
- [x] MCP server with the live text tool surface
|
||||
- [x] User search and voice overview
|
||||
- [x] Nocturne debug web UI (7a shell, Takeover read-only gate, Settings)
|
||||
- [x] Voice join/leave/move (joins muted) over the validated core bridge
|
||||
- [x] Nocturne debug web UI (7a shell, DM tab, favourites, speaking marks, HTTPS, process control)
|
||||
- [x] Outbound audio pipeline (virtual mic, TTS local/API, ducking, browser monitoring)
|
||||
- [x] Inbound audio capture pipeline (VAD segmentation, WAV, transcript store, Vosk STT)
|
||||
- [~] Reaction polls and chat command help (paused)
|
||||
- [ ] Browser microphone puppeting (browser mic into the bot's voice)
|
||||
- [ ] Voice auto-leave and the `stay @user` command
|
||||
- [ ] Invite-link joins
|
||||
- [ ] Notification parsing: edits, deletes, replies, references
|
||||
- [ ] Reply-to-puppet as a trigger
|
||||
- [ ] Per-user rate limits and command cooldowns
|
||||
- [ ] Voice join/leave and invite-link joins
|
||||
- [ ] Voice auto-leave and the `stay @user` command
|
||||
- [ ] Collector account for voice membership fallback
|
||||
- [~] MCP event delivery (parked)
|
||||
- [ ] Outbound audio pipeline
|
||||
- [ ] Inbound audio capture pipeline
|
||||
- [ ] Video stream pipeline
|
||||
- [ ] Proxy mode + event bus
|
||||
- [ ] Container image (`veskslop:slop-latest`)
|
||||
|
||||
@@ -32,21 +32,24 @@ Ordered backlog. `CURRENT.md` tracks the live session state.
|
||||
|
||||
## Deferred
|
||||
|
||||
- Voice join and leave, invite-link joins, audio in and out, and video streaming. The plan is in
|
||||
`docs/voice.md`. Voice state reads and events exist. Voice actions do not.
|
||||
- Voice auto-leave, `stay @user`, invite-link joins, audio in and out, and video streaming. The
|
||||
plan is in `docs/voice.md`. The basic voice actions exist; auto-leave and audio do not.
|
||||
- Client lightening. Low priority. The client must stay stable.
|
||||
- Web UI authentication and remote accounts.
|
||||
|
||||
## Next
|
||||
|
||||
1. Voice backend: `voice.join` (muted), `voice.leave`, `voice.move`, the auto-leave timer, and the
|
||||
`stay @user` command.
|
||||
2. Live verification: `reaction.add`, `user.search`, `voice.overview`, `VOICE_STATE_UPDATE`, core
|
||||
start on a real display, and a full QR login approval.
|
||||
3. Notification parsing: edits, deletes, reactions, and message references.
|
||||
4. Reply-to-puppet counts as a trigger.
|
||||
5. Per-user rate limit and command cooldowns.
|
||||
6. Collector account: stripped-down read-only modules. Use it when the puppet cannot find a user's
|
||||
1. Voice auto-leave (`voiceLeaveDelaySeconds`) and the `stay @user` command.
|
||||
2. Live verification on the audio host: TTS local + API, ducking, capture, `voice.join`/
|
||||
`voice.leave` end to end, and a full QR approval.
|
||||
3. Browser microphone puppeting: a mix sink so the browser mic and TTS share `veskslop_mic`, a
|
||||
`pacat` input path, and echo handling.
|
||||
4. Speech to text: Vosk (small CPU model in `poc/models/`) is wired and verified; add more
|
||||
languages or a larger model when needed. `VESKSLOP_STT_CMD`/`_API_URL` stay available.
|
||||
5. Notification parsing: edits, deletes, reactions, and message references.
|
||||
6. Reply-to-puppet counts as a trigger.
|
||||
7. Per-user rate limit and command cooldowns.
|
||||
8. Collector account: stripped-down read-only modules. Use it when the puppet cannot find a user's
|
||||
voice channel or the channel members.
|
||||
|
||||
## Web UI follow-ups
|
||||
@@ -55,7 +58,8 @@ Ordered after the voice backend unless the owner asks first.
|
||||
|
||||
- Favourites, presence dots, and real member counts.
|
||||
- MCP tool-call tracing in the console Tool calls tab.
|
||||
- Transcript tab once voice audio lands.
|
||||
- Transcript tab fed by the audio capture and STT.
|
||||
- Yield (stop mid-line) for TTS when a human speaks.
|
||||
- DM scope in the header (the guild picker becomes "Direct messages").
|
||||
- Message editing, deletion, reactions, and history pagination.
|
||||
- The remaining mockup screens: 7b/7c console details, 6b history window, 4b group chat, 2c state
|
||||
|
||||
@@ -37,6 +37,7 @@ mention the puppet first.
|
||||
| `whitelist list` | Show the owner and the trusted users with their tiers. |
|
||||
| `whitelist add @user [user\|trusted\|admin]` | Add one or more users. The default tier is `user`. |
|
||||
| `whitelist remove @user` | Remove one or more users from the trust list. |
|
||||
| `speak <text>` | Speak text into the bot's current voice call. `speak stop` stops it. |
|
||||
| `poll ...` | Start a reaction poll. Paused at this time. |
|
||||
| `stay @user` | Planned with the voice backend. Pin the bot to a voice user. See [voice.md](voice.md). |
|
||||
|
||||
@@ -178,6 +179,9 @@ event. The log rotates at the size limit.
|
||||
| `continuation` | Follow-up message ID and the parent entry ID |
|
||||
| `command` | Command text, requester, result |
|
||||
| `reply` | Command, mode, target, result, error |
|
||||
| `voice` | Voice action, channel, mute |
|
||||
| `audio` | Audio action (speak, stop, devices, capture), provider, size |
|
||||
| `manual` | Web UI send: mode, channel or user, content length |
|
||||
| `error` | Scope and message |
|
||||
|
||||
Secret-looking strings are redacted before the write. Discord token shapes and `mfa.` strings
|
||||
|
||||
@@ -59,6 +59,11 @@ Event line:
|
||||
| `state.get` | none | The current user, the guilds, and the channels. Cache-only. Includes user avatars and guild icons. |
|
||||
| `voice.get` | optional `userId`, optional `guildId` | The voice channel and its members, or `channelId: null`. Without `guildId` it searches all guilds. |
|
||||
| `voice.overview` | `guildId` | The occupied voice channels in the guild and their members |
|
||||
| `voice.join` | `channelId`, optional `mute` (default true) | Joins the voice channel. Sets self-mute first. |
|
||||
| `voice.leave` | none | Leaves the current voice channel. |
|
||||
| `voice.move` | `channelId` | Moves the client to another voice channel. |
|
||||
| `voice.devices` | none | The client's input and output audio devices. |
|
||||
| `voice.setDevices` | optional `inputDeviceId`, optional `outputDeviceId` | Sets the client devices. |
|
||||
| `user.search` | `query`, optional `guildId`, optional `limit` | Users from the client cache. Guild members when `guildId` is set. |
|
||||
| `dm.list` | optional `limit` | Cached DM and group DM channels with recipients, icons, and avatars. Cache-only. |
|
||||
| `message.send` | `channelId`, `content` | `{ channelId, messageId, sent: true }` |
|
||||
@@ -77,6 +82,7 @@ The renderer normalizes client events and pushes them to subscribers.
|
||||
| `MESSAGE_REACTION_ADD` | Message ID, channel, guild, user, emoji |
|
||||
| `MESSAGE_REACTION_REMOVE` | Same fields as add |
|
||||
| `VOICE_STATE_UPDATE` | User, channel, guild, session, mute, deaf, self-mute, self-deaf |
|
||||
| `VOICE_SPEAKING` | User, speaking flag, channel |
|
||||
|
||||
## Validation rules
|
||||
|
||||
@@ -94,6 +100,12 @@ The core validates all input. The daemon cannot bypass these rules.
|
||||
to walk history.
|
||||
- `voice.get` accepts snowflakes only. It reads the client cache. It never joins a call.
|
||||
- `voice.overview` requires a guild snowflake. It reads the client cache.
|
||||
- `voice.join` and `voice.move` require a channel snowflake and a voice channel type (`2` or `13`).
|
||||
The join sets self-mute when the action is available. The internal lookups
|
||||
(`selectVoiceChannel`, `setSelfMute`) were verified live on 2026-09-13.
|
||||
- `voice.leave` takes no parameters. It disconnects the client.
|
||||
- `voice.setDevices` takes short device ID strings. The device list comes from
|
||||
`navigator.mediaDevices.enumerateDevices()`.
|
||||
- `user.search` takes a query of 1 to 100 characters and a limit of 1 to 25. It reads the client
|
||||
cache only. An exact snowflake query returns that user when cached.
|
||||
- `dm.list` takes a limit of 1 to 50. It reads `ChannelStore.getSortedPrivateChannels()` only.
|
||||
@@ -126,6 +138,7 @@ The core rate limit is a safety net. It does not replace the daemon pacing.
|
||||
| `NOT_SENDABLE` | The channel type cannot receive messages. |
|
||||
| `SEND_FAILED` | Discord rejected the message. |
|
||||
| `REACTION_FAILED` | Discord rejected the reaction. |
|
||||
| `VOICE_FAILED` | Discord rejected the voice action. |
|
||||
| `DM_UNAVAILABLE` | The client cannot create a DM channel. |
|
||||
| `NOT_SUPPORTED` | The client does not offer the internal action. |
|
||||
| `RATE_LIMITED` | The core rate limit rejected the action. |
|
||||
|
||||
@@ -61,6 +61,16 @@ The daemon exposes MCP over Streamable HTTP. The default address is `http://127.
|
||||
| `send_dm` | Send a DM to a user. |
|
||||
| `get_voice_state` | Which voice channel a user is in and who else is there. |
|
||||
| `get_voice_overview` | Occupied voice channels in a guild and their members. |
|
||||
| `voice_join` | Join a voice channel, muted by default. |
|
||||
| `voice_leave` | Leave the current voice channel. |
|
||||
| `voice_move` | Move to another voice channel. |
|
||||
| `get_audio_status` | Audio engine, TTS, queue, ducking, and capture state. |
|
||||
| `speak` | Queue text to speech into the virtual microphone. |
|
||||
| `stop_audio` | Stop speech and clear the queue. |
|
||||
| `start_capture` / `stop_capture` | Record or stop recording the client output sink. |
|
||||
| `list_audio_devices` | The client's input and output audio devices. |
|
||||
| `set_audio_devices` | Set the client's input and output devices. |
|
||||
| `get_transcript` | Read captured call segments with speaker, WAV path, and text. |
|
||||
| `search_users` | Search the client user cache by username, global name, or nickname. |
|
||||
| `list_dms` | List cached DM and group DM channels with recipients. |
|
||||
| `get_message_log` | Read the trigger log. |
|
||||
@@ -135,6 +145,7 @@ Verified live on 2026-09-13: the web UI started the client under auto-Xvfb, the
|
||||
| Setting | Default | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `VESKSLOP_SOCKET` | `$XDG_RUNTIME_DIR/vesktop-puppet.sock` | Core bridge socket |
|
||||
| `VESKSLOP_REQUEST_TIMEOUT_MS` | `30000` | How long a core request waits before it fails. |
|
||||
| `VESKSLOP_STATE_FILE` | `poc/.veskslop-state.json` | Owner, trust list, and config |
|
||||
| `VESKSLOP_ACTION_LOG` | `<state dir>/actions.jsonl` | Persistent action log |
|
||||
| `VESKSLOP_ACTION_LOG_MAX_BYTES` | 5 MiB | Action log rotation limit |
|
||||
@@ -142,9 +153,23 @@ Verified live on 2026-09-13: the web UI started the client under auto-Xvfb, the
|
||||
| `VESKSLOP_CACHE_MAX_BYTES` | 50 MiB | Context cache size limit |
|
||||
| `MCP_HOST` / `MCP_PORT` | `127.0.0.1` / `8788` | MCP listen address |
|
||||
| `WEBUI_HOST` / `WEBUI_PORT` | `0.0.0.0` / `8789` | Web UI listen address. No authentication. |
|
||||
| `WEBUI_TLS` / `WEBUI_TLS_KEY` / `WEBUI_TLS_CERT` | `1` / unset | Self-signed HTTPS, or an existing certificate. |
|
||||
| `VESKSLOP_CORE_CMD` | unset | Full command that starts the client. Overrides the default. |
|
||||
| `VESKSLOP_CORE_ARGS` | `--no-sandbox --disable-gpu --disable-dev-shm-usage` | Flags for the default Electron command. |
|
||||
| `VESKSLOP_XVFB` | unset | Set to `0` to disable automatic Xvfb start on Linux. |
|
||||
| `VESKSLOP_AUDIO_AUTOCONNECT` | `1` | Point the client at the virtual devices on connect. |
|
||||
| `VESKSLOP_AUDIO_MANAGE_SERVER` | `1` | Start `pulseaudio` when `pactl` cannot reach a server. |
|
||||
| `VESKSLOP_AUDIO_SINK` / `_SOURCE` / `_CAPTURE_SINK` | `veskslop_tts` / `veskslop_mic` / `veskslop_capture` | Virtual device names. |
|
||||
| `VESKSLOP_AUDIO_DUCK` | `30` | Sink volume percent while others speak. |
|
||||
| `VESKSLOP_FFPLAY` / `VESKSLOP_FFMPEG` | `ffplay` / `ffmpeg` | Playback and capture binaries. |
|
||||
| `VESKSLOP_TTS_CMD` | unset | Local TTS command with `{out}` and `{text_file}`. |
|
||||
| `VESKSLOP_TTS_API_URL` / `_API_KEY` / `_PROVIDER` | unset | HTTP TTS provider and bearer token. |
|
||||
| `VESKSLOP_TRANSCRIPT` | `<state dir>/transcripts.jsonl` | Transcript store path. |
|
||||
| `VESKSLOP_PREFERENCES` | `<state dir>/preferences.json` | Web UI preferences (visible guilds, TTS). |
|
||||
| `VESKSLOP_CAPTURE_AUTO` | `0` | Start capture when the bot joins a call. |
|
||||
| `VESKSLOP_VAD_THRESHOLD_DB` / `_MIN_SPEECH_MS` / `_SILENCE_MS` / `_PRE_ROLL_MS` / `_MAX_SEGMENT_MS` | `-35` / `120` / `600` / `300` / `30000` | VAD tuning. |
|
||||
| `VESKSLOP_STT_CMD` / `_API_URL` / `_API_KEY` | unset | Speech-to-text provider. |
|
||||
| `VESKSLOP_VOSK_MODEL` / `_VOSK_DIR` | `poc/models/vosk-model*` | Local Vosk model, auto-detected when missing. |
|
||||
|
||||
## How to add an MCP tool
|
||||
|
||||
|
||||
+3
-1
@@ -6,12 +6,14 @@ This document describes the test setup, the run commands, and the known gaps.
|
||||
|
||||
The pure daemon logic has unit tests. The tests cover the ring buffers, the mention helpers, the
|
||||
command parser, the poll parser, the trust helpers, the context assembly, the redaction, and the
|
||||
pacing policy (`planSend`). The cache tests cover merge, dedupe, trimming, and eviction.
|
||||
pacing policy (`planSend`). The cache tests cover merge, dedupe, trimming, and eviction. The audio
|
||||
tests cover the VAD, the WAV writer, the transcript store, and a local STT command.
|
||||
|
||||
```sh
|
||||
cd poc
|
||||
pnpm test:comm
|
||||
pnpm test:cache
|
||||
pnpm test:audio
|
||||
```
|
||||
|
||||
The files are `poc/src/test-communication.ts` and `poc/src/test-cache.ts`. The tests do not need
|
||||
|
||||
+128
-13
@@ -1,6 +1,7 @@
|
||||
# Voice
|
||||
|
||||
Voice state reads exist. Voice actions do not. This document records the plan and the decisions.
|
||||
Voice state reads and the basic actions exist. Auto-leave, audio, and video do not. This document
|
||||
records the state and the plan.
|
||||
|
||||
## Current
|
||||
|
||||
@@ -9,31 +10,145 @@ Voice state reads exist. Voice actions do not. This document records the plan an
|
||||
| `voice.get` | Implemented. One user's voice channel and its members. Without `guildId` it searches all guilds. |
|
||||
| `voice.overview` | Implemented. Occupied voice channels and members for one guild. |
|
||||
| `VOICE_STATE_UPDATE` events | Normalized and pushed. Mock-verified. Live check pending. |
|
||||
| Join and leave | Not started. |
|
||||
| Audio in and out | Not started. |
|
||||
| `voice.join` | Implemented. Joins muted. Internal lookups verified live. |
|
||||
| `voice.leave` | Implemented. Disconnects the client. |
|
||||
| `voice.move` | Implemented. Moves to another channel. |
|
||||
| Auto-leave | Not started. |
|
||||
| `stay @user` | Not started. |
|
||||
| Capture and transcript | Implemented. VAD segments, WAV per utterance, JSONL store, live events. |
|
||||
| Speech to text | Hook implemented (local command or HTTP API); provider not chosen yet. |
|
||||
| Audio in and out | Output and capture work. Browser mic puppeting is not started. |
|
||||
| Video | Not started. |
|
||||
|
||||
The core reads `VoiceStateStore` only. It never joins a call on its own.
|
||||
|
||||
## Planned actions
|
||||
## Actions
|
||||
|
||||
| Action | Parameters | Result |
|
||||
| --- | --- | --- |
|
||||
| `voice.join` | `channelId`, optional `mute` (default true) | `{ channelId, joined: true }` |
|
||||
| `voice.join` | `channelId`, optional `mute` (default true) | `{ channelId, joined: true, muted }` |
|
||||
| `voice.leave` | none | `{ left: true }` |
|
||||
| `voice.move` | `channelId` | `{ channelId, moved: true }` |
|
||||
| `voice.move` | `channelId` | `{ channelId, joined: true }` |
|
||||
|
||||
Rules:
|
||||
|
||||
- The channel must be a voice channel. The core validates the snowflake and the channel type.
|
||||
- Join joins the call muted. The bot does not capture or send audio.
|
||||
- Voice actions are state changes, not send floods. They stay out of the message pacing queue, but
|
||||
the daemon rate limits them separately.
|
||||
- The join implementation goes through Discord's internal channel action, located at runtime. The
|
||||
lookup is unverified until the first live run. This is the same pattern as `reaction.add`.
|
||||
- Voice actions are state changes, not send floods. They stay out of the message pacing queue. The
|
||||
daemon spaces them 250-750 ms (`pacedVoice`).
|
||||
- The join goes through Discord's internal channel action, located at runtime.
|
||||
`selectVoiceChannel` and `setSelfMute` were verified live on 2026-09-13. The end-to-end join was
|
||||
not run.
|
||||
|
||||
The plan for later audio: the bot unmutes only while it plays an audio stream, then mutes again.
|
||||
Audio playback is a later feature.
|
||||
|
||||
## Audio
|
||||
|
||||
Voice audio uses the system audio stack, not Discord internals. The daemon owns the wiring.
|
||||
|
||||
### Devices
|
||||
|
||||
| Device | Kind | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `veskslop_tts` | null sink | Playback target for TTS |
|
||||
| `veskslop_mic` | remap source of `veskslop_tts.monitor` | The client microphone |
|
||||
| `veskslop_capture` | null sink | Client output, recorded by the daemon |
|
||||
|
||||
The host needs a sound server. On Debian and Ubuntu:
|
||||
|
||||
```sh
|
||||
apt install pulseaudio pulseaudio-utils ffmpeg espeak-ng
|
||||
```
|
||||
|
||||
The daemon starts `pulseaudio` automatically when `pactl` cannot reach a server, unless
|
||||
`VESKSLOP_AUDIO_MANAGE_SERVER=0`. If `pactl` is missing, the status reports the install hint. The
|
||||
Web UI Audio section has a **Connect devices** button that rechecks the client device list.
|
||||
|
||||
On startup the daemon creates missing devices with `pactl` and unloads the modules it created on
|
||||
exit. When the client connects, the daemon finds the devices by label and sets them through
|
||||
`voice.setDevices`, unless `VESKSLOP_AUDIO_AUTOCONNECT=0`. Names are configurable with
|
||||
`VESKSLOP_AUDIO_SINK`, `VESKSLOP_AUDIO_SOURCE`, and `VESKSLOP_AUDIO_CAPTURE_SINK`.
|
||||
|
||||
If the stack is unavailable, playback/speak calls return the reason and the rest of the daemon
|
||||
works as before.
|
||||
|
||||
### Playback
|
||||
|
||||
- TTS has two providers:
|
||||
- **local**: `VESKSLOP_TTS_CMD` with `{out}` and `{text_file}` placeholders. Example:
|
||||
`piper --model sv_SE-nst-medium.onnx --output_file {out} -f {text_file}`.
|
||||
- **api**: `VESKSLOP_TTS_API_URL` receives a JSON body `{ text, voice? }` and returns audio
|
||||
bytes. `VESKSLOP_TTS_API_KEY` adds a bearer token.
|
||||
- `ffplay` decodes and plays into `veskslop_tts` (`PULSE_SINK`). `VESKSLOP_FFPLAY` overrides the
|
||||
binary. Lines play one at a time in order.
|
||||
|
||||
### Browser monitor (Client audio)
|
||||
|
||||
The header **Client audio** button streams the call to the browser:
|
||||
|
||||
- The daemon reads `veskslop_capture.monitor` with `ffmpeg` and sends raw 48 kHz mono PCM over
|
||||
the WebSocket at `/audio`.
|
||||
- The browser plays the PCM with the Web Audio API and can send it to a chosen output device.
|
||||
- This is separate from the bot's microphone and TTS. Nothing the browser plays affects the call.
|
||||
|
||||
Browser microphone puppeting (talking through the bot) is not implemented. It needs a mix sink so
|
||||
the browser mic and TTS share `veskslop_mic`, a `pacat` input, and echo handling.
|
||||
|
||||
### Ducking
|
||||
|
||||
When another person speaks, the daemon lowers the `veskslop_tts` sink volume to
|
||||
`VESKSLOP_AUDIO_DUCK` percent (default 30) and restores it when they stop. Speaking comes from the
|
||||
`VOICE_SPEAKING` event. Staying quiet while someone else talks is a queue property, not a switch.
|
||||
Yield (stop mid-line) is not implemented.
|
||||
|
||||
### Capture and transcript
|
||||
|
||||
The daemon records the client output and splits it into utterances.
|
||||
|
||||
1. `ffmpeg` reads `veskslop_capture.monitor` as 16 kHz mono PCM (`VESKSLOP_FFMPEG` overrides).
|
||||
2. The VAD opens a segment after `VESKSLOP_VAD_MIN_SPEECH_MS` above `VESKSLOP_VAD_THRESHOLD_DB`,
|
||||
keeps `VESKSLOP_VAD_PRE_ROLL_MS` of audio from before, and closes after
|
||||
`VESKSLOP_VAD_SILENCE_MS` of quiet.
|
||||
3. Each segment becomes a WAV under `<cache>/audio/captures/` and a transcript entry in
|
||||
`VESKSLOP_TRANSCRIPT` (default `<state dir>/transcripts.jsonl`).
|
||||
4. When `VOICE_SPEAKING` covers the segment, the dominant speaker is attached.
|
||||
5. When STT is configured, the segment is transcribed and the text is attached.
|
||||
|
||||
STT providers:
|
||||
|
||||
- **Vosk (default)**: CPU, small and light. Install it and drop a model in `poc/models/`
|
||||
(auto-detected), or set `VESKSLOP_VOSK_MODEL`:
|
||||
```sh
|
||||
pip install vosk
|
||||
curl -LO https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip
|
||||
unzip vosk-model-small-en-us-0.15.zip -d poc/models
|
||||
```
|
||||
The daemon runs `poc/scripts/stt-vosk.py` for each segment. Quality is robotic but light.
|
||||
- **local command**: `VESKSLOP_STT_CMD` with `{wav}`; the command prints text on stdout.
|
||||
- **api**: `VESKSLOP_STT_API_URL` receives WAV bytes and returns text or JSON `{ text }`.
|
||||
`VESKSLOP_STT_API_KEY` adds a bearer token.
|
||||
|
||||
TTS engine and voice are chosen in Settings → Client → Speech (stored in the daemon preferences).
|
||||
The local engine defaults to `espeak-ng` when installed; the voice list comes from
|
||||
`espeak-ng --voices`.
|
||||
|
||||
Start and stop with `POST /api/audio/capture`, the Audio settings button, or the MCP tools. Set
|
||||
`VESKSLOP_CAPTURE_AUTO=1` to start capture when the bot joins a call and stop it on leave.
|
||||
|
||||
Transcript events go over the SSE stream: `TRANSCRIPT_SESSION`, `TRANSCRIPT_SEGMENT`,
|
||||
`TRANSCRIPT_TEXT`, `TRANSCRIPT_SESSION_END`, and `TRANSCRIPT_ERROR`. The agent console's
|
||||
Transcript tab shows the live list; `GET /api/transcript` and the `get_transcript` MCP tool read
|
||||
the store. Segments without text still carry the WAV path, so an LLM can transcribe them later.
|
||||
|
||||
### Controls
|
||||
|
||||
- Chat: `speak <text>` and `speak stop` from a trusted user.
|
||||
- MCP: `speak`, `stop_audio`, `get_audio_status`, `start_capture`, `stop_capture`,
|
||||
`list_audio_devices`, `set_audio_devices`, `get_transcript`.
|
||||
- HTTP: `/api/audio`, `/api/audio/speak`, `/api/audio/stop`, `/api/audio/capture`,
|
||||
`/api/audio/devices`.
|
||||
- Web UI: the TTS panel (provider, Speak, Stop) and the Audio settings section.
|
||||
|
||||
|
||||
## Lifecycle
|
||||
|
||||
@@ -60,8 +175,8 @@ when the puppet cannot see a user's voice channel or the channel members. It sta
|
||||
|
||||
A live run must confirm these points.
|
||||
|
||||
1. The internal `selectVoiceChannel` lookup exists and joins the given channel.
|
||||
2. The internal disconnect action leaves the call.
|
||||
1. `voice.join` joins the given channel end to end. (The `selectVoiceChannel` lookup is verified.)
|
||||
2. `voice.leave` disconnects the call.
|
||||
3. `VOICE_STATE_UPDATE` carries the fields that the core normalizes.
|
||||
4. The join is muted and Discord shows the mute state.
|
||||
5. The auto-leave timer works across a client restart. The timer is daemon state, not core state.
|
||||
|
||||
+78
-12
@@ -32,6 +32,19 @@ only. Do not expose the port to the public internet. See [daemon.md](daemon.md).
|
||||
|
||||
The MCP server stays on `127.0.0.1:8788`. Only the UI binds outside localhost.
|
||||
|
||||
## TLS
|
||||
|
||||
The UI serves HTTPS with a self-signed certificate so browser media APIs have a secure context.
|
||||
The daemon generates the certificate with `openssl` on first run (localhost plus the host's LAN
|
||||
addresses) and stores it under `<state dir>/tls/`. The browser shows a warning once; accept it.
|
||||
|
||||
| Setting | Default | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `WEBUI_TLS` | `1` | Set to `0` to serve plain HTTP. |
|
||||
| `WEBUI_TLS_KEY` / `WEBUI_TLS_CERT` | unset | Use an existing certificate instead. |
|
||||
|
||||
With plain HTTP, `navigator.mediaDevices` only exists on localhost; remote browsers must use HTTPS.
|
||||
|
||||
## Page
|
||||
|
||||
- Plain HTML, CSS, and JavaScript in `poc/public/`. No build step and no frontend framework.
|
||||
@@ -39,12 +52,21 @@ The MCP server stays on `127.0.0.1:8788`. Only the UI binds outside localhost.
|
||||
in `poc/WebUIMockup.zip`. Implemented screen: **7a**, the docked three-column debug shell.
|
||||
- Layout: 50px header, conditional call strip, `236px | 1fr | 306px` body, and the agent console
|
||||
bar (35px collapsed, resizable drawer expanded).
|
||||
- Header: wordmark, guild picker, health indicators, Takeover switch, Settings.
|
||||
- Left rail: guild switcher, text channels, voice channels with inline participants, direct
|
||||
messages, and the account footer.
|
||||
- Center: channel header, message list, composer, and the TTS panel.
|
||||
- Right column: In voice, Favourites, Open DMs & groups.
|
||||
- Console: Tool calls, Events (live SSE), Transcript. The ticker shows the newest event.
|
||||
- Header: wordmark, **Guild / DM** scope tabs, guild picker, health indicators, **Client audio**,
|
||||
Takeover switch, Settings. While the bridge is down a **Start server** button appears with a
|
||||
pulsing bloom; if the whole daemon is unreachable it becomes **Refresh** and reloads the page.
|
||||
The body dims and stops taking input while the bridge is down.
|
||||
- Left rail (Guild tab): guild switcher, text channels, voice channels with inline participants,
|
||||
and **People** — users recently seen in a voice channel (click opens their cached DM). Clicking a
|
||||
voice channel joins it (muted); clicking the active channel leaves.
|
||||
- Left rail (DM tab): every cached DM and group with unread badge, avatar, name, last-message time,
|
||||
and a favourite star.
|
||||
- Right column: **In voice** (favourited users in a call, grouped by call), **Favourites**, and
|
||||
**Open DMs & groups** as quick access. The guild's own occupied channels live in the left rail.
|
||||
- Speaking users show animated level bars and a bold name (design 2c: audio → video → share marks).
|
||||
Nicknames come from the guild member store; DMs fall back to the username.
|
||||
- Console: Tool calls, Events (live SSE), Transcript. The ticker shows the newest event. The
|
||||
Transcript tab lists captured segments (time, speaker, text, WAV) and updates live.
|
||||
- Opening a chat fetches the latest 50 messages from Discord into the client cache. The refresh
|
||||
button by the channel header re-fetches. Live message events read the cache.
|
||||
- Attachments render from their Discord CDN links. Images display inline; other files are link
|
||||
@@ -53,9 +75,23 @@ The MCP server stays on `127.0.0.1:8788`. Only the UI binds outside localhost.
|
||||
- Composer and TTS drafts persist in `localStorage` per channel. They survive re-renders, the
|
||||
bridge connecting, and a page reload. A successful send clears the channel draft.
|
||||
- When the bridge connects after the page loaded, the page reloads state without touching drafts.
|
||||
The reload retries while the client boots, and the health poll keeps retrying if the first
|
||||
attempt failed.
|
||||
- Roadmap gaps are shown, not hidden: TTS, transcript, tool-call tracing, favourites, and voice
|
||||
join/monitor appear as marked placeholders.
|
||||
|
||||
## DMs and favourites
|
||||
|
||||
- The **DM** scope tab shows all cached DMs in the rail and the conversation in the centre.
|
||||
- Unread badges sit left of the avatar: purple for unread, orange for mentions while Takeover is
|
||||
on; while Takeover is off they are grey, meaning the bot saw them and you did not.
|
||||
- The hollow star on a row adds it to favourites, stored in the daemon preferences. Starring a 1:1
|
||||
DM favourites that user; starring a group DM favourites the channel.
|
||||
- Favourited users get presence tracking. If they are in a call, they appear under **In voice**,
|
||||
grouped per call. The right column's **Favourites** shows their status when they are not.
|
||||
- The last-message time comes from the channel's last message snowflake: a time today, `yesterday`,
|
||||
or a short date.
|
||||
|
||||
## Takeover
|
||||
|
||||
Takeover is a read-only switch. Off, the composer is locked (a `read-only` tag sits next to the
|
||||
@@ -65,12 +101,29 @@ only; the API itself does not enforce it.
|
||||
|
||||
## Settings
|
||||
|
||||
The Settings button opens a slide-over with the server-side controls:
|
||||
The Settings button opens a slide-over:
|
||||
|
||||
- **Visible guilds** — top level. Checkboxes filter the guild switcher and picker live. **Save**
|
||||
writes the selection to the daemon preferences (`/api/preferences`); **Hide all** and **Show all**
|
||||
select none or every guild. All visible is stored as `null`, none as `[]`. All browsers share
|
||||
the saved state, and Save works while the client is disconnected.
|
||||
- **Client** — a collapsible group with:
|
||||
- **Bot audio** — engine status, TTS provider state, a **Connect devices** button, and the
|
||||
capture toggle. These are the daemon host's devices.
|
||||
- **Speech** — the TTS engine (Auto / Local / API), language, and voice. Voices are grouped by
|
||||
base language and filtered by the chosen language; English, Swedish, German, Danish,
|
||||
Norwegian, Russian, and Cherokee are pinned on top. Stored in the daemon preferences.
|
||||
- **Display** — the timestamp timezone. Defaults to the browser's zone
|
||||
(`Intl.DateTimeFormat().resolvedOptions().timeZone`); pick any IANA zone to override. Stored in
|
||||
`localStorage` and applied to messages, the transcript, the event log, the action log, and the
|
||||
core status.
|
||||
- **Browser audio** — puppeting controls: the browser output device for Client audio, a
|
||||
placeholder mic select, and the call-audio state. **Enable browser audio** asks for the
|
||||
microphone permission, which browsers require before they list output devices. Separate from
|
||||
the bot's devices.
|
||||
- **Server** — a collapsible group with:
|
||||
- **Core process** — state, last exit, output, Start / Stop / Restart.
|
||||
- **Account login** — token file state, QR login, captcha.
|
||||
- **Visible guilds** — checkboxes for the guild switcher and picker. Stored in `localStorage`;
|
||||
"Show all" clears the filter.
|
||||
- **Action log** — the persistent JSONL log, newest first.
|
||||
|
||||
## Core process
|
||||
@@ -126,8 +179,21 @@ replaceable.
|
||||
| GET | `/api/dms` | `limit?` | Cached DM and group DM channels with recipients |
|
||||
| GET | `/api/voice` | `userId?`, `guildId?` | One user's voice state |
|
||||
| GET | `/api/voice/overview` | `guildId` | Occupied voice channels and members |
|
||||
| POST | `/api/voice/join` | `channelId` | Reserved. Returns `501` |
|
||||
| POST | `/api/voice/leave` | none | Reserved. Returns `501` |
|
||||
| POST | `/api/voice/join` | `channelId`, `mute?` | Join a voice channel (muted by default) |
|
||||
| POST | `/api/voice/leave` | none | Leave the current voice channel |
|
||||
| POST | `/api/voice/move` | `channelId` | Move to another voice channel |
|
||||
| GET | `/api/preferences` | none | Web UI preferences (`visibleGuilds`, `tts`) |
|
||||
| POST | `/api/preferences` | `visibleGuilds?`, `tts?` | Update preferences. Stored in `<state dir>/preferences.json`. |
|
||||
| GET | `/api/audio` | none | Audio engine, TTS, queue, ducking, capture state |
|
||||
| GET | `/api/audio/devices` | none | The client's input and output devices |
|
||||
| GET | `/api/audio/tts/voices` | none | Local TTS voices (espeak-ng) |
|
||||
| POST | `/api/audio/connect` | none | Re-apply the virtual devices to the client |
|
||||
| WS | `/audio` | `{"type":"listen","on":bool}` | Raw 48 kHz mono PCM of the call |
|
||||
| POST | `/api/audio/devices` | `inputDeviceId?`, `outputDeviceId?` | Set the client's devices |
|
||||
| POST | `/api/audio/speak` | `text`, `provider?`, `voice?` | Queue text to speech |
|
||||
| POST | `/api/audio/stop` | none | Stop speech and clear the queue |
|
||||
| POST | `/api/audio/capture` | `start` | Start or stop capture |
|
||||
| GET | `/api/transcript` | `limit?`, `sessionId?` | Sessions and captured segments |
|
||||
| GET | `/api/logs` | `limit` | The persistent action log |
|
||||
| GET | `/api/events` | none | Server-sent events |
|
||||
|
||||
@@ -140,7 +206,7 @@ gap. See [daemon.md](daemon.md).
|
||||
## Not started
|
||||
|
||||
- Web UI authentication and user accounts.
|
||||
- Voice join and leave. The UI elements exist and call the reserved routes.
|
||||
- Voice auto-leave and the `stay @user` command.
|
||||
- TTS playback, transcript capture, and MCP tool-call tracing.
|
||||
- Favourites, presence, member counts, and message editing/deletion/reactions.
|
||||
- Pagination of channel history.
|
||||
|
||||
Reference in New Issue
Block a user