Update docs and README for voice audio, capture, and the web UI

This commit is contained in:
Fisto
2026-09-14 11:19:11 +02:00
parent c003d9532f
commit c7aad7547f
10 changed files with 356 additions and 66 deletions
+18 -11
View File
@@ -36,7 +36,7 @@
## 🚀 What is Veskslop
Veskslop takes the beloved Vesktop base and deletes the part where a person has to click things. What remains is a **headless Discord client** driven through a **validated capability layer**, so an **AI agent can run a real Discord account over MCP**. Text automation works today. Audio pipes and video streams are the next descent into madness.
Veskslop takes the beloved Vesktop base and deletes the part where a person has to click things. What remains is a **headless Discord client** driven through a **validated capability layer**, so an **AI agent can run a real Discord account over MCP**. Text automation and the first audio pipes work today. Video streams are the next descent into madness.
It is Discord, but as infrastructure. It is the account you already have, plus `curl`, minus dignity.
@@ -45,11 +45,11 @@ Think of it as:
- 🖥️ **A Discord client with the client removed.** Runs on a server, in a container, on a Raspberry Pi, in your weird home lab, wherever.
- 🧠 **An agent body (text tools live).** Your MCP-capable AI gets a real account with real permissions and real consequences.
- 🔀 **A proxy (planned).** Messages, events, voice state, attachments — everything forwarded, interceptable, scriptable, and API-shaped.
- 🔊 **An audio router (planned).** Play anything into voice. Capture anything out. TTS in, STT out, pipes for days.
- 🔊 **An audio router (partly live).** TTS into the call, ducking, browser monitoring, and captured call audio with Vosk STT. Browser microphone puppeting is next.
- 📹 **A streaming endpoint (planned).** Push files, cameras, generative visuals, or pure slop into a voice channel.
- 🧪 **A science experiment that will absolutely not get your account banned** *(allegedly)*.
> **Status: text layer works.** The real Vesktop client runs headless and is driven through a validated capability layer. Triggers, trust tiers, conversation context, response policy, persistent logging, MCP tools, user search, voice state reads, and the Nocturne-style debug web UI are live. Reaction polls are paused. Voice actions, audio, and video are still the vision. The design docs live in [docs/INDEX.md](docs/INDEX.md).
> **Status: text and voice basics work.** The real Vesktop client runs headless and is driven through a validated capability layer. Triggers, trust tiers, conversation context, response policy, persistent logging, MCP tools, user search, voice state, voice join/leave/move, a virtual-mic TTS pipeline with ducking, browser call monitoring, VAD-segmented call capture with Vosk STT, core process control, web QR login, and the Nocturne-style debug web UI over self-signed HTTPS are live. Reaction polls are paused. Auto-leave, browser microphone puppeting, and video are still ahead. The design docs live in [docs/INDEX.md](docs/INDEX.md).
## 🗿 Why though
@@ -74,7 +74,11 @@ Veskslop ships an **MCP server** so any MCP-capable agent (Claude, GPT-based age
| `get_state` | User, guilds, and channels. Cache-only, zero queries. |
| `get_voice_state` | Which voice channel a user is in and who else is suffering there. |
| `get_voice_overview` | Occupied voice channels in a guild and their members. |
| `voice_join` / `voice_leave` / `voice_move` | Join muted, leave, or move the account's call. |
| `get_audio_status` / `speak` / `stop_audio` | Virtual-mic TTS state, queue text, stop playback. |
| `start_capture` / `stop_capture` / `get_transcript` | Segment call audio, stop, read the transcript. |
| `search_users` | Find a user ID from a username, nickname, or cached profile. |
| `list_dms` | Cached DMs and group DMs with recipients, unread counts, and last-message time. |
| `send_message` | Send a message as the account. Send-paced. |
| `read_messages` | Read the client's message cache. |
| `send_dm` | Slide into someone's DMs. |
@@ -85,10 +89,11 @@ Veskslop ships an **MCP server** so any MCP-capable agent (Claude, GPT-based age
| `whitelist_list` / `whitelist_add` / `whitelist_remove` | Trust tiers: user, trusted, admin. |
| `run_commands` | Run several read-only tools in one call. |
There is also a small web control UI on `http://<host>:8789/` with a JSON API and an SSE event
stream. See [docs/webui.md](docs/webui.md).
There is also a debug web UI on `https://<host>:8789/` (self-signed certificate) with a JSON API,
an SSE event stream, a DM/favourites tab, and browser call audio. See [docs/webui.md](docs/webui.md).
**Planned**: voice join/leave, audio play/capture, video streams, message edit/delete/upload, presence and voice state control.
**Planned**: voice auto-leave and `stay @user`, browser microphone puppeting, video streams, message
edit/delete/upload, notification parsing, presence control.
Every action is **safe by default**: strict input validation in the client, structured errors, a hard outbound rate limit, and paced sends. A buggy agent cannot crash the client or spam Discord.
@@ -210,17 +215,19 @@ The full setup, including the headless client and the mock core, is in
- [x] Response policy, hard rate limits, persistent action log
- [x] MCP server with the live text tool surface
- [x] User search and voice overview
- [x] Nocturne debug web UI (7a shell, Takeover read-only gate, Settings)
- [x] Voice join/leave/move (joins muted) over the validated core bridge
- [x] Nocturne debug web UI (7a shell, DM tab, favourites, speaking marks, HTTPS, process control)
- [x] Outbound audio pipeline (virtual mic, TTS local/API, ducking, browser monitoring)
- [x] Inbound audio capture pipeline (VAD segmentation, WAV, transcript store, Vosk STT)
- [~] Reaction polls and chat command help (paused)
- [ ] Browser microphone puppeting (browser mic into the bot's voice)
- [ ] Voice auto-leave and the `stay @user` command
- [ ] Invite-link joins
- [ ] Notification parsing: edits, deletes, replies, references
- [ ] Reply-to-puppet as a trigger
- [ ] Per-user rate limits and command cooldowns
- [ ] Voice join/leave and invite-link joins
- [ ] Voice auto-leave and the `stay @user` command
- [ ] Collector account for voice membership fallback
- [~] MCP event delivery (parked)
- [ ] Outbound audio pipeline
- [ ] Inbound audio capture pipeline
- [ ] Video stream pipeline
- [ ] Proxy mode + event bus
- [ ] Container image (`veskslop:slop-latest`)