Update docs and README for voice audio, capture, and the web UI
This commit is contained in:
@@ -36,7 +36,7 @@
|
||||
|
||||
## 🚀 What is Veskslop
|
||||
|
||||
Veskslop takes the beloved Vesktop base and deletes the part where a person has to click things. What remains is a **headless Discord client** driven through a **validated capability layer**, so an **AI agent can run a real Discord account over MCP**. Text automation works today. Audio pipes and video streams are the next descent into madness.
|
||||
Veskslop takes the beloved Vesktop base and deletes the part where a person has to click things. What remains is a **headless Discord client** driven through a **validated capability layer**, so an **AI agent can run a real Discord account over MCP**. Text automation and the first audio pipes work today. Video streams are the next descent into madness.
|
||||
|
||||
It is Discord, but as infrastructure. It is the account you already have, plus `curl`, minus dignity.
|
||||
|
||||
@@ -45,11 +45,11 @@ Think of it as:
|
||||
- 🖥️ **A Discord client with the client removed.** Runs on a server, in a container, on a Raspberry Pi, in your weird home lab, wherever.
|
||||
- 🧠 **An agent body (text tools live).** Your MCP-capable AI gets a real account with real permissions and real consequences.
|
||||
- 🔀 **A proxy (planned).** Messages, events, voice state, attachments — everything forwarded, interceptable, scriptable, and API-shaped.
|
||||
- 🔊 **An audio router (planned).** Play anything into voice. Capture anything out. TTS in, STT out, pipes for days.
|
||||
- 🔊 **An audio router (partly live).** TTS into the call, ducking, browser monitoring, and captured call audio with Vosk STT. Browser microphone puppeting is next.
|
||||
- 📹 **A streaming endpoint (planned).** Push files, cameras, generative visuals, or pure slop into a voice channel.
|
||||
- 🧪 **A science experiment that will absolutely not get your account banned** *(allegedly)*.
|
||||
|
||||
> **Status: text layer works.** The real Vesktop client runs headless and is driven through a validated capability layer. Triggers, trust tiers, conversation context, response policy, persistent logging, MCP tools, user search, voice state reads, and the Nocturne-style debug web UI are live. Reaction polls are paused. Voice actions, audio, and video are still the vision. The design docs live in [docs/INDEX.md](docs/INDEX.md).
|
||||
> **Status: text and voice basics work.** The real Vesktop client runs headless and is driven through a validated capability layer. Triggers, trust tiers, conversation context, response policy, persistent logging, MCP tools, user search, voice state, voice join/leave/move, a virtual-mic TTS pipeline with ducking, browser call monitoring, VAD-segmented call capture with Vosk STT, core process control, web QR login, and the Nocturne-style debug web UI over self-signed HTTPS are live. Reaction polls are paused. Auto-leave, browser microphone puppeting, and video are still ahead. The design docs live in [docs/INDEX.md](docs/INDEX.md).
|
||||
|
||||
## 🗿 Why though
|
||||
|
||||
@@ -74,7 +74,11 @@ Veskslop ships an **MCP server** so any MCP-capable agent (Claude, GPT-based age
|
||||
| `get_state` | User, guilds, and channels. Cache-only, zero queries. |
|
||||
| `get_voice_state` | Which voice channel a user is in and who else is suffering there. |
|
||||
| `get_voice_overview` | Occupied voice channels in a guild and their members. |
|
||||
| `voice_join` / `voice_leave` / `voice_move` | Join muted, leave, or move the account's call. |
|
||||
| `get_audio_status` / `speak` / `stop_audio` | Virtual-mic TTS state, queue text, stop playback. |
|
||||
| `start_capture` / `stop_capture` / `get_transcript` | Segment call audio, stop, read the transcript. |
|
||||
| `search_users` | Find a user ID from a username, nickname, or cached profile. |
|
||||
| `list_dms` | Cached DMs and group DMs with recipients, unread counts, and last-message time. |
|
||||
| `send_message` | Send a message as the account. Send-paced. |
|
||||
| `read_messages` | Read the client's message cache. |
|
||||
| `send_dm` | Slide into someone's DMs. |
|
||||
@@ -85,10 +89,11 @@ Veskslop ships an **MCP server** so any MCP-capable agent (Claude, GPT-based age
|
||||
| `whitelist_list` / `whitelist_add` / `whitelist_remove` | Trust tiers: user, trusted, admin. |
|
||||
| `run_commands` | Run several read-only tools in one call. |
|
||||
|
||||
There is also a small web control UI on `http://<host>:8789/` with a JSON API and an SSE event
|
||||
stream. See [docs/webui.md](docs/webui.md).
|
||||
There is also a debug web UI on `https://<host>:8789/` (self-signed certificate) with a JSON API,
|
||||
an SSE event stream, a DM/favourites tab, and browser call audio. See [docs/webui.md](docs/webui.md).
|
||||
|
||||
**Planned**: voice join/leave, audio play/capture, video streams, message edit/delete/upload, presence and voice state control.
|
||||
**Planned**: voice auto-leave and `stay @user`, browser microphone puppeting, video streams, message
|
||||
edit/delete/upload, notification parsing, presence control.
|
||||
|
||||
Every action is **safe by default**: strict input validation in the client, structured errors, a hard outbound rate limit, and paced sends. A buggy agent cannot crash the client or spam Discord.
|
||||
|
||||
@@ -210,17 +215,19 @@ The full setup, including the headless client and the mock core, is in
|
||||
- [x] Response policy, hard rate limits, persistent action log
|
||||
- [x] MCP server with the live text tool surface
|
||||
- [x] User search and voice overview
|
||||
- [x] Nocturne debug web UI (7a shell, Takeover read-only gate, Settings)
|
||||
- [x] Voice join/leave/move (joins muted) over the validated core bridge
|
||||
- [x] Nocturne debug web UI (7a shell, DM tab, favourites, speaking marks, HTTPS, process control)
|
||||
- [x] Outbound audio pipeline (virtual mic, TTS local/API, ducking, browser monitoring)
|
||||
- [x] Inbound audio capture pipeline (VAD segmentation, WAV, transcript store, Vosk STT)
|
||||
- [~] Reaction polls and chat command help (paused)
|
||||
- [ ] Browser microphone puppeting (browser mic into the bot's voice)
|
||||
- [ ] Voice auto-leave and the `stay @user` command
|
||||
- [ ] Invite-link joins
|
||||
- [ ] Notification parsing: edits, deletes, replies, references
|
||||
- [ ] Reply-to-puppet as a trigger
|
||||
- [ ] Per-user rate limits and command cooldowns
|
||||
- [ ] Voice join/leave and invite-link joins
|
||||
- [ ] Voice auto-leave and the `stay @user` command
|
||||
- [ ] Collector account for voice membership fallback
|
||||
- [~] MCP event delivery (parked)
|
||||
- [ ] Outbound audio pipeline
|
||||
- [ ] Inbound audio capture pipeline
|
||||
- [ ] Video stream pipeline
|
||||
- [ ] Proxy mode + event bus
|
||||
- [ ] Container image (`veskslop:slop-latest`)
|
||||
|
||||
Reference in New Issue
Block a user