zib/Veskslop-t
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update docs for the debug web UI, voice plan, and current status

Browse Source
This commit is contained in:
Fisto
2026-09-13 15:47:51 +02:00
parent 2bcd8c6d7a
commit 65f0788502
14 changed files with 594 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+20 -12
View File
@@ -14,6 +14,8 @@ before you stop.
- [architecture.md](docs/architecture.md) — system parts and boundaries - [architecture.md](docs/architecture.md) — system parts and boundaries
- [core-bridge.md](docs/core-bridge.md) — the Vesktop-side capability layer - [core-bridge.md](docs/core-bridge.md) — the Vesktop-side capability layer
- [daemon.md](docs/daemon.md) — the external MCP daemon - [daemon.md](docs/daemon.md) — the external MCP daemon
- [webui.md](docs/webui.md) — the web control UI and its JSON API
- [voice.md](docs/voice.md) — voice state reads, planned actions, and lifecycle
- [communication.md](docs/communication.md) — trust model, commands, trigger log - [communication.md](docs/communication.md) — trust model, commands, trigger log
- [auth.md](docs/auth.md) — QR login and token persistence - [auth.md](docs/auth.md) — QR login and token persistence
- [testing.md](docs/testing.md) — tests and live test setup - [testing.md](docs/testing.md) — tests and live test setup
@@ -29,8 +31,9 @@ before you stop.
`src/main/puppetBridge.ts`. Do not spread logic through upstream files. `src/main/puppetBridge.ts`. Do not spread logic through upstream files.
- The daemon holds the logic: triggers, trust list, command parsing, trigger log, and pacing. - The daemon holds the logic: triggers, trust list, command parsing, trigger log, and pacing.
- The daemon binds to localhost. Remote access goes over LAN or Tailscale only. - The daemon binds to localhost. Remote access goes over LAN or Tailscale only.
- Pacing: the daemon delays every command by 500 to 2800 ms. The core adds a hard limit of 500 ms - Pacing: the daemon sends the first action of a burst in 30-200 ms, then spaces sends 500-2800 ms.
between sends and 30 sends per minute. A burst resets after a random 5-20 s of quiet. The core adds a hard limit of 500 ms between sends
and 30 sends per minute.
## Guardrails ## Guardrails
@@ -50,13 +53,18 @@ before you stop.
| Main bridge, event push, hooks | Implemented | | Main bridge, event push, hooks | Implemented |
| Daemon on the core-bridge transport | Implemented and verified live | | Daemon on the core-bridge transport | Implemented and verified live |
| QR login, pacing, trigger log | Implemented | | QR login, pacing, trigger log | Implemented |
| Human pacing (fast first send, burst, variable reset) | Implemented |
| Trusted-only triggers, trust tiers | Implemented | | Trusted-only triggers, trust tiers | Implemented |
| Context cache, fetch cooldown, continuation | Implemented | | Context cache, fetch cooldown, continuation | Implemented |
| Response policy (channel or DM) | Implemented | | Response policy (channel or DM) | Implemented |
| Chat commands (`poll`) | Implemented | | Chat commands (`whitelist`) | Implemented. `poll` is paused. |
| Persistent action log | Implemented | | Persistent action log | Implemented |
| Voice state read | Implemented | | Web UI and JSON API | Nocturne 7a debug shell + roadmap placeholders. No auth; LAN or Tailscale only. |
| Voice actions, notification parsing | Not started | | Core process control and web QR login | Implemented. Core start/stop verified live. |
| User search (cache and guild members) | Implemented |
| Voice state read and `VOICE_STATE_UPDATE` events | Implemented |
| Voice actions, audio, video | Not started. Plan in [docs/voice.md](docs/voice.md). |
| Notification parsing | Not started |
| Auth persistence in the core | Implemented and verified live | | Auth persistence in the core | Implemented and verified live |
| CDP transport | Kept as a development fallback | | CDP transport | Kept as a development fallback |
@@ -64,15 +72,15 @@ before you stop.
See [CURRENT.md](CURRENT.md) and [TODO.md](TODO.md) for the live list. The main items are: See [CURRENT.md](CURRENT.md) and [TODO.md](TODO.md) for the live list. The main items are:
1. Notification parsing: edits, deletes, reactions, and replies. 1. Voice backend behind the Web UI Join/Leave buttons.
2. Reply-to-puppet as a trigger. 2. Voice: `voice.join`, `voice.leave`, `voice.move`, auto-leave, and the `stay @user` command.
3. Per-user rate limit and command cooldowns. 3. Notification parsing: edits, deletes, reactions, and replies.
4. Setup command with reaction-based multiple choice. 4. Reply-to-puppet as a trigger.
5. MCP event delivery decision. 5. Per-user rate limit and command cooldowns.
6. Collector account with read-only voice membership. 6. Collector account with read-only voice membership.
Open decisions wait in [CURRENT.md](CURRENT.md) → Open discussions: MCP event delivery, untrusted Paused: the `poll` command and the setup command flow. Parked: MCP event delivery and the LLM
logging, the privacy pass, the setup command flow, and the first live reaction check. attachment. Open decisions wait in [CURRENT.md](CURRENT.md) → Open discussions.
## Resume checklist ## Resume checklist
CURRENT.md
+55 -28
View File
@@ -1,17 +1,26 @@
# CURRENT.md — session state # CURRENT.md — session state
Last updated: 2026-09-11 Last updated: 2026-09-13
This file tracks the work state. Read it first. Update it before you stop. This file tracks the work state. Read it first. Update it before you stop.
## Current focus (broad) ## Current focus (broad)
Build the text-based communication layer: trusted tiers, context, response routing, MCP tools, and The text layer is done. The web UI follows the Nocturne design handoff. The voice backend is the
persistent logging. Voice, audio, and video stay deferred. next feature; live verification and a full QR approval are pending. The LLM attachment and MCP
event delivery are parked.
## In progress ## In progress
- None. The documentation pass is complete. The project is ready to resume. - The web UI follows the Nocturne design handoff (`poc/WebUIMockup.zip`). The 7a shell is
implemented and rendered headless for review: header, call strip, three-column body, composer,
and the agent console.
- Takeover is a client-side read-only gate. Off, the composer is locked. Manual sends are logged.
- Core process control is implemented and verified live: the daemon starts the client under
auto-Xvfb, the bridge connects, and stop exits cleanly.
- Web QR login is implemented. The session reached `pending` against the real gateway; a full
phone approval is pending.
- Voice actions are designed in [docs/voice.md](docs/voice.md). The backend is not started.
## Done ## Done
@@ -24,38 +33,54 @@ persistent logging. Voice, audio, and video stay deferred.
- Continuation window: a rushed follow-up joins the request context. - Continuation window: a rushed follow-up joins the request context.
- Response policy: origin channel or DM. Sensitive commands default to DM. - Response policy: origin channel or DM. Sensitive commands default to DM.
- Persistent action log: triggers, commands, replies, errors, rotation, and secret redaction. - Persistent action log: triggers, commands, replies, errors, rotation, and secret redaction.
- Chat command: `poll` with number-emoji reactions, one vote per user, and auto results. - `poll` is paused (`POLL_ENABLED = false`). The parser and its tests stay.
- MCP tools: context, config, voice state, action log, and a read-only batch tool. - Human send pacing: first send of a burst waits 30-200 ms, later sends 500-2800 ms, and the burst
- Event processing is serialized. Replayed events are dropped. resets after a random 5-20 s of quiet. Reads take 100-300 ms and do not reset the burst.
- Unit tests for the pure logic and the cache. Mock end-to-end tests pass. - `user.search`: cache and guild-member lookup. `voice.get` searches all guilds without a `guildId`.
- Live verification: state, DM send, and a synthetic trigger through the real event path. - `voice.overview`: occupied voice channels and members for a guild.
- `VOICE_STATE_UPDATE` events: normalized and pushed. Mock-verified.
- Web UI: Nocturne design tokens in `poc/public/nocturne.css`, the 7a debug shell, live guild and
channel rail, messages, DMs, voice membership, agent console with a live event tab, and a
Settings slide-over for core process, login, visible guilds, and the action log. Roadmap
surfaces show as placeholders.
- Web UI data: opening a chat fetches the latest 50 messages into the client cache; attachments
hotlink Discord CDN URLs and are never stored; avatars and guild icons load from the CDN with a
monogram fallback; the page reloads itself when the bridge connects.
- Web UI drafts: composer and TTS text persist per channel in `localStorage` and survive
re-renders, reconnects, and page reloads.
- Visible guilds: the Settings panel filters the guild switcher and picker.
- Takeover: client-side read-only switch with `localStorage` persistence; manual sends are written
to the action log with `type: "manual"`.
- `dm.list`: cached DM and group DM channels, exposed over MCP as `list_dms`.
- Core process control: the daemon starts, stops, and restarts the client, reconnects the bridge
after a restart, auto-starts Xvfb when `DISPLAY` is missing, and stops what it started on exit.
Override with `VESKSLOP_CORE_CMD`/`ARGS`; disable Xvfb with `VESKSLOP_XVFB=0`.
- Web QR login: the Settings panel runs the remote-auth session and renders the QR. The token file
is written on approval and a running managed core restarts with the new token.
- Unit tests cover the pacing policy. `pnpm lint` and `pnpm testTypes` pass at the root.
## Next (in order) ## Next (in order)
1. Notification parsing: edits, deletes, reactions, and message references. 1. Voice backend: `voice.join`, `voice.leave`, `voice.move`, auto-leave, and the `stay @user` command.
2. Reply-to-puppet counts as a trigger. 2. Live verification: `user.search`, `voice.overview`, `VOICE_STATE_UPDATE`, and a full web QR login
3. Per-user rate limit and command cooldowns. approval on the real client.
4. Setup command with reaction-based multiple choice. 3. Notification parsing: edits, deletes, reactions, and message references.
5. MCP event delivery decision: poll, stateful session, or a local event stream. 4. Reply-to-puppet counts as a trigger.
5. Per-user rate limit and command cooldowns.
6. Collector account: read-only modules for voice membership fallback. 6. Collector account: read-only modules for voice membership fallback.
7. Later: voice actions, audio, video, client lightening, web debug UI, remote authentication.
## Open discussions (need the user) ## Open discussions (need the user)
These items were discussed and wait for a decision or a live check. 1. **Web UI auth.** Accepted: no auth, LAN or Tailscale only. Revisit before any wider exposure.
2. **MCP event delivery.** Parked with the LLM attachment. Options stay in `TODO.md`.
1. **MCP event delivery.** The MCP server is stateless, so it cannot push events. Options: (a) poll 3. **Untrusted logging.** Untrusted interactions are ignored at this time. Confirm whether to log
the logs with a `since` cursor, (b) a stateful Streamable HTTP session with server notifications, untrusted DMs and mentions again.
(c) a separate local SSE or WebSocket event stream. Option (a) works today. Decide before an LLM 4. **Privacy pass.** Only token-shaped and `mfa.` strings are redacted, and only in the persistent
attaches.
2. **Untrusted logging.** Untrusted interactions are ignored at this time. The separate 30-entry
buffer exists but receives nothing. Confirm whether to log untrusted DMs and mentions again.
3. **Privacy pass.** Only token-shaped and `mfa.` strings are redacted, and only in the persistent
action log. A fuller policy (retention, cache clearing, message redaction) is deferred. action log. A fuller policy (retention, cache clearing, message redaction) is deferred.
4. **First live run of reactions.** `reaction.add` and the message-ID return are mock-verified only. 5. **Live checks.** `reaction.add`, `user.search`, `voice.overview`, and the voice state event
The first real-client poll must confirm Discord's internal `addReaction` lookup. normalization are mock-verified only. The QR login session reached `pending` and `cancel`
5. **Setup command flow.** The reaction-based multiple-choice design is recorded in `TODO.md`. The against the real gateway; the approval path needs the phone. Confirm on the first real-client
poll command now provides the reaction machinery. run.
## Blocked / needs input ## Blocked / needs input
@@ -69,7 +94,9 @@ These items were discussed and wait for a decision or a live check.
- Backlog: `TODO.md` - Backlog: `TODO.md`
- Design documents: `docs/INDEX.md` - Design documents: `docs/INDEX.md`
- Design handoff: `poc/WebUIMockup.zip` (screen 7a is implemented)
- Agent rules: `AGENTS.md` - Agent rules: `AGENTS.md`
- Daemon code: `poc/src/` - Daemon code: `poc/src/`
- Web UI code: `poc/src/webui.ts`, `poc/public/`
- Core bridge code: `src/shared/puppetProtocol.ts`, `src/renderer/puppet/`, - Core bridge code: `src/shared/puppetProtocol.ts`, `src/renderer/puppet/`,
`src/main/puppetBridge.ts` `src/main/puppetBridge.ts`
README.md
+13 -6
View File
@@ -49,7 +49,7 @@ Think of it as:
- 📹 **A streaming endpoint (planned).** Push files, cameras, generative visuals, or pure slop into a voice channel. - 📹 **A streaming endpoint (planned).** Push files, cameras, generative visuals, or pure slop into a voice channel.
- 🧪 **A science experiment that will absolutely not get your account banned** *(allegedly)*. - 🧪 **A science experiment that will absolutely not get your account banned** *(allegedly)*.
> **Status: text layer works.** The real Vesktop client runs headless and is driven through a validated capability layer. Triggers, trust tiers, conversation context, response policy, polls, persistent logging, and MCP tools are live. Voice actions, audio, and video are still the vision. The design docs live in [docs/INDEX.md](docs/INDEX.md). > **Status: text layer works.** The real Vesktop client runs headless and is driven through a validated capability layer. Triggers, trust tiers, conversation context, response policy, persistent logging, MCP tools, user search, voice state reads, and the Nocturne-style debug web UI are live. Reaction polls are paused. Voice actions, audio, and video are still the vision. The design docs live in [docs/INDEX.md](docs/INDEX.md).
## 🗿 Why though ## 🗿 Why though
@@ -73,7 +73,9 @@ Veskslop ships an **MCP server** so any MCP-capable agent (Claude, GPT-based age
| --- | --- | | --- | --- |
| `get_state` | User, guilds, and channels. Cache-only, zero queries. | | `get_state` | User, guilds, and channels. Cache-only, zero queries. |
| `get_voice_state` | Which voice channel a user is in and who else is suffering there. | | `get_voice_state` | Which voice channel a user is in and who else is suffering there. |
| `send_message` | Send a message as the account. Query-paced. | | `get_voice_overview` | Occupied voice channels in a guild and their members. |
| `search_users` | Find a user ID from a username, nickname, or cached profile. |
| `send_message` | Send a message as the account. Send-paced. |
| `read_messages` | Read the client's message cache. | | `read_messages` | Read the client's message cache. |
| `send_dm` | Slide into someone's DMs. | | `send_dm` | Slide into someone's DMs. |
| `get_context` | Recent conversation context for a channel. Fetches new messages only. | | `get_context` | Recent conversation context for a channel. Fetches new messages only. |
@@ -83,6 +85,9 @@ Veskslop ships an **MCP server** so any MCP-capable agent (Claude, GPT-based age
| `whitelist_list` / `whitelist_add` / `whitelist_remove` | Trust tiers: user, trusted, admin. | | `whitelist_list` / `whitelist_add` / `whitelist_remove` | Trust tiers: user, trusted, admin. |
| `run_commands` | Run several read-only tools in one call. | | `run_commands` | Run several read-only tools in one call. |
There is also a small web control UI on `http://<host>:8789/` with a JSON API and an SSE event
stream. See [docs/webui.md](docs/webui.md).
**Planned**: voice join/leave, audio play/capture, video streams, message edit/delete/upload, presence and voice state control. **Planned**: voice join/leave, audio play/capture, video streams, message edit/delete/upload, presence and voice state control.
Every action is **safe by default**: strict input validation in the client, structured errors, a hard outbound rate limit, and paced sends. A buggy agent cannot crash the client or spam Discord. Every action is **safe by default**: strict input validation in the client, structured errors, a hard outbound rate limit, and paced sends. A buggy agent cannot crash the client or spam Discord.
@@ -204,14 +209,16 @@ The full setup, including the headless client and the mock core, is in
- [x] Trigger log, trust tiers, conversation context, continuation window - [x] Trigger log, trust tiers, conversation context, continuation window
- [x] Response policy, hard rate limits, persistent action log - [x] Response policy, hard rate limits, persistent action log
- [x] MCP server with the live text tool surface - [x] MCP server with the live text tool surface
- [x] Reaction polls and chat command help - [x] User search and voice overview
- [x] Nocturne debug web UI (7a shell, Takeover read-only gate, Settings)
- [~] Reaction polls and chat command help (paused)
- [ ] Notification parsing: edits, deletes, replies, references - [ ] Notification parsing: edits, deletes, replies, references
- [ ] Reply-to-puppet as a trigger - [ ] Reply-to-puppet as a trigger
- [ ] Per-user rate limits and command cooldowns - [ ] Per-user rate limits and command cooldowns
- [ ] Setup command with reaction-based multiple choice
- [ ] MCP event delivery (poll, stateful session, or local event stream)
- [ ] Collector account for voice membership fallback
- [ ] Voice join/leave and invite-link joins - [ ] Voice join/leave and invite-link joins
- [ ] Voice auto-leave and the `stay @user` command
- [ ] Collector account for voice membership fallback
- [~] MCP event delivery (parked)
- [ ] Outbound audio pipeline - [ ] Outbound audio pipeline
- [ ] Inbound audio capture pipeline - [ ] Inbound audio capture pipeline
- [ ] Video stream pipeline - [ ] Video stream pipeline
TODO.md
+44 -37
View File
@@ -2,66 +2,73 @@
Ordered backlog. `CURRENT.md` tracks the live session state. Ordered backlog. `CURRENT.md` tracks the live session state.
## Where the project finished (2026-09-11) ## Where the project finished (2026-09-13)
- The validated core bridge is live: a fixed action list, input validation, structured errors, and a - The validated core bridge is live: a fixed action list, input validation, structured errors, and a
hard outbound rate limit. See `docs/core-bridge.md`. hard outbound rate limit. See `docs/core-bridge.md`.
- The daemon uses the core bridge. QR login, pacing, the trigger log, trust tiers, context, the - The daemon uses the core bridge. QR login, pacing, the trigger log, trust tiers, context, the
response policy, the continuation window, the persistent action log, and the read-only voice response policy, the continuation window, the persistent action log, and the read-only voice
state all work. state all work.
- Human pacing is live: a fast first send (30-200 ms), burst sends (500-2800 ms), and a variable
5-20 s burst reset. Reads take a small gap. See `docs/daemon.md`.
- User search reads the client cache and guild members. `voice.get` searches all guilds without a
`guildId`. `voice.overview` lists occupied channels per guild.
- The web UI follows the Nocturne design handoff: the 7a debug shell with the guild rail, message
view, voice membership, agent console, Takeover read-only gate, and a Settings slide-over for
core process, login, visible guilds, and the action log. Messages fetch the latest 50 on open;
attachments use CDN links; avatars and guild icons load from the CDN. Drafts survive reloads.
Roadmap surfaces show as placeholders. See `docs/webui.md`.
- `dm.list` lists cached DM and group DM channels. `list_dms` exposes it over MCP.
- The daemon starts, stops, and restarts the client, and the Settings panel runs the QR login.
- Mock end-to-end tests pass: tiers, continuation, sensitive replies, batch MCP, and voice state. - Mock end-to-end tests pass: tiers, continuation, sensitive replies, batch MCP, and voice state.
- Repo note: use `zib/Veskslop-t` (remote `veskslop-t`) until the original repo's server hook is - Repo note: use `zib/Veskslop-t` (remote `veskslop-t`) until the original repo's server hook is
fixed. fixed.
## Paused
- The `poll` command. `POLL_ENABLED` is `false` in `poc/src/puppet.ts`. The parser and tests stay.
- The setup command flow. The reaction-based multiple-choice design is on hold.
- The LLM attachment and MCP event delivery. Parked by the owner.
## Deferred ## Deferred
- Voice join and leave, invite-link joins, audio in and out, and video streaming. Voice state - Voice join and leave, invite-link joins, audio in and out, and video streaming. The plan is in
reads exist. Voice actions do not. `docs/voice.md`. Voice state reads and events exist. Voice actions do not.
- Client lightening. Low priority. The client must stay stable. - Client lightening. Low priority. The client must stay stable.
- Web debug UI and remote client authentication. - Web UI authentication and remote accounts.
## Text features ## Next
Done: 1. Voice backend: `voice.join` (muted), `voice.leave`, `voice.move`, the auto-leave timer, and the
`stay @user` command.
- Trusted-only triggers: direct messages and direct mentions. 2. Live verification: `reaction.add`, `user.search`, `voice.overview`, `VOICE_STATE_UPDATE`, core
- Trust tiers: `user`, `trusted`, `admin`. The owner is an admin. start on a real display, and a full QR login approval.
- Context: the last 15 messages or 4000 characters, whichever is lower. File names only. 3. Notification parsing: edits, deletes, reactions, and message references.
Time stamps. Incremental fetch with an old-message cache. 4. Reply-to-puppet counts as a trigger.
- Fetch cooldown: `fetchCooldownSeconds`, default 5. 5. Per-user rate limit and command cooldowns.
- Continuation window: `continuationSeconds`, default 10. A rushed follow-up joins the request 6. Collector account: stripped-down read-only modules. Use it when the puppet cannot find a user's
context.
- Response policy: origin channel or DM. Sensitive commands default to DM.
- Persistent action log with rotation and secret redaction.
- MCP batch tool for read-only commands, plus a voice state tool.
- Reaction poll command: `poll "Question" opt 1 // opt 2 [// opt 3 [// opt 4]]` or the pipe form.
Number emojis, multiple reactions per user, bare command shows help, auto results after
`pollSeconds`.
- Bare `whitelist` and bare `poll` show help with usage and formatting.
Next:
- Notification parsing: edits, deletes, reactions, and message references.
- Reply-to-puppet counts as a trigger.
- Per-user rate limit and command cooldowns.
- Setup command with reaction-based multiple choice. The puppet reacts 1, 2, 3, 4 to its own
message and waits for the user reaction. This also serves as a small poll for group chats.
- More chat commands: `ask`, `summarize`, `context`, `config`.
- Collector account: stripped-down read-only modules. Use it when the puppet cannot find a user's
voice channel or the channel members. voice channel or the channel members.
## Web UI follow-ups
Ordered after the voice backend unless the owner asks first.
- Favourites, presence dots, and real member counts.
- MCP tool-call tracing in the console Tool calls tab.
- Transcript tab once voice audio lands.
- DM scope in the header (the guild picker becomes "Direct messages").
- Message editing, deletion, reactions, and history pagination.
- The remaining mockup screens: 7b/7c console details, 6b history window, 4b group chat, 2c state
legend, 1c phone layout.
## Open decisions ## Open decisions
- **MCP event delivery.** The current Streamable HTTP mode is stateless. The server cannot push - **Web UI auth.** Accepted: no auth, LAN or Tailscale only. Revisit before any wider exposure.
events. Options: - **MCP event delivery.** Parked. Options when it returns:
1. Poll `get_message_log` and `get_action_log` with a `since` cursor. No new transport work. 1. Poll `get_message_log` and `get_action_log` with a `since` cursor. No new transport work.
2. A stateful Streamable HTTP session. The server can send notifications. 2. A stateful Streamable HTTP session. The server can send notifications.
3. A separate local event stream over SSE or a WebSocket. 3. A separate local event stream over SSE or a WebSocket.
Option 1 works today. Options 2 and 3 need a decision before an LLM attaches.
- **Untrusted logging.** Untrusted interactions are ignored at this time. The separate 30-entry - **Untrusted logging.** Untrusted interactions are ignored at this time. The separate 30-entry
buffer exists but receives nothing. Decide whether to log untrusted DMs and mentions again. buffer exists but receives nothing. Decide whether to log untrusted DMs and mentions again.
- **Privacy pass.** Only token-shaped and `mfa.` strings are redacted, and only in the persistent - **Privacy pass.** Only token-shaped and `mfa.` strings are redacted, and only in the persistent
action log. Decide the fuller policy (retention, cache clearing, message redaction) later. action log. Decide the fuller policy (retention, cache clearing, message redaction) later.
- **Live reaction check.** `reaction.add` and the message-ID return are mock-verified only. Confirm
Discord's internal `addReaction` lookup on the first real-client run.
docs/INDEX.md
+2
View File
@@ -9,6 +9,8 @@ This file lists the project documents.
| [architecture.md](architecture.md) | System parts, boundaries, and data flow | | [architecture.md](architecture.md) | System parts, boundaries, and data flow |
| [core-bridge.md](core-bridge.md) | The Vesktop-side capability layer | | [core-bridge.md](core-bridge.md) | The Vesktop-side capability layer |
| [daemon.md](daemon.md) | The external MCP daemon | | [daemon.md](daemon.md) | The external MCP daemon |
| [webui.md](webui.md) | The web control UI and its JSON API |
| [voice.md](voice.md) | Voice state reads, planned actions, and lifecycle |
| [communication.md](communication.md) | Trust model, commands, and the trigger log | | [communication.md](communication.md) | Trust model, commands, and the trigger log |
| [auth.md](auth.md) | QR login and token persistence | | [auth.md](auth.md) | QR login and token persistence |
| [testing.md](testing.md) | Tests and the live test setup | | [testing.md](testing.md) | Tests and the live test setup |
docs/architecture.md
+13 -8
View File
@@ -7,18 +7,20 @@ real Vesktop client, so the traffic comes from the official client.
## Parts ## Parts
The system has four parts: The system has five parts:
1. **Vesktop core** — the real Vesktop client. It runs headless. It exposes a small capability layer. 1. **Vesktop core** — the real Vesktop client. It runs headless. It exposes a small capability layer.
2. **Core bridge** — the capability layer. It accepts validated actions and pushes events. 2. **Core bridge** — the capability layer. It accepts validated actions and pushes events.
3. **Veskslop daemon** — the external process. It holds all logic. It exposes MCP. 3. **Veskslop daemon** — the external process. It holds all logic. It exposes MCP.
4. **MCP clients** — agents and tools. They call the daemon. 4. **Web UI** — a small browser control surface served by the daemon. See [webui.md](webui.md).
5. **MCP clients** — agents and tools. They call the daemon.
``` ```
MCP clients (agents) MCP clients (agents) Web browser (owner)
│ Streamable HTTP (MCP) │ Streamable HTTP (MCP) │ HTTP + SSE
▼ ▼ ▼
Veskslop daemon ──── JSON lines over a Unix socket ────▶ Vesktop core Veskslop daemon ◀───────────────────── web UI listener
│ ──── JSON lines over a Unix socket ────▶ Vesktop core
│ │ │ │
│ QR login, pacing, trigger log, │ validated actions + events │ QR login, pacing, trigger log, │ validated actions + events
│ trust list, MCP tools ▼ │ trust list, MCP tools ▼
@@ -55,9 +57,12 @@ when the action list changes.
| Pacing | `poc/src/pacing.ts` | Implemented | | Pacing | `poc/src/pacing.ts` | Implemented |
| Trigger log and trust list | `poc/src/communication.ts`, `poc/src/puppet.ts` | Implemented and verified live | | Trigger log and trust list | `poc/src/communication.ts`, `poc/src/puppet.ts` | Implemented and verified live |
| Context cache and response policy | `poc/src/message-cache.ts`, `poc/src/puppet.ts` | Implemented and verified with the mock | | Context cache and response policy | `poc/src/message-cache.ts`, `poc/src/puppet.ts` | Implemented and verified with the mock |
| Chat commands and polls | `poc/src/puppet.ts`, `poc/src/communication.ts` | Implemented and verified with the mock | | Chat commands and polls | `poc/src/puppet.ts`, `poc/src/communication.ts` | Implemented. Poll is paused (`POLL_ENABLED = false`). |
| Persistent action log | `poc/src/action-log.ts` | Implemented | | Persistent action log | `poc/src/action-log.ts` | Implemented |
| Voice, audio, video | Not started | Not started | | Web UI and JSON API | `poc/src/webui.ts`, `poc/public/` | Nocturne 7a debug shell + roadmap placeholders |
| User search | `src/renderer/puppet/actions.ts` (`user.search`) | Implemented |
| Voice state reads | `src/renderer/puppet/actions.ts`, `src/renderer/puppet/events.ts` | Implemented |
| Voice actions, audio, video | Not started | Not started |
## Data flow ## Data flow
docs/auth.md
+13
View File
@@ -18,6 +18,19 @@ captcha, the tool shows the challenge and reads a key from `HCAPTCHA_KEY` or fro
The flow was verified end-to-end on 2026-09-11. The flow was verified end-to-end on 2026-09-11.
## Web UI login
The web UI Login tab runs the same session (`poc/src/qr-session.ts`). See [webui.md](webui.md).
- `POST /api/auth/qr` starts the session. `GET /api/auth` returns the state and the QR image.
- On success the token is written to the token file. A running managed core restarts with
`VESKSLOP_FORCE_TOKEN=1`, so the new account takes effect.
- A captcha challenge shows the sitekey. Paste the hCaptcha response token to finish.
- The API never returns the token.
To log in a second account, set a different `TOKEN_FILE` for the daemon (and thus the spawned core)
and restart the daemon. One daemon manages one account at this time.
## Token file ## Token file
- Default path: `poc/.token`, relative to the `poc/` directory. - Default path: `poc/.token`, relative to the `poc/` directory.
docs/communication.md
+5 -1
View File
@@ -37,7 +37,8 @@ mention the puppet first.
| `whitelist list` | Show the owner and the trusted users with their tiers. | | `whitelist list` | Show the owner and the trusted users with their tiers. |
| `whitelist add @user [user\|trusted\|admin]` | Add one or more users. The default tier is `user`. | | `whitelist add @user [user\|trusted\|admin]` | Add one or more users. The default tier is `user`. |
| `whitelist remove @user` | Remove one or more users from the trust list. | | `whitelist remove @user` | Remove one or more users from the trust list. |
| `poll <question> \| <opt 1> \| <opt 2> [\| opt 3 [\| opt 4]]` | Start a reaction poll. | | `poll ...` | Start a reaction poll. Paused at this time. |
| `stay @user` | Planned with the voice backend. Pin the bot to a voice user. See [voice.md](voice.md). |
Notes: Notes:
@@ -49,6 +50,9 @@ Notes:
## Polls ## Polls
**Paused.** `POLL_ENABLED` is `false` in `poc/src/puppet.ts`. The parser, the tests, and the code
stay in place. The daemon does not run a `poll` command.
The `poll` command posts a message and adds one number emoji per option. Discord users vote by The `poll` command posts a message and adds one number emoji per option. Discord users vote by
reacting. The poll closes after `pollSeconds` (default 120) and the daemon posts the result. reacting. The poll closes after `pollSeconds` (default 120) and the daemon posts the result.
docs/core-bridge.md
+22 -4
View File
@@ -56,14 +56,28 @@ Event line:
| --- | --- | --- | | --- | --- | --- |
| `ping` | none | `"pong"` | | `ping` | none | `"pong"` |
| `events.subscribe` | none | `{ "subscribed": true }`. The socket becomes an event subscriber. | | `events.subscribe` | none | `{ "subscribed": true }`. The socket becomes an event subscriber. |
| `state.get` | none | The current user, the guilds, and the channels. Cache-only. | | `state.get` | none | The current user, the guilds, and the channels. Cache-only. Includes user avatars and guild icons. |
| `voice.get` | optional `userId`, optional `guildId` | The voice channel and its members, or `channelId: null` | | `voice.get` | optional `userId`, optional `guildId` | The voice channel and its members, or `channelId: null`. Without `guildId` it searches all guilds. |
| `voice.overview` | `guildId` | The occupied voice channels in the guild and their members |
| `user.search` | `query`, optional `guildId`, optional `limit` | Users from the client cache. Guild members when `guildId` is set. |
| `dm.list` | optional `limit` | Cached DM and group DM channels with recipients, icons, and avatars. Cache-only. |
| `message.send` | `channelId`, `content` | `{ channelId, messageId, sent: true }` | | `message.send` | `channelId`, `content` | `{ channelId, messageId, sent: true }` |
| `message.read` | `channelId`, `limit` | `{ cached, messages }`. Cache-only. | | `message.read` | `channelId`, `limit` | `{ cached, messages }`. Cache-only. Messages include the author avatar and attachment CDN links. |
| `message.fetch` | `channelId`, `limit`, optional `before` or `after` | `{ fetched, messages }`. Fetches from Discord. | | `message.fetch` | `channelId`, `limit`, optional `before` or `after` | `{ fetched, messages }`. Fetches from Discord. Same message shape. |
| `reaction.add` | `channelId`, `messageId`, `emoji` | `{ added: true }` | | `reaction.add` | `channelId`, `messageId`, `emoji` | `{ added: true }` |
| `dm.send` | `userId`, `content` | `{ userId, channelId, sent: true }` | | `dm.send` | `userId`, `content` | `{ userId, channelId, sent: true }` |
## Events
The renderer normalizes client events and pushes them to subscribers.
| Event | Payload |
| --- | --- |
| `MESSAGE_CREATE` | Message ID, channel, guild, channel type, author, content, mentions, timestamp |
| `MESSAGE_REACTION_ADD` | Message ID, channel, guild, user, emoji |
| `MESSAGE_REACTION_REMOVE` | Same fields as add |
| `VOICE_STATE_UPDATE` | User, channel, guild, session, mute, deaf, self-mute, self-deaf |
## Validation rules ## Validation rules
The core validates all input. The daemon cannot bypass these rules. The core validates all input. The daemon cannot bypass these rules.
@@ -79,6 +93,10 @@ The core validates all input. The daemon cannot bypass these rules.
- `before` and `after` must be snowflakes. They are mutually exclusive. `message.fetch` uses them - `before` and `after` must be snowflakes. They are mutually exclusive. `message.fetch` uses them
to walk history. to walk history.
- `voice.get` accepts snowflakes only. It reads the client cache. It never joins a call. - `voice.get` accepts snowflakes only. It reads the client cache. It never joins a call.
- `voice.overview` requires a guild snowflake. It reads the client cache.
- `user.search` takes a query of 1 to 100 characters and a limit of 1 to 25. It reads the client
cache only. An exact snowflake query returns that user when cached.
- `dm.list` takes a limit of 1 to 50. It reads `ChannelStore.getSortedPrivateChannels()` only.
- `reaction.add` accepts a short emoji string without line breaks. It counts as an outbound action - `reaction.add` accepts a short emoji string without line breaks. It counts as an outbound action
for the rate limit. for the rate limit.
- A request line must not be larger than 64 KiB. - A request line must not be larger than 64 KiB.
docs/daemon.md
+52 -8
View File
@@ -6,12 +6,17 @@ The daemon runs outside Vesktop. It holds all logic. It talks to the core bridge
| File | Role | Status | | File | Role | Status |
| --- | --- | --- | | --- | --- | --- |
| `poc/src/control-bridge.ts` | MCP server, tool list, health endpoint | Implemented | | `poc/src/control-bridge.ts` | MCP server, tool list, health endpoint, core reconnect | Implemented |
| `poc/src/core-process.ts` | Start, stop, and restart the Vesktop core process | Implemented |
| `poc/src/qr-session.ts` | Remote-auth QR login session (shared with the web UI) | Implemented |
| `poc/src/auth-service.ts` | Auth state for the web UI | Implemented |
| `poc/src/puppet.ts` | Trust list, command handling, trigger log, response policy | Implemented | | `poc/src/puppet.ts` | Trust list, command handling, trigger log, response policy | Implemented |
| `poc/src/communication.ts` | Pure log, parser, and context logic | Implemented | | `poc/src/communication.ts` | Pure log, parser, and context logic | Implemented |
| `poc/src/message-cache.ts` | Channel message cache in a tmp folder | Implemented | | `poc/src/message-cache.ts` | Channel message cache in a tmp folder | Implemented |
| `poc/src/action-log.ts` | Persistent JSONL action log with rotation | Implemented | | `poc/src/action-log.ts` | Persistent JSONL action log with rotation | Implemented |
| `poc/src/pacing.ts` | Paced queue and query coalescing | Implemented | | `poc/src/pacing.ts` | Paced send queue and read queries | Implemented |
| `poc/src/webui.ts` | Web UI listener and JSON API | First draft |
| `poc/public/` | Static web UI (HTML, CSS, JS) | First draft |
| `poc/src/core-client.ts` | Client actions over the core bridge | Implemented (primary transport) | | `poc/src/core-client.ts` | Client actions over the core bridge | Implemented (primary transport) |
| `poc/src/cdp-client.ts` | Chrome DevTools Protocol client | Development fallback | | `poc/src/cdp-client.ts` | Chrome DevTools Protocol client | Development fallback |
| `poc/src/qr-login.ts` | Terminal QR login | Implemented and verified | | `poc/src/qr-login.ts` | Terminal QR login | Implemented and verified |
@@ -33,9 +38,14 @@ the page, so it has no core-side guardrails. Use it for debugging only.
## Pacing ## Pacing
The daemon paces all commands. The policy is a random delay of 500 to 2800 ms. The queue The daemon paces all outbound sends. A send is `message.send` or `dm.send`.
serializes the commands. The queue coalesces identical concurrent reads. The daemon also prefers
cached state over queries. - The first send of a burst waits 30 to 200 ms.
- Later sends in the same burst wait 500 to 2800 ms.
- A burst ends after a random 5 to 20 seconds of quiet. The next send is a fast first action.
Read queries are serialized with a small 100 to 300 ms gap. Reads do not reset the send burst.
Identical concurrent reads are coalesced. The daemon prefers cached state over queries.
The core adds a second, hard rate limit. See [core-bridge.md](core-bridge.md). The core adds a second, hard rate limit. See [core-bridge.md](core-bridge.md).
@@ -50,6 +60,9 @@ The daemon exposes MCP over Streamable HTTP. The default address is `http://127.
| `read_messages` | Read cached messages from a channel. | | `read_messages` | Read cached messages from a channel. |
| `send_dm` | Send a DM to a user. | | `send_dm` | Send a DM to a user. |
| `get_voice_state` | Which voice channel a user is in and who else is there. | | `get_voice_state` | Which voice channel a user is in and who else is there. |
| `get_voice_overview` | Occupied voice channels in a guild and their members. |
| `search_users` | Search the client user cache by username, global name, or nickname. |
| `list_dms` | List cached DM and group DM channels with recipients. |
| `get_message_log` | Read the trigger log. | | `get_message_log` | Read the trigger log. |
| `get_action_log` | Read the persistent action log. | | `get_action_log` | Read the persistent action log. |
| `run_commands` | Run several read-only tools in one call, in order. | | `run_commands` | Run several read-only tools in one call, in order. |
@@ -63,9 +76,12 @@ The daemon exposes MCP over Streamable HTTP. The default address is `http://127.
## Run commands ## Run commands
```sh ```sh
# Start the daemon. It connects to the core bridge. # Start the daemon. It connects to the core bridge and starts the web UI.
pnpm control:bridge pnpm control:bridge
# Start the daemon with a different web UI port.
WEBUI_PORT=18889 pnpm control:bridge
# Start a mock core for tests without Electron. # Start a mock core for tests without Electron.
VESKSLOP_SOCKET=/tmp/vesktop-puppet-mock.sock pnpm control:mock VESKSLOP_SOCKET=/tmp/vesktop-puppet-mock.sock pnpm control:mock
@@ -84,12 +100,34 @@ pnpm cdp:probe
CDP_EXPRESSION='...' pnpm cdp:dispatch CDP_EXPRESSION='...' pnpm cdp:dispatch
``` ```
## Core process
The daemon can start, stop, and restart the Vesktop client (`poc/src/core-process.ts`). The web UI
Core tab and the `/api/core/*` routes use it.
- The spawn sets `VESKSLOP_HEADLESS=1` and `VESKSLOP_TOKEN_FILE` (the daemon token file).
- The default command is `node_modules/.bin/electron .` from the repo root. `VESKSLOP_CORE_CMD`
replaces it. `VESKSLOP_CORE_ARGS` replaces the default flags.
- On Linux without a `DISPLAY`, the daemon starts Xvfb on the first free display from `:99` and
points the client at it. An existing `DISPLAY` wins. `VESKSLOP_XVFB=0` disables auto-start.
- The status holds the last exit code and signal, and the last 20 output lines. The Core tab shows
both, so a failed start explains itself.
- The daemon stops a client that it started, and any Xvfb it started, when the daemon exits. An
externally started client is reported as `external` and is not managed.
- When the core bridge drops, the daemon retries every 2 s. A restart moves the bridge connection
to the new client. The in-memory trigger log resets; the state file and the action log survive.
- QR login lives in `poc/src/qr-session.ts` and is shared by the terminal tool and the web UI.
See [auth.md](auth.md).
Verified live on 2026-09-13: the web UI started the client under auto-Xvfb, the bridge connected,
`state.get` returned the real account and guilds, and stop exited with code 0.
## Health endpoint ## Health endpoint
`GET /health` returns the CDP state, the owner ID, and the query count. `GET /health` returns the core connection state, the socket path, the owner ID, and the query count.
```json ```json
{ "ok": true, "cdpConnected": true, "ownerId": "177117296140091393", "queries": 4 } { "ok": true, "coreConnected": true, "socketPath": "/run/user/1000/vesktop-puppet.sock", "ownerId": "177117296140091393", "queries": 4 }
``` ```
## Settings and paths ## Settings and paths
@@ -103,6 +141,10 @@ CDP_EXPRESSION='...' pnpm cdp:dispatch
| `VESKSLOP_CACHE_DIR` | `<tmp>/veskslop-cache` | Context cache folder | | `VESKSLOP_CACHE_DIR` | `<tmp>/veskslop-cache` | Context cache folder |
| `VESKSLOP_CACHE_MAX_BYTES` | 50 MiB | Context cache size limit | | `VESKSLOP_CACHE_MAX_BYTES` | 50 MiB | Context cache size limit |
| `MCP_HOST` / `MCP_PORT` | `127.0.0.1` / `8788` | MCP listen address | | `MCP_HOST` / `MCP_PORT` | `127.0.0.1` / `8788` | MCP listen address |
| `WEBUI_HOST` / `WEBUI_PORT` | `0.0.0.0` / `8789` | Web UI listen address. No authentication. |
| `VESKSLOP_CORE_CMD` | unset | Full command that starts the client. Overrides the default. |
| `VESKSLOP_CORE_ARGS` | `--no-sandbox --disable-gpu --disable-dev-shm-usage` | Flags for the default Electron command. |
| `VESKSLOP_XVFB` | unset | Set to `0` to disable automatic Xvfb start on Linux. |
## How to add an MCP tool ## How to add an MCP tool
@@ -115,6 +157,8 @@ CDP_EXPRESSION='...' pnpm cdp:dispatch
## Security notes ## Security notes
- The MCP server binds to localhost by default. - The MCP server binds to localhost by default.
- The web UI binds to all interfaces by default and has no authentication. Use it over LAN or
Tailscale only. Do not expose the port to the public internet. See [webui.md](webui.md).
- Remote access must go over LAN or Tailscale. Do not expose the daemon to the public internet. - Remote access must go over LAN or Tailscale. Do not expose the daemon to the public internet.
- The daemon holds no Discord credentials. The client holds the session. - The daemon holds no Discord credentials. The client holds the session.
- The daemon token file is separate from the client session. See [auth.md](auth.md). - The daemon token file is separate from the client session. See [auth.md](auth.md).
docs/decisions.md
+49 -2
View File
@@ -45,8 +45,8 @@ already does this work.
## ADR-5: Two pacing layers ## ADR-5: Two pacing layers
**Decision.** The daemon paces all commands with a random delay of 500 to 2800 ms. The core adds a **Decision.** The daemon paces all outbound sends with a human-emulation policy (ADR-12). The core
hard limit of 500 ms between sends and 30 sends per minute. adds a hard limit of 500 ms between sends and 30 sends per minute.
**Reason.** The daemon policy looks human. The core limit is a safety net. A daemon bug cannot cause **Reason.** The daemon policy looks human. The core limit is a safety net. A daemon bug cannot cause
a send flood. a send flood.
@@ -97,3 +97,50 @@ Redact token-shaped strings before the write.
**Reason.** The in-memory trigger log resets at restart. The action log gives a durable audit trail **Reason.** The in-memory trigger log resets at restart. The action log gives a durable audit trail
without storing raw credentials. without storing raw credentials.
## ADR-12: Human send pacing
**Decision.** The first outbound send of a burst waits 30 to 200 ms. Later sends wait 500 to
2800 ms. A burst ends after a random 5 to 20 seconds of quiet. Reads take a small 100 to 300 ms gap
and do not reset the burst.
**Reason.** A real client sends the first message quickly and then slows down. Reads are not
visible in the same way and must not consume the send budget. The core hard limit stays as the
safety net. See [daemon.md](daemon.md).
## ADR-13: Web UI on a separate listener
**Decision.** The daemon serves the web UI on its own listener (`0.0.0.0:8789`) with no
authentication. The MCP server stays on localhost. The page is plain HTML, CSS, and JS behind a
JSON API.
**Reason.** Remote control over LAN or Tailscale needs a reachable port. A separate listener keeps
the MCP surface local. The JSON API is the stable boundary, so a future frontend replaces the page
without a backend change. No auth fits a private network, but the port must never be exposed to the
public internet.
## ADR-14: Voice joins muted, presence first
**Decision.** `voice.join` joins muted. Audio playback is a later feature and unmutes only while
playing. The bot leaves on its own after the owner or the stay target leaves, plus a delay.
**Reason.** A muted bot captures no audio and does not disturb the call. The lifecycle keeps the
bot from lingering in empty calls. See [voice.md](voice.md).
## ADR-15: Nocturne design and the debug shell
**Decision.** The web UI follows the Nocturne design system and the 7a debug shell from the design
handoff (`poc/WebUIMockup.zip`). `poc/public/nocturne.css` carries the token set. The page stays
plain HTML, CSS, and JavaScript with no build step. The JSON API stays the migration boundary.
**Reason.** The owner chose the design direction. Vanilla keeps the no-build constraint. Roadmap
gaps (TTS, transcript, tool-call tracing, favourites, voice actions) show as marked placeholders so
the shape of the finished UI is visible while the backend catches up.
## ADR-16: Takeover is a read-only gate
**Decision.** The Takeover switch is client-side. Off, the composer is locked and the page cannot
send. On, manual sends are allowed and each one is written to the action log as a `manual` entry.
**Reason.** The switch exists to stop accidental use of the bot account, not to model agent
ownership. The API does not enforce it, so other clients can still send. See [webui.md](webui.md).
docs/testing.md
+90 -7
View File
@@ -6,7 +6,7 @@ This document describes the test setup, the run commands, and the known gaps.
The pure daemon logic has unit tests. The tests cover the ring buffers, the mention helpers, the The pure daemon logic has unit tests. The tests cover the ring buffers, the mention helpers, the
command parser, the poll parser, the trust helpers, the context assembly, the redaction, and the command parser, the poll parser, the trust helpers, the context assembly, the redaction, and the
pacing range. The cache tests cover merge, dedupe, trimming, and eviction. pacing policy (`planSend`). The cache tests cover merge, dedupe, trimming, and eviction.
```sh ```sh
cd poc cd poc
@@ -50,12 +50,68 @@ VESKSLOP_SOCKET=/tmp/vesktop-puppet-mock.sock TEST_TRIGGER_CONTENT="whitelist li
``` ```
Expected result: the daemon log shows one trusted entry with the parsed command. The mock log shows Expected result: the daemon log shows one trusted entry with the parsed command. The mock log shows
the reply (`message.send`). The reply wait is inside the 500 to 2800 ms window. the reply (`message.send`). A first reply after a quiet period waits 30 to 200 ms. Further replies
in the same burst wait 500 to 2800 ms.
The mock also covers tiers, continuation, polls, reactions, and the action log. Set config values The mock also covers tiers, continuation, polls, reactions, and the action log. Set config values
with `set_puppet_config` before a test. Inject a reaction with with `set_puppet_config` before a test. Inject a reaction with
`TEST_TRIGGER_REACTION=1 TEST_REACTION_MESSAGE_ID=msg-1 TEST_REACTION_EMOJI="1️⃣" pnpm test:trigger`. `TEST_TRIGGER_REACTION=1 TEST_REACTION_MESSAGE_ID=msg-1 TEST_REACTION_EMOJI="1️⃣" pnpm test:trigger`.
## Web UI smoke test
The web UI starts with the daemon. The mock core is enough for this test.
```sh
# Terminal 1: mock core
VESKSLOP_SOCKET=/tmp/vesktop-puppet-mock.sock pnpm control:mock
# Terminal 2: daemon (web UI on a free port)
VESKSLOP_SOCKET=/tmp/vesktop-puppet-mock.sock WEBUI_PORT=18789 pnpm control:bridge
# Terminal 3: check the API
curl -s http://127.0.0.1:18789/api/health
curl -s http://127.0.0.1:18789/api/state
curl -s "http://127.0.0.1:18789/api/users?q=zz"
curl -s http://127.0.0.1:18789/api/dms
curl -s "http://127.0.0.1:18789/api/voice/overview?guildId=100000000000000001"
curl -s http://127.0.0.1:18789/api/events # server-sent events
```
The page itself can be rendered headless for a visual check. With Xvfb running:
```sh
DISPLAY=:99 node_modules/.bin/electron /tmp/check.js --no-sandbox
```
where `check.js` loads `http://127.0.0.1:18789/` in a `BrowserWindow` and logs page errors. This
was used to verify the 7a shell, the locked/unlocked composer, and the Settings slide-over.
The voice join and leave routes return `501` until the voice backend lands. See [webui.md](webui.md).
Process control and login work without the mock core:
```sh
curl -s http://127.0.0.1:18789/api/core/status
curl -s http://127.0.0.1:18789/api/auth
curl -s -X POST http://127.0.0.1:18789/api/auth/qr
curl -s -X POST http://127.0.0.1:18789/api/auth/qr/cancel
```
Starting `/api/auth/qr` opens a real remote-auth gateway session. Cancel it if you do not intend to
log in. Approving a login replaces the token file.
To test start and stop without Electron, run the daemon with a stand-in command:
```sh
VESKSLOP_SOCKET=/tmp/vesktop-puppet-mock.sock VESKSLOP_CORE_CMD="sleep 300" WEBUI_PORT=18789 pnpm control:bridge
curl -s -X POST http://127.0.0.1:18789/api/core/start
curl -s -X POST http://127.0.0.1:18789/api/core/stop
```
A real client start needs a display and the Electron shared libraries. On Linux, start Xvfb first.
The container needs `libasound.so.2` for Electron. Extract the library locally when the system
package is not available.
## CDP probe ## CDP probe
The probe checks the CDP fallback transport. Start the client with The probe checks the CDP fallback transport. Start the client with
@@ -74,7 +130,23 @@ event.
## Live test setup ## Live test setup
The container has no display and no PipeWire. Use a virtual display. The daemon can start the client for you, including a virtual display. This is the easy path.
```sh
cd poc
pnpm control:bridge
# Open http://<host>:8789/ and press Start in Settings → Core process.
# Or from the shell:
curl -s -X POST http://127.0.0.1:8789/api/core/start
curl -s http://127.0.0.1:8789/api/core/status
```
The daemon starts Xvfb when `DISPLAY` is unset, spawns `electron .`, and connects the bridge. The
token file seeds the login. The Settings slide-over shows the process state, the last exit code,
and the last 20 output lines. Set `VESKSLOP_XVFB=0` to turn off auto-Xvfb.
Manual setup (no auto-start):
```sh ```sh
# 1. Start a virtual display. On Linux, install Xvfb or extract it locally. # 1. Start a virtual display. On Linux, install Xvfb or extract it locally.
@@ -97,8 +169,9 @@ Notes:
- `VENCORD_USER_DATA_DIR` pins the Electron profile. The profile holds the client session. - `VENCORD_USER_DATA_DIR` pins the Electron profile. The profile holds the client session.
- Write `{"firstLaunch": false}` to `<DATA_DIR>/state.json` before the first start. This step skips - Write `{"firstLaunch": false}` to `<DATA_DIR>/state.json` before the first start. This step skips
the first-launch window. The `VESKSLOP_HEADLESS=1` flag does the same job. the first-launch window. The `VESKSLOP_HEADLESS=1` flag does the same job.
- The container needs `libasound.so.2` for Electron. Extract the library locally when the system - The container needs `libasound.so.2` for Electron. Install `libasound2` or extract the library
package is not available. locally and point `LD_LIBRARY_PATH` at it.
- The live start was verified on 2026-09-13 through the web UI route.
## Sanctioned live target ## Sanctioned live target
@@ -120,6 +193,14 @@ The core bridge was verified live on 2026-09-11:
the parsed command. the parsed command.
- The client ran headless under Xvfb with the token seeded from the token file. - The client ran headless under Xvfb with the token seeded from the token file.
Core process control and the web QR session were verified live on 2026-09-13:
- `POST /api/core/start` started the client, including auto-Xvfb, and the bridge connected.
- `state.get` returned the real account and guilds.
- `POST /api/core/stop` exited with code 0 and released Xvfb.
- The QR session reached `pending` with a scannable QR and cancelled cleanly. A full approval needs
a phone and was not run.
The CDP transport was verified earlier with the same checks. The CDP transport was verified earlier with the same checks.
## Known gaps ## Known gaps
@@ -128,6 +209,8 @@ The CDP transport was verified earlier with the same checks.
| --- | --- | | --- | --- |
| Live trigger tests need an incoming message. | Use the mock core, or send from zzib's account. | | Live trigger tests need an incoming message. | Use the mock core, or send from zzib's account. |
| `reaction.add` and the message ID return are mock-verified only. | Confirm Discord's internal `addReaction` lookup on the first live run. | | `reaction.add` and the message ID return are mock-verified only. | Confirm Discord's internal `addReaction` lookup on the first live run. |
| Polls and other chat commands are mock-verified only. | Run them on the live client. | | Polls and other chat commands are mock-verified only. | Poll is paused; the tests still cover the parser. |
| `message.read` is cache-only. | A channel that the client never opened returns zero messages. | | `message.read` is cache-only. | A channel that the client never opened returns zero messages. |
| Voice, audio, and video are not started. | No coverage. | | `VOICE_STATE_UPDATE` normalization is mock-verified only. | Confirm the event fields on the first live run. |
| The web UI has no authentication. | Keep it on LAN or Tailscale only. |
| Voice actions, audio, and video are not started. | No coverage. |
docs/voice.md
+67
View File
@@ -0,0 +1,67 @@
# Voice
Voice state reads exist. Voice actions do not. This document records the plan and the decisions.
## Current
| Part | Status |
| --- | --- |
| `voice.get` | Implemented. One user's voice channel and its members. Without `guildId` it searches all guilds. |
| `voice.overview` | Implemented. Occupied voice channels and members for one guild. |
| `VOICE_STATE_UPDATE` events | Normalized and pushed. Mock-verified. Live check pending. |
| Join and leave | Not started. |
| Audio in and out | Not started. |
| Video | Not started. |
The core reads `VoiceStateStore` only. It never joins a call on its own.
## Planned actions
| Action | Parameters | Result |
| --- | --- | --- |
| `voice.join` | `channelId`, optional `mute` (default true) | `{ channelId, joined: true }` |
| `voice.leave` | none | `{ left: true }` |
| `voice.move` | `channelId` | `{ channelId, moved: true }` |
Rules:
- The channel must be a voice channel. The core validates the snowflake and the channel type.
- Join joins the call muted. The bot does not capture or send audio.
- Voice actions are state changes, not send floods. They stay out of the message pacing queue, but
the daemon rate limits them separately.
- The join implementation goes through Discord's internal channel action, located at runtime. The
lookup is unverified until the first live run. This is the same pattern as `reaction.add`.
The plan for later audio: the bot unmutes only while it plays an audio stream, then mutes again.
Audio playback is a later feature.
## Lifecycle
The bot leaves a call on its own in two cases.
1. **Owner leaves.** When the owner leaves the call, the bot waits `voiceLeaveDelaySeconds` and
then leaves. A rejoin inside the window cancels the leave.
2. **A stay target leaves.** The `stay @user` command pins the bot to a user. When that user
leaves, the bot waits the same delay and then leaves.
`stay @user` needs a chat command. Chat commands are paused at this time, so the command waits for
the voice backend. The default delay is proposed as 30 seconds and lives in the config.
The bot never leaves while the owner or the stay target is present. A manual `voice.leave` wins
over both rules.
## Collector account
The collector account is a stripped-down client with read-only voice membership. The daemon uses it
when the puppet cannot see a user's voice channel or the channel members. It stays deferred. See
[TODO.md](../TODO.md).
## Verification list
A live run must confirm these points.
1. The internal `selectVoiceChannel` lookup exists and joins the given channel.
2. The internal disconnect action leaves the call.
3. `VOICE_STATE_UPDATE` carries the fields that the core normalizes.
4. The join is muted and Discord shows the mute state.
5. The auto-leave timer works across a client restart. The timer is daemon state, not core state.
docs/webui.md
+147
View File
@@ -0,0 +1,147 @@
# Web UI
The daemon serves the debug web control UI. The owner uses it to observe the account the agent
sees, read and send messages, search users, and see voice membership. The shell follows the design
handoff in `poc/WebUIMockup.zip`; several panels are roadmap placeholders.
## Run
The UI starts with the daemon. No extra command is needed.
```sh
cd poc
pnpm control:bridge
# Page
http://<host>:8789/
# API
http://<host>:8789/api/health
```
| Setting | Default | Purpose |
| --- | --- | --- |
| `WEBUI_HOST` | `0.0.0.0` | Listen address |
| `WEBUI_PORT` | `8789` | Listen port |
## Security
The UI has **no authentication**. Anyone who can reach the port controls the bot, can start and
stop the client, and can start a QR login. Bind the daemon to a private LAN or a Tailscale address
only. Do not expose the port to the public internet. See [daemon.md](daemon.md).
The MCP server stays on `127.0.0.1:8788`. Only the UI binds outside localhost.
## Page
- Plain HTML, CSS, and JavaScript in `poc/public/`. No build step and no frontend framework.
- The look follows the Nocturne design system (`poc/public/nocturne.css`) from the design handoff
in `poc/WebUIMockup.zip`. Implemented screen: **7a**, the docked three-column debug shell.
- Layout: 50px header, conditional call strip, `236px | 1fr | 306px` body, and the agent console
bar (35px collapsed, resizable drawer expanded).
- Header: wordmark, guild picker, health indicators, Takeover switch, Settings.
- Left rail: guild switcher, text channels, voice channels with inline participants, direct
messages, and the account footer.
- Center: channel header, message list, composer, and the TTS panel.
- Right column: In voice, Favourites, Open DMs & groups.
- Console: Tool calls, Events (live SSE), Transcript. The ticker shows the newest event.
- Opening a chat fetches the latest 50 messages from Discord into the client cache. The refresh
button by the channel header re-fetches. Live message events read the cache.
- Attachments render from their Discord CDN links. Images display inline; other files are link
chips. The daemon never downloads or stores attachment bytes.
- Avatars and guild icons load from the Discord CDN with a monogram fallback.
- Composer and TTS drafts persist in `localStorage` per channel. They survive re-renders, the
bridge connecting, and a page reload. A successful send clears the channel draft.
- When the bridge connects after the page loaded, the page reloads state without touching drafts.
- Roadmap gaps are shown, not hidden: TTS, transcript, tool-call tracing, favourites, and voice
join/monitor appear as marked placeholders.
## Takeover
Takeover is a read-only switch. Off, the composer is locked (a `read-only` tag sits next to the
lock) and the page cannot send. On, the composer unlocks and every manual send is written to the
action log as a `manual` entry. The choice is stored in `localStorage`. The setting is client-side
only; the API itself does not enforce it.
## Settings
The Settings button opens a slide-over with the server-side controls:
- **Core process** — state, last exit, output, Start / Stop / Restart.
- **Account login** — token file state, QR login, captcha.
- **Visible guilds** — checkboxes for the guild switcher and picker. Stored in `localStorage`;
"Show all" clears the filter.
- **Action log** — the persistent JSONL log, newest first.
## Core process
The Settings panel starts, stops, and restarts the Vesktop client. The daemon spawns the process
from the repo root and labels its output with `[core]` in the daemon log.
- The default command is `node_modules/.bin/electron .` with
`--no-sandbox --disable-gpu --disable-dev-shm-usage`.
- Override the command with `VESKSLOP_CORE_CMD` and the flags with `VESKSLOP_CORE_ARGS`.
- The spawn sets `VESKSLOP_HEADLESS=1` and points `VESKSLOP_TOKEN_FILE` at the daemon token file.
- On Linux without a `DISPLAY`, the daemon starts Xvfb automatically. `VESKSLOP_XVFB=0` disables
this. A real `DISPLAY` wins.
- Only a process started by the current daemon is managed. An externally started client shows as
`external` and cannot be stopped from the UI.
- The status shows the last exit code or signal and the last 20 output lines. A failed start is
visible on the page instead of only in the daemon log.
## Login
The Settings panel runs the same QR flow as `pnpm poc:qr`, with the QR rendered as an image.
1. Click **Start QR login**. The daemon opens the remote-auth gateway and shows the QR.
2. Scan the QR with the Discord mobile app and approve.
3. On success the token is written to the token file with mode 600. A running managed core
restarts with `VESKSLOP_FORCE_TOKEN=1` so the new account takes effect.
4. If Discord asks for a captcha, the panel shows the sitekey and a field for the hCaptcha response
token. Paste the token and submit.
The token never leaves the daemon. The API reports the account name and ID, not the token.
## API
The JSON API is the migration boundary. A future frontend calls the same routes. The page code is
replaceable.
| Method | Path | Body or query | Result |
| --- | --- | --- | --- |
| GET | `/api/health` | none | Core state, process state, owner, uptime, event count, query count |
| GET | `/api/core/status` | none | Bridge and core process state |
| POST | `/api/core/start` | `forceToken?` | Start the client |
| POST | `/api/core/stop` | none | Stop the managed client |
| POST | `/api/core/restart` | `forceToken?` | Restart the managed client |
| GET | `/api/auth` | none | Token file state and the QR session, with `qrDataUrl` when pending |
| POST | `/api/auth/qr` | none | Start a QR login session |
| POST | `/api/auth/qr/cancel` | none | Cancel the session |
| POST | `/api/auth/qr/captcha` | `key` | Submit an hCaptcha response token |
| GET | `/api/state` | none | User, guilds, and channels |
| GET | `/api/messages` | `channelId`, `limit`, `refresh?` | Messages from a channel. `refresh=1` fetches the latest from Discord and fills the client cache; otherwise cache-only. |
| POST | `/api/messages` | `channelId`, `content` | Send a message. Records a `manual` action. |
| POST | `/api/dm` | `userId` or `username`, `content` | Send a DM. Records a `manual` action. |
| GET | `/api/users` | `q`, `guildId?`, `limit?` | Search the client user cache |
| GET | `/api/dms` | `limit?` | Cached DM and group DM channels with recipients |
| GET | `/api/voice` | `userId?`, `guildId?` | One user's voice state |
| GET | `/api/voice/overview` | `guildId` | Occupied voice channels and members |
| POST | `/api/voice/join` | `channelId` | Reserved. Returns `501` |
| POST | `/api/voice/leave` | none | Reserved. Returns `501` |
| GET | `/api/logs` | `limit` | The persistent action log |
| GET | `/api/events` | none | Server-sent events |
Every response is JSON: `{ "ok": true, "result": ... }` or `{ "ok": false, "error": "..." }`. A
core validation error adds `code`.
The channel detail routes use the same pacing as the MCP tools. Sends are paced; reads take a small
gap. See [daemon.md](daemon.md).
## Not started
- Web UI authentication and user accounts.
- Voice join and leave. The UI elements exist and call the reserved routes.
- TTS playback, transcript capture, and MCP tool-call tracing.
- Favourites, presence, member counts, and message editing/deletion/reactions.
- Pagination of channel history.
- The full DM scope from the mockup: the rail always shows text, voice, and direct lists.