zib/Veskslop-t
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update docs for the debug web UI, voice plan, and current status

Browse Source
This commit is contained in:
Fisto
2026-09-13 15:47:51 +02:00
parent 2bcd8c6d7a
commit 65f0788502
14 changed files with 594 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/INDEX.md
+2
View File
@@ -9,6 +9,8 @@ This file lists the project documents.
| [architecture.md](architecture.md) | System parts, boundaries, and data flow |
| [core-bridge.md](core-bridge.md) | The Vesktop-side capability layer |
| [daemon.md](daemon.md) | The external MCP daemon |
| [webui.md](webui.md) | The web control UI and its JSON API |
| [voice.md](voice.md) | Voice state reads, planned actions, and lifecycle |
| [communication.md](communication.md) | Trust model, commands, and the trigger log |
| [auth.md](auth.md) | QR login and token persistence |
| [testing.md](testing.md) | Tests and the live test setup |
docs/architecture.md
+17 -12
View File
@@ -7,22 +7,24 @@ real Vesktop client, so the traffic comes from the official client.
## Parts
The system has four parts:
The system has five parts:
1. **Vesktop core** — the real Vesktop client. It runs headless. It exposes a small capability layer.
2. **Core bridge** — the capability layer. It accepts validated actions and pushes events.
3. **Veskslop daemon** — the external process. It holds all logic. It exposes MCP.
4. **MCP clients** — agents and tools. They call the daemon.
4. **Web UI** — a small browser control surface served by the daemon. See [webui.md](webui.md).
5. **MCP clients** — agents and tools. They call the daemon.
```
MCP clients (agents)
│ Streamable HTTP (MCP)
▼
Veskslop daemon ──── JSON lines over a Unix socket ────▶ Vesktop core
│ │
│ QR login, pacing, trigger log, │ validated actions + events
│ trust list, MCP tools ▼
│ Discord (official client traffic)
MCP clients (agents) Web browser (owner)
│ Streamable HTTP (MCP) │ HTTP + SSE
▼ ▼
Veskslop daemon ◀───────────────────── web UI listener
│ ──── JSON lines over a Unix socket ────▶ Vesktop core
│ │
│ QR login, pacing, trigger log, │ validated actions + events
│ trust list, MCP tools ▼
│ Discord (official client traffic)
▼
Local state files (.token, .veskslop-state.json)
```
@@ -55,9 +57,12 @@ when the action list changes.
| Pacing | `poc/src/pacing.ts` | Implemented |
| Trigger log and trust list | `poc/src/communication.ts`, `poc/src/puppet.ts` | Implemented and verified live |
| Context cache and response policy | `poc/src/message-cache.ts`, `poc/src/puppet.ts` | Implemented and verified with the mock |
| Chat commands and polls | `poc/src/puppet.ts`, `poc/src/communication.ts` | Implemented and verified with the mock |
| Chat commands and polls | `poc/src/puppet.ts`, `poc/src/communication.ts` | Implemented. Poll is paused (`POLL_ENABLED = false`). |
| Persistent action log | `poc/src/action-log.ts` | Implemented |
| Voice, audio, video | Not started | Not started |
| Web UI and JSON API | `poc/src/webui.ts`, `poc/public/` | Nocturne 7a debug shell + roadmap placeholders |
| User search | `src/renderer/puppet/actions.ts` (`user.search`) | Implemented |
| Voice state reads | `src/renderer/puppet/actions.ts`, `src/renderer/puppet/events.ts` | Implemented |
| Voice actions, audio, video | Not started | Not started |
## Data flow
docs/auth.md
+13
View File
@@ -18,6 +18,19 @@ captcha, the tool shows the challenge and reads a key from `HCAPTCHA_KEY` or fro
The flow was verified end-to-end on 2026-09-11.
## Web UI login
The web UI Login tab runs the same session (`poc/src/qr-session.ts`). See [webui.md](webui.md).
- `POST /api/auth/qr` starts the session. `GET /api/auth` returns the state and the QR image.
- On success the token is written to the token file. A running managed core restarts with
`VESKSLOP_FORCE_TOKEN=1`, so the new account takes effect.
- A captcha challenge shows the sitekey. Paste the hCaptcha response token to finish.
- The API never returns the token.
To log in a second account, set a different `TOKEN_FILE` for the daemon (and thus the spawned core)
and restart the daemon. One daemon manages one account at this time.
## Token file
- Default path: `poc/.token`, relative to the `poc/` directory.
docs/communication.md
+5 -1
View File
@@ -37,7 +37,8 @@ mention the puppet first.
| `whitelist list` | Show the owner and the trusted users with their tiers. |
| `whitelist add @user [user\|trusted\|admin]` | Add one or more users. The default tier is `user`. |
| `whitelist remove @user` | Remove one or more users from the trust list. |
| `poll <question> \| <opt 1> \| <opt 2> [\| opt 3 [\| opt 4]]` | Start a reaction poll. |
| `poll ...` | Start a reaction poll. Paused at this time. |
| `stay @user` | Planned with the voice backend. Pin the bot to a voice user. See [voice.md](voice.md). |
Notes:
@@ -49,6 +50,9 @@ Notes:
## Polls
**Paused.** `POLL_ENABLED` is `false` in `poc/src/puppet.ts`. The parser, the tests, and the code
stay in place. The daemon does not run a `poll` command.
The `poll` command posts a message and adds one number emoji per option. Discord users vote by
reacting. The poll closes after `pollSeconds` (default 120) and the daemon posts the result.
docs/core-bridge.md
+22 -4
View File
@@ -56,14 +56,28 @@ Event line:
| --- | --- | --- |
| `ping` | none | `"pong"` |
| `events.subscribe` | none | `{ "subscribed": true }`. The socket becomes an event subscriber. |
| `state.get` | none | The current user, the guilds, and the channels. Cache-only. |
| `voice.get` | optional `userId`, optional `guildId` | The voice channel and its members, or `channelId: null` |
| `state.get` | none | The current user, the guilds, and the channels. Cache-only. Includes user avatars and guild icons. |
| `voice.get` | optional `userId`, optional `guildId` | The voice channel and its members, or `channelId: null`. Without `guildId` it searches all guilds. |
| `voice.overview` | `guildId` | The occupied voice channels in the guild and their members |
| `user.search` | `query`, optional `guildId`, optional `limit` | Users from the client cache. Guild members when `guildId` is set. |
| `dm.list` | optional `limit` | Cached DM and group DM channels with recipients, icons, and avatars. Cache-only. |
| `message.send` | `channelId`, `content` | `{ channelId, messageId, sent: true }` |
| `message.read` | `channelId`, `limit` | `{ cached, messages }`. Cache-only. |
| `message.fetch` | `channelId`, `limit`, optional `before` or `after` | `{ fetched, messages }`. Fetches from Discord. |
| `message.read` | `channelId`, `limit` | `{ cached, messages }`. Cache-only. Messages include the author avatar and attachment CDN links. |
| `message.fetch` | `channelId`, `limit`, optional `before` or `after` | `{ fetched, messages }`. Fetches from Discord. Same message shape. |
| `reaction.add` | `channelId`, `messageId`, `emoji` | `{ added: true }` |
| `dm.send` | `userId`, `content` | `{ userId, channelId, sent: true }` |
## Events
The renderer normalizes client events and pushes them to subscribers.
| Event | Payload |
| --- | --- |
| `MESSAGE_CREATE` | Message ID, channel, guild, channel type, author, content, mentions, timestamp |
| `MESSAGE_REACTION_ADD` | Message ID, channel, guild, user, emoji |
| `MESSAGE_REACTION_REMOVE` | Same fields as add |
| `VOICE_STATE_UPDATE` | User, channel, guild, session, mute, deaf, self-mute, self-deaf |
## Validation rules
The core validates all input. The daemon cannot bypass these rules.
@@ -79,6 +93,10 @@ The core validates all input. The daemon cannot bypass these rules.
- `before` and `after` must be snowflakes. They are mutually exclusive. `message.fetch` uses them
to walk history.
- `voice.get` accepts snowflakes only. It reads the client cache. It never joins a call.
- `voice.overview` requires a guild snowflake. It reads the client cache.
- `user.search` takes a query of 1 to 100 characters and a limit of 1 to 25. It reads the client
cache only. An exact snowflake query returns that user when cached.
- `dm.list` takes a limit of 1 to 50. It reads `ChannelStore.getSortedPrivateChannels()` only.
- `reaction.add` accepts a short emoji string without line breaks. It counts as an outbound action
for the rate limit.
- A request line must not be larger than 64 KiB.
docs/daemon.md
+52 -8
View File
@@ -6,12 +6,17 @@ The daemon runs outside Vesktop. It holds all logic. It talks to the core bridge
| File | Role | Status |
| --- | --- | --- |
| `poc/src/control-bridge.ts` | MCP server, tool list, health endpoint | Implemented |
| `poc/src/control-bridge.ts` | MCP server, tool list, health endpoint, core reconnect | Implemented |
| `poc/src/core-process.ts` | Start, stop, and restart the Vesktop core process | Implemented |
| `poc/src/qr-session.ts` | Remote-auth QR login session (shared with the web UI) | Implemented |
| `poc/src/auth-service.ts` | Auth state for the web UI | Implemented |
| `poc/src/puppet.ts` | Trust list, command handling, trigger log, response policy | Implemented |
| `poc/src/communication.ts` | Pure log, parser, and context logic | Implemented |
| `poc/src/message-cache.ts` | Channel message cache in a tmp folder | Implemented |
| `poc/src/action-log.ts` | Persistent JSONL action log with rotation | Implemented |
| `poc/src/pacing.ts` | Paced queue and query coalescing | Implemented |
| `poc/src/pacing.ts` | Paced send queue and read queries | Implemented |
| `poc/src/webui.ts` | Web UI listener and JSON API | First draft |
| `poc/public/` | Static web UI (HTML, CSS, JS) | First draft |
| `poc/src/core-client.ts` | Client actions over the core bridge | Implemented (primary transport) |
| `poc/src/cdp-client.ts` | Chrome DevTools Protocol client | Development fallback |
| `poc/src/qr-login.ts` | Terminal QR login | Implemented and verified |
@@ -33,9 +38,14 @@ the page, so it has no core-side guardrails. Use it for debugging only.
## Pacing
The daemon paces all commands. The policy is a random delay of 500 to 2800 ms. The queue
serializes the commands. The queue coalesces identical concurrent reads. The daemon also prefers
cached state over queries.
The daemon paces all outbound sends. A send is `message.send` or `dm.send`.
- The first send of a burst waits 30 to 200 ms.
- Later sends in the same burst wait 500 to 2800 ms.
- A burst ends after a random 5 to 20 seconds of quiet. The next send is a fast first action.
Read queries are serialized with a small 100 to 300 ms gap. Reads do not reset the send burst.
Identical concurrent reads are coalesced. The daemon prefers cached state over queries.
The core adds a second, hard rate limit. See [core-bridge.md](core-bridge.md).
@@ -50,6 +60,9 @@ The daemon exposes MCP over Streamable HTTP. The default address is `http://127.
| `read_messages` | Read cached messages from a channel. |
| `send_dm` | Send a DM to a user. |
| `get_voice_state` | Which voice channel a user is in and who else is there. |
| `get_voice_overview` | Occupied voice channels in a guild and their members. |
| `search_users` | Search the client user cache by username, global name, or nickname. |
| `list_dms` | List cached DM and group DM channels with recipients. |
| `get_message_log` | Read the trigger log. |
| `get_action_log` | Read the persistent action log. |
| `run_commands` | Run several read-only tools in one call, in order. |
@@ -63,9 +76,12 @@ The daemon exposes MCP over Streamable HTTP. The default address is `http://127.
## Run commands
```sh
# Start the daemon. It connects to the core bridge.
# Start the daemon. It connects to the core bridge and starts the web UI.
pnpm control:bridge
# Start the daemon with a different web UI port.
WEBUI_PORT=18889 pnpm control:bridge
# Start a mock core for tests without Electron.
VESKSLOP_SOCKET=/tmp/vesktop-puppet-mock.sock pnpm control:mock
@@ -84,12 +100,34 @@ pnpm cdp:probe
CDP_EXPRESSION='...' pnpm cdp:dispatch
```
## Core process
The daemon can start, stop, and restart the Vesktop client (`poc/src/core-process.ts`). The web UI
Core tab and the `/api/core/*` routes use it.
- The spawn sets `VESKSLOP_HEADLESS=1` and `VESKSLOP_TOKEN_FILE` (the daemon token file).
- The default command is `node_modules/.bin/electron .` from the repo root. `VESKSLOP_CORE_CMD`
replaces it. `VESKSLOP_CORE_ARGS` replaces the default flags.
- On Linux without a `DISPLAY`, the daemon starts Xvfb on the first free display from `:99` and
points the client at it. An existing `DISPLAY` wins. `VESKSLOP_XVFB=0` disables auto-start.
- The status holds the last exit code and signal, and the last 20 output lines. The Core tab shows
both, so a failed start explains itself.
- The daemon stops a client that it started, and any Xvfb it started, when the daemon exits. An
externally started client is reported as `external` and is not managed.
- When the core bridge drops, the daemon retries every 2 s. A restart moves the bridge connection
to the new client. The in-memory trigger log resets; the state file and the action log survive.
- QR login lives in `poc/src/qr-session.ts` and is shared by the terminal tool and the web UI.
See [auth.md](auth.md).
Verified live on 2026-09-13: the web UI started the client under auto-Xvfb, the bridge connected,
`state.get` returned the real account and guilds, and stop exited with code 0.
## Health endpoint
`GET /health` returns the CDP state, the owner ID, and the query count.
`GET /health` returns the core connection state, the socket path, the owner ID, and the query count.
```json
{ "ok": true, "cdpConnected": true, "ownerId": "177117296140091393", "queries": 4 }
{ "ok": true, "coreConnected": true, "socketPath": "/run/user/1000/vesktop-puppet.sock", "ownerId": "177117296140091393", "queries": 4 }
```
## Settings and paths
@@ -103,6 +141,10 @@ CDP_EXPRESSION='...' pnpm cdp:dispatch
| `VESKSLOP_CACHE_DIR` | `<tmp>/veskslop-cache` | Context cache folder |
| `VESKSLOP_CACHE_MAX_BYTES` | 50 MiB | Context cache size limit |
| `MCP_HOST` / `MCP_PORT` | `127.0.0.1` / `8788` | MCP listen address |
| `WEBUI_HOST` / `WEBUI_PORT` | `0.0.0.0` / `8789` | Web UI listen address. No authentication. |
| `VESKSLOP_CORE_CMD` | unset | Full command that starts the client. Overrides the default. |
| `VESKSLOP_CORE_ARGS` | `--no-sandbox --disable-gpu --disable-dev-shm-usage` | Flags for the default Electron command. |
| `VESKSLOP_XVFB` | unset | Set to `0` to disable automatic Xvfb start on Linux. |
## How to add an MCP tool
@@ -115,6 +157,8 @@ CDP_EXPRESSION='...' pnpm cdp:dispatch
## Security notes
- The MCP server binds to localhost by default.
- The web UI binds to all interfaces by default and has no authentication. Use it over LAN or
Tailscale only. Do not expose the port to the public internet. See [webui.md](webui.md).
- Remote access must go over LAN or Tailscale. Do not expose the daemon to the public internet.
- The daemon holds no Discord credentials. The client holds the session.
- The daemon token file is separate from the client session. See [auth.md](auth.md).
docs/decisions.md
+49 -2
View File
@@ -45,8 +45,8 @@ already does this work.
## ADR-5: Two pacing layers
**Decision.** The daemon paces all commands with a random delay of 500 to 2800 ms. The core adds a
hard limit of 500 ms between sends and 30 sends per minute.
**Decision.** The daemon paces all outbound sends with a human-emulation policy (ADR-12). The core
adds a hard limit of 500 ms between sends and 30 sends per minute.
**Reason.** The daemon policy looks human. The core limit is a safety net. A daemon bug cannot cause
a send flood.
@@ -97,3 +97,50 @@ Redact token-shaped strings before the write.
**Reason.** The in-memory trigger log resets at restart. The action log gives a durable audit trail
without storing raw credentials.
## ADR-12: Human send pacing
**Decision.** The first outbound send of a burst waits 30 to 200 ms. Later sends wait 500 to
2800 ms. A burst ends after a random 5 to 20 seconds of quiet. Reads take a small 100 to 300 ms gap
and do not reset the burst.
**Reason.** A real client sends the first message quickly and then slows down. Reads are not
visible in the same way and must not consume the send budget. The core hard limit stays as the
safety net. See [daemon.md](daemon.md).
## ADR-13: Web UI on a separate listener
**Decision.** The daemon serves the web UI on its own listener (`0.0.0.0:8789`) with no
authentication. The MCP server stays on localhost. The page is plain HTML, CSS, and JS behind a
JSON API.
**Reason.** Remote control over LAN or Tailscale needs a reachable port. A separate listener keeps
the MCP surface local. The JSON API is the stable boundary, so a future frontend replaces the page
without a backend change. No auth fits a private network, but the port must never be exposed to the
public internet.
## ADR-14: Voice joins muted, presence first
**Decision.** `voice.join` joins muted. Audio playback is a later feature and unmutes only while
playing. The bot leaves on its own after the owner or the stay target leaves, plus a delay.
**Reason.** A muted bot captures no audio and does not disturb the call. The lifecycle keeps the
bot from lingering in empty calls. See [voice.md](voice.md).
## ADR-15: Nocturne design and the debug shell
**Decision.** The web UI follows the Nocturne design system and the 7a debug shell from the design
handoff (`poc/WebUIMockup.zip`). `poc/public/nocturne.css` carries the token set. The page stays
plain HTML, CSS, and JavaScript with no build step. The JSON API stays the migration boundary.
**Reason.** The owner chose the design direction. Vanilla keeps the no-build constraint. Roadmap
gaps (TTS, transcript, tool-call tracing, favourites, voice actions) show as marked placeholders so
the shape of the finished UI is visible while the backend catches up.
## ADR-16: Takeover is a read-only gate
**Decision.** The Takeover switch is client-side. Off, the composer is locked and the page cannot
send. On, manual sends are allowed and each one is written to the action log as a `manual` entry.
**Reason.** The switch exists to stop accidental use of the bot account, not to model agent
ownership. The API does not enforce it, so other clients can still send. See [webui.md](webui.md).
docs/testing.md
+90 -7
View File
@@ -6,7 +6,7 @@ This document describes the test setup, the run commands, and the known gaps.
The pure daemon logic has unit tests. The tests cover the ring buffers, the mention helpers, the
command parser, the poll parser, the trust helpers, the context assembly, the redaction, and the
pacing range. The cache tests cover merge, dedupe, trimming, and eviction.
pacing policy (`planSend`). The cache tests cover merge, dedupe, trimming, and eviction.
```sh
cd poc
@@ -50,12 +50,68 @@ VESKSLOP_SOCKET=/tmp/vesktop-puppet-mock.sock TEST_TRIGGER_CONTENT="whitelist li
```
Expected result: the daemon log shows one trusted entry with the parsed command. The mock log shows
the reply (`message.send`). The reply wait is inside the 500 to 2800 ms window.
the reply (`message.send`). A first reply after a quiet period waits 30 to 200 ms. Further replies
in the same burst wait 500 to 2800 ms.
The mock also covers tiers, continuation, polls, reactions, and the action log. Set config values
with `set_puppet_config` before a test. Inject a reaction with
`TEST_TRIGGER_REACTION=1 TEST_REACTION_MESSAGE_ID=msg-1 TEST_REACTION_EMOJI="1️⃣" pnpm test:trigger`.
## Web UI smoke test
The web UI starts with the daemon. The mock core is enough for this test.
```sh
# Terminal 1: mock core
VESKSLOP_SOCKET=/tmp/vesktop-puppet-mock.sock pnpm control:mock
# Terminal 2: daemon (web UI on a free port)
VESKSLOP_SOCKET=/tmp/vesktop-puppet-mock.sock WEBUI_PORT=18789 pnpm control:bridge
# Terminal 3: check the API
curl -s http://127.0.0.1:18789/api/health
curl -s http://127.0.0.1:18789/api/state
curl -s "http://127.0.0.1:18789/api/users?q=zz"
curl -s http://127.0.0.1:18789/api/dms
curl -s "http://127.0.0.1:18789/api/voice/overview?guildId=100000000000000001"
curl -s http://127.0.0.1:18789/api/events # server-sent events
```
The page itself can be rendered headless for a visual check. With Xvfb running:
```sh
DISPLAY=:99 node_modules/.bin/electron /tmp/check.js --no-sandbox
```
where `check.js` loads `http://127.0.0.1:18789/` in a `BrowserWindow` and logs page errors. This
was used to verify the 7a shell, the locked/unlocked composer, and the Settings slide-over.
The voice join and leave routes return `501` until the voice backend lands. See [webui.md](webui.md).
Process control and login work without the mock core:
```sh
curl -s http://127.0.0.1:18789/api/core/status
curl -s http://127.0.0.1:18789/api/auth
curl -s -X POST http://127.0.0.1:18789/api/auth/qr
curl -s -X POST http://127.0.0.1:18789/api/auth/qr/cancel
```
Starting `/api/auth/qr` opens a real remote-auth gateway session. Cancel it if you do not intend to
log in. Approving a login replaces the token file.
To test start and stop without Electron, run the daemon with a stand-in command:
```sh
VESKSLOP_SOCKET=/tmp/vesktop-puppet-mock.sock VESKSLOP_CORE_CMD="sleep 300" WEBUI_PORT=18789 pnpm control:bridge
curl -s -X POST http://127.0.0.1:18789/api/core/start
curl -s -X POST http://127.0.0.1:18789/api/core/stop
```
A real client start needs a display and the Electron shared libraries. On Linux, start Xvfb first.
The container needs `libasound.so.2` for Electron. Extract the library locally when the system
package is not available.
## CDP probe
The probe checks the CDP fallback transport. Start the client with
@@ -74,7 +130,23 @@ event.
## Live test setup
The container has no display and no PipeWire. Use a virtual display.
The daemon can start the client for you, including a virtual display. This is the easy path.
```sh
cd poc
pnpm control:bridge
# Open http://<host>:8789/ and press Start in Settings → Core process.
# Or from the shell:
curl -s -X POST http://127.0.0.1:8789/api/core/start
curl -s http://127.0.0.1:8789/api/core/status
```
The daemon starts Xvfb when `DISPLAY` is unset, spawns `electron .`, and connects the bridge. The
token file seeds the login. The Settings slide-over shows the process state, the last exit code,
and the last 20 output lines. Set `VESKSLOP_XVFB=0` to turn off auto-Xvfb.
Manual setup (no auto-start):
```sh
# 1. Start a virtual display. On Linux, install Xvfb or extract it locally.
@@ -97,8 +169,9 @@ Notes:
- `VENCORD_USER_DATA_DIR` pins the Electron profile. The profile holds the client session.
- Write `{"firstLaunch": false}` to `<DATA_DIR>/state.json` before the first start. This step skips
the first-launch window. The `VESKSLOP_HEADLESS=1` flag does the same job.
- The container needs `libasound.so.2` for Electron. Extract the library locally when the system
package is not available.
- The container needs `libasound.so.2` for Electron. Install `libasound2` or extract the library
locally and point `LD_LIBRARY_PATH` at it.
- The live start was verified on 2026-09-13 through the web UI route.
## Sanctioned live target
@@ -120,6 +193,14 @@ The core bridge was verified live on 2026-09-11:
the parsed command.
- The client ran headless under Xvfb with the token seeded from the token file.
Core process control and the web QR session were verified live on 2026-09-13:
- `POST /api/core/start` started the client, including auto-Xvfb, and the bridge connected.
- `state.get` returned the real account and guilds.
- `POST /api/core/stop` exited with code 0 and released Xvfb.
- The QR session reached `pending` with a scannable QR and cancelled cleanly. A full approval needs
a phone and was not run.
The CDP transport was verified earlier with the same checks.
## Known gaps
@@ -128,6 +209,8 @@ The CDP transport was verified earlier with the same checks.
| --- | --- |
| Live trigger tests need an incoming message. | Use the mock core, or send from zzib's account. |
| `reaction.add` and the message ID return are mock-verified only. | Confirm Discord's internal `addReaction` lookup on the first live run. |
| Polls and other chat commands are mock-verified only. | Run them on the live client. |
| Polls and other chat commands are mock-verified only. | Poll is paused; the tests still cover the parser. |
| `message.read` is cache-only. | A channel that the client never opened returns zero messages. |
| Voice, audio, and video are not started. | No coverage. |
| `VOICE_STATE_UPDATE` normalization is mock-verified only. | Confirm the event fields on the first live run. |
| The web UI has no authentication. | Keep it on LAN or Tailscale only. |
| Voice actions, audio, and video are not started. | No coverage. |
docs/voice.md
+67
View File
@@ -0,0 +1,67 @@
# Voice
Voice state reads exist. Voice actions do not. This document records the plan and the decisions.
## Current
| Part | Status |
| --- | --- |
| `voice.get` | Implemented. One user's voice channel and its members. Without `guildId` it searches all guilds. |
| `voice.overview` | Implemented. Occupied voice channels and members for one guild. |
| `VOICE_STATE_UPDATE` events | Normalized and pushed. Mock-verified. Live check pending. |
| Join and leave | Not started. |
| Audio in and out | Not started. |
| Video | Not started. |
The core reads `VoiceStateStore` only. It never joins a call on its own.
## Planned actions
| Action | Parameters | Result |
| --- | --- | --- |
| `voice.join` | `channelId`, optional `mute` (default true) | `{ channelId, joined: true }` |
| `voice.leave` | none | `{ left: true }` |
| `voice.move` | `channelId` | `{ channelId, moved: true }` |
Rules:
- The channel must be a voice channel. The core validates the snowflake and the channel type.
- Join joins the call muted. The bot does not capture or send audio.
- Voice actions are state changes, not send floods. They stay out of the message pacing queue, but
the daemon rate limits them separately.
- The join implementation goes through Discord's internal channel action, located at runtime. The
lookup is unverified until the first live run. This is the same pattern as `reaction.add`.
The plan for later audio: the bot unmutes only while it plays an audio stream, then mutes again.
Audio playback is a later feature.
## Lifecycle
The bot leaves a call on its own in two cases.
1. **Owner leaves.** When the owner leaves the call, the bot waits `voiceLeaveDelaySeconds` and
then leaves. A rejoin inside the window cancels the leave.
2. **A stay target leaves.** The `stay @user` command pins the bot to a user. When that user
leaves, the bot waits the same delay and then leaves.
`stay @user` needs a chat command. Chat commands are paused at this time, so the command waits for
the voice backend. The default delay is proposed as 30 seconds and lives in the config.
The bot never leaves while the owner or the stay target is present. A manual `voice.leave` wins
over both rules.
## Collector account
The collector account is a stripped-down client with read-only voice membership. The daemon uses it
when the puppet cannot see a user's voice channel or the channel members. It stays deferred. See
[TODO.md](../TODO.md).
## Verification list
A live run must confirm these points.
1. The internal `selectVoiceChannel` lookup exists and joins the given channel.
2. The internal disconnect action leaves the call.
3. `VOICE_STATE_UPDATE` carries the fields that the core normalizes.
4. The join is muted and Discord shows the mute state.
5. The auto-leave timer works across a client restart. The timer is daemon state, not core state.
docs/webui.md
+147
View File
@@ -0,0 +1,147 @@
# Web UI
The daemon serves the debug web control UI. The owner uses it to observe the account the agent
sees, read and send messages, search users, and see voice membership. The shell follows the design
handoff in `poc/WebUIMockup.zip`; several panels are roadmap placeholders.
## Run
The UI starts with the daemon. No extra command is needed.
```sh
cd poc
pnpm control:bridge
# Page
http://<host>:8789/
# API
http://<host>:8789/api/health
```
| Setting | Default | Purpose |
| --- | --- | --- |
| `WEBUI_HOST` | `0.0.0.0` | Listen address |
| `WEBUI_PORT` | `8789` | Listen port |
## Security
The UI has **no authentication**. Anyone who can reach the port controls the bot, can start and
stop the client, and can start a QR login. Bind the daemon to a private LAN or a Tailscale address
only. Do not expose the port to the public internet. See [daemon.md](daemon.md).
The MCP server stays on `127.0.0.1:8788`. Only the UI binds outside localhost.
## Page
- Plain HTML, CSS, and JavaScript in `poc/public/`. No build step and no frontend framework.
- The look follows the Nocturne design system (`poc/public/nocturne.css`) from the design handoff
in `poc/WebUIMockup.zip`. Implemented screen: **7a**, the docked three-column debug shell.
- Layout: 50px header, conditional call strip, `236px | 1fr | 306px` body, and the agent console
bar (35px collapsed, resizable drawer expanded).
- Header: wordmark, guild picker, health indicators, Takeover switch, Settings.
- Left rail: guild switcher, text channels, voice channels with inline participants, direct
messages, and the account footer.
- Center: channel header, message list, composer, and the TTS panel.
- Right column: In voice, Favourites, Open DMs & groups.
- Console: Tool calls, Events (live SSE), Transcript. The ticker shows the newest event.
- Opening a chat fetches the latest 50 messages from Discord into the client cache. The refresh
button by the channel header re-fetches. Live message events read the cache.
- Attachments render from their Discord CDN links. Images display inline; other files are link
chips. The daemon never downloads or stores attachment bytes.
- Avatars and guild icons load from the Discord CDN with a monogram fallback.
- Composer and TTS drafts persist in `localStorage` per channel. They survive re-renders, the
bridge connecting, and a page reload. A successful send clears the channel draft.
- When the bridge connects after the page loaded, the page reloads state without touching drafts.
- Roadmap gaps are shown, not hidden: TTS, transcript, tool-call tracing, favourites, and voice
join/monitor appear as marked placeholders.
## Takeover
Takeover is a read-only switch. Off, the composer is locked (a `read-only` tag sits next to the
lock) and the page cannot send. On, the composer unlocks and every manual send is written to the
action log as a `manual` entry. The choice is stored in `localStorage`. The setting is client-side
only; the API itself does not enforce it.
## Settings
The Settings button opens a slide-over with the server-side controls:
- **Core process** — state, last exit, output, Start / Stop / Restart.
- **Account login** — token file state, QR login, captcha.
- **Visible guilds** — checkboxes for the guild switcher and picker. Stored in `localStorage`;
"Show all" clears the filter.
- **Action log** — the persistent JSONL log, newest first.
## Core process
The Settings panel starts, stops, and restarts the Vesktop client. The daemon spawns the process
from the repo root and labels its output with `[core]` in the daemon log.
- The default command is `node_modules/.bin/electron .` with
`--no-sandbox --disable-gpu --disable-dev-shm-usage`.
- Override the command with `VESKSLOP_CORE_CMD` and the flags with `VESKSLOP_CORE_ARGS`.
- The spawn sets `VESKSLOP_HEADLESS=1` and points `VESKSLOP_TOKEN_FILE` at the daemon token file.
- On Linux without a `DISPLAY`, the daemon starts Xvfb automatically. `VESKSLOP_XVFB=0` disables
this. A real `DISPLAY` wins.
- Only a process started by the current daemon is managed. An externally started client shows as
`external` and cannot be stopped from the UI.
- The status shows the last exit code or signal and the last 20 output lines. A failed start is
visible on the page instead of only in the daemon log.
## Login
The Settings panel runs the same QR flow as `pnpm poc:qr`, with the QR rendered as an image.
1. Click **Start QR login**. The daemon opens the remote-auth gateway and shows the QR.
2. Scan the QR with the Discord mobile app and approve.
3. On success the token is written to the token file with mode 600. A running managed core
restarts with `VESKSLOP_FORCE_TOKEN=1` so the new account takes effect.
4. If Discord asks for a captcha, the panel shows the sitekey and a field for the hCaptcha response
token. Paste the token and submit.
The token never leaves the daemon. The API reports the account name and ID, not the token.
## API
The JSON API is the migration boundary. A future frontend calls the same routes. The page code is
replaceable.
| Method | Path | Body or query | Result |
| --- | --- | --- | --- |
| GET | `/api/health` | none | Core state, process state, owner, uptime, event count, query count |
| GET | `/api/core/status` | none | Bridge and core process state |
| POST | `/api/core/start` | `forceToken?` | Start the client |
| POST | `/api/core/stop` | none | Stop the managed client |
| POST | `/api/core/restart` | `forceToken?` | Restart the managed client |
| GET | `/api/auth` | none | Token file state and the QR session, with `qrDataUrl` when pending |
| POST | `/api/auth/qr` | none | Start a QR login session |
| POST | `/api/auth/qr/cancel` | none | Cancel the session |
| POST | `/api/auth/qr/captcha` | `key` | Submit an hCaptcha response token |
| GET | `/api/state` | none | User, guilds, and channels |
| GET | `/api/messages` | `channelId`, `limit`, `refresh?` | Messages from a channel. `refresh=1` fetches the latest from Discord and fills the client cache; otherwise cache-only. |
| POST | `/api/messages` | `channelId`, `content` | Send a message. Records a `manual` action. |
| POST | `/api/dm` | `userId` or `username`, `content` | Send a DM. Records a `manual` action. |
| GET | `/api/users` | `q`, `guildId?`, `limit?` | Search the client user cache |
| GET | `/api/dms` | `limit?` | Cached DM and group DM channels with recipients |
| GET | `/api/voice` | `userId?`, `guildId?` | One user's voice state |
| GET | `/api/voice/overview` | `guildId` | Occupied voice channels and members |
| POST | `/api/voice/join` | `channelId` | Reserved. Returns `501` |
| POST | `/api/voice/leave` | none | Reserved. Returns `501` |
| GET | `/api/logs` | `limit` | The persistent action log |
| GET | `/api/events` | none | Server-sent events |
Every response is JSON: `{ "ok": true, "result": ... }` or `{ "ok": false, "error": "..." }`. A
core validation error adds `code`.
The channel detail routes use the same pacing as the MCP tools. Sends are paced; reads take a small
gap. See [daemon.md](daemon.md).
## Not started
- Web UI authentication and user accounts.
- Voice join and leave. The UI elements exist and call the reserved routes.
- TTS playback, transcript capture, and MCP tool-call tracing.
- Favourites, presence, member counts, and message editing/deletion/reactions.
- Pagination of channel history.
- The full DM scope from the mockup: the rail always shows text, voice, and direct lists.