214 lines
12 KiB
Markdown
214 lines
12 KiB
Markdown
# Web UI
|
|
|
|
The daemon serves the debug web control UI. The owner uses it to observe the account the agent
|
|
sees, read and send messages, search users, and see voice membership. The shell follows the design
|
|
handoff in `poc/WebUIMockup.zip`; several panels are roadmap placeholders.
|
|
|
|
## Run
|
|
|
|
The UI starts with the daemon. No extra command is needed.
|
|
|
|
```sh
|
|
cd poc
|
|
pnpm control:bridge
|
|
|
|
# Page
|
|
http://<host>:8789/
|
|
|
|
# API
|
|
http://<host>:8789/api/health
|
|
```
|
|
|
|
| Setting | Default | Purpose |
|
|
| --- | --- | --- |
|
|
| `WEBUI_HOST` | `0.0.0.0` | Listen address |
|
|
| `WEBUI_PORT` | `8789` | Listen port |
|
|
|
|
## Security
|
|
|
|
The UI has **no authentication**. Anyone who can reach the port controls the bot, can start and
|
|
stop the client, and can start a QR login. Bind the daemon to a private LAN or a Tailscale address
|
|
only. Do not expose the port to the public internet. See [daemon.md](daemon.md).
|
|
|
|
The MCP server stays on `127.0.0.1:8788`. Only the UI binds outside localhost.
|
|
|
|
## TLS
|
|
|
|
The UI serves HTTPS with a self-signed certificate so browser media APIs have a secure context.
|
|
The daemon generates the certificate with `openssl` on first run (localhost plus the host's LAN
|
|
addresses) and stores it under `<state dir>/tls/`. The browser shows a warning once; accept it.
|
|
|
|
| Setting | Default | Purpose |
|
|
| --- | --- | --- |
|
|
| `WEBUI_TLS` | `1` | Set to `0` to serve plain HTTP. |
|
|
| `WEBUI_TLS_KEY` / `WEBUI_TLS_CERT` | unset | Use an existing certificate instead. |
|
|
|
|
With plain HTTP, `navigator.mediaDevices` only exists on localhost; remote browsers must use HTTPS.
|
|
|
|
## Page
|
|
|
|
- Plain HTML, CSS, and JavaScript in `poc/public/`. No build step and no frontend framework.
|
|
- The look follows the Nocturne design system (`poc/public/nocturne.css`) from the design handoff
|
|
in `poc/WebUIMockup.zip`. Implemented screen: **7a**, the docked three-column debug shell.
|
|
- Layout: 50px header, conditional call strip, `236px | 1fr | 306px` body, and the agent console
|
|
bar (35px collapsed, resizable drawer expanded).
|
|
- Header: wordmark, **Guild / DM** scope tabs, guild picker, health indicators, **Client audio**,
|
|
Takeover switch, Settings. While the bridge is down a **Start server** button appears with a
|
|
pulsing bloom; if the whole daemon is unreachable it becomes **Refresh** and reloads the page.
|
|
The body dims and stops taking input while the bridge is down.
|
|
- Left rail (Guild tab): guild switcher, text channels, voice channels with inline participants,
|
|
and **People** — users recently seen in a voice channel (click opens their cached DM). Clicking a
|
|
voice channel joins it (muted); clicking the active channel leaves.
|
|
- Left rail (DM tab): every cached DM and group with unread badge, avatar, name, last-message time,
|
|
and a favourite star.
|
|
- Right column: **In voice** (favourited users in a call, grouped by call), **Favourites**, and
|
|
**Open DMs & groups** as quick access. The guild's own occupied channels live in the left rail.
|
|
- Speaking users show animated level bars and a bold name (design 2c: audio → video → share marks).
|
|
Nicknames come from the guild member store; DMs fall back to the username.
|
|
- Console: Tool calls, Events (live SSE), Transcript. The ticker shows the newest event. The
|
|
Transcript tab lists captured segments (time, speaker, text, WAV) and updates live.
|
|
- Opening a chat fetches the latest 50 messages from Discord into the client cache. The refresh
|
|
button by the channel header re-fetches. Live message events read the cache.
|
|
- Attachments render from their Discord CDN links. Images display inline; other files are link
|
|
chips. The daemon never downloads or stores attachment bytes.
|
|
- Avatars and guild icons load from the Discord CDN with a monogram fallback.
|
|
- Composer and TTS drafts persist in `localStorage` per channel. They survive re-renders, the
|
|
bridge connecting, and a page reload. A successful send clears the channel draft.
|
|
- When the bridge connects after the page loaded, the page reloads state without touching drafts.
|
|
The reload retries while the client boots, and the health poll keeps retrying if the first
|
|
attempt failed.
|
|
- Roadmap gaps are shown, not hidden: TTS, transcript, tool-call tracing, favourites, and voice
|
|
join/monitor appear as marked placeholders.
|
|
|
|
## DMs and favourites
|
|
|
|
- The **DM** scope tab shows all cached DMs in the rail and the conversation in the centre.
|
|
- Unread badges sit left of the avatar: purple for unread, orange for mentions while Takeover is
|
|
on; while Takeover is off they are grey, meaning the bot saw them and you did not.
|
|
- The hollow star on a row adds it to favourites, stored in the daemon preferences. Starring a 1:1
|
|
DM favourites that user; starring a group DM favourites the channel.
|
|
- Favourited users get presence tracking. If they are in a call, they appear under **In voice**,
|
|
grouped per call. The right column's **Favourites** shows their status when they are not.
|
|
- The last-message time comes from the channel's last message snowflake: a time today, `yesterday`,
|
|
or a short date.
|
|
|
|
## Takeover
|
|
|
|
Takeover is a read-only switch. Off, the composer is locked (a `read-only` tag sits next to the
|
|
lock) and the page cannot send. On, the composer unlocks and every manual send is written to the
|
|
action log as a `manual` entry. The choice is stored in `localStorage`. The setting is client-side
|
|
only; the API itself does not enforce it.
|
|
|
|
## Settings
|
|
|
|
The Settings button opens a slide-over:
|
|
|
|
- **Visible guilds** — top level. Checkboxes filter the guild switcher and picker live. **Save**
|
|
writes the selection to the daemon preferences (`/api/preferences`); **Hide all** and **Show all**
|
|
select none or every guild. All visible is stored as `null`, none as `[]`. All browsers share
|
|
the saved state, and Save works while the client is disconnected.
|
|
- **Client** — a collapsible group with:
|
|
- **Bot audio** — engine status, TTS provider state, a **Connect devices** button, and the
|
|
capture toggle. These are the daemon host's devices.
|
|
- **Speech** — the TTS engine (Auto / Local / API), language, and voice. Voices are grouped by
|
|
base language and filtered by the chosen language; English, Swedish, German, Danish,
|
|
Norwegian, Russian, and Cherokee are pinned on top. Stored in the daemon preferences.
|
|
- **Display** — the timestamp timezone. Defaults to the browser's zone
|
|
(`Intl.DateTimeFormat().resolvedOptions().timeZone`); pick any IANA zone to override. Stored in
|
|
`localStorage` and applied to messages, the transcript, the event log, the action log, and the
|
|
core status.
|
|
- **Browser audio** — puppeting controls: the browser output device for Client audio, a
|
|
placeholder mic select, and the call-audio state. **Enable browser audio** asks for the
|
|
microphone permission, which browsers require before they list output devices. Separate from
|
|
the bot's devices.
|
|
- **Server** — a collapsible group with:
|
|
- **Core process** — state, last exit, output, Start / Stop / Restart.
|
|
- **Account login** — token file state, QR login, captcha.
|
|
- **Action log** — the persistent JSONL log, newest first.
|
|
|
|
## Core process
|
|
|
|
The Settings panel starts, stops, and restarts the Vesktop client. The daemon spawns the process
|
|
from the repo root and labels its output with `[core]` in the daemon log.
|
|
|
|
- The default command is `node_modules/.bin/electron .` with
|
|
`--no-sandbox --disable-gpu --disable-dev-shm-usage`.
|
|
- Override the command with `VESKSLOP_CORE_CMD` and the flags with `VESKSLOP_CORE_ARGS`.
|
|
- The spawn sets `VESKSLOP_HEADLESS=1` and points `VESKSLOP_TOKEN_FILE` at the daemon token file.
|
|
- On Linux without a `DISPLAY`, the daemon starts Xvfb automatically. `VESKSLOP_XVFB=0` disables
|
|
this. A real `DISPLAY` wins.
|
|
- Only a process started by the current daemon is managed. An externally started client shows as
|
|
`external` and cannot be stopped from the UI.
|
|
- The status shows the last exit code or signal and the last 20 output lines. A failed start is
|
|
visible on the page instead of only in the daemon log.
|
|
|
|
## Login
|
|
|
|
The Settings panel runs the same QR flow as `pnpm poc:qr`, with the QR rendered as an image.
|
|
|
|
1. Click **Start QR login**. The daemon opens the remote-auth gateway and shows the QR.
|
|
2. Scan the QR with the Discord mobile app and approve.
|
|
3. On success the token is written to the token file with mode 600. A running managed core
|
|
restarts with `VESKSLOP_FORCE_TOKEN=1` so the new account takes effect.
|
|
4. If Discord asks for a captcha, the panel shows the sitekey and a field for the hCaptcha response
|
|
token. Paste the token and submit.
|
|
|
|
The token never leaves the daemon. The API reports the account name and ID, not the token.
|
|
|
|
## API
|
|
|
|
The JSON API is the migration boundary. A future frontend calls the same routes. The page code is
|
|
replaceable.
|
|
|
|
| Method | Path | Body or query | Result |
|
|
| --- | --- | --- | --- |
|
|
| GET | `/api/health` | none | Core state, process state, owner, uptime, event count, query count |
|
|
| GET | `/api/core/status` | none | Bridge and core process state |
|
|
| POST | `/api/core/start` | `forceToken?` | Start the client |
|
|
| POST | `/api/core/stop` | none | Stop the managed client |
|
|
| POST | `/api/core/restart` | `forceToken?` | Restart the managed client |
|
|
| GET | `/api/auth` | none | Token file state and the QR session, with `qrDataUrl` when pending |
|
|
| POST | `/api/auth/qr` | none | Start a QR login session |
|
|
| POST | `/api/auth/qr/cancel` | none | Cancel the session |
|
|
| POST | `/api/auth/qr/captcha` | `key` | Submit an hCaptcha response token |
|
|
| GET | `/api/state` | none | User, guilds, and channels |
|
|
| GET | `/api/messages` | `channelId`, `limit`, `refresh?` | Messages from a channel. `refresh=1` fetches the latest from Discord and fills the client cache; otherwise cache-only. |
|
|
| POST | `/api/messages` | `channelId`, `content` | Send a message. Records a `manual` action. |
|
|
| POST | `/api/dm` | `userId` or `username`, `content` | Send a DM. Records a `manual` action. |
|
|
| GET | `/api/users` | `q`, `guildId?`, `limit?` | Search the client user cache |
|
|
| GET | `/api/dms` | `limit?` | Cached DM and group DM channels with recipients |
|
|
| GET | `/api/voice` | `userId?`, `guildId?` | One user's voice state |
|
|
| GET | `/api/voice/overview` | `guildId` | Occupied voice channels and members |
|
|
| POST | `/api/voice/join` | `channelId`, `mute?` | Join a voice channel (muted by default) |
|
|
| POST | `/api/voice/leave` | none | Leave the current voice channel |
|
|
| POST | `/api/voice/move` | `channelId` | Move to another voice channel |
|
|
| GET | `/api/preferences` | none | Web UI preferences (`visibleGuilds`, `tts`) |
|
|
| POST | `/api/preferences` | `visibleGuilds?`, `tts?` | Update preferences. Stored in `<state dir>/preferences.json`. |
|
|
| GET | `/api/audio` | none | Audio engine, TTS, queue, ducking, capture state |
|
|
| GET | `/api/audio/devices` | none | The client's input and output devices |
|
|
| GET | `/api/audio/tts/voices` | none | Local TTS voices (espeak-ng) |
|
|
| POST | `/api/audio/connect` | none | Re-apply the virtual devices to the client |
|
|
| WS | `/audio` | `{"type":"listen","on":bool}` | Raw 48 kHz mono PCM of the call |
|
|
| POST | `/api/audio/devices` | `inputDeviceId?`, `outputDeviceId?` | Set the client's devices |
|
|
| POST | `/api/audio/speak` | `text`, `provider?`, `voice?` | Queue text to speech |
|
|
| POST | `/api/audio/stop` | none | Stop speech and clear the queue |
|
|
| POST | `/api/audio/capture` | `start` | Start or stop capture |
|
|
| GET | `/api/transcript` | `limit?`, `sessionId?` | Sessions and captured segments |
|
|
| GET | `/api/logs` | `limit` | The persistent action log |
|
|
| GET | `/api/events` | none | Server-sent events |
|
|
|
|
Every response is JSON: `{ "ok": true, "result": ... }` or `{ "ok": false, "error": "..." }`. A
|
|
core validation error adds `code`.
|
|
|
|
The channel detail routes use the same pacing as the MCP tools. Sends are paced; reads take a small
|
|
gap. See [daemon.md](daemon.md).
|
|
|
|
## Not started
|
|
|
|
- Web UI authentication and user accounts.
|
|
- Voice auto-leave and the `stay @user` command.
|
|
- TTS playback, transcript capture, and MCP tool-call tracing.
|
|
- Favourites, presence, member counts, and message editing/deletion/reactions.
|
|
- Pagination of channel history.
|
|
- The full DM scope from the mockup: the rail always shows text, voice, and direct lists.
|