291 lines
19 KiB
Markdown
291 lines
19 KiB
Markdown
# Web UI
|
||
|
||
The daemon serves the debug web control UI. The owner uses it to observe the account the agent
|
||
sees, read and send messages, search users, and see voice membership. The shell follows the design
|
||
handoff in `poc/WebUIMockup.zip`; several panels are roadmap placeholders.
|
||
|
||
## Run
|
||
|
||
The UI starts with the daemon. No extra command is needed.
|
||
|
||
```sh
|
||
cd poc
|
||
pnpm control:bridge
|
||
|
||
# Page
|
||
http://<host>:8789/
|
||
|
||
# API
|
||
http://<host>:8789/api/health
|
||
```
|
||
|
||
| Setting | Default | Purpose |
|
||
| --- | --- | --- |
|
||
| `WEBUI_HOST` | `0.0.0.0` | Listen address |
|
||
| `WEBUI_PORT` | `8789` | Listen port |
|
||
|
||
## Security
|
||
|
||
The UI has **no authentication**. Anyone who can reach the port controls the bot, can start and
|
||
stop the client, and can start a QR login. Bind the daemon to a private LAN or a Tailscale address
|
||
only. Do not expose the port to the public internet. See [daemon.md](daemon.md).
|
||
|
||
The MCP server stays on `127.0.0.1:8788`. Only the UI binds outside localhost.
|
||
|
||
## TLS
|
||
|
||
The UI serves HTTPS with a self-signed certificate so browser media APIs have a secure context.
|
||
The daemon generates the certificate with `openssl` on first run (localhost plus the host's LAN
|
||
addresses) and stores it under `<state dir>/tls/`. The browser shows a warning once; accept it.
|
||
|
||
| Setting | Default | Purpose |
|
||
| --- | --- | --- |
|
||
| `WEBUI_TLS` | `1` | Set to `0` to serve plain HTTP. |
|
||
| `WEBUI_TLS_KEY` / `WEBUI_TLS_CERT` | unset | Use an existing certificate instead. |
|
||
|
||
With plain HTTP, `navigator.mediaDevices` only exists on localhost; remote browsers must use HTTPS.
|
||
|
||
## Page
|
||
|
||
- Plain HTML, CSS, and JavaScript in `poc/public/`. No build step and no frontend framework.
|
||
- The look follows the Nocturne design system (`poc/public/nocturne.css`) from the design handoff
|
||
in `poc/WebUIMockup.zip`. Implemented screen: **7a**, the docked three-column debug shell.
|
||
- Layout: 50px header, conditional call strip, `236px | 1fr | 306px` body, and the agent console
|
||
bar (35px collapsed, resizable drawer expanded).
|
||
- Header: wordmark, **Guild / DM** scope tabs, guild picker, health indicators, **Client audio**,
|
||
Takeover switch, Settings. While the bridge is down a **Start server** button appears with a
|
||
pulsing bloom; if the whole daemon is unreachable it becomes **Refresh** and reloads the page.
|
||
The body dims and stops taking input while the bridge is down.
|
||
- Left rail (Guild tab): guild switcher, text channels, voice channels with inline participants,
|
||
and **People** — users recently seen in a voice channel (click opens their cached DM). Clicking a
|
||
voice channel joins it (muted); clicking the active channel leaves.
|
||
- Left rail (DM tab): every cached DM and group with unread badge, avatar, name, last-message time,
|
||
and a favourite star.
|
||
- Right column: **In voice** (favourited users in a call, grouped by call), **Favourites**, and
|
||
**Open DMs & groups** as quick access. The guild's own occupied channels live in the left rail.
|
||
- Speaking users show animated level bars and a bold name (design 2c: audio → video → share marks).
|
||
The bars scale up when speaking starts and stay on until an explicit stop, a mute, or a channel
|
||
change. A missing stop event leaves them on by design; they do not decay on their own. The
|
||
fade-out runs the equalizer inside a shrinking container, so the amplitude drops to nothing
|
||
instead of the block collapsing at full height. Muted members never show bars.
|
||
Nicknames come from the guild member store; DMs fall back to the username.
|
||
- Console: Tool calls, Events (live SSE), Transcript. The ticker shows the newest event. The
|
||
Transcript tab lists captured segments (time, speaker, text, WAV) and updates live. It has
|
||
**Start**, **Stop**, and **Refresh** buttons; the status line shows `capturing` or `capture off`
|
||
plus the segment count.
|
||
- Opening a chat fetches the latest 50 messages from Discord into the client cache. The refresh
|
||
button by the channel header re-fetches. Live message events read the cache. Edited messages show
|
||
an `edited` marker and replies show `↳ replying to <name>`.
|
||
- Attachments render from their Discord CDN links. Images display inline; other files are link
|
||
chips. The daemon never downloads or stores attachment bytes.
|
||
- Avatars and guild icons load from the Discord CDN with a monogram fallback.
|
||
- Composer and TTS drafts persist in `localStorage` per channel. They survive re-renders, the
|
||
bridge connecting, and a page reload. A successful send clears the channel draft.
|
||
- When the bridge connects after the page loaded, the page reloads state without touching drafts.
|
||
The reload retries while the client boots, and the health poll keeps retrying if the first
|
||
attempt failed.
|
||
- Roadmap gaps are shown, not hidden: TTS, transcript, tool-call tracing, favourites, and voice
|
||
join/monitor appear as marked placeholders.
|
||
|
||
## Mobile
|
||
|
||
At 768px and below the shell becomes the 1c phone layout. One pane shows at a time behind a
|
||
four-tab bar (Users, Chat, Call, Activity), with 44px hit targets and the call bar pinned above
|
||
the content.
|
||
|
||
- **Users** is the left rail: guild switcher, channels, voice, and people.
|
||
- **Chat** is the centre column: messages, composer, TTS panel.
|
||
- **Call** is the live call: the whole right column, with the current call's members listed first
|
||
(not only favourites).
|
||
- **Activity** is the full agent console (Tool calls, Events, Transcript).
|
||
- The Chat tab carries the total DM unread count. The header compresses to one line: scope tabs,
|
||
guild picker, health dot and event rate, icon-only Client audio and Settings, and the Takeover
|
||
switch.
|
||
- The selected tab persists in `localStorage`. Picking a channel from Users switches to Chat;
|
||
picking a voice channel switches to Call.
|
||
|
||
The first draft renders headless at 390x844. Remaining gaps against 1c: the transcript and the TTS
|
||
box live in Chat, not Call, and the connection stats are not chips yet.
|
||
|
||
## DMs and favourites
|
||
|
||
- The **DM** scope tab shows all cached DMs in the rail and the conversation in the centre.
|
||
- Unread badges sit left of the avatar: purple for unread, orange for mentions while Takeover is
|
||
on; while Takeover is off they are grey, meaning the bot saw them and you did not.
|
||
- The hollow star on a row adds it to favourites, stored in the daemon preferences. Starring a 1:1
|
||
DM favourites that user; starring a group DM favourites the channel.
|
||
- Favourited users get presence tracking. If they are in a call, they appear under **In voice**,
|
||
grouped per call. The right column's **Favourites** shows their status when they are not.
|
||
- The last-message time comes from the channel's last message snowflake: a time today, `yesterday`,
|
||
or a short date.
|
||
|
||
## Media and Bot
|
||
|
||
- **Media** and **Bot** are scope buttons beside Guild and DM. They open a full-window view below
|
||
the header and the call strip (the shell is covered; only the header, and the call strip when it
|
||
is visible, stay on screen). Settings slides over the media view. Mobile is out of scope for the
|
||
media feature.
|
||
- Media lists scraped items with kind chips (All, Images, GIFs, Videos, Audio) and filters for
|
||
source (any, attachments, stickers, embeds), source channel, poster, aspect ratio, resolution,
|
||
and a posted-date range (from/to). **New scrape…** opens a popup with **Server** and **Group
|
||
chat** tabs: servers expand into their text channels, each channel (and each group chat) has
|
||
**Links** and **Download** buttons, and group chats always use the puppet. **History** lists
|
||
each scrape with start time, duration, pages, messages, media counts, bytes, files, and previews.
|
||
The scrape strip shows live progress with a pulse and **Stop**. Images and GIFs render from the
|
||
cached preview when one exists, then the local file, then the CDN URL; videos and audio are link
|
||
cards with the poster and posted time in the tooltip.
|
||
- Bot shows the slopbot connection state and the scrape queue.
|
||
- The views are self-contained: `poc/public/media.js` and `media.css`, over the `/api/media/*`
|
||
routes in the table below. See [media.md](media.md).
|
||
|
||
## Takeover
|
||
|
||
Takeover is a read-only switch. Off, the composer is locked (a `read-only` tag sits next to the
|
||
lock) and the page cannot send. On, the composer unlocks and every manual send is written to the
|
||
action log as a `manual` entry. The choice is stored in `localStorage`. The setting is client-side
|
||
only; the API itself does not enforce it.
|
||
|
||
## Settings
|
||
|
||
The Settings button opens a slide-over:
|
||
|
||
- **Visible guilds** — top level. Checkboxes filter the guild switcher and picker live. **Save**
|
||
writes the selection to the daemon preferences (`/api/preferences`); **Hide all** and **Show all**
|
||
select none or every guild. All visible is stored as `null`, none as `[]`. All browsers share
|
||
the saved state, and Save works while the client is disconnected.
|
||
- **Client** — a collapsible group with:
|
||
- **Bot audio** — engine status, TTS provider state, a **Connect devices** button, and the
|
||
capture toggle. These are the daemon host's devices.
|
||
- **Speech** — the TTS engine (Auto plus every configured engine: espeak-ng, the command engine
|
||
by its `VESKSLOP_TTS_NAME`, and the HTTP API), language, and voice. **Start engine** warms the
|
||
selected engine (starts Xvfb, Wine, and the host for the command engine). Selecting an engine
|
||
switches the voice list to that engine; the language row hides for single-language engines.
|
||
Voices are grouped by base language and filtered by the chosen language; English, Swedish,
|
||
German, Danish, Norwegian, Russian, and Cherokee are pinned on top. Stored in the daemon
|
||
preferences. Speed (0.5x–2.0x with 0.5/0.75/1/1.25/1.5 presets), pitch (-12 to +12
|
||
semitones), and an Expression slider (0–100, Flextalk voices only) live in the TTS panel and
|
||
save to the same preferences; controls the selected voice does not support stay hidden. For
|
||
Flextalk voices the TTS panel also opens the **Flextalk voice profile** window: a movable panel
|
||
that recenters on each open, with character sliders, five named presets, radio-kit effects
|
||
(high-pass, echo, word mode), and test lines (Attention, Laboratory, Critical, Health,
|
||
Radiation, plus two custom) that speak immediately.
|
||
- **Display** — the timestamp timezone. Defaults to the browser's zone
|
||
(`Intl.DateTimeFormat().resolvedOptions().timeZone`); pick any IANA zone to override. Stored in
|
||
`localStorage` and applied to messages, the transcript, the event log, the action log, and the
|
||
core status.
|
||
- **Browser audio** — puppeting controls: the browser output device for Client audio, a
|
||
placeholder mic select, and the call-audio state. **Enable browser audio** asks for the
|
||
microphone permission, which browsers require before they list output devices. Separate from
|
||
the bot's devices. Clicking **Client audio** in the header does the same: it asks for the
|
||
permission, unlocks the audio context in the same click, and then starts listening. A second
|
||
click stops it.
|
||
- **Bot** — slopbot connection state, the media storage limit in GiB with an info tooltip (5% overrun
|
||
and the warning rule), the usage bar segmented by image, GIF, video, audio, and other, and
|
||
**Clean media…** with a 5 second confirmation countdown. See [media.md](media.md).
|
||
- **Server** — a collapsible group with:
|
||
- **Core process** — state, last exit, output, Start / Stop / Restart.
|
||
- **Account login** — token file state, QR login, captcha.
|
||
- **Action log** — the persistent JSONL log, newest first.
|
||
|
||
## Core process
|
||
|
||
The Settings panel starts, stops, and restarts the Vesktop client. The daemon spawns the process
|
||
from the repo root and labels its output with `[core]` in the daemon log.
|
||
|
||
- The default command is `node_modules/.bin/electron .` with
|
||
`--no-sandbox --disable-gpu --disable-dev-shm-usage`.
|
||
- Override the command with `VESKSLOP_CORE_CMD` and the flags with `VESKSLOP_CORE_ARGS`.
|
||
- The spawn sets `VESKSLOP_HEADLESS=1` and points `VESKSLOP_TOKEN_FILE` at the daemon token file.
|
||
- On Linux without a `DISPLAY`, the daemon starts Xvfb automatically. `VESKSLOP_XVFB=0` disables
|
||
this. A real `DISPLAY` wins.
|
||
- Only a process started by the current daemon is managed. An externally started client shows as
|
||
`external` and cannot be stopped from the UI.
|
||
- The status shows the last exit code or signal and the last 20 output lines. A failed start is
|
||
visible on the page instead of only in the daemon log.
|
||
|
||
## Login
|
||
|
||
The Settings panel runs the same QR flow as `pnpm poc:qr`, with the QR rendered as an image.
|
||
|
||
1. Click **Start QR login**. The daemon opens the remote-auth gateway and shows the QR.
|
||
2. Scan the QR with the Discord mobile app and approve.
|
||
3. On success the token is written to the token file with mode 600. A running managed core
|
||
restarts with `VESKSLOP_FORCE_TOKEN=1` so the new account takes effect.
|
||
4. If Discord asks for a captcha, the panel shows the sitekey and a field for the hCaptcha response
|
||
token. Paste the token and submit.
|
||
|
||
The token never leaves the daemon. The API reports the account name and ID, not the token.
|
||
|
||
## API
|
||
|
||
The JSON API is the migration boundary. A future frontend calls the same routes. The page code is
|
||
replaceable.
|
||
|
||
| Method | Path | Body or query | Result |
|
||
| --- | --- | --- | --- |
|
||
| GET | `/api/health` | none | Core state, process state, owner, uptime, event count, query count |
|
||
| GET | `/api/core/status` | none | Bridge and core process state |
|
||
| POST | `/api/core/start` | `forceToken?` | Start the client |
|
||
| POST | `/api/core/stop` | none | Stop the managed client |
|
||
| POST | `/api/core/restart` | `forceToken?` | Restart the managed client |
|
||
| GET | `/api/auth` | none | Token file state and the QR session, with `qrDataUrl` when pending |
|
||
| POST | `/api/auth/qr` | none | Start a QR login session |
|
||
| POST | `/api/auth/qr/cancel` | none | Cancel the session |
|
||
| POST | `/api/auth/qr/captcha` | `key` | Submit an hCaptcha response token |
|
||
| GET | `/api/state` | none | User, guilds, and channels |
|
||
| GET | `/api/messages` | `channelId`, `limit`, `refresh?` | Messages from a channel. `refresh=1` fetches the latest from Discord and fills the client cache; otherwise cache-only. |
|
||
| POST | `/api/messages` | `channelId`, `content` | Send a message. Records a `manual` action. |
|
||
| POST | `/api/dm` | `userId` or `username`, `content` | Send a DM. Records a `manual` action. |
|
||
| GET | `/api/users` | `q`, `guildId?`, `limit?` | Search the client user cache |
|
||
| GET | `/api/dms` | `limit?` | Cached DM and group DM channels with recipients |
|
||
| GET | `/api/voice` | `userId?`, `guildId?` | One user's voice state |
|
||
| GET | `/api/voice/overview` | `guildId` | Occupied voice channels and members |
|
||
| GET | `/api/voice/volume` | `userIds?` comma-separated | Per-user volumes plus output volume; defaults to the current call |
|
||
| POST | `/api/voice/volume` | `userId`, `volume` | Set a user's playback volume (0 to 200) |
|
||
| POST | `/api/voice/join` | `channelId`, `mute?` | Join a voice channel (muted by default) |
|
||
| POST | `/api/voice/leave` | none | Leave the current voice channel |
|
||
| POST | `/api/voice/move` | `channelId` | Move to another voice channel |
|
||
| GET | `/api/preferences` | none | Web UI preferences (`visibleGuilds`, `tts`) |
|
||
| POST | `/api/preferences` | `visibleGuilds?`, `tts?` | Update preferences. Stored in `<state dir>/preferences.json`. |
|
||
| GET | `/api/audio` | none | Audio engine, TTS, queue, ducking, capture state |
|
||
| GET | `/api/audio/devices` | none | The client's input and output devices |
|
||
| GET | `/api/audio/tts/voices` | `engine?` | Voices for the selected engine (espeak-ng or `VESKSLOP_TTS_VOICES`) |
|
||
| POST | `/api/audio/connect` | none | Re-apply the virtual devices to the client |
|
||
| WS | `/audio` | `{"type":"listen","on":bool}` | Raw 48 kHz mono PCM of the call |
|
||
| POST | `/api/audio/devices` | `inputDeviceId?`, `outputDeviceId?` | Set the client's devices |
|
||
| POST | `/api/audio/speak` | `text`, `provider?`, `voice?`, `speed?`, `pitch?` | Queue text to speech |
|
||
| POST | `/api/audio/stop` | none | Stop speech and clear the queue |
|
||
| POST | `/api/audio/capture` | `start` | Start or stop capture |
|
||
| GET | `/api/transcript` | `limit?`, `sessionId?` | Sessions and captured segments |
|
||
| GET | `/api/media` | `kind?`, `source?`, `channelId?`, `authorId?`, `aspect?`, `resolution?`, `after?`, `before?`, `limit?`, `offset?` | Media items across every scrape, newest first. `source` is `attachment`, `sticker`, or `embed`; `aspect` and `resolution` are the preset keys; `after`/`before` are ISO times on `posted_at`. Items carry `authorName`, `postedAt`, `width`, `height`, `aspectRatio`, `contentType`, `durationSecs`; `localUrl`/`thumbUrl` when a file or preview exists |
|
||
| GET | `/api/media/:scrapeId/:itemId/file` | none | The downloaded file, content-typed from its extension |
|
||
| GET | `/api/media/:scrapeId/:itemId/thumb` | none | The cached WebP preview |
|
||
| GET | `/api/media/scrapes` | none | Every scrape with state and counters |
|
||
| GET | `/api/media/status` | none | Slopbot connection state and the scrape list |
|
||
| GET | `/api/media/authors` | none | Distinct posters across every scrape, with counts |
|
||
| GET | `/api/media/stats` | same filters as `/api/media` | Counts: `total`, `downloaded`, `cached`, `notDownloaded` |
|
||
| GET | `/api/media/storage` | none | Usage against the limit, per-kind sizes, `nearLimit`; marks old scrapes when over |
|
||
| POST | `/api/media/scrape` | `channelId`, `guildId?`, `mode?`, `backend?`, `confirm?`, `downloadDir?`, `after?`, `before?` | Start a media scrape. `backend` is `auto` (default), `slopbot`, or `puppet`; `downloadDir` is an absolute path for downloaded files (previews stay local); `after`/`before` are ISO dates bounding the walk. Near the limit: `409` with code `MEDIA_LIMIT` unless `confirm: true` |
|
||
| POST | `/api/media/count` | `channelId`, `guildId?`, `after?`, `before?` | Discord search total for the channel or range. Returns `{ count }` or `{ count: null, reason }` (slopbot guild channels only) |
|
||
| GET | `/api/media/errors` | none | Bot and scrape errors, newest first |
|
||
| POST | `/api/media/errors/clear` | none | Clear the error list |
|
||
| POST | `/api/media/scrape/stop` | `id` | Stop a media scrape |
|
||
| POST | `/api/media/clean` | `ids?` | Delete the given scrapes, or every scrape when `ids` is absent |
|
||
| GET | `/api/logs` | `limit` | The persistent action log |
|
||
| GET | `/api/events` | none | Server-sent events |
|
||
|
||
Every response is JSON: `{ "ok": true, "result": ... }` or `{ "ok": false, "error": "..." }`. A
|
||
core validation error adds `code`.
|
||
|
||
The channel detail routes use the same pacing as the MCP tools. Sends are paced; reads take a small
|
||
gap. See [daemon.md](daemon.md).
|
||
|
||
## Not started
|
||
|
||
- Web UI authentication and user accounts.
|
||
- Voice auto-leave and the `stay @user` command.
|
||
- TTS playback, transcript capture, and MCP tool-call tracing.
|
||
- Favourites, presence, member counts, and message editing/deletion/reactions.
|
||
- Pagination of channel history.
|
||
- The full DM scope from the mockup: the rail always shows text, voice, and direct lists.
|