CURRENT.md — session state

Last updated: 2026-09-14

This file tracks the work state. Read it first. Update it before you stop.

Current focus (broad)

The text layer is done. Voice actions and audio work. The local LLM harness (Veskslop agent, Go + Eino) is scaffolded and smoke-tested; the daemon-side task push is next. The web UI has a first mobile draft at 390px.

In progress

The Veskslop agent lives in agent/ (Go 1.25+, Eino v0.9.19, remote llama.cpp over the OpenAI-compatible API, daemon tools over MCP Streamable HTTP). It is disabled by default (VESKSLOP_AGENT_ENABLED=1), takes tasks on POST /task, and logs every tool call. Smoke-tested on 2026-09-14: 28 tools loaded, one ReAct round, final message returned. The daemon does not push tasks to it yet. Plan and env vars: docs/agent.md.
Remote agents use the daemon's MCP endpoint directly (query state, send notifications). A notification tool set and a documented LAN/Tailscale bind are planned.
Mobile web UI first draft: four tabs (Users, Chat, Call, Activity), one pane at a time, 44px targets, tab badge, call bar pinned, Activity becomes the full console. Verified headless at 390x844. Next: transcript and TTS box in the Call tab, per the 1c design.
Fixed a pre-existing web UI bug: loadDms called the nonexistent renderDirect, so the DM rail only rendered from other paths. It now calls renderDmRail().
Voice audio is implemented: daemon-managed virtual devices (veskslop_tts, veskslop_mic, veskslop_capture), TTS providers (local command + HTTP API), queued playback through ffplay, ducking on VOICE_SPEAKING, and WAV capture. The daemon starts pulseaudio when pactl cannot reach a server and reports install hints. Verified live on 2026-09-13 after installing pulseaudio/ffmpeg: engine ok, devices created, PCM streamed.
The Web UI serves self-signed HTTPS (openssl, stored under the state dir) so browser media APIs have a secure context. WEBUI_TLS=0 disables it.
Capture and transcript are implemented: ffmpeg reads the client output, a VAD cuts it into utterances, each becomes a WAV plus a transcript entry with the dominant speaker, and a pluggable STT hook (local command or HTTP API) fills the text. Live events feed the console Transcript tab; get_transcript and /api/transcript read the store. VESKSLOP_CAPTURE_AUTO=1 records calls automatically. Verified end to end with real PulseAudio on 2026-09-13.
STT uses Vosk (CPU, light): poc/scripts/stt-vosk.py plus a small model in poc/models/ (auto-detected). Verified with a TTS→capture→Vosk round trip. Other providers remain available via VESKSLOP_STT_CMD/_API_URL.
TTS defaults to espeak-ng when installed; engine and voice live in Settings → Client → Speech (daemon preferences, so all browsers share them).
Preferences moved to <state dir>/preferences.json and no longer depend on the core, so guild visibility saves even while the client is down. API responses send Cache-Control: no-store.
Browser audio has an Enable browser audio button that requests the microphone permission, which browsers require before listing output devices.
Browser microphone puppeting (talking through the bot) is next; it needs a mix sink, a pacat input, and echo handling.
The web UI follows the Nocturne design handoff (poc/WebUIMockup.zip). The 7a shell is implemented and rendered headless for review: header, call strip, three-column body, composer, and the agent console.
Takeover is a client-side read-only gate. Off, the composer is locked. Manual sends are logged.
Core process control is implemented and verified live: the daemon starts the client under auto-Xvfb, the bridge connects, and stop exits cleanly.
Web QR login is implemented. The session reached pending against the real gateway; a full phone approval is pending.

Done

Validated core bridge: protocol, actions, validation, structured errors, hard outbound rate limit.
Auth persistence: pinned DATA_DIR, token file, preload seeding, VESKSLOP_HEADLESS.
Daemon on the core-bridge transport. CDP stays as a development fallback.
Trusted-only triggers: DMs and direct mentions. Trust tiers: user, trusted, admin.
Context: 15 messages or 4000 characters, time stamps, file names only, incremental fetch, tmp cache with a size limit, fetch cooldown.
Continuation window: a rushed follow-up joins the request context.
Response policy: origin channel or DM. Sensitive commands default to DM.
Persistent action log: triggers, commands, replies, errors, rotation, and secret redaction.
poll is paused (POLL_ENABLED = false). The parser and its tests stay.
Human send pacing: first send of a burst waits 30-200 ms, later sends 500-2800 ms, and the burst resets after a random 5-20 s of quiet. Reads take 100-300 ms and do not reset the burst.
user.search: cache and guild-member lookup. voice.get searches all guilds without a guildId.
voice.overview: occupied voice channels and members for a guild.
VOICE_STATE_UPDATE events: normalized and pushed. Mock-verified.
Voice actions: voice.join (muted), voice.leave, and voice.move in the core, with daemon routes (/api/voice/*), MCP tools, and Web UI wiring. Voice actions are paced 250-750 ms and written to the action log. The internal lookups were probed on the live client.
Web UI voice rail: every voice channel is clickable. Clicking joins (muted); clicking the active channel leaves. Verified with the stateful mock.
Web UI startup: a Start server button appears left of Takeover while the bridge is down and hides when it connects. State reloads retry while the client boots, so guilds appear without a manual refresh. Core requests now time out after VESKSLOP_REQUEST_TIMEOUT_MS instead of hanging forever on a missing reply.
Settings: "Visible guilds" stays top level; Audio sits under a collapsible "Client" group; Core process, Account login, and Action log sit under a collapsible "Server" group.
Web UI timestamps use the browser timezone by default; Settings → Client → Display can override it with any IANA zone. The choice is per browser (localStorage).
Web UI preferences: visible guilds persist server-side in the daemon state file (/api/preferences), so every browser shares them. The Settings panel has an explicit Save button; selections filter the UI live and persist only on Save.
DM list: recipients resolve from IDs through the user store, so names and avatars load.
Guild menu: fixed the light default button background.
Web UI: Nocturne design tokens in poc/public/nocturne.css, the 7a debug shell, live guild and channel rail, messages, DMs, voice membership, agent console with a live event tab, and a Settings slide-over for core process, login, visible guilds, and the action log. Roadmap surfaces show as placeholders.
Web UI data: opening a chat fetches the latest 50 messages into the client cache; attachments hotlink Discord CDN URLs and are never stored; avatars and guild icons load from the CDN with a monogram fallback; the page reloads itself when the bridge connects.
Web UI drafts: composer and TTS text persist per channel in localStorage and survive re-renders, reconnects, and page reloads.
Visible guilds: the Settings panel filters the guild switcher and picker.
Takeover: client-side read-only switch with localStorage persistence; manual sends are written to the action log with type: "manual".
dm.list: cached DM and group DM channels, exposed over MCP as list_dms.
Core process control: the daemon starts, stops, and restarts the client, reconnects the bridge after a restart, auto-starts Xvfb when DISPLAY is missing, and stops what it started on exit. Override with VESKSLOP_CORE_CMD/ARGS; disable Xvfb with VESKSLOP_XVFB=0.
Web QR login: the Settings panel runs the remote-auth session and renders the QR. The token file is written on approval and a running managed core restarts with the new token.
Unit tests cover the pacing policy. pnpm lint and pnpm testTypes pass at the root.
Web UI navigation: Guild/DM scope tabs, a DM rail with unread badges (purple/orange, grey while Takeover is off), last-message times, favourite stars, and a People list of users seen in voice. Favourited users get presence and call tracking, grouped by call in In voice. Speaking shows level bars and bold names (2c marks for mute/deaf/video/share). The body dims while the bridge is down; the Start button becomes Refresh when the daemon is unreachable.
TTS settings are split into language and voice, grouped by base language with the requested languages pinned on top.
Text notifications are parsed: edits update the context cache, deletes drop from it, trusted reactions are logged, and replies are kept in the request context. A reply to a puppet message counts as a trigger without a mention. The web UI marks edited messages and replies.
Voice auto-leave is implemented: when the owner (zzib) leaves the bot's call, the bot leaves after voiceLeaveDelaySeconds (default 30). A rejoin cancels it. Verified with the mock, including cancellation. stay @user is deferred.
Audio verified end to end on 2026-09-14: TTS played into the virtual mic (recording veskslop_mic peaked at -4.1 dB) and the browser monitor streamed PCM over the TLS WebSocket.
Next: per-user limits and the collector account. The agent attachment (remote llama.cpp, MCP transport, event delivery) waits for a design discussion; the draft prompt has open issues to settle first.

Next (in order)

Live verification: voice.join/voice.leave end to end, user.search, voice.overview, VOICE_STATE_UPDATE, and a full web QR login approval on the real client.
Agent: daemon task push for free-form triggers (commands stay local), tool allowlist, per-conversation serialization, and tests against a fake model.
Collector account: read-only modules for voice membership fallback.
Per-user rate limit and command cooldowns.
Mobile web UI: transcript and TTS in the Call tab; verify on a phone.

Open discussions (need the user)

Web UI auth. Accepted: no auth, LAN or Tailscale only. Revisit before any wider exposure.
MCP event delivery. Decided for the local harness: the daemon pushes tasks to the agent's POST /task. The transport details (retry, dedupe IDs, queue) are still open. Remote agents poll or call tools directly; they do not receive pushes.
Untrusted logging. Untrusted interactions are ignored at this time. Confirm whether to log untrusted DMs and mentions again.
Privacy pass. Only token-shaped and mfa. strings are redacted, and only in the persistent action log. A fuller policy (retention, cache clearing, message redaction) is deferred.
Live checks. reaction.add, user.search, voice.overview, and the voice state event normalization are mock-verified only. The QR login session reached pending and cancel against the real gateway; the approval path needs the phone. Confirm on the first real-client run.

Blocked / needs input

The original repo zib/Veskslop has a broken server-side git setup. Fetches and clones fail with bad pack header because a script (/home/git/.sys_health_s3) runs on the Veskslop read path. The admin must fix it. Use zib/Veskslop-t (remote veskslop-t) until then.
main on the original server is 68fc8bb (zib, "Upload files to docs/images"). It diverged from the local main (24447f5). Reconcile after the server is fixed.

Where to look

Backlog: TODO.md
Design documents: docs/INDEX.md
Design handoff: poc/WebUIMockup.zip (screen 7a is implemented)
Agent rules: AGENTS.md
Daemon code: poc/src/
Web UI code: poc/src/webui.ts, poc/public/
Core bridge code: src/shared/puppetProtocol.ts, src/renderer/puppet/, src/main/puppetBridge.ts

12 KiB Raw Blame History