Veskslop-t/CURRENT.md

# CURRENT.md — session state

Last updated: 2026-09-14

This file tracks the work state. Read it first. Update it before you stop.

## Current focus (broad)

The text layer is done. Voice actions and audio work. The local LLM harness (Veskslop agent, Go +
Eino) is scaffolded and smoke-tested; the daemon-side task push is next. The web UI has a first
mobile draft at 390px.

## In progress

- The Veskslop agent lives in `agent/` (Go 1.25+, Eino v0.9.19, remote llama.cpp over the
  OpenAI-compatible API, daemon tools over MCP Streamable HTTP). It is disabled by default
  (`VESKSLOP_AGENT_ENABLED=1`), takes tasks on `POST /task`, and logs every tool call. Smoke-tested
  on 2026-09-14: 28 tools loaded, one ReAct round, final message returned. The daemon does not push
  tasks to it yet. Plan and env vars: [docs/agent.md](docs/agent.md).
- Remote agents use the daemon's MCP endpoint directly (query state, send notifications). A
  notification tool set and a documented LAN/Tailscale bind are planned.
- Mobile web UI first draft: four tabs (Users, Chat, Call, Activity), one pane at a time, 44px
  targets, tab badge, call bar pinned, Activity becomes the full console. Verified headless at
  390x844. Next: transcript and TTS box in the Call tab, per the 1c design.
- Fixed a pre-existing web UI bug: `loadDms` called the nonexistent `renderDirect`, so the DM rail
  only rendered from other paths. It now calls `renderDmRail()`.
- Voice audio is implemented: daemon-managed virtual devices (`veskslop_tts`, `veskslop_mic`,
  `veskslop_capture`), TTS providers (local command + HTTP API), queued playback through `ffplay`,
  ducking on `VOICE_SPEAKING`, and WAV capture. The daemon starts `pulseaudio` when `pactl` cannot
  reach a server and reports install hints. Verified live on 2026-09-13 after installing
  `pulseaudio`/`ffmpeg`: engine ok, devices created, PCM streamed.
- The Web UI serves self-signed HTTPS (`openssl`, stored under the state dir) so browser media
  APIs have a secure context. `WEBUI_TLS=0` disables it.
- Capture and transcript are implemented: ffmpeg reads the client output, a VAD cuts it into
  utterances, each becomes a WAV plus a transcript entry with the dominant speaker, and a
  pluggable STT hook (local command or HTTP API) fills the text. Live events feed the console
  Transcript tab; `get_transcript` and `/api/transcript` read the store. `VESKSLOP_CAPTURE_AUTO=1`
  records calls automatically. Verified end to end with real PulseAudio on 2026-09-13.
- STT uses Vosk (CPU, light): `poc/scripts/stt-vosk.py` plus a small model in `poc/models/`
  (auto-detected). Verified with a TTS→capture→Vosk round trip. Other providers remain available
  via `VESKSLOP_STT_CMD`/`_API_URL`.
- TTS defaults to `espeak-ng` when installed; engine and voice live in Settings → Client → Speech
  (daemon preferences, so all browsers share them).
- Preferences moved to `<state dir>/preferences.json` and no longer depend on the core, so guild
  visibility saves even while the client is down. API responses send `Cache-Control: no-store`.
- Browser audio has an **Enable browser audio** button that requests the microphone permission,
  which browsers require before listing output devices.
- Browser microphone puppeting (talking through the bot) is next; it needs a mix sink, a `pacat`
  input, and echo handling.
- The web UI follows the Nocturne design handoff (`poc/WebUIMockup.zip`). The 7a shell is
  implemented and rendered headless for review: header, call strip, three-column body, composer,
  and the agent console.
- Takeover is a client-side read-only gate. Off, the composer is locked. Manual sends are logged.
- Core process control is implemented and verified live: the daemon starts the client under
  auto-Xvfb, the bridge connects, and stop exits cleanly.
- Web QR login is implemented. The session reached `pending` against the real gateway; a full
  phone approval is pending.

## Done

- Validated core bridge: protocol, actions, validation, structured errors, hard outbound rate limit.
- Auth persistence: pinned `DATA_DIR`, token file, preload seeding, `VESKSLOP_HEADLESS`.
- Daemon on the core-bridge transport. CDP stays as a development fallback.
- Trusted-only triggers: DMs and direct mentions. Trust tiers: user, trusted, admin.
- Context: 15 messages or 4000 characters, time stamps, file names only, incremental fetch, tmp
  cache with a size limit, fetch cooldown.
- Continuation window: a rushed follow-up joins the request context.
- Response policy: origin channel or DM. Sensitive commands default to DM.
- Persistent action log: triggers, commands, replies, errors, rotation, and secret redaction.
- `poll` is paused (`POLL_ENABLED = false`). The parser and its tests stay.
- Human send pacing: first send of a burst waits 30-200 ms, later sends 500-2800 ms, and the burst
  resets after a random 5-20 s of quiet. Reads take 100-300 ms and do not reset the burst.
- `user.search`: cache and guild-member lookup. `voice.get` searches all guilds without a `guildId`.
- `voice.overview`: occupied voice channels and members for a guild.
- `VOICE_STATE_UPDATE` events: normalized and pushed. Mock-verified.
- Voice actions: `voice.join` (muted), `voice.leave`, and `voice.move` in the core, with daemon
  routes (`/api/voice/*`), MCP tools, and Web UI wiring. Voice actions are paced 250-750 ms and
  written to the action log. The internal lookups were probed on the live client.
- Web UI voice rail: every voice channel is clickable. Clicking joins (muted); clicking the active
  channel leaves. Verified with the stateful mock.
- Web UI startup: a **Start server** button appears left of Takeover while the bridge is down and
  hides when it connects. State reloads retry while the client boots, so guilds appear without a
  manual refresh. Core requests now time out after `VESKSLOP_REQUEST_TIMEOUT_MS` instead of
  hanging forever on a missing reply.
- Settings: "Visible guilds" stays top level; Audio sits under a collapsible "Client" group;
  Core process, Account login, and Action log sit under a collapsible "Server" group.
- Web UI timestamps use the browser timezone by default; Settings → Client → Display can override
  it with any IANA zone. The choice is per browser (`localStorage`).
- Web UI preferences: visible guilds persist server-side in the daemon state file
  (`/api/preferences`), so every browser shares them. The Settings panel has an explicit **Save**
  button; selections filter the UI live and persist only on Save.
- DM list: recipients resolve from IDs through the user store, so names and avatars load.
- Guild menu: fixed the light default button background.
- Web UI: Nocturne design tokens in `poc/public/nocturne.css`, the 7a debug shell, live guild and
  channel rail, messages, DMs, voice membership, agent console with a live event tab, and a
  Settings slide-over for core process, login, visible guilds, and the action log. Roadmap
  surfaces show as placeholders.
- Web UI data: opening a chat fetches the latest 50 messages into the client cache; attachments
  hotlink Discord CDN URLs and are never stored; avatars and guild icons load from the CDN with a
  monogram fallback; the page reloads itself when the bridge connects.
- Web UI drafts: composer and TTS text persist per channel in `localStorage` and survive
  re-renders, reconnects, and page reloads.
- Visible guilds: the Settings panel filters the guild switcher and picker.
- Takeover: client-side read-only switch with `localStorage` persistence; manual sends are written
  to the action log with `type: "manual"`.
- `dm.list`: cached DM and group DM channels, exposed over MCP as `list_dms`.
- Core process control: the daemon starts, stops, and restarts the client, reconnects the bridge
  after a restart, auto-starts Xvfb when `DISPLAY` is missing, and stops what it started on exit.
  Override with `VESKSLOP_CORE_CMD`/`ARGS`; disable Xvfb with `VESKSLOP_XVFB=0`.
- Web QR login: the Settings panel runs the remote-auth session and renders the QR. The token file
  is written on approval and a running managed core restarts with the new token.
- Unit tests cover the pacing policy. `pnpm lint` and `pnpm testTypes` pass at the root.

- Web UI navigation: Guild/DM scope tabs, a DM rail with unread badges (purple/orange, grey while
  Takeover is off), last-message times, favourite stars, and a People list of users seen in voice.
  Favourited users get presence and call tracking, grouped by call in **In voice**. Speaking shows
  level bars and bold names (2c marks for mute/deaf/video/share). The body dims while the bridge
  is down; the Start button becomes Refresh when the daemon is unreachable.
- TTS settings are split into language and voice, grouped by base language with the requested
  languages pinned on top.

- Text notifications are parsed: edits update the context cache, deletes drop from it, trusted
  reactions are logged, and replies are kept in the request context. A reply to a puppet message
  counts as a trigger without a mention. The web UI marks edited messages and replies.
- Voice auto-leave is implemented: when the owner (zzib) leaves the bot's call, the bot leaves
  after `voiceLeaveDelaySeconds` (default 30). A rejoin cancels it. Verified with the mock,
  including cancellation. `stay @user` is deferred.
- Audio verified end to end on 2026-09-14: TTS played into the virtual mic (recording
  `veskslop_mic` peaked at -4.1 dB) and the browser monitor streamed PCM over the TLS WebSocket.
- Next: per-user limits and the collector account. The agent attachment (remote llama.cpp, MCP
  transport, event delivery) waits for a design discussion; the draft prompt has open issues to
  settle first.

## Next (in order)

1. Live verification: `voice.join`/`voice.leave` end to end, `user.search`, `voice.overview`,
   `VOICE_STATE_UPDATE`, and a full web QR login approval on the real client.
2. Agent: daemon task push for free-form triggers (commands stay local), tool allowlist,
   per-conversation serialization, and tests against a fake model.
3. Collector account: read-only modules for voice membership fallback.
4. Per-user rate limit and command cooldowns.
5. Mobile web UI: transcript and TTS in the Call tab; verify on a phone.

## Open discussions (need the user)

1. **Web UI auth.** Accepted: no auth, LAN or Tailscale only. Revisit before any wider exposure.
2. **MCP event delivery.** Decided for the local harness: the daemon pushes tasks to the agent's
   `POST /task`. The transport details (retry, dedupe IDs, queue) are still open. Remote agents
   poll or call tools directly; they do not receive pushes.
3. **Untrusted logging.** Untrusted interactions are ignored at this time. Confirm whether to log
   untrusted DMs and mentions again.
4. **Privacy pass.** Only token-shaped and `mfa.` strings are redacted, and only in the persistent
   action log. A fuller policy (retention, cache clearing, message redaction) is deferred.
5. **Live checks.** `reaction.add`, `user.search`, `voice.overview`, and the voice state event
   normalization are mock-verified only. The QR login session reached `pending` and `cancel`
   against the real gateway; the approval path needs the phone. Confirm on the first real-client
   run.

## Blocked / needs input

- The original repo `zib/Veskslop` has a broken server-side git setup. Fetches and clones fail with
  `bad pack header` because a script (`/home/git/.sys_health_s3`) runs on the Veskslop read path.
  The admin must fix it. Use `zib/Veskslop-t` (remote `veskslop-t`) until then.
- `main` on the original server is `68fc8bb` (zib, "Upload files to docs/images"). It diverged from
  the local `main` (`24447f5`). Reconcile after the server is fixed.

## Where to look

- Backlog: `TODO.md`
- Design documents: `docs/INDEX.md`
- Design handoff: `poc/WebUIMockup.zip` (screen 7a is implemented)
- Agent rules: `AGENTS.md`
- Daemon code: `poc/src/`
- Web UI code: `poc/src/webui.ts`, `poc/public/`
- Core bridge code: `src/shared/puppetProtocol.ts`, `src/renderer/puppet/`,
  `src/main/puppetBridge.ts`