Add the agent scaffold, mobile web UI, voice auto-leave, and text notifications
This commit is contained in:
+37
-15
@@ -1,21 +1,29 @@
|
||||
# CURRENT.md — session state
|
||||
|
||||
Last updated: 2026-09-13
|
||||
Last updated: 2026-09-14
|
||||
|
||||
This file tracks the work state. Read it first. Update it before you stop.
|
||||
|
||||
## Current focus (broad)
|
||||
|
||||
The text layer is done. The web UI follows the Nocturne design handoff. Voice actions are in
|
||||
progress: join/leave/move work; auto-leave and `stay @user` are next. Live verification and a full
|
||||
QR approval are pending. The LLM attachment and MCP event delivery are parked.
|
||||
The text layer is done. Voice actions and audio work. The local LLM harness (Veskslop agent, Go +
|
||||
Eino) is scaffolded and smoke-tested; the daemon-side task push is next. The web UI has a first
|
||||
mobile draft at 390px.
|
||||
|
||||
## In progress
|
||||
|
||||
- Voice actions are started: `voice.join` (muted), `voice.leave`, and `voice.move` are implemented
|
||||
in the core, wired through the daemon, Web UI, and MCP, and mock-verified. The internal
|
||||
`selectVoiceChannel` and `setSelfMute` lookups were probed live on the real client.
|
||||
- Auto-leave and `stay @user` are the next voice step.
|
||||
- The Veskslop agent lives in `agent/` (Go 1.25+, Eino v0.9.19, remote llama.cpp over the
|
||||
OpenAI-compatible API, daemon tools over MCP Streamable HTTP). It is disabled by default
|
||||
(`VESKSLOP_AGENT_ENABLED=1`), takes tasks on `POST /task`, and logs every tool call. Smoke-tested
|
||||
on 2026-09-14: 28 tools loaded, one ReAct round, final message returned. The daemon does not push
|
||||
tasks to it yet. Plan and env vars: [docs/agent.md](docs/agent.md).
|
||||
- Remote agents use the daemon's MCP endpoint directly (query state, send notifications). A
|
||||
notification tool set and a documented LAN/Tailscale bind are planned.
|
||||
- Mobile web UI first draft: four tabs (Users, Chat, Call, Activity), one pane at a time, 44px
|
||||
targets, tab badge, call bar pinned, Activity becomes the full console. Verified headless at
|
||||
390x844. Next: transcript and TTS box in the Call tab, per the 1c design.
|
||||
- Fixed a pre-existing web UI bug: `loadDms` called the nonexistent `renderDirect`, so the DM rail
|
||||
only rendered from other paths. It now calls `renderDmRail()`.
|
||||
- Voice audio is implemented: daemon-managed virtual devices (`veskslop_tts`, `veskslop_mic`,
|
||||
`veskslop_capture`), TTS providers (local command + HTTP API), queued playback through `ffplay`,
|
||||
ducking on `VOICE_SPEAKING`, and WAV capture. The daemon starts `pulseaudio` when `pactl` cannot
|
||||
@@ -111,20 +119,34 @@ QR approval are pending. The LLM attachment and MCP event delivery are parked.
|
||||
- TTS settings are split into language and voice, grouped by base language with the requested
|
||||
languages pinned on top.
|
||||
|
||||
- Text notifications are parsed: edits update the context cache, deletes drop from it, trusted
|
||||
reactions are logged, and replies are kept in the request context. A reply to a puppet message
|
||||
counts as a trigger without a mention. The web UI marks edited messages and replies.
|
||||
- Voice auto-leave is implemented: when the owner (zzib) leaves the bot's call, the bot leaves
|
||||
after `voiceLeaveDelaySeconds` (default 30). A rejoin cancels it. Verified with the mock,
|
||||
including cancellation. `stay @user` is deferred.
|
||||
- Audio verified end to end on 2026-09-14: TTS played into the virtual mic (recording
|
||||
`veskslop_mic` peaked at -4.1 dB) and the browser monitor streamed PCM over the TLS WebSocket.
|
||||
- Next: per-user limits and the collector account. The agent attachment (remote llama.cpp, MCP
|
||||
transport, event delivery) waits for a design discussion; the draft prompt has open issues to
|
||||
settle first.
|
||||
|
||||
## Next (in order)
|
||||
|
||||
1. Voice auto-leave (`voiceLeaveDelaySeconds`) and the `stay @user` command.
|
||||
2. Live verification: `voice.join`/`voice.leave` end to end, `user.search`, `voice.overview`,
|
||||
1. Live verification: `voice.join`/`voice.leave` end to end, `user.search`, `voice.overview`,
|
||||
`VOICE_STATE_UPDATE`, and a full web QR login approval on the real client.
|
||||
3. Notification parsing: edits, deletes, reactions, and message references.
|
||||
4. Reply-to-puppet counts as a trigger.
|
||||
5. Per-user rate limit and command cooldowns.
|
||||
6. Collector account: read-only modules for voice membership fallback.
|
||||
2. Agent: daemon task push for free-form triggers (commands stay local), tool allowlist,
|
||||
per-conversation serialization, and tests against a fake model.
|
||||
3. Collector account: read-only modules for voice membership fallback.
|
||||
4. Per-user rate limit and command cooldowns.
|
||||
5. Mobile web UI: transcript and TTS in the Call tab; verify on a phone.
|
||||
|
||||
## Open discussions (need the user)
|
||||
|
||||
1. **Web UI auth.** Accepted: no auth, LAN or Tailscale only. Revisit before any wider exposure.
|
||||
2. **MCP event delivery.** Parked with the LLM attachment. Options stay in `TODO.md`.
|
||||
2. **MCP event delivery.** Decided for the local harness: the daemon pushes tasks to the agent's
|
||||
`POST /task`. The transport details (retry, dedupe IDs, queue) are still open. Remote agents
|
||||
poll or call tools directly; they do not receive pushes.
|
||||
3. **Untrusted logging.** Untrusted interactions are ignored at this time. Confirm whether to log
|
||||
untrusted DMs and mentions again.
|
||||
4. **Privacy pass.** Only token-shaped and `mfa.` strings are redacted, and only in the persistent
|
||||
|
||||
Reference in New Issue
Block a user