zib/Veskslop-t
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
0ba51e06bad191806253e17b355f465c9d91f7e1
Veskslop-t/CURRENT.md
T
Fisto 0ba51e06ba Cycle previews through the walk
2026-09-17 00:20:00 +02:00

37 KiB
Raw Blame History

CURRENT.md — session state

Last updated: 2026-09-16

This file tracks the work state. Read it first. Update it before you stop.

Current focus (broad)

The text layer is done. Voice actions and audio work. The local LLM harness (Veskslop agent, Go + Eino) is scaffolded and smoke-tested; the daemon-side task push is next. The web UI has a first mobile draft at 390px.

In progress

  • Media archive project, branch media-scrape-project on veskslop-t. Design approved 2026-09-16 (docs: media.md, media-scraper.md, slopbot.md, media-mcp.md). Decisions: scrape the channel the command is sent in; slopbot (discord.js bot core) for guild channels and the puppet core bridge for group DMs; one self-contained SQLite + media dir per scrape under media-scrape/db/ (gitignored); links indexed by default, downloads optional, third-party embeds never downloaded; 30 GiB default limit with a 90% warning and 5% no-delete overrun, usage bar and clean button in Settings → Bot; Media and Bot top-level tabs; MCP hooks designed for C8. The slopbot scaffold (config loader, invite helper) and its discord.js dependency were added 2026-09-16. C5 smoke test: /api/media/status and /api/media answer on a daemon with no core.

    Checkpoints, push after each:

    • C1 design docs, AGENTS/INDEX pointers, .gitignore
    • C2 core message shape: embeds, forwards, attachment metadata
    • C3 media-scrape store and puppet walker, tests
    • C4 slopbot backend, tests
    • C5 daemon wiring: scrape media command, MediaService, /api/media/*
    • C6 Media and Bot tabs in the Web UI
    • C7a storage limits, retention, warning/confirm flow, storage and clean API
    • C7b Settings group: limit field with info hover, usage bar, clean modal
    • C8 MCP service hooks (scrape_media, scrape reads, storage, clean; reads in run_commands)
    • C10 downloader: attachment and sticker bytes, hash-addressed, serial, size-capped, retries; embeds excluded in SQL; local files served at /api/media/<scrape>/<item>/file and preferred by the Media tab. Live run 2026-09-16: scrape ...-20260916083401 fetched the 784 KB attachment; storage counted actual bytes.
    • C11 preview caching: Discord-hosted image/GIF previews (WebP 320px, hash-addressed, 5 MiB cap, one attempt) run after every walk in both modes. Live run ...-20260916090722: previewed: 2; thumbs served over /api/media/<scrape>/<item>/thumb (200).
    • C35 Walk/preview cycling. A live scrape now alternates: walk 5 pages, preview up to one batch (25, with the cooldown), walk on. Tiles fill in during a long walk instead of only at the end, and preview traffic is spread through the walk. Configurable with VESKSLOP_MEDIA_WALK_CYCLE_PAGES (0 defers previews to the end). The final pass still drains whatever remains.
    • C34 Resume from cursor. A paused scrape can continue where it stopped instead of re-walking the top: POST /api/media/scrape/resume, MCP resume_media_scrape, and a Resume button in History. Previews never needed the walk — backfill fetches them straight from the stored URLs; the walk is only for discovery. Live: the paused gernal_chut scrape resumed at cursor 365256427247239181, fetched its final page, and entered the preview pass.
    • C33 Puppet history pagination. The core's message.fetch read the client store, which keeps only the newest messages plus the channel's first — every page was 10 messages and the walk stopped after one page (a 10-message, 4-media scrape of an ancient group with thousands of messages). It now fetches through the client's REST layer first (contiguous 100-message pages, transport: "rest"), store as fallback. Rebuilt and restarted the managed client; live re-run of channel 591000745164406784: 21+ pages / 2,100+ messages and climbing, with hundreds of images/videos being indexed.
    • C32 Group-chat clarity and puppet refresh. Group DM scrapes already worked (live-verified against the running daemon: zıbb & fred and a 1:1 DM, backend puppet, done, previews cached); the Group chat tab had shown an empty list while the core was offline. It now says so explicitly and offers Retry. Expired attachment URLs in DM/group-DM scrapes refresh through the core bridge (message.fetch with before: id+1, which returns exactly the target).
    • C31 Batched previews. The preview pass now runs in batches (default 25) with a cooldown (default 60 s), configurable via VESKSLOP_MEDIA_PREVIEW_BATCH and VESKSLOP_MEDIA_PREVIEW_BATCH_PAUSE_MS; batching counts rows, not successes. Live log: preview batch of 25 done, pausing 60000 ms.
    • C30 Preview completeness. Audio attachments and voice messages get an ffmpeg waveform strip; oversized animated GIFs fall back to an ffmpeg-read first frame instead of too large; transient network failures stay pending for the next backfill instead of becoming permanent. Live: a scrape with 58 audio items backfilled 13 waveforms in 40 s, serving image/webp. Remaining non-cached: Lottie stickers/custom emoji (not indexed as media) and embeds whose pages refuse bots (marked PRIVATE/GONE by their 403/404).
    • C29 Attachment video previews. Discord's image proxy renders a frame for video files (?width=320&height=320&format=webp), so previewUrlFor now resizes attachment mp4/webm in links mode too and pendingPreviews includes non-embed videos. Live: paused-scrape videos backfilled to image/webp thumbs, pending queue drained to 0.
    • C28 Media grid usability. Infinite scroll (240 px threshold) with a Load more fallback, previewError exposed so tiles show PRIVATE (403) or GONE (404/410) with the reason in the tooltip, and the scrape strip lists only queued/running jobs — paused scrapes move to History.
    • C27 Non-media stream guard. Rows awaiting backfill fell back to first-frame <video> with a provider page URL (YouTube embed, redirectors), and the stream route piped the HTML through (text/html errors). Now only real video files (mp4/webm/m4v) are framed, the stream route returns 415 for non-media upstreams, the viewer's still-frame is gated the same way, and "Open in new tab" prefers play_url. Live: YouTube item → 415 JSON; backfill reached 184/271 previews during the run.
    • C26 YouTube preview root cause. Dedupe picked the newest row per URL; a paused scrape (gernal_chut, stopped mid-walk) had no previews yet and hid an older scrape that did. Fixes: dedupe prefers the richer row (downloaded > preview > newer), paused walks still run the download/preview passes for collected pages, daemon start backfills previews for scrapes that paused early, and preview gaps are now 200-600 ms. Live: backfill cached 24 previews in the paused scrape within 45 s and a YouTube item then served thumbUrl.
    • C25 Live scrape progress. The media tab polls /api/media/scrapes every 4 s while open, so the running row counts update; pages show X/Yp (~Z%) from estimatedMessages when the search total is available (some channels deny the bot search with Missing Access, so it stays optional). Live: pages/messages incremented 2→10 during a walk on gernal_chut.
    • C24 Embeddable items open straight into their player: clicking a YouTube/Vimeo/Twitch/ Streamable card shows the iframe immediately, no extra play press. Non-embeddable videos keep the still + play icon.
    • C23 Grid and viewer source fixes. The grid now prefers the cached still (or local file) for every kind, and only falls back to a first-frame video, served through the same-origin stream endpoint instead of hotlinked CDN/redirector URLs (fixes fxtwitter/YouTube previews that ORB blocked while the cached thumb sat unused). The viewer shows a muted first frame for videos without a still, and stream refreshes the signed URL only when the stored one fails (4xx). Live: stream returned 206 video/webm for a thumbnail-less video; YouTube rows keep their cached thumb.
    • C22 Media grid polish: per-kind status row ((Images/Videos) | Downloaded I/V | Cached I/V | Loaded I/V), GIF tiles with a translucent GIF label that play the animated original on hover (cached previews are static ffmpeg frames), Tenor oEmbed preview resolution, and first-frame/inline playback for embeds that carry Discord's videoUrl (Tenor/gifv now classify as video with play_url). remotePreviewUrl exposes the animated source to the UI. Live: stats per kind correct; Tenor embed stored with play_url.
    • C21 Playback of non-downloaded and expired media. play_url stores the embed's direct video (gifv/Tenor mp4); a new /api/media/<scrape>/<item>/stream route serves the local file or re-fetches the source message through slopbot to refresh expired signed CDN URLs and streams it with Range support. The viewer and View full use it, so Discord attachment videos play without a prior download and old links no longer dead-end. Live: 2.5 MB webm streamed 200 with a refreshed URL; play_url populated for a media.discordapp.net gifv mp4 and provider redirectors.
    • C20 Cross-scrape dedupe and viewer polish. queryMedia and stats deduplicate by URL (newest scrape wins), so overlapping scrapes no longer duplicate cards or counters (live: 17 items / 17 unique). Viewer: big play icon top-right; play embeds a provider player or a direct mp4/webm/m4v, otherwise opens the external warning; View full opens the embedded media/player in a new tab while the link button opens the original page; duration is stored (Discord attachment duration, ffprobe for downloads, 0 normalized to unknown); favicon added.
    • C19 External embed previews and MIME fixes. Embeds with no thumbnail in Discord's data are resolved via provider oEmbed/fxtwitter and an og:image fallback; preview bytes keep their real extension (preview_ext), so JPEG thumbnails serve as image/jpeg instead of mislabeled WebP (the browser "Content-Type image/webp is not supported" errors). The grid no longer uses embed page URLs as media, and first-frame <video> is limited to playable direct files (mp4/webm/m4v) with an on-error placeholder. Live: fxtwitter/vxReddit/TikTok previews cached; 7 jpg + 4 webp thumbs served with the right types.
    • C18 Video previews and the status row. Uploaded videos have no Discord thumbnails: the grid renders a first-frame <video> for them, and downloads generate a poster frame with ffmpeg into thumbs/ (live: both video attachments in the test channel got posters). The status row is now (Images/Videos) | Downloaded A/B | Cached C/B (excl Downl) | Loaded L/B with B = total for the active filters.
    • C17 Media grid and viewer. Embed videos now render their cached thumbnail with a play icon; a Metadata dropdown picks banner fields (User, Type, Date, Source, Resolution, Length); nothing shows until Load media; the status row reads downloaded A/B | cached C/D (excludes downloaded) | N loaded from /api/media/stats; clicking a tile opens a viewer with wide-below / vertical-right metadata, full link with an external-site warning, View full, and an embedded player for YouTube/Vimeo/Twitch/Streamable when possible.
    • C16 Message counting and date-bounded scrapes. Discord search (total_results, one guild request, slopbot only) powers POST /api/media/count and MCP count_media_messages; the confirm dialog has From/To dates plus a Check Message Count button; the walk honors afterId/beforeId; estimatedMessages, fromDate, and toDate land in the scrape meta and History. Live: count 17 all / 17 after 2026-09-16 / 0 before, DM returns a reason, and a from-dated scrape stored 17 messages with estimatedMessages: 17.
    • C15 Scrape UX: custom download destination (absolute path per scrape; files there, previews local, embed thumbnails copied to both, custom files not served), confirmation popup before Links/Download, New scrape/History right-aligned with New scrape furthest right, modal X and backdrop close, active-only progress strip, and a Bot error feed with a red unseen badge that clears on open plus a Clear button. Live run ...-20260916095029: 6 files and 7 embed thumbnails in /tmp/opencode/media-dest/<id>/, 11 local previews, 0 local files, localUrl null; error record and clear verified over the API.
    • C14 Links-mode embed thumbnails, item metadata, filters, and the scrape popup. Metadata: poster (author_id/author_name), posted_at, width/height, aspect_ratio, with an ffprobe fallback after downloads. Filters: poster, aspect preset (vertical/square/4:3/16:9/ 21:9/other), resolution preset (SD/HD/FHD/QHD/4K+/unknown, by shortest side), and a posted-date range. UI: scrape popup with Server and Group-chat trees (server -> channels, each with Links/Download), History popup (time, duration, pages/messages, media, bytes, files, previews), live progress pulse, full-window view. Live run ...-20260916094120: 11 previews in links mode (embed thumbnails included), poster zzib, a 9:16 short stored as 0.563, and the 16:9 filter matched.
    • C13 Embed thumbnails in download mode: embed rows cache their stored thumbnail URL; the full third-party media is never fetched. Live run ...-20260916091458 on the test channel: 6 files downloaded, 11 previews cached (YouTube, Reddit, Tenor, TikTok, X). Also fixed the state transition: the job stays running through download and preview passes so clients do not see done early.
    • C12 Media tab filters and full-window layout: kind chips, source filter (attachment/sticker/embed), source-channel filter, and the view now fills the window below the header and call strip. API/MCP gained source and channelId on query_media. Smoke-tested over /api/media?source= and ?channelId=.
    • [~] C9 live verification. Slopbot path verified live on 2026-09-16: login Slopbot#5775, guild 177931418704281600. Test channel links scrape: 17 messages, 4 images + 13 videos, 0 errors; download scrape fetched the 784 KB attachment. Large channel gernal_chut: 11 pages, 1088 messages, 96 images + 1 GIF + 203 videos, 0 errors (no 429 at the current pacing). Puppet path verified live on a group DM (zıbb & fred) and a 1:1 DM: backend puppet, done, 0 errors. The puppet field check is done too: the group DM returned attachments plus mapped embeds (YouTube, Instagram), and the large slopbot channel produced YouTube, Gfycat, Imgur, Steam, and Twitch embeds alongside attachments. Still owed: a stress walk on the second guild (559346566754729995, channel 559346566754729997).

  • Flextalk voice profile, end to end. The profile window is a movable panel that recenters on each open; it carries the character sliders with engine defaults and +/-2 snapping, five named presets, radio-kit effects (high-pass 40-400 Hz, echo 0-100, word mode with a gap in ms), and test lines (Attention, Laboratory, Critical, Health, Radiation, plus two custom lines in localStorage) that post the current profile straight to /api/audio/speak. Fixed the test buttons reading a stale expression from saved state; they now read the slider like Speak does.

  • Root cause of the profile never reaching the engine (2026-09-15): the wrapper wrote the Flextalk .reg with echo, and dash's echo treats \2 in ...\Watson\2.0\... as an escape, which corrupted the key so wine reg import was a silent no-op (it still returns 0). The wrapper now uses printf, validates that every profile value is numeric, and rejects a .reg containing control characters. Documented in TTS-DESIGN.md.

  • The Veskslop agent lives in agent/ (Go 1.25+, Eino v0.9.19, remote llama.cpp over the OpenAI-compatible API, daemon tools over MCP Streamable HTTP). It is disabled by default (VESKSLOP_AGENT_ENABLED=1), takes tasks on POST /task, and logs every tool call. Smoke-tested on 2026-09-14: 28 tools loaded, one ReAct round, final message returned. The daemon does not push tasks to it yet. Plan and env vars: docs/agent.md.

  • Remote agents use the daemon's MCP endpoint directly (query state, send notifications). A notification tool set and a documented LAN/Tailscale bind are planned.

  • Mobile web UI first draft: four tabs (Users, Chat, Call, Activity), one pane at a time, 44px targets, tab badge, call bar pinned, Activity becomes the full console. Verified headless at 390x844. Next: transcript and TTS box in the Call tab, per the 1c design.

  • Fixed a pre-existing web UI bug: loadDms called the nonexistent renderDirect, so the DM rail only rendered from other paths. It now calls renderDmRail().

  • Fixed the speaking marks: the core turned Discord's numeric speaking: 0 into true, so the equalizer bars never stopped. The bars now fade in on start and decay out on stop (500 ms), with a 2.5 s idle timeout for missed stops; a mute or channel change ends the state at once. Muted members show no bars.

  • Browser audio: Client audio now runs the browser unlock in the click (microphone permission, AudioContext created and resumed, output sink applied) before it starts listening. Verified headless: 52 PCM frames played through a running context. The button label no longer wipes the icon markup.

  • Capture routing: nothing moved the client's playback streams into veskslop_capture, so the browser monitor streamed digital silence. The audio manager now adopts matching streams (VESKSLOP_AUDIO_CLIENT_MATCH, default vesktop,electron,chromium,discord) on device setup, monitor start, and every 5 s while listening or capturing.

  • LAN audio verified end to end on 2026-09-14: a tone stream on veskslop_tts was moved to veskslop_capture (daemon log: moved stream 81 to veskpop_capture) and the browser, connected to https://192.168.0.178:18801/, played 256+ PCM frames with a peak of 4096 (a stream already on the capture sink peaked at 29274).

  • Speaking bars are always in the layout now: scaleY(0) and transparent when idle, they scale up with the fade-in at start, run the equalizer animation while speaking, and stay on until an explicit stop, a mute, or a channel change. Live data showed repeated speaking: true pulses and no stop events inside an 8 s window, so the idle decay is removed: bars stick rather than disappear. The fade runs the equalizer inside a shrinking container, so the amplitude drops smoothly instead of the block collapsing at full height. The core treats a missing speaking flag as a start pulse and accepts user_id. Core-side changes need a rebuild plus a client restart.

  • Added poc/scripts/audio-doctor.sh (pnpm audio:doctor): default sink, sink inputs, audio status with adoption counters, a 5 s volumedetect on veskslop_capture.monitor, and a live event peek. Run it in a real call to trace a silent browser monitor.

  • Live doctor output (2026-09-14): the vesktop/electron streams sit on veskslop_capture (sink 40) already, adopted: 0, and VOICE_SPEAKING events arrive. Client audio routing is therefore in place. Bot TTS works. Capture was off (capturing: false), so the transcript was empty; Vosk transcribes correctly when invoked with the model env.

  • Live call audio works now (the user confirmed TTS is audible and the silent call was partly a per-user volume set to zero for some members, plus another issue). The volume read returns null because this client build exposes no UserVolumeStore; the setter reports NOT_SUPPORTED. Volume is not a usable control here.

  • External TTS engine support (external project, first target is a command wrapper): the contract lives in TTS-DESIGN.md. Veskslop now has nullable tts.speed (0.5-2.0) and tts.pitch (-12..+12) preferences, per-line overrides on /api/audio/speak and MCP speak, {speed}/ {pitch} placeholders, VESKSLOP_TTS_SPEED/_PITCH exports, and speed/pitch sliders in the TTS panel. A custom command advertises its voices through VESKSLOP_TTS_VOICES; the espeak list is hidden while a custom command is set. Verified with a wrapper that logged argv and env: preferences and overrides both arrive. Reading the engine's GET /voices for the HTTP provider remains future work.

  • The MicroslopSam engine project is now part of the tree in microslopsam/ (copied from the separate project; its build/ output and voice data stay ignored). The wrapper was renamed setup/vekslop-tts.sh -> setup/veskslop-tts.sh and every reference and configuration export was fixed. The engine also corrected the Flextalk pitch finding (REG_DWORD bug), so Keith Bell and Kerry Watson now support pitch and carry no no-pitch flag.

  • Implemented the Flextalk Expression profile parameter end to end: VESKSLOP_TTS_VOICES marks the two voices as expression=59 / expression=100; the wrapper writes the registry value before synthesis; Veskslop has tts.expression (0-100 nullable), the {expression} placeholder, VESKSLOP_TTS_EXPRESSION, an HTTP/MCP field, and an Expression slider that appears only for those voices (default shown until moved, null leaves the registry untouched). Verified: 0/59/100 produce distinct audio, the registry reaches 0x14 for 20, and the UI shows 25/100/default states.

  • Fixed the null-value coercion in the preference normalizers (Number(null) === 0 turned a cleared speed/expression into 0.5/0); unset now stays null.

  • Renderer-down handling polished: the bridge logs one warning per outage instead of a stack per polled command, and the navigator.languages request catches the new timeout rejection instead of raising an unhandled promise warning. (The exitCode: 9 renderer kills in the live log were client restarts; the daemon's retry handled the startup race as designed.)

  • The TTS panel has speed presets (0.5x, 0.75x, 1x, 1.25x, 1.5x) under the fine-adjust slider, and the pitch control is hidden entirely for voices that report supportsPitch: false. (Flextalk voices no longer report that; see the pitch correction above.) Verified: preset clicks update the slider/label and save.

  • TTS playback has a lead-in of silence before each line (VESKSLOP_TTS_LEAD_MS, default 150 ms, max 2000, added with adelay in ffplay; status reports ttsLeadMs) so Discord's voice pipeline is settled before the first syllable. Verified on the sink monitor: first sound at 0.02 s with 0 ms and 0.62 s with 600 ms.

  • Speech settings have a Start engine button that warms the selected engine (POST /api/audio/tts/warm, MCP start_tts_engine): it runs one throwaway synthesis so the command engine's Xvfb/Wine/host are up. Verified with MicroslopSam: cold 1732 ms, warm 505 ms, and the UI shows "MicroslopSam ready in 835 ms". The warm text is a full sentence because their two-pass loudnorm fails with inf on very short input.

  • Startup and crash robustness: sendRendererCommand now rejects immediately when the client renderer is destroyed or crashed, and times out after 15 s, so a dead frame no longer logs Electron's "Render frame was disposed" stack once per poll. Device auto-connect retries up to 10 times every 2.5 s, because the bridge socket is ready before the renderer's modules load (the live error was Reflect.get called on non-object from a lazy lookup at startup). Verified with the mock: attempt 1 failed, attempt 2 set the devices.

  • TTS engines are selectable: the Speech settings and the TTS panel list espeak-ng, the command engine (VESKSLOP_TTS_NAME, default "custom engine"), and the HTTP API; unavailable entries are disabled and Auto resolves to the command engine first. Selecting an engine switches the voice list (/api/audio/tts/voices?engine=) and hides the language row for single-language engines. The TTS panel now has a voice select next to the speed/pitch sliders. Fixed a slider bug where a null speed preference (Number(null) === 0) clamped the slider to 0.5 and then persisted it.

  • TTS controls: the TTS panel has a working duck on voice toggle (stored in preferences, reported as duckEnabled). Ducking was verified live with the mock: the veskslop_tts sink drops to 30% while TTS plays and someone speaks, and returns to 100% after. The Transcript tab in the console has Start and Stop buttons; the status line shows capturing/capture off. The espeak-ng voice list now includes the m1–m7, f1–f5, croak, and whisper variants (<voice>+<variant>), so each language has about 15 voices instead of one.

  • Speaking state is now authoritative: the daemon polls the core's voice.speaking action (ChannelRTCStore.getSpeakingParticipants) every second and synthesizes the missing stop/start events; dispatcher VOICE_SPEAKING hints feed the same set so stale users clear. This fixes the bars sticking for everyone and the transcript attribution, which credited the first member because speaker intervals never closed. Verified with the mock: a speaking: true event cleared within one poll.

  • The client streams sit uncorked, unmuted, and at 100% on veskslop_capture, yet the monitor is silent. Gateway VOICE_SPEAKING events do not prove inbound RTP, and the TTS path only proves outbound, so the remaining suspect is the client's inbound voice state (stuck deafen or a broken voice connection), not the sink routing. /health now returns selfId so the bot's own member flags can be picked out of the voice member list; the doctor prints it.

  • Client output is still silent at the sink (-91 dB) although the streams are uncorked, unmuted, and at 100%, and a headless Chromium tone into the same sink measured -7.4 dB. Added per-user volume reading and writing to find Discord-side silencing: user.volume / user.setVolume in the core, /api/voice/volume (GET and POST), MCP get_user_volume / set_user_volume, and volume in every voice member. A volume of 0 (or an output volume of 0) silences a user even when Pulse shows a healthy stream. voice.join also clears persisted self-deafen now. The doctor prints output and per-user volumes.

  • Voice audio is implemented: daemon-managed virtual devices (veskslop_tts, veskslop_mic, veskslop_capture), TTS providers (local command + HTTP API), queued playback through ffplay, ducking on VOICE_SPEAKING, and WAV capture. The daemon starts pulseaudio when pactl cannot reach a server and reports install hints. Verified live on 2026-09-13 after installing pulseaudio/ffmpeg: engine ok, devices created, PCM streamed.

  • The Web UI serves self-signed HTTPS (openssl, stored under the state dir) so browser media APIs have a secure context. WEBUI_TLS=0 disables it.

  • Capture and transcript are implemented: ffmpeg reads the client output, a VAD cuts it into utterances, each becomes a WAV plus a transcript entry with the dominant speaker, and a pluggable STT hook (local command or HTTP API) fills the text. Live events feed the console Transcript tab; get_transcript and /api/transcript read the store. VESKSLOP_CAPTURE_AUTO=1 records calls automatically. Verified end to end with real PulseAudio on 2026-09-13.

  • STT uses Vosk (CPU, light): poc/scripts/stt-vosk.py plus a small model in poc/models/ (auto-detected). Verified with a TTS→capture→Vosk round trip. Other providers remain available via VESKSLOP_STT_CMD/_API_URL.

  • TTS defaults to espeak-ng when installed; engine and voice live in Settings → Client → Speech (daemon preferences, so all browsers share them).

  • Preferences moved to <state dir>/preferences.json and no longer depend on the core, so guild visibility saves even while the client is down. API responses send Cache-Control: no-store.

  • Browser audio has an Enable browser audio button that requests the microphone permission, which browsers require before listing output devices.

  • Browser microphone puppeting (talking through the bot) is next; it needs a mix sink, a pacat input, and echo handling.

  • The web UI follows the Nocturne design handoff (poc/WebUIMockup.zip). The 7a shell is implemented and rendered headless for review: header, call strip, three-column body, composer, and the agent console.

  • Takeover is a client-side read-only gate. Off, the composer is locked. Manual sends are logged.

  • Core process control is implemented and verified live: the daemon starts the client under auto-Xvfb, the bridge connects, and stop exits cleanly.

  • Web QR login is implemented. The session reached pending against the real gateway; a full phone approval is pending.

Done

  • Validated core bridge: protocol, actions, validation, structured errors, hard outbound rate limit.

  • Auth persistence: pinned DATA_DIR, token file, preload seeding, VESKSLOP_HEADLESS.

  • Daemon on the core-bridge transport. CDP stays as a development fallback.

  • Trusted-only triggers: DMs and direct mentions. Trust tiers: user, trusted, admin.

  • Context: 15 messages or 4000 characters, time stamps, file names only, incremental fetch, tmp cache with a size limit, fetch cooldown.

  • Continuation window: a rushed follow-up joins the request context.

  • Response policy: origin channel or DM. Sensitive commands default to DM.

  • Persistent action log: triggers, commands, replies, errors, rotation, and secret redaction.

  • poll is paused (POLL_ENABLED = false). The parser and its tests stay.

  • Human send pacing: first send of a burst waits 30-200 ms, later sends 500-2800 ms, and the burst resets after a random 5-20 s of quiet. Reads take 100-300 ms and do not reset the burst.

  • user.search: cache and guild-member lookup. voice.get searches all guilds without a guildId.

  • voice.overview: occupied voice channels and members for a guild.

  • VOICE_STATE_UPDATE events: normalized and pushed. Mock-verified.

  • Voice actions: voice.join (muted), voice.leave, and voice.move in the core, with daemon routes (/api/voice/*), MCP tools, and Web UI wiring. Voice actions are paced 250-750 ms and written to the action log. The internal lookups were probed on the live client.

  • Web UI voice rail: every voice channel is clickable. Clicking joins (muted); clicking the active channel leaves. Verified with the stateful mock.

  • Web UI startup: a Start server button appears left of Takeover while the bridge is down and hides when it connects. State reloads retry while the client boots, so guilds appear without a manual refresh. Core requests now time out after VESKSLOP_REQUEST_TIMEOUT_MS instead of hanging forever on a missing reply.

  • Settings: "Visible guilds" stays top level; Audio sits under a collapsible "Client" group; Core process, Account login, and Action log sit under a collapsible "Server" group.

  • Web UI timestamps use the browser timezone by default; Settings → Client → Display can override it with any IANA zone. The choice is per browser (localStorage).

  • Web UI preferences: visible guilds persist server-side in the daemon state file (/api/preferences), so every browser shares them. The Settings panel has an explicit Save button; selections filter the UI live and persist only on Save.

  • DM list: recipients resolve from IDs through the user store, so names and avatars load.

  • Guild menu: fixed the light default button background.

  • Web UI: Nocturne design tokens in poc/public/nocturne.css, the 7a debug shell, live guild and channel rail, messages, DMs, voice membership, agent console with a live event tab, and a Settings slide-over for core process, login, visible guilds, and the action log. Roadmap surfaces show as placeholders.

  • Web UI data: opening a chat fetches the latest 50 messages into the client cache; attachments hotlink Discord CDN URLs and are never stored; avatars and guild icons load from the CDN with a monogram fallback; the page reloads itself when the bridge connects.

  • Web UI drafts: composer and TTS text persist per channel in localStorage and survive re-renders, reconnects, and page reloads.

  • Visible guilds: the Settings panel filters the guild switcher and picker.

  • Takeover: client-side read-only switch with localStorage persistence; manual sends are written to the action log with type: "manual".

  • dm.list: cached DM and group DM channels, exposed over MCP as list_dms.

  • Core process control: the daemon starts, stops, and restarts the client, reconnects the bridge after a restart, auto-starts Xvfb when DISPLAY is missing, and stops what it started on exit. Override with VESKSLOP_CORE_CMD/ARGS; disable Xvfb with VESKSLOP_XVFB=0.

  • Web QR login: the Settings panel runs the remote-auth session and renders the QR. The token file is written on approval and a running managed core restarts with the new token.

  • Unit tests cover the pacing policy. pnpm lint and pnpm testTypes pass at the root.

  • Web UI navigation: Guild/DM scope tabs, a DM rail with unread badges (purple/orange, grey while Takeover is off), last-message times, favourite stars, and a People list of users seen in voice. Favourited users get presence and call tracking, grouped by call in In voice. Speaking shows level bars and bold names (2c marks for mute/deaf/video/share). The body dims while the bridge is down; the Start button becomes Refresh when the daemon is unreachable.

  • TTS settings are split into language and voice, grouped by base language with the requested languages pinned on top.

  • Text notifications are parsed: edits update the context cache, deletes drop from it, trusted reactions are logged, and replies are kept in the request context. A reply to a puppet message counts as a trigger without a mention. The web UI marks edited messages and replies.

  • Voice auto-leave is implemented: when the owner (zzib) leaves the bot's call, the bot leaves after voiceLeaveDelaySeconds (default 30). A rejoin cancels it. Verified with the mock, including cancellation. stay @user is deferred.

  • Audio verified end to end on 2026-09-14: TTS played into the virtual mic (recording veskslop_mic peaked at -4.1 dB) and the browser monitor streamed PCM over the TLS WebSocket.

  • Next: per-user limits and the collector account. The agent attachment (remote llama.cpp, MCP transport, event delivery) waits for a design discussion; the draft prompt has open issues to settle first.

Next (in order)

  1. Live verification: voice.join/voice.leave end to end, user.search, voice.overview, VOICE_STATE_UPDATE, and a full web QR login approval on the real client.
  2. Agent: daemon task push for free-form triggers (commands stay local), tool allowlist, per-conversation serialization, and tests against a fake model.
  3. Collector account: read-only modules for voice membership fallback.
  4. Per-user rate limit and command cooldowns.
  5. Mobile web UI: transcript and TTS in the Call tab; verify on a phone.

Open discussions (need the user)

  1. Web UI auth. Accepted: no auth, LAN or Tailscale only. Revisit before any wider exposure.
  2. MCP event delivery. Decided for the local harness: the daemon pushes tasks to the agent's POST /task. The transport details (retry, dedupe IDs, queue) are still open. Remote agents poll or call tools directly; they do not receive pushes.
  3. Untrusted logging. Untrusted interactions are ignored at this time. Confirm whether to log untrusted DMs and mentions again.
  4. Privacy pass. Only token-shaped and mfa. strings are redacted, and only in the persistent action log. A fuller policy (retention, cache clearing, message redaction) is deferred.
  5. Live checks. reaction.add, user.search, voice.overview, and the voice state event normalization are mock-verified only. The QR login session reached pending and cancel against the real gateway; the approval path needs the phone. Confirm on the first real-client run.

Blocked / needs input

  • Slopbot: the bot is invited to the test guild and the intent is enabled. The token lives in poc/.env (gitignored, mode 600) and will be replaced when the project goes to prod. The daemon must be restarted to pick it up.
  • The original repo zib/Veskslop has a broken server-side git setup. Fetches and clones fail with bad pack header because a script (/home/git/.sys_health_s3) runs on the Veskslop read path. The admin must fix it. Use zib/Veskslop-t (remote veskslop-t) until then. That remote has no stored credential; use the one-off token URL in GITEA.md. Rechecked 2026-09-16: the original still fails with bad pack header.
  • main on the original server is 68fc8bb (zib, "Upload files to docs/images"). It diverged from the local main (24447f5). Reconcile after the server is fixed.

Where to look

  • Backlog: TODO.md
  • Design documents: docs/INDEX.md
  • Design handoff: poc/WebUIMockup.zip (screen 7a is implemented)
  • Agent rules: AGENTS.md
  • Daemon code: poc/src/
  • Web UI code: poc/src/webui.ts, poc/public/
  • Core bridge code: src/shared/puppetProtocol.ts, src/renderer/puppet/, src/main/puppetBridge.ts
Reference in New Issue View Git Blame Copy Permalink