zib/Veskslop-t
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update docs and README for voice audio, capture, and the web UI

Browse Source
This commit is contained in:
Fisto
2026-09-14 11:19:11 +02:00
parent c003d9532f
commit c7aad7547f
10 changed files with 356 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TODO.md
+15 -11
View File
@@ -32,21 +32,24 @@ Ordered backlog. `CURRENT.md` tracks the live session state.
## Deferred
- Voice join and leave, invite-link joins, audio in and out, and video streaming. The plan is in
`docs/voice.md`. Voice state reads and events exist. Voice actions do not.
- Voice auto-leave, `stay @user`, invite-link joins, audio in and out, and video streaming. The
plan is in `docs/voice.md`. The basic voice actions exist; auto-leave and audio do not.
- Client lightening. Low priority. The client must stay stable.
- Web UI authentication and remote accounts.
## Next
1. Voice backend: `voice.join` (muted), `voice.leave`, `voice.move`, the auto-leave timer, and the
`stay @user` command.
2. Live verification: `reaction.add`, `user.search`, `voice.overview`, `VOICE_STATE_UPDATE`, core
start on a real display, and a full QR login approval.
3. Notification parsing: edits, deletes, reactions, and message references.
4. Reply-to-puppet counts as a trigger.
5. Per-user rate limit and command cooldowns.
6. Collector account: stripped-down read-only modules. Use it when the puppet cannot find a user's
1. Voice auto-leave (`voiceLeaveDelaySeconds`) and the `stay @user` command.
2. Live verification on the audio host: TTS local + API, ducking, capture, `voice.join`/
`voice.leave` end to end, and a full QR approval.
3. Browser microphone puppeting: a mix sink so the browser mic and TTS share `veskslop_mic`, a
`pacat` input path, and echo handling.
4. Speech to text: Vosk (small CPU model in `poc/models/`) is wired and verified; add more
languages or a larger model when needed. `VESKSLOP_STT_CMD`/`_API_URL` stay available.
5. Notification parsing: edits, deletes, reactions, and message references.
6. Reply-to-puppet counts as a trigger.
7. Per-user rate limit and command cooldowns.
8. Collector account: stripped-down read-only modules. Use it when the puppet cannot find a user's
voice channel or the channel members.
## Web UI follow-ups
@@ -55,7 +58,8 @@ Ordered after the voice backend unless the owner asks first.
- Favourites, presence dots, and real member counts.
- MCP tool-call tracing in the console Tool calls tab.
- Transcript tab once voice audio lands.
- Transcript tab fed by the audio capture and STT.
- Yield (stop mid-line) for TTS when a human speaks.
- DM scope in the header (the guild picker becomes "Direct messages").
- Message editing, deletion, reactions, and history pagination.
- The remaining mockup screens: 7b/7c console details, 6b history window, 4b group chat, 2c state