zib/Veskslop-t
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add daemon restart, preview diagnostics, browse caching, and rate-limit pauses

Browse Source
This commit is contained in:
Fisto
2026-09-17 07:33:35 +02:00
parent cba7ae8143
commit dea1724eb3
13 changed files with 367 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/daemon.md
+1
View File
@@ -181,6 +181,7 @@ Verified live on 2026-09-13: the web UI started the client under auto-Xvfb, the
| `VESKSLOP_MEDIA_PREVIEW_BATCH` | `25` | Previews per batch before the cooldown. |
| `VESKSLOP_MEDIA_PREVIEW_BATCH_PAUSE_MS` | `60000` | Cooldown between preview batches. `0` disables batching. |
| `VESKSLOP_MEDIA_WALK_CYCLE_PAGES` | `5` | Walk pages between preview cycles on a live scrape. `0` defers previews to the end. |
| `VESKSLOP_MEDIA_RATELIMIT_PAUSE_MS` | `60000` | Minimum pause after a `429` before media traffic resumes. |
| `VESKSLOP_FFPROBE` | `ffprobe` | Probe binary used to fill missing dimensions after a download. |
| `VESKSLOP_MEDIA_DOWNLOAD_MIN_MS` / `_MAX_MS` | `1000` / `3000` | Gap between downloaded files. |
| `VESKSLOP_AUDIO_DUCK` | `30` | Sink volume percent while others speak. |
docs/media-scraper.md
+19
View File
@@ -157,6 +157,25 @@ Previews run after every walk:
(`VESKSLOP_MEDIA_PREVIEW_BATCH`, `VESKSLOP_MEDIA_PREVIEW_BATCH_PAUSE_MS`), because bursts of
thumbnail fetches trigger Discord and provider rate limits. A batch counts rows, not successes.
## Rate limits
Discord and the provider APIs throttle bursts. The engine detects limiting and holds all media
traffic:
- A `429` from a preview or download fetch pauses the pass; the row stays pending (never marked as a
permanent error) and is retried later.
- A walk page that fails with a `429` or a rate-limit message registers a cooldown, and every page
and preview waits it out before continuing.
- The pause is `VESKSLOP_MEDIA_RATELIMIT_PAUSE_MS` (default 60 s), or the server's `Retry-After`
when it is longer. Detection is logged as `rate limit detected; pausing media traffic until …`.
## Browsing cache
Browsing never calls Discord. The grid renders cached previews or local files only; videos without
a cached still show a labeled placeholder, and playback streams only when the user presses play.
GIF hover plays the animated GIF cached in `thumbs/` (the original is kept alongside the static
frame and referenced by `anim_hash`), never the provider, so scrolling cannot spam anyone.
## Job states
`queued`, `running`, `paused`, `stopping`, `done`, `failed`, `pending-delete`, `deleting`, `deleted`.
docs/webui.md
+3
View File
@@ -271,6 +271,9 @@ replaceable.
| POST | `/api/media/count` | `channelId`, `guildId?`, `after?`, `before?` | Discord search total for the channel or range. Returns `{ count }` or `{ count: null, reason }` (slopbot guild channels only) |
| GET | `/api/media/errors` | none | Bot and scrape errors, newest first |
| POST | `/api/media/errors/clear` | none | Clear the error list |
| GET | `/api/media/preview-failures` | `limit?` | Rows whose preview failed, with the reason |
| GET | `/api/media/:scrapeId/:itemId/anim` | none | The cached animated GIF for hover playback |
| POST | `/api/daemon/restart` | none | Replace the daemon process with a fresh copy (the client keeps running) |
| POST | `/api/media/scrape/stop` | `id` | Stop a media scrape |
| POST | `/api/media/scrape/resume` | `id` | Continue a paused scrape from its cursor |
| POST | `/api/media/clean` | `ids?` | Delete the given scrapes, or every scrape when `ids` is absent |