Add the Bot-tab backfill control room and media strip rules

docs/media-scraper.md

@@ -106,7 +106,7 @@ filled with `ffprobe` after a download when `VESKSLOP_FFPROBE` is available.
 Downloads run after a finished walk when the scrape used `mode: "download"`:
 
 - A serial queue, one file at a time, 1-3 s apart, with a size cap
-  (`VESKSLOP_MEDIA_MAX_FILE_BYTES`, default 50 MiB) and 2 retries on network or 5xx errors.
+  (the **Scraping** settings, default 50 MiB) and 2 retries on network or 5xx errors.
 - `4xx` from the skip set (400, 401, 403, 404, 410) marks the URL expired; the row is not retried.
 - The file name is the SHA-256 of the bytes under `media/`. `file_path`, `file_hash`, and
   `file_bytes` are written to the row, so the run resumes and dedupes by content.
@@ -121,8 +121,8 @@ Downloads run after a finished walk when the scrape used `mode: "download"`:
   durations come from Discord; the 0 that Discord sends for non-audio files is treated as unknown.
 - Progress counters: `downloaded`, `downloadErrors`.
 
-Previews cycle with the walk: after every `VESKSLOP_MEDIA_WALK_CYCLE_PAGES` pages (default 5) the
-engine previews up to one batch (`VESKSLOP_MEDIA_PREVIEW_BATCH`, default 25), then keeps walking,
+Previews cycle with the walk: after every **Preview every N walk pages** setting (default 5) pages
+the engine previews up to one batch (the **Backfilling** batch size, default 25), then keeps walking,
 so tiles fill in during a long walk instead of only at the end. Set the cycle to `0` to defer all
 previews to the end.
 
@@ -154,15 +154,19 @@ Previews run after every walk:
   counter is `previewed`. Previews count toward the storage limit.
 - The job stays `running` through the download and preview passes, so a client never sees `done`
   on a half-archived scrape. A paused walk still gets these passes for the pages it collected.
-- At daemon start, `backfillPreviews` runs the same pass over scrapes that paused before their
-  previews were cached, so a stopped scrape does not leave blank tiles.
-- Transient fetch failures keep the row pending and count up `preview_attempts`. After
-  `VESKSLOP_MEDIA_PREVIEW_MAX_ATTEMPTS` tries (default 3) the row is marked
-  `unreachable after N attempt(s)`, so dead hosts (Gfycat, Streamable) stop burning batches forever.
-- Preview gaps default to 200-600 ms (`VESKSLOP_MEDIA_PREVIEW_MIN_MS`/`_MAX_MS`), much faster than
-  file downloads. Previews are fetched in **batches of 25 with a 60 s cooldown** between batches
-  (`VESKSLOP_MEDIA_PREVIEW_BATCH`, `VESKSLOP_MEDIA_PREVIEW_BATCH_PAUSE_MS`), because bursts of
-  thumbnail fetches trigger Discord and provider rate limits. A batch counts rows, not successes.
+- The **backfill queue** (`media-scrape/src/backfill.ts`) fills in scrapes that paused or finished
+  before their previews were cached. One job per chat groups every paused, done, or failed scrape
+  of that chat, so a paused-and-resumed scrape stays in one job. Jobs run one at a time, in
+  batches, and report pending/cached/failed counts, per-kind pending, throughput, and an ETA. The
+  Bot tab drives them: **Start backfill**, per-job Start/Pause, and **Pause all**. On daemon boot
+  jobs start automatically when the Backfilling setting **Start automatically on daemon boot** is
+  on.
+- Transient fetch failures keep the row pending and count up `preview_attempts`. After the
+  **Tries before giving up** setting (default 3) the row is marked `unreachable after N attempt(s)`,
+  so dead hosts (Gfycat, Streamable) stop burning batches forever.
+- Preview gaps and batching live in the Bot tab settings (**Backfilling**): gap 200-600 ms, batches
+  of 25 with a 60 s cooldown between them, because bursts of thumbnail fetches trigger Discord and
+  provider rate limits. A batch counts rows, not successes.
 
 ## Rate limits
 
@@ -173,8 +177,9 @@ traffic:
   permanent error) and is retried later.
 - A walk page that fails with a `429` or a rate-limit message registers a cooldown, and every page
   and preview waits it out before continuing.
-- The pause is `VESKSLOP_MEDIA_RATELIMIT_PAUSE_MS` (default 60 s), or the server's `Retry-After`
-  when it is longer. Detection is logged as `rate limit detected; pausing media traffic until …`.
+- The pause is the **Timers** setting **Rate-limit cooldown** (default 60 s), or the server's
+  `Retry-After` when it is longer. Detection is logged as `rate limit detected; pausing media traffic
+  until …`. The Bot tab banner reads `API timed out, increase timers` while media traffic is held.
 
 ## Browsing cache
 
@@ -183,6 +188,27 @@ a cached still show a labeled placeholder, and playback streams only when the us
 GIF hover plays the animated GIF cached in `thumbs/` (the original is kept alongside the static
 frame and referenced by `anim_hash`), never the provider, so scrolling cannot spam anyone.
 
+## Bot tab settings
+
+The Bot tab owns the tunables; they persist in `preferences.json` and apply to live jobs at once.
+
+| Group | Setting | Default |
+| --- | --- | --- |
+| Backfilling | Batch size (rows) | 25 |
+| Backfilling | Cooldown between batches (ms) | 60000 |
+| Backfilling | Tries before giving up | 3 |
+| Backfilling | Preview gap min / max (ms) | 200 / 600 |
+| Backfilling | Max preview size (MiB) | 5 |
+| Backfilling | Start automatically on daemon boot | on |
+| Scraping | Preview every N walk pages | 5 |
+| Scraping | Max file size (MiB) | 50 |
+| Scraping | Download gap min / max (ms) | 1000 / 3000 |
+| Scraping | Page fetch retries | 2 |
+| Timers | Walk page gap min / max (ms) | 1200 / 3500 |
+| Timers | Rate-limit cooldown (ms) | 60000 |
+
+The old `VESKSLOP_MEDIA_*` tuning variables are gone; the values above were their defaults.
+
 ## Job states
 
 `queued`, `running`, `paused`, `stopping`, `done`, `failed`, `pending-delete`, `deleting`, `deleted`.