Backfill previews for paused scrapes and prefer richer dedupe rows
This commit is contained in:
@@ -176,6 +176,7 @@ Verified live on 2026-09-13: the web UI started the client under auto-Xvfb, the
|
||||
| `VESKSLOP_MEDIA_DB_DIR` | `<repo>/media-scrape/db` | Media archive root: per-scrape databases, downloads, previews. |
|
||||
| `VESKSLOP_MEDIA_MAX_FILE_BYTES` | `52428800` | Download size cap (50 MiB). Larger files are skipped. |
|
||||
| `VESKSLOP_MEDIA_PREVIEW_MAX_BYTES` | `5242880` | Preview size cap (5 MiB). Larger previews are skipped. |
|
||||
| `VESKSLOP_MEDIA_PREVIEW_MIN_MS` / `_MAX_MS` | `200` / `600` | Gap between preview fetches. |
|
||||
| `VESKSLOP_FFPROBE` | `ffprobe` | Probe binary used to fill missing dimensions after a download. |
|
||||
| `VESKSLOP_MEDIA_DOWNLOAD_MIN_MS` / `_MAX_MS` | `1000` / `3000` | Gap between downloaded files. |
|
||||
| `VESKSLOP_AUDIO_DUCK` | `30` | Sink volume percent while others speak. |
|
||||
|
||||
@@ -133,7 +133,11 @@ Previews run after every walk:
|
||||
(`preview_ext`, default webp); `preview_hash` and `preview_bytes` land on the row, and the
|
||||
counter is `previewed`. Previews count toward the storage limit.
|
||||
- The job stays `running` through the download and preview passes, so a client never sees `done`
|
||||
on a half-archived scrape.
|
||||
on a half-archived scrape. A paused walk still gets these passes for the pages it collected.
|
||||
- At daemon start, `backfillPreviews` runs the same pass over scrapes that paused before their
|
||||
previews were cached, so a stopped scrape does not leave blank tiles.
|
||||
- Preview gaps default to 200-600 ms (`VESKSLOP_MEDIA_PREVIEW_MIN_MS`/`_MAX_MS`), much faster than
|
||||
file downloads.
|
||||
|
||||
## Job states
|
||||
|
||||
|
||||
Reference in New Issue
Block a user