Document the media archive in the README

README.md

@@ -22,6 +22,7 @@
 [![Headless](https://img.shields.io/badge/headless-100%25-success)](#-why-though)
 [![MCP](https://img.shields.io/badge/MCP-agentic%20banger-8a2be2)](#-mcp-the-brain)
 [![Audio](https://img.shields.io/badge/audio-in%20%2F%20out%2F%20sideways-1db954)](#-audio-pipes)
+[![Media](https://img.shields.io/badge/media%20archive-scraped-2ecc71)](#-media-archive)
 [![Video](https://img.shields.io/badge/video-rtx%20on-ff4500)](#-video-streams)
 [![Based On](https://img.shields.io/badge/based%20on-Vesktop-black)](https://github.com/Vencord/Vesktop)
 [![License](https://img.shields.io/badge/license-GPL--3.0--or--later-blue)](LICENSE)
@@ -46,10 +47,11 @@ Think of it as:
 - 🧠 **An agent body (text tools live).** Your MCP-capable AI gets a real account with real permissions and real consequences.
 - 🔀 **A proxy (planned).** Messages, events, voice state, attachments — everything forwarded, interceptable, scriptable, and API-shaped.
 - 🔊 **An audio router (partly live).** TTS into the call, ducking, browser monitoring, and captured call audio with Vosk STT. Browser microphone puppeting is next.
+- 📼 **A media archive (live).** Walks a channel's whole history, indexes every image, GIF, video, and audio item, caches previews, and makes it all browsable and searchable in the web UI.
 - 📹 **A streaming endpoint (planned).** Push files, cameras, generative visuals, or pure slop into a voice channel.
 - 🧪 **A science experiment that will absolutely not get your account banned** *(allegedly)*.
 
-> **Status: text and voice basics work.** The real Vesktop client runs headless and is driven through a validated capability layer. Triggers, trust tiers, conversation context, response policy, persistent logging, MCP tools, user search, voice state, voice join/leave/move, a virtual-mic TTS pipeline with ducking, browser call monitoring, VAD-segmented call capture with Vosk STT, core process control, web QR login, and the Nocturne-style debug web UI over self-signed HTTPS are live. Reaction polls are paused. Auto-leave, browser microphone puppeting, and video are still ahead. The design docs live in [docs/INDEX.md](docs/INDEX.md).
+> **Status: text, voice, and the media archive work.** The real Vesktop client runs headless and is driven through a validated capability layer. Triggers, trust tiers, conversation context, response policy, persistent logging, MCP tools, user search, voice state, voice join/leave/move, a virtual-mic TTS pipeline with ducking, browser call monitoring, VAD-segmented call capture with Vosk STT, core process control, web QR login, and the Nocturne-style debug web UI over self-signed HTTPS are live. The media archive walks channel history on both a dedicated bot and the puppet, caches previews, backfills titles and previews in the background, and ships its own Media and Bot tabs. Reaction polls are paused. Browser microphone puppeting and video are still ahead. The design docs live in [docs/INDEX.md](docs/INDEX.md).
 
 ## 🗿 Why though
 
@@ -88,12 +90,18 @@ Veskslop ships an **MCP server** so any MCP-capable agent (Claude, GPT-based age
 | `get_puppet_config` / `set_puppet_config` | Response policy, context limits, poll duration. |
 | `whitelist_list` / `whitelist_add` / `whitelist_remove` | Trust tiers: user, trusted, admin. |
 | `run_commands` | Run several read-only tools in one call. |
+| `scrape_media` | Start a media scrape of a channel or group chat (links or download mode). |
+| `stop_media_scrape` / `resume_media_scrape` / `delete_media_scrape` | Control and clean up scrapes. |
+| `list_media_scrapes` / `get_media_scrape` | Scrape states, counters, and history. |
+| `query_media` | Query the archive: kind, source, provider, extensions, size/length, title search, sort. |
+| `get_media_item` | One item with its local file, preview, and message link. |
+| `count_media_messages` / `get_media_storage` | Discord search totals and storage usage. |
 
 There is also a debug web UI on `https://<host>:8789/` (self-signed certificate) with a JSON API,
 an SSE event stream, a DM/favourites tab, and browser call audio. See [docs/webui.md](docs/webui.md).
 
-**Planned**: voice auto-leave and `stay @user`, browser microphone puppeting, video streams, message
+**Planned**: `stay @user`, invite-link joins, browser microphone puppeting, video streams, message
-edit/delete/upload, notification parsing, presence control.
+edit/delete/upload, presence control.
 
 Every action is **safe by default**: strict input validation in the client, structured errors, a hard outbound rate limit, and paced sends. A buggy agent cannot crash the client or spam Discord.
 
@@ -134,6 +142,45 @@ voice channel ──▶ per-user decode ──▶ mix ──▶ PCM/file/STT ─
 <img src="docs/images/veskslop-audio.png" alt="Veskslop audio pipeline" width="80%" />
 <br />
 
+## 📼 Media archive
+
+Point it at a channel and it walks the history, indexes every image, GIF, video, and audio item it
+finds, caches previews (Discord's own proxy for attachments, provider oEmbed/og:image for embeds),
+downloads bytes on request, and makes the whole thing browsable in the **Media** tab.
+
+```sh
+@puppet scrape media           # index links (default)
+@puppet scrape media download  # also download attachment bytes
+@puppet scrape media status
+@puppet scrape media stop
+```
+
+- **Two backends, picked automatically.** A dedicated bot account walks guild channels through the
+  official API; the puppet handles DMs, group chats, and anything the bot cannot see. A scrape
+  remembers which backend it used, and the daemon serializes walks and backfills so Discord's rate
+  limits are never multiplied.
+- **Media tab.** Kind/source/origin filters, a People popover, aspect and resolution presets,
+  file-type chips, **Do not include** and **Must include** groups (providers like Discord,
+  YouTube, Giphy/Tenor, Reddit, Twitter, Instagram, TikTok, Streamable, …, sizes, lengths), a
+  title search with order-free word prefixes and `*` wildcards, sorting by newest/oldest/user/
+  size/length, and an **Infinite/Page** toggle (page mode pre-loads the next page and lets you
+  bookmark a filter+page).
+- **Bot tab.** Backfill queue with per-chat jobs and ETAs, a YouTube title catch-up lane, a
+  one-shot recovery task for deleted-file previews, a live failure-policy hint, and all the
+  pacing knobs (batch sizes, cooldowns, timers) with server-side persistence.
+- **Nothing is thrown away on one bad answer.** A failed fetch on Discord's CDN is retried with a
+  freshly signed URL from the source message; still-failing rows are parked for one delayed retry
+  and then left unverified, so a finished backfill never churns over genuinely deleted media.
+  Recovering a stale archive has recovered 90%+ of previously "gone" previews.
+- **Re-scrapes are incremental.** Starting a scrape of a channel that already has a complete walk
+  starts where that walk began, so only new messages are fetched.
+- **Verify it.** `pnpm verify:scrapes` (from `poc/`) checks every index for completeness: done
+  state, message counts against Discord's search totals, and the oldest message against the
+  source's creation time.
+
+Full details: [docs/media.md](docs/media.md), [docs/media-scraper.md](docs/media-scraper.md),
+[docs/slopbot.md](docs/slopbot.md), [docs/media-mcp.md](docs/media-mcp.md).
+
 ## 📹 Video streams (planned)
 
 Send video where Discord expects a camera:
@@ -223,6 +270,9 @@ The full setup, including the headless client and the mock core, is in
 - [ ] Browser microphone puppeting (browser mic into the bot's voice)
 - [x] Voice auto-leave when the owner leaves (with a delay and rejoin cancellation)
 - [ ] `stay @user` and invite-link joins
+- [x] Media archive: full-history walk, dedupe, previews, downloads, Media and Bot tabs
+- [x] Backfill queue per chat, YouTube title catch-up, Discord preview recovery, suspect retries
+- [x] Media filters, title search with wildcards, sorting, infinite/page modes, message links
 - [x] Notification parsing: edits, deletes, reactions, replies (reply-to-puppet triggers too)
 - [x] Reply-to-puppet as a trigger
 - [ ] Per-user rate limits and command cooldowns
@@ -264,6 +314,12 @@ Start here:
 - [poc/README.md](poc/README.md) — daemon run commands
 - [docs/core-bridge.md](docs/core-bridge.md) — the capability layer
 - [docs/daemon.md](docs/daemon.md) — the MCP daemon
+- [docs/media.md](docs/media.md) — the media archive and its two web tabs
+- [docs/media-scraper.md](docs/media-scraper.md) — the scrape engine, backfills, failure policy
+
+After `pnpm install`, the usual loop is `pnpm testTypes`, `pnpm lint`, `pnpm build`, then
+`cd poc && pnpm check && pnpm test:comm && pnpm test:media`. Optional extras: `pnpm test:ui`
+(browser smoke test for the Media tab) and `pnpm verify:scrapes` (scrape completeness report).
 
 <div align="center">