zib/Veskslop-t
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add the media downloader with hash storage and local file serving

Browse Source
This commit is contained in:
Fisto
2026-09-16 10:35:55 +02:00
parent 3ea72f587d
commit f7664256d1
15 changed files with 431 additions and 28 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/media.md
+11 -7
View File
@@ -1,10 +1,12 @@
# Media archive
Status: design approved 2026-09-16. The engine, the slopbot backend, the `scrape media` command,
the `/api/media/*` routes, storage limits with retention, the Settings group, and the MCP tools are
implemented (C3-C8). The Media and Bot tabs are C6. The slopbot path is live-verified: the test
channel produced 17 messages and 4 images + 13 videos with no errors. The puppet path and a
`download` run are still owed (C9). Checkpoints live in [CURRENT.md](../CURRENT.md).
the `/api/media/*` routes, storage limits with retention, the Settings group, the MCP tools, and
the byte downloader are implemented (C3-C8, C10). The Media and Bot tabs are C6. Live-verified on
the test channel: links mode produced 17 messages (4 images, 13 videos) with no errors, and
download mode fetched the attachment (784 KB) and served it locally. Remaining: preview caching
(C11), the puppet `message.fetch` field check, and a 429 observation. Checkpoints live in
[CURRENT.md](../CURRENT.md).
The media archive saves the media history of a channel. A trusted user runs `scrape media` in the
channel. The daemon walks the history through a scrape backend and indexes every image, GIF,
@@ -72,9 +74,11 @@ media-scrape/db/
- `links` is the default. The archive stores the Discord CDN URL and the metadata. Nothing large is
downloaded.
- `download` fetches attachment bytes into `media/`. Deduplicated by hash. **Not implemented yet**
(checkpoint C10): the command accepts the mode and the schema has the columns, but the download
queue is still owed.
- `download` fetches attachment bytes into `media/`, one file at a time, hash-addressed. Embed rows
are never download targets. The Media tab prefers the local file when one exists, so the archive
survives CDN URL expiry.
- Cached image previews are still planned (C11). Until then, the Media tab hotlinks Discord CDN URLs
in `links` mode.
- Third-party embeds are **never downloaded**: YouTube, Instagram, X, TikTok, and every other
embed stays a link with metadata.
- Discord-hosted image and sticker previews are cached small (`thumbs/`, WebP, via the CDN