Add custom download destinations, confirmations, and the bot error feed
This commit is contained in:
+1
-1
@@ -35,7 +35,7 @@ preset keys from `filter-presets.ts`; `after`/`before` are ISO times on `posted_
|
||||
|
||||
| Tool | Access | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `scrape_media` | trusted | Start a scrape (`channelId`, `mode`) |
|
||||
| `scrape_media` | trusted | Start a scrape (`channelId`, `guildId?`, `mode?`, `backend?`, `downloadDir?`) |
|
||||
| `list_media_scrapes` | trusted | Every scrape with state, size, and counts |
|
||||
| `get_media_scrape` | trusted | One scrape, live counters and cursor |
|
||||
| `stop_media_scrape` | trusted | Stop the active scrape |
|
||||
|
||||
@@ -103,6 +103,9 @@ Downloads run after a finished walk when the scrape used `mode: "download"`:
|
||||
`file_bytes` are written to the row, so the run resumes and dedupes by content.
|
||||
- Only attachment and sticker rows are targets. Embed rows are excluded in SQL
|
||||
(`source != 'embed'`).
|
||||
- With a custom destination, files go to `<destination>/<scrape-id>/media/` and the row stores the
|
||||
absolute path; without one they stay in `<scrape>/media/`. Embed previews are copied to
|
||||
`<destination>/<scrape-id>/thumbnails/` while the local copy stays for the Media tab.
|
||||
- Progress counters: `downloaded`, `downloadErrors`.
|
||||
|
||||
Previews run after every walk:
|
||||
|
||||
+11
-4
@@ -80,6 +80,10 @@ media-scrape/db/
|
||||
- `download` fetches attachment bytes into `media/`, one file at a time, hash-addressed. Embed rows
|
||||
are never download targets. The Media tab prefers the local file when one exists, so the archive
|
||||
survives CDN URL expiry.
|
||||
- A custom download destination sends the bytes to `<dir>/<scrape-id>/media/` instead. Previews
|
||||
stay local, and embed thumbnails are copied to `<dir>/<scrape-id>/thumbnails/` so the download
|
||||
has a poster. Files in a custom destination are not served or linked by the Media tab; the user
|
||||
owns that path.
|
||||
- Third-party embeds are **never downloaded as media**: YouTube, Instagram, X, TikTok, and every
|
||||
other embed stays a link. Its thumbnail is cached (below); the video itself is never fetched.
|
||||
- Previews are cached small (WebP, 320px) in `thumbs/` after every walk, in both modes: Discord
|
||||
@@ -134,10 +138,13 @@ that deletion is irreversible. The delete button is disabled for a 5 second coun
|
||||
21:9, Other), resolution (SD, HD, FHD, QHD, 4K+, Unknown), and a posted-date range (from/to).
|
||||
Grid with lazy loading; previews come from the cache, then the local file, then the CDN; each
|
||||
tile's tooltip shows the poster and posted time. A scrape strip shows live progress with Stop.
|
||||
**New scrape…** opens a popup with Server and Group chat trees (servers expand to channels; each
|
||||
row has Links and Download), and **History** lists each scrape with its start time, duration,
|
||||
pages, messages, media counts, bytes, files, and previews. The view fills the window below the
|
||||
header and the call strip.
|
||||
**New scrape…** (rightmost) opens a popup with Server and Group chat trees (servers expand to
|
||||
channels; each row has Links and Download). Every scrape button asks for confirmation, and
|
||||
Download can target a custom absolute destination. **History** lists each scrape with its start
|
||||
time, duration, pages, messages, media counts, bytes, files, and previews. The popups close on
|
||||
backdrop click, X, or Close. Only active scrapes show in the strip; the view fills the window
|
||||
below the header and the call strip. The Bot tab lists bot and scrape errors with a red badge
|
||||
that disappears when the tab is opened and a Clear button.
|
||||
- **Bot** tab: slopbot connection state, the scrape queue, and the storage limit with a usage bar.
|
||||
The bar is segmented and colour coded by media kind (images, GIFs, video, audio, other) plus the
|
||||
database overhead.
|
||||
|
||||
+3
-1
@@ -263,7 +263,9 @@ replaceable.
|
||||
| GET | `/api/media/status` | none | Slopbot connection state and the scrape list |
|
||||
| GET | `/api/media/authors` | none | Distinct posters across every scrape, with counts |
|
||||
| GET | `/api/media/storage` | none | Usage against the limit, per-kind sizes, `nearLimit`; marks old scrapes when over |
|
||||
| POST | `/api/media/scrape` | `channelId`, `guildId?`, `mode?`, `backend?`, `confirm?` | Start a media scrape. `backend` is `auto` (default), `slopbot`, or `puppet`. Near the limit: `409` with code `MEDIA_LIMIT` unless `confirm: true` |
|
||||
| POST | `/api/media/scrape` | `channelId`, `guildId?`, `mode?`, `backend?`, `confirm?`, `downloadDir?` | Start a media scrape. `backend` is `auto` (default), `slopbot`, or `puppet`; `downloadDir` is an absolute path for downloaded files (previews stay local). Near the limit: `409` with code `MEDIA_LIMIT` unless `confirm: true` |
|
||||
| GET | `/api/media/errors` | none | Bot and scrape errors, newest first |
|
||||
| POST | `/api/media/errors/clear` | none | Clear the error list |
|
||||
| POST | `/api/media/scrape/stop` | `id` | Stop a media scrape |
|
||||
| POST | `/api/media/clean` | `ids?` | Delete the given scrapes, or every scrape when `ids` is absent |
|
||||
| GET | `/api/logs` | `limit` | The persistent action log |
|
||||
|
||||
Reference in New Issue
Block a user