Deduplicate media across scrapes and polish the viewer controls
This commit is contained in:
+11
-3
@@ -67,6 +67,10 @@ media-scrape/db/
|
||||
- The database is an index. It holds the scrape state, the messages seen, the media items, and the
|
||||
link between a media item and its source messages. Source channel, author, and timestamp are
|
||||
indexed so filters can be added later without a re-scrape.
|
||||
- Scrapes stay self-contained, but the API and UI **deduplicate by URL across scrapes**: one card
|
||||
per URL, the newest scrape wins; stats count unique URLs too. Re-scraping a channel does not
|
||||
duplicate anything in the grid or the counters. (Files are still stored per scrape, so the
|
||||
on-disk size doubles with overlapping scrapes; a shared content store is a possible later step.)
|
||||
- Storage uses `node:sqlite` (experimental in Node 22, prints a warning). If its API changes,
|
||||
`better-sqlite3` is the fallback; it needs a native build, so that switch is an owner install.
|
||||
- The file name in `media/` is the SHA-256 of the bytes, never the original name. The original
|
||||
@@ -147,9 +151,13 @@ that deletion is irreversible. The delete button is disabled for a 5 second coun
|
||||
tiles carry a translucent play icon top-right. Discord serves no thumbnails for uploaded videos,
|
||||
so a video without a cached preview renders a first-frame `<video>` in the grid, and downloaded
|
||||
videos get an ffmpeg poster cached in `thumbs/`. Clicking a tile opens a viewer: wide media
|
||||
shows the metadata banner below, vertical media shows it to the right; the banner has the full
|
||||
link (with an external-site warning) and **View full**, and the play action embeds a YouTube,
|
||||
Vimeo, Twitch, or Streamable player when possible. The status row reads
|
||||
shows the metadata banner below, vertical media shows it to the right, and videos carry a large
|
||||
play icon top-right. The play action embeds a YouTube, Vimeo, Twitch, or Streamable player, or
|
||||
plays a direct mp4/webm/m4v; when neither is possible it opens the external-site warning, since
|
||||
the page is the only way to watch. **View full** opens the embedded media (player, local file,
|
||||
or cached preview) in a new tab; the full link is a separate button that opens the original
|
||||
page with the warning. Length comes from Discord's attachment duration and, for downloads, an
|
||||
ffprobe probe. The status row reads
|
||||
`(Images/Videos) | Downloaded A/B | Cached C/B (excl Downl) | Loaded L/B`, where B is the total
|
||||
for the active filters. A scrape strip shows live progress with Stop.
|
||||
**New scrape…** (rightmost) opens a popup with Server and Group chat trees (servers expand to
|
||||
|
||||
Reference in New Issue
Block a user