Store embed titles and search them
This commit is contained in:
@@ -88,7 +88,7 @@ One SQLite file per scrape (`index.db`, `node:sqlite`).
|
||||
| Table | Columns |
|
||||
| --- | --- |
|
||||
| `messages` | `id`, `author_id`, `author_name`, `timestamp`, `content` |
|
||||
| `media` | `id`, `kind`, `source`, `author_id`, `author_name`, `posted_at`, `url`, `provider`, `file_hash`, `preview_hash`, `content_type`, `size`, `width`, `height`, `aspect_ratio`, `duration_secs`, `downloaded`, `file_path`, `created_at` |
|
||||
| `media` | `id`, `kind`, `source`, `author_id`, `author_name`, `posted_at`, `url`, `provider`, `title`, `title_key`, `file_hash`, `preview_hash`, `content_type`, `size`, `width`, `height`, `aspect_ratio`, `duration_secs`, `downloaded`, `file_path`, `created_at` |
|
||||
| `media_sources` | `media_id`, `message_id` |
|
||||
| `downloads` | `media_id`, `state`, `attempts`, `error` |
|
||||
|
||||
@@ -96,7 +96,11 @@ One SQLite file per scrape (`index.db`, `node:sqlite`).
|
||||
slopbot backend could ask for it (one request; null otherwise). Indexes: `kind`, `posted_at`, and
|
||||
`media_sources(message_id)`. The database is an index; the media
|
||||
folder holds bytes. Every item carries its poster (`author_id`, `author_name`), the post time
|
||||
(`posted_at`), and its dimensions (`width`, `height`, `aspect_ratio`). Filter presets live in
|
||||
(`posted_at`), and its dimensions (`width`, `height`, `aspect_ratio`). Embed rows also store
|
||||
Discord's `title` (YouTube and friends) plus `title_key`, the lowercased, punctuation-collapsed
|
||||
form used by the title search: multi-word prefix tokens by default ("Ma Ca Ju" finds *Man who jumps
|
||||
over car*), one phrase with the exact-match toggle, and rows without a title drop out while a
|
||||
title query is set. Filter presets live in
|
||||
`media-scrape/src/filter-presets.ts`: aspect buckets (vertical, square, 4:3, 16:9, 21:9, other) and
|
||||
resolution buckets by the shortest side (SD, HD, FHD, QHD, 4K+, unknown). A missing resolution is
|
||||
filled with `ffprobe` after a download when `VESKSLOP_FFPROBE` is available.
|
||||
|
||||
+3
-1
@@ -149,7 +149,9 @@ that deletion is irreversible. The delete button is disabled for a 5 second coun
|
||||
and the multi-select People popover. The **Advanced search** drawer has aspect ratio, resolution,
|
||||
source channel, posted-date range, file-type chips, cached-only, a default-on hide for 4chan
|
||||
links (they 404 almost always), a sort select (newest, oldest, user, size, length), and a **Do not
|
||||
include** and **Must include** groups with nested collapsibles for sources (the common providers
|
||||
include** and **Must include** groups with nested collapsibles, a title search for embed titles
|
||||
(multi-word prefix matching, with an **Exact match (quotes)** toggle) for sources (the common
|
||||
providers
|
||||
in the archive plus embeds, attachments, and stickers), file types, size (with B/KB/MB/GB units),
|
||||
and length (seconds), with **Apply filters** / **Reset** in the bar. Size and length ranges drop
|
||||
rows with no known value. The bar and drawer show only on the Media tab.
|
||||
|
||||
+5
-2
@@ -137,7 +137,10 @@ box live in Chat, not Call, and the connection stats are not chips yet.
|
||||
2ch.hk, Imgur, FunnyJunk, Internet Archive — plus embeds, attachments, stickers) and file types.
|
||||
**Must include** requires them (same provider list and kinds, file-type chips, and size/length
|
||||
bounds with B/KB/MB/GB units or seconds). A provider entry may carry several substrings, shown as
|
||||
`Giphy/Tenor`. A sort select orders the grid by newest, oldest, user, size, or length. Size and
|
||||
`Giphy/Tenor`. A title field searches embed titles (YouTube): default is multi-word prefix
|
||||
matching (each token must start a word of the title, order-free), and **Exact match (quotes)**
|
||||
treats the input as one phrase. A sort select orders the grid by newest, oldest, user, size, or
|
||||
length. Size and
|
||||
length bounds drop rows with no known value. The filter bar and the drawer are hidden on the Bot
|
||||
tab. Filters do not
|
||||
apply until **Apply filters** is pressed; **Reset** clears every filter and the zoom, unloads the
|
||||
@@ -289,7 +292,7 @@ replaceable.
|
||||
| POST | `/api/audio/stop` | none | Stop speech and clear the queue |
|
||||
| POST | `/api/audio/capture` | `start` | Start or stop capture |
|
||||
| GET | `/api/transcript` | `limit?`, `sessionId?` | Sessions and captured segments |
|
||||
| GET | `/api/media` | `kind?`, `source?`, `channelId?`, `authorId?` (comma list), `origin?` (`guild`/`dm`), `cachedOnly?`, `ext?` (comma list), `excludeHosts?` (comma list), `excludeExt?`, `excludeProviders?`, `excludeSources?`, `includeProviders?`, `includeSources?` (comma lists), `sizeLt?`/`sizeGt?` (bytes), `lenLt?`/`lenGt?` (seconds), `sort?` (`newest` default, `oldest`, `user`, `size`, `length`), `aspect?`, `resolution?`, `after?`, `before?`, `limit?`, `offset?` | Media items across every scrape, newest first. `source` is `attachment`, `sticker`, or `embed`; `aspect` and `resolution` are the preset keys; `after`/`before` are ISO times on `posted_at`. Items carry `authorName`, `postedAt`, `width`, `height`, `aspectRatio`, `contentType`, `durationSecs`, `previewError`, `animUrl`; `localUrl`/`thumbUrl` when a file or preview exists |
|
||||
| GET | `/api/media` | `kind?`, `source?`, `channelId?`, `authorId?` (comma list), `origin?` (`guild`/`dm`), `cachedOnly?`, `ext?` (comma list), `excludeHosts?` (comma list), `excludeExt?`, `excludeProviders?`, `excludeSources?`, `includeProviders?`, `includeSources?` (comma lists), `title?` (200 chars max), `titleExact?`, `sizeLt?`/`sizeGt?` (bytes), `lenLt?`/`lenGt?` (seconds), `sort?` (`newest` default, `oldest`, `user`, `size`, `length`), `aspect?`, `resolution?`, `after?`, `before?`, `limit?`, `offset?` | Media items across every scrape, newest first. `source` is `attachment`, `sticker`, or `embed`; `aspect` and `resolution` are the preset keys; `after`/`before` are ISO times on `posted_at`. Items carry `authorName`, `postedAt`, `width`, `height`, `aspectRatio`, `contentType`, `durationSecs`, `previewError`, `animUrl`; `localUrl`/`thumbUrl` when a file or preview exists |
|
||||
| GET | `/api/media/:scrapeId/:itemId/file` | none | The downloaded file, content-typed from its extension |
|
||||
| GET | `/api/media/:scrapeId/:itemId/thumb` | none | The cached preview (type from `preview_ext`) |
|
||||
| GET | `/api/media/:scrapeId/:itemId/stream` | none | Playback/full-view source: the local file, or the remote URL refreshed from the source message. Supports `Range`. Non-media upstream responses (provider pages) return `415` |
|
||||
|
||||
Reference in New Issue
Block a user