Add source filters and a full-window media view

This commit is contained in:
Fisto
2026-09-16 11:11:50 +02:00
parent 2573dd4d63
commit 902646224f
12 changed files with 179 additions and 32 deletions
+4
View File
@@ -40,6 +40,10 @@ mobile draft at 390px.
- [x] C11 preview caching: Discord-hosted image/GIF previews (WebP 320px, hash-addressed, 5 MiB - [x] C11 preview caching: Discord-hosted image/GIF previews (WebP 320px, hash-addressed, 5 MiB
cap, one attempt) run after every walk in both modes. Live run `...-20260916090722`: cap, one attempt) run after every walk in both modes. Live run `...-20260916090722`:
`previewed: 2`; thumbs served over `/api/media/<scrape>/<item>/thumb` (200). `previewed: 2`; thumbs served over `/api/media/<scrape>/<item>/thumb` (200).
- [x] C12 Media tab filters and full-window layout: kind chips, source filter
(attachment/sticker/embed), source-channel filter, and the view now fills the window below the
header and call strip. API/MCP gained `source` and `channelId` on `query_media`. Smoke-tested
over `/api/media?source=` and `?channelId=`.
- [~] C9 live verification. Slopbot path verified live on 2026-09-16: login Slopbot#5775, guild - [~] C9 live verification. Slopbot path verified live on 2026-09-16: login Slopbot#5775, guild
`177931418704281600`. Test channel links scrape: 17 messages, 4 images + 13 videos, 0 errors; `177931418704281600`. Test channel links scrape: 17 messages, 4 images + 13 videos, 0 errors;
download scrape fetched the 784 KB attachment. Large channel `gernal_chut`: 11 pages, 1088 download scrape fetched the 784 KB attachment. Large channel `gernal_chut`: 11 pages, 1088
+1 -1
View File
@@ -83,7 +83,7 @@ The daemon exposes MCP over Streamable HTTP. The default address is `http://127.
| `get_media_scrape` | One scrape by ID. | | `get_media_scrape` | One scrape by ID. |
| `stop_media_scrape` | Stop a scrape; it pauses after the current page. | | `stop_media_scrape` | Stop a scrape; it pauses after the current page. |
| `get_media_storage` | Usage against the limit, per-kind sizes, and the scrape list. | | `get_media_storage` | Usage against the limit, per-kind sizes, and the scrape list. |
| `query_media` | Media items across scrapes (`kind?`, `scrapeId?`, `limit?`, `offset?`). | | `query_media` | Media items across scrapes (`kind?`, `source?`, `scrapeId?`, `channelId?`, `limit?`, `offset?`). |
| `get_media_item` | One media item by `scrapeId` and `itemId`. | | `get_media_item` | One media item by `scrapeId` and `itemId`. |
| `delete_media_scrape` | Delete a scrape and its files. Irreversible. | | `delete_media_scrape` | Delete a scrape and its files. Irreversible. |
| `run_commands` | Run several read-only tools in one call, in order. | | `run_commands` | Run several read-only tools in one call, in order. |
+1 -1
View File
@@ -23,7 +23,7 @@ interface MediaService {
} }
``` ```
`MediaQuery` (implemented): `kind`, `scrapeId`, `limit`, `offset`. Planned: `channelId`, `MediaQuery` (implemented): `kind`, `source`, `scrapeId`, `channelId`, `limit`, `offset`. Planned:
`guildId`, `authorId`, `after`, `before`, `text`. `guildId`, `authorId`, `after`, `before`, `text`.
`StorageStatus`: `usedBytes`, `limitBytes`, `warnPercent`, `overrunPercent`, `byKind`, `StorageStatus`: `usedBytes`, `limitBytes`, `warnPercent`, `overrunPercent`, `byKind`,
+5 -3
View File
@@ -129,9 +129,11 @@ that deletion is irreversible. The delete button is disabled for a 5 second coun
## Web UI ## Web UI
- **Media** tab: filter chips (All, Images, GIFs, Videos, Audio), source channel, date, and text - **Media** tab: filter chips (All, Images, GIFs, Videos, Audio), a source filter (any,
search. Grid with lazy loading; click opens a lightbox with the source message link, the attachments, stickers, embeds), and a source-channel filter built from the scrape list. Date and
original URL, and a download button. A scrape-status strip shows progress with Start and Stop. text search are planned. Grid with lazy loading; previews come from the cache, then the local
file, then the CDN. A scrape-status strip shows progress with Start and Stop. The view fills the
window below the header and the call strip.
- **Bot** tab: slopbot connection state, the scrape queue, and the storage limit with a usage bar. - **Bot** tab: slopbot connection state, the scrape queue, and the storage limit with a usage bar.
The bar is segmented and colour coded by media kind (images, GIFs, video, audio, other) plus the The bar is segmented and colour coded by media kind (images, GIFs, video, audio, other) plus the
database overhead. database overhead.
+9 -6
View File
@@ -121,11 +121,14 @@ box live in Chat, not Call, and the connection stats are not chips yet.
## Media and Bot ## Media and Bot
- **Media** and **Bot** are scope buttons beside Guild and DM. They open overlays over the shell. - **Media** and **Bot** are scope buttons beside Guild and DM. They open a full-window view below
Mobile is out of scope for the media feature. the header and the call strip (the shell is covered; only the header, and the call strip when it
- Media lists scraped items with kind chips (All, Images, GIFs, Videos, Audio), a channel picker is visible, stay on screen). Mobile is out of scope for the media feature.
with **Start links** / **Start download**, and the active scrapes with **Stop**. Images and GIFs - Media lists scraped items with kind chips (All, Images, GIFs, Videos, Audio), a source filter
render from their Discord CDN URLs; videos and audio are link cards. (any, attachments, stickers, embeds), a source-channel filter built from the scrape list, a
channel picker with **Start links** / **Start download**, and the active scrapes with **Stop**.
Images and GIFs render from the cached preview when one exists, then the local file, then the
Discord CDN URL; videos and audio are link cards.
- Bot shows the slopbot connection state and the scrape queue. - Bot shows the slopbot connection state and the scrape queue.
- The views are self-contained: `poc/public/media.js` and `media.css`, over the `/api/media/*` - The views are self-contained: `poc/public/media.js` and `media.css`, over the `/api/media/*`
routes in the table below. See [media.md](media.md). routes in the table below. See [media.md](media.md).
@@ -249,7 +252,7 @@ replaceable.
| POST | `/api/audio/stop` | none | Stop speech and clear the queue | | POST | `/api/audio/stop` | none | Stop speech and clear the queue |
| POST | `/api/audio/capture` | `start` | Start or stop capture | | POST | `/api/audio/capture` | `start` | Start or stop capture |
| GET | `/api/transcript` | `limit?`, `sessionId?` | Sessions and captured segments | | GET | `/api/transcript` | `limit?`, `sessionId?` | Sessions and captured segments |
| GET | `/api/media` | `kind?`, `limit?`, `offset?` | Media items across every scrape, newest first; `localUrl` when a file was downloaded | | GET | `/api/media` | `kind?`, `source?`, `channelId?`, `limit?`, `offset?` | Media items across every scrape, newest first. `source` is `attachment`, `sticker`, or `embed`; `channelId` filters by source channel. `localUrl`/`thumbUrl` when a file or preview exists |
| GET | `/api/media/:scrapeId/:itemId/file` | none | The downloaded file, content-typed from its extension | | GET | `/api/media/:scrapeId/:itemId/file` | none | The downloaded file, content-typed from its extension |
| GET | `/api/media/:scrapeId/:itemId/thumb` | none | The cached WebP preview | | GET | `/api/media/:scrapeId/:itemId/thumb` | none | The cached WebP preview |
| GET | `/api/media/scrapes` | none | Every scrape with state and counters | | GET | `/api/media/scrapes` | none | Every scrape with state and counters |
+17 -7
View File
@@ -320,17 +320,27 @@ export class ScrapeStore {
return row ?? null; return row ?? null;
} }
queryMedia(options: { kind?: MediaKind; limit?: number; offset?: number } = {}): MediaRow[] { queryMedia(
options: { kind?: MediaKind; source?: string; limit?: number; offset?: number } = {}
): MediaRow[] {
const limit = Math.max(1, Math.min(options.limit ?? 50, 500)); const limit = Math.max(1, Math.min(options.limit ?? 50, 500));
const offset = Math.max(0, options.offset ?? 0); const offset = Math.max(0, options.offset ?? 0);
const statement = options.kind const conditions: string[] = [];
? this.db.prepare("SELECT * FROM media WHERE kind = ? ORDER BY id DESC LIMIT ? OFFSET ?") const values: Array<string | number> = [];
: this.db.prepare("SELECT * FROM media ORDER BY id DESC LIMIT ? OFFSET ?"); if (options.kind) {
conditions.push("kind = ?");
values.push(options.kind);
}
if (options.source) {
conditions.push("source = ?");
values.push(options.source);
}
const rows = options.kind const where = conditions.length ? `WHERE ${conditions.join(" AND ")}` : "";
? statement.all(options.kind, limit, offset) const rows = this.db
: statement.all(limit, offset); .prepare(`SELECT * FROM media ${where} ORDER BY id DESC LIMIT ? OFFSET ?`)
.all(...values, limit, offset);
return rows as unknown as MediaRow[]; return rows as unknown as MediaRow[];
} }
+30
View File
@@ -396,6 +396,36 @@ void (async () => {
assert.equal(existsSync(join(root, started.id, row.file_path)), true); assert.equal(existsSync(join(root, started.id, row.file_path)), true);
}); });
await test("store filters by source and kind", () => {
const dir = join(root, "store-source");
mkdirSync(dir, { recursive: true });
const store = new ScrapeStore(dir);
store.insertPage([
message("700", {
attachments: [attachment({ filename: "pic.png", contentType: "image/png" })],
embeds: [
{
type: "video",
url: "https://www.youtube.com/watch?v=x",
title: null,
provider: "YouTube",
videoUrl: "https://www.youtube.com/embed/x",
imageUrl: null,
thumbnailUrl: null
}
]
})
]);
assert.equal(store.queryMedia({ source: "attachment" }).length, 1);
assert.equal(store.queryMedia({ source: "embed" }).length, 1);
assert.equal(store.queryMedia({ source: "sticker" }).length, 0);
assert.equal(store.queryMedia({ kind: "image", source: "attachment" }).length, 1);
assert.equal(store.queryMedia({ kind: "video", source: "attachment" }).length, 0);
store.close();
});
await test("preview urls are Discord-only resizes", () => { await test("preview urls are Discord-only resizes", () => {
assert.equal( assert.equal(
previewUrlFor({ url: "https://cdn.discordapp.com/attachments/1/2/pic.png", kind: "image" }), previewUrlFor({ url: "https://cdn.discordapp.com/attachments/1/2/pic.png", kind: "image" }),
+11 -7
View File
@@ -2,10 +2,13 @@
.vk-media-overlay { .vk-media-overlay {
position: fixed; position: fixed;
inset: 0; left: 0;
right: 0;
top: var(--media-top, 50px);
bottom: 0;
z-index: 90; z-index: 90;
display: flex; display: flex;
background: color-mix(in srgb, var(--color-neutral-900) 70%, transparent); background: var(--color-bg);
} }
.vk-media-overlay[hidden] { .vk-media-overlay[hidden] {
@@ -16,13 +19,14 @@
display: flex; display: flex;
flex-direction: column; flex-direction: column;
gap: 10px; gap: 10px;
width: min(1100px, 100%); width: 100%;
margin: 24px auto; height: 100%;
padding: 14px; margin: 0;
padding: 14px 18px;
overflow: hidden; overflow: hidden;
border-radius: var(--radius-lg, 12px); border-radius: 0;
background: var(--color-bg); background: var(--color-bg);
box-shadow: inset 0 0 0 1px var(--color-neutral-800); box-shadow: none;
} }
.vk-media-head { .vk-media-head {
+73 -2
View File
@@ -18,7 +18,8 @@
items: [], items: [],
scrapes: [], scrapes: [],
channels: [], channels: [],
loadedChannels: false loadedChannels: false,
filters: { source: "", channelId: "" }
}; };
const overlay = document.createElement("div"); const overlay = document.createElement("div");
@@ -63,8 +64,18 @@
return node; return node;
} }
/** Start the view below the header and the call strip, filling the rest. */
function layoutTop() {
const header = document.querySelector(".vk-header");
const callStrip = document.querySelector("#call-strip");
let top = header ? header.getBoundingClientRect().bottom : 0;
if (callStrip && !callStrip.hidden) top = Math.max(top, callStrip.getBoundingClientRect().bottom);
overlay.style.setProperty("--media-top", `${Math.round(top)}px`);
}
function show(view) { function show(view) {
state.view = view; state.view = view;
layoutTop();
overlay.hidden = false; overlay.hidden = false;
document.querySelector("#scope-media")?.classList.toggle("active", view === "media"); document.querySelector("#scope-media")?.classList.toggle("active", view === "media");
document.querySelector("#scope-bot")?.classList.toggle("active", view === "bot"); document.querySelector("#scope-bot")?.classList.toggle("active", view === "bot");
@@ -128,6 +139,7 @@
setStatus(`media status failed: ${error.message}`); setStatus(`media status failed: ${error.message}`);
} }
renderScrapes(); renderScrapes();
if (state.view === "media") renderToolbar();
} }
async function loadMedia(reset) { async function loadMedia(reset) {
@@ -137,7 +149,11 @@
} }
try { try {
const kind = state.kind === "all" ? "" : `&kind=${state.kind}`; const kind = state.kind === "all" ? "" : `&kind=${state.kind}`;
const items = await window.VK.api.get(`/api/media?limit=60&offset=${state.offset}${kind}`); const source = state.filters.source ? `&source=${state.filters.source}` : "";
const channel = state.filters.channelId ? `&channelId=${state.filters.channelId}` : "";
const items = await window.VK.api.get(
`/api/media?limit=60&offset=${state.offset}${kind}${source}${channel}`
);
state.items = state.items.concat(items); state.items = state.items.concat(items);
state.offset += items.length; state.offset += items.length;
setStatus(`${state.items.length} items loaded`); setStatus(`${state.items.length} items loaded`);
@@ -236,6 +252,50 @@
toolbar.append(button("Start links", () => void startScrape("links"))); toolbar.append(button("Start links", () => void startScrape("links")));
toolbar.append(button("Start download", () => void startScrape("download"))); toolbar.append(button("Start download", () => void startScrape("download")));
const sourceSelect = document.createElement("select");
sourceSelect.className = "vk-chip vk-chip-off";
for (const [value, label] of [
["", "Any source"],
["attachment", "Attachments"],
["sticker", "Stickers"],
["embed", "Embeds"]
]) {
const option = document.createElement("option");
option.value = value;
option.textContent = label;
sourceSelect.append(option);
}
sourceSelect.value = state.filters.source;
sourceSelect.addEventListener("change", () => {
state.filters.source = sourceSelect.value;
void loadMedia(true);
});
toolbar.append(sourceSelect);
const channelSelect = document.createElement("select");
channelSelect.className = "vk-chip vk-chip-off";
const anyChannel = document.createElement("option");
anyChannel.value = "";
anyChannel.textContent = "Any channel";
channelSelect.append(anyChannel);
const seen = new Set();
for (const scrape of state.scrapes) {
if (seen.has(scrape.channelId)) continue;
seen.add(scrape.channelId);
const option = document.createElement("option");
option.value = scrape.channelId;
option.textContent =
state.channels.find(channel => channel.id === scrape.channelId)?.label ?? scrape.channelId;
channelSelect.append(option);
}
channelSelect.value = state.filters.channelId;
channelSelect.addEventListener("change", () => {
state.filters.channelId = channelSelect.value;
void loadMedia(true);
});
toolbar.append(channelSelect);
} else { } else {
toolbar.append(el("span", null, "Slopbot is the dedicated bot core. Setup: docs/slopbot.md.")); toolbar.append(el("span", null, "Slopbot is the dedicated bot core. Setup: docs/slopbot.md."));
} }
@@ -436,6 +496,17 @@
document.querySelector("#media-clean")?.addEventListener("click", openCleanModal); document.querySelector("#media-clean")?.addEventListener("click", openCleanModal);
window.addEventListener("resize", () => {
if (state.view) layoutTop();
});
const callStrip = document.querySelector("#call-strip");
if (callStrip) {
new MutationObserver(() => {
if (state.view) layoutTop();
}).observe(callStrip, { attributes: true, attributeFilter: ["hidden"] });
}
const settingsOverlay = document.querySelector("#settings-overlay"); const settingsOverlay = document.querySelector("#settings-overlay");
if (settingsOverlay) { if (settingsOverlay) {
new MutationObserver(() => { new MutationObserver(() => {
+7 -2
View File
@@ -823,13 +823,18 @@ function buildServer(): McpServer {
description: "Query media items across scrapes, newest scrape first.", description: "Query media items across scrapes, newest scrape first.",
inputSchema: { inputSchema: {
kind: z.enum(["image", "gif", "video", "audio"]).optional(), kind: z.enum(["image", "gif", "video", "audio"]).optional(),
source: z
.enum(["attachment", "sticker", "embed"])
.optional()
.describe("Where the item came from"),
scrapeId: z.string().optional().describe("Restrict to one scrape"), scrapeId: z.string().optional().describe("Restrict to one scrape"),
channelId: z.string().optional().describe("Restrict to one source channel"),
limit: z.number().int().min(1).max(500).optional(), limit: z.number().int().min(1).max(500).optional(),
offset: z.number().int().min(0).optional() offset: z.number().int().min(0).optional()
} }
}, },
async ({ kind, scrapeId, limit, offset }) => async ({ kind, source, scrapeId, channelId, limit, offset }) =>
jsonText(media.queryMedia({ kind, scrapeId, limit, offset })) jsonText(media.queryMedia({ kind, source, scrapeId, channelId, limit, offset }))
); );
server.registerTool( server.registerTool(
+16 -2
View File
@@ -92,6 +92,7 @@ export interface MediaItemResult {
itemId: number; itemId: number;
channelId: string; channelId: string;
kind: MediaKind; kind: MediaKind;
source: string | null;
url: string; url: string;
localUrl: string | null; localUrl: string | null;
thumbUrl: string | null; thumbUrl: string | null;
@@ -352,7 +353,14 @@ export class MediaService {
/** Read media rows across every scrape, newest scrape first. */ /** Read media rows across every scrape, newest scrape first. */
queryMedia( queryMedia(
options: { kind?: MediaKind; scrapeId?: string; limit?: number; offset?: number } = {} options: {
kind?: MediaKind;
source?: string;
scrapeId?: string;
channelId?: string;
limit?: number;
offset?: number;
} = {}
): MediaItemResult[] { ): MediaItemResult[] {
const limit = Math.max(1, Math.min(options.limit ?? 60, 500)); const limit = Math.max(1, Math.min(options.limit ?? 60, 500));
const offset = Math.max(0, options.offset ?? 0); const offset = Math.max(0, options.offset ?? 0);
@@ -362,10 +370,15 @@ export class MediaService {
for (const meta of this.engine.list()) { for (const meta of this.engine.list()) {
if (meta.state === "deleted") continue; if (meta.state === "deleted") continue;
if (options.scrapeId && meta.id !== options.scrapeId) continue; if (options.scrapeId && meta.id !== options.scrapeId) continue;
if (options.channelId && meta.channelId !== options.channelId) continue;
const store = new ScrapeStore(scrapeDir(this.rootDir, meta.id)); const store = new ScrapeStore(scrapeDir(this.rootDir, meta.id));
try { try {
const rows = store.queryMedia({ kind: options.kind, limit: limit + offset }); const rows = store.queryMedia({
kind: options.kind,
source: options.source,
limit: limit + offset
});
for (const row of rows) { for (const row of rows) {
if (skipped++ < offset) continue; if (skipped++ < offset) continue;
if (results.length >= limit) break; if (results.length >= limit) break;
@@ -374,6 +387,7 @@ export class MediaService {
itemId: row.id, itemId: row.id,
channelId: meta.channelId, channelId: meta.channelId,
kind: row.kind, kind: row.kind,
source: row.source,
url: row.url, url: row.url,
localUrl: row.file_path ? `/api/media/${meta.id}/${row.id}/file` : null, localUrl: row.file_path ? `/api/media/${meta.id}/${row.id}/file` : null,
thumbUrl: row.preview_hash ? `/api/media/${meta.id}/${row.id}/thumb` : null, thumbUrl: row.preview_hash ? `/api/media/${meta.id}/${row.id}/thumb` : null,
+5 -1
View File
@@ -795,10 +795,14 @@ export function startWebUi(deps: WebUiDeps): WebUiHandle {
const kind = ["image", "gif", "video", "audio"].includes(String(req.query.kind ?? "")) const kind = ["image", "gif", "video", "audio"].includes(String(req.query.kind ?? ""))
? (String(req.query.kind) as "image" | "gif" | "video" | "audio") ? (String(req.query.kind) as "image" | "gif" | "video" | "audio")
: undefined; : undefined;
const source = ["attachment", "sticker", "embed"].includes(String(req.query.source ?? ""))
? String(req.query.source)
: undefined;
const channelId = req.query.channelId ? String(req.query.channelId) : undefined;
const limit = clampLimit(req.query.limit, 60, 500); const limit = clampLimit(req.query.limit, 60, 500);
const offset = Math.max(0, Number(req.query.offset ?? 0) || 0); const offset = Math.max(0, Number(req.query.offset ?? 0) || 0);
res.json({ ok: true, result: deps.media.queryMedia({ kind, limit, offset }) }); res.json({ ok: true, result: deps.media.queryMedia({ kind, source, channelId, limit, offset }) });
}) })
); );