Add media archive and slopbot design docs

2026-09-16 07:47:28 +02:00
parent 677afc747f
commit b59b9db461
10 changed files with 453 additions and 197 deletions
@@ -0,0 +1,71 @@
+# Slopbot
+
+Slopbot is the dedicated bot account for media scraping. It is separate from the puppet and from
+the Vesktop core. It uses the official Discord bot API through
+[discord.js](https://github.com/discordjs/discord.js) (Apache-2.0), the external core the feature
+is built on.
+
+## Why a bot
+
+For guild channels, a bot scrapes better than a user account:
+
+- The official bot REST API has documented, per-route rate limits. discord.js queues by bucket and
+  honors `429` automatically.
+- No Electron client, no Xvfb, no renderer round-trip. The walk continues while the puppet is
+  offline.
+- The bot token is a separate credential with its own lifecycle.
+
+Slopbot cannot scrape user DMs or group DMs. Discord does not expose them to bots. The puppet
+backend covers those, and covers guild channels where slopbot is not present. This split is a
+Discord fact, not a design choice.
+
+Slopbot is a real bot account. It is not a selfbot: the locked decision against selfbot protocol
+libraries stands.
+
+## Setup
+
+1. Create an application in the Discord Developer Portal and add a bot.
+2. Enable the **Message Content** privileged intent. Without it Discord blanks `content`,
+   `attachments`, and `embeds` in both gateway events and REST history responses. Unverified apps
+   in under 100 servers can enable it in the portal.
+3. Invite the bot with the `bot` and `applications.commands` scopes and the minimum permissions:
+   View Channels, Read Message History, Send Messages (for status replies), Embed Links.
+4. Store the token in the daemon state file `slopbot.json` (mode 600) or `VESKSLOP_SLOPBOT_TOKEN`.
+   Never commit it. The action log redacts token-shaped strings as before.
+
+Slopbot needs no slash commands: it is a scraping sensor, not a chat citizen. It never replies in
+channels.
+
+## Configuration
+
+| Setting | Default | Purpose |
+| --- | --- | --- |
+| `VESKSLOP_SLOPBOT_ENABLED` | `0` | Kill switch, off by default |
+| `VESKSLOP_SLOPBOT_TOKEN` / `slopbot.json` | unset | Bot token |
+| `VESKSLOP_SLOPBOT_CLIENT_ID` | unset | For the invite URL helper |
+
+## Code
+
+- `media-scrape/slopbot/` holds the bot core: client lifecycle, the `ScrapeBackend` implementation,
+  and the invite helper.
+- discord.js is a dependency of the daemon package (`poc/package.json`), the same way the MCP SDK
+  is. The code stays in `media-scrape/`; the daemon only mounts it.
+- The Bot tab shows connection state, the guilds slopbot is in, the active scrape, and the queue.
+  It also renders the invite URL from the client ID.
+
+## Rate limits
+
+discord.js handles per-route buckets and retries `429` and `5xx` internally. The scrape engine adds
+its own budgets on top (see [media-scraper.md](media-scraper.md)): page gap, hourly and daily caps.
+Slopbot runs with the faster bot budget; the puppet backend uses the slower user budget.
+
+## Security
+
+- The bot token is a credential. It is redacted from logs, never committed, and loaded from the
+  state dir or the environment.
+- The bot gets minimum permissions and only the guilds the owner invites it to.
+- A guild channel the bot cannot see is a normal fallback to the puppet, not an error.
+
+## Status
+
+Design approved 2026-09-16. Implementation at checkpoint C4 in [CURRENT.md](../CURRENT.md).