93 lines
4.1 KiB
Markdown
93 lines
4.1 KiB
Markdown
# Slopbot
|
|
|
|
Slopbot is the dedicated bot account for media scraping. It is separate from the puppet and from
|
|
the Vesktop core. It uses the official Discord bot API through
|
|
[discord.js](https://github.com/discordjs/discord.js) (Apache-2.0), the external core the feature
|
|
is built on.
|
|
|
|
## Why a bot
|
|
|
|
For guild channels, a bot scrapes better than a user account:
|
|
|
|
- The official bot REST API has documented, per-route rate limits. discord.js queues by bucket and
|
|
honors `429` automatically.
|
|
- No Electron client, no Xvfb, no renderer round-trip. The walk continues while the puppet is
|
|
offline.
|
|
- The bot token is a separate credential with its own lifecycle.
|
|
|
|
Slopbot cannot scrape user DMs or group DMs. Discord does not expose them to bots. The puppet
|
|
backend covers those, and covers guild channels where slopbot is not present. This split is a
|
|
Discord fact, not a design choice.
|
|
|
|
Slopbot is a real bot account. It is not a selfbot: the locked decision against selfbot protocol
|
|
libraries stands.
|
|
|
|
## Setup
|
|
|
|
1. Create an application in the Discord Developer Portal and add a bot.
|
|
2. Enable the **Message Content** privileged intent. Without it Discord blanks `content`,
|
|
`attachments`, and `embeds` in both gateway events and REST history responses. Unverified apps
|
|
in under 100 servers can enable it in the portal.
|
|
3. Invite the bot with the `bot` and `applications.commands` scopes and the minimum permissions:
|
|
View Channels and Read Message History. The bot never sends, so it needs no more.
|
|
4. Store the token in the daemon state file `slopbot.json` (mode 600) or `VESKSLOP_SLOPBOT_TOKEN`.
|
|
Never commit it. The action log redacts token-shaped strings as before.
|
|
|
|
Slopbot needs no slash commands: it is a scraping sensor, not a chat citizen. It never replies in
|
|
channels.
|
|
|
|
## Configuration
|
|
|
|
| Setting | Default | Purpose |
|
|
| --- | --- | --- |
|
|
| `VESKSLOP_SLOPBOT_ENABLED` | `0` | Kill switch, off by default |
|
|
| `VESKSLOP_SLOPBOT_TOKEN` | unset | Bot token (preferred) |
|
|
| `VESKSLOP_SLOPBOT_STATE` | unset | JSON file with `token` and `clientId` |
|
|
| `VESKSLOP_SLOPBOT_CLIENT_ID` | unset | For the invite URL helper |
|
|
|
|
## Code
|
|
|
|
| File | Role |
|
|
| --- | --- |
|
|
| `media-scrape/slopbot/config.ts` | Environment and state-file settings |
|
|
| `media-scrape/slopbot/client.ts` | Login, status, and the live fetcher |
|
|
| `media-scrape/slopbot/backend.ts` | discord.js messages to the core message shape |
|
|
| `media-scrape/slopbot/index.ts` | Module surface |
|
|
|
|
- discord.js lives in `media-scrape/package.json`; the folder is its own pnpm workspace root.
|
|
- The mapping is duck-typed, so the tests run without a token. The live Discord shapes were checked
|
|
against the discord.js 14 typings, but a real login is still owed at C8.
|
|
- The Bot tab shows connection state, the guilds slopbot is in, the active scrape, and the queue.
|
|
It also renders the invite URL from the client ID.
|
|
|
|
## Live check
|
|
|
|
```sh
|
|
cd poc
|
|
VESKSLOP_SLOPBOT_ENABLED=1 VESKSLOP_SLOPBOT_STATE=/path/to/slopbot.json \
|
|
pnpm test:slopbot:live <channelId> [guildId]
|
|
```
|
|
|
|
The check logs in, prints the bot user and guild count, then fetches one history page and maps it.
|
|
The token is read from the state file or the environment and is never printed. The bot must be
|
|
invited to the guild first (View Channels + Read Message History).
|
|
|
|
## Rate limits
|
|
|
|
discord.js handles per-route buckets and retries `429` and `5xx` internally. The scrape engine adds
|
|
its own budgets on top (see [media-scraper.md](media-scraper.md)): page gap, hourly and daily caps.
|
|
Slopbot runs with the faster bot budget; the puppet backend uses the slower user budget.
|
|
|
|
## Security
|
|
|
|
- The bot token is a credential. It is redacted from logs, never committed, and loaded from the
|
|
state dir or the environment.
|
|
- The bot gets minimum permissions and only the guilds the owner invites it to.
|
|
- A guild channel the bot cannot see is a normal fallback to the puppet, not an error.
|
|
|
|
## Status
|
|
|
|
Backend implemented 2026-09-16 (C4), wired into the daemon at C5: `MediaService` logs the client in
|
|
on the first scrape and picks it for guild channels it is in. A live login with a real token is owed
|
|
at C9. Checkpoints: [CURRENT.md](../CURRENT.md).
|