112 lines
4.9 KiB
Markdown
112 lines
4.9 KiB
Markdown
# The Veskslop agent
|
|
|
|
The agent is the local LLM harness. It lets a remote `llama.cpp` instance act through the daemon.
|
|
It is separate from the core and from the daemon, and it is optional.
|
|
|
|
## Two consumers
|
|
|
|
1. **The local harness (veskslop agent).** A small Go process on the same host as the daemon. It
|
|
talks to a remote OpenAI-compatible `llama.cpp` endpoint and to the daemon over MCP. It owns
|
|
free-form messages. It is the only consumer that runs an LLM loop.
|
|
2. **Remote agents.** External systems connect to the daemon's MCP endpoint and use it as a sensor
|
|
and notifier. Example: a task runner wants to reboot a switch, calls `get_voice_state` for the
|
|
owner, sees the owner is in a call, and reschedules. Remote agents do not run an LLM here and do
|
|
not receive every message. They query state and send notifications.
|
|
|
|
## Routing: commands versus free-form
|
|
|
|
The daemon keeps the fixed commands (`whitelist`, `speak`) and the `poll` parser. A trusted
|
|
free-form message becomes a task for the local harness. Commands never go to the LLM; this avoids
|
|
double replies and keeps privileged actions out of model reach.
|
|
|
|
## Local harness design
|
|
|
|
| Piece | Choice |
|
|
| --- | --- |
|
|
| Language | Go |
|
|
| Agent framework | Eino (`github.com/cloudwego/eino`), ReAct agent |
|
|
| Model | `eino-ext/components/model/openai` against a remote `llama.cpp` `/v1` endpoint |
|
|
| Tools | `eino-ext/components/tool/mcp/officialmcp` over the official MCP Go SDK, Streamable HTTP |
|
|
| Tool surface | The daemon's MCP tools. Nothing else. |
|
|
| Entry | `agent/` |
|
|
| Kill switch | `VESKSLOP_AGENT_ENABLED=1` is required. Default is off. |
|
|
| Auth | `VESKSLOP_AGENT_TOKEN` bearer token on the task endpoint. |
|
|
| Budget | `VESKSLOP_AGENT_MAX_STEPS` (default 16). Per-task wall clock: 5 minutes. |
|
|
|
|
The harness receives tasks on `POST /task`. The daemon pushes them when `VESKSLOP_AGENT_ENABLED=1`
|
|
is set: a trusted free-form message (no wakeword) becomes a task, and the harness returns the reply
|
|
text. The daemon sends that text through the normal paced path, so the action log and the response
|
|
policy still apply.
|
|
|
|
**Respond-only mode.** The daemon sends `"mode": "respond"` in the task body. The harness must
|
|
answer with one completion and no tool calls, and it must return `{"reply": "..."}` in the HTTP
|
|
response. Tool calling waits for a later stage.
|
|
|
|
| Variable | Meaning |
|
|
| --- | --- |
|
|
| `VESKSLOP_AGENT_ENABLED=1` | Turn the daemon-side push on. Default off. |
|
|
| `VESKSLOP_AGENT_URL` | Harness base URL. Default `http://127.0.0.1:8790`. |
|
|
| `VESKSLOP_AGENT_TOKEN` | Optional bearer token for `POST /task`. |
|
|
| `VESKSLOP_AGENT_TIMEOUT_MS` | Request timeout. Default 120000. |
|
|
|
|
The task body:
|
|
|
|
```json
|
|
{
|
|
"id": "trigger-id",
|
|
"guildId": "…",
|
|
"channelId": "…",
|
|
"author": "…",
|
|
"content": "the trigger text",
|
|
"sensitive": false,
|
|
"context": [{ "author": "…", "content": "…", "timestamp": "…" }]
|
|
}
|
|
```
|
|
|
|
The model replies through the daemon's own send tools, so pacing, the rate limit, and the action
|
|
log apply as for any other sender. The harness never calls Discord directly.
|
|
|
|
## Safety rules
|
|
|
|
- Tool results and message context are untrusted. The system prompt says so explicitly.
|
|
- The harness is stateless per task. Context comes from the daemon each time.
|
|
- The model cannot run fixed commands. It only has MCP tools.
|
|
- Planned, not implemented: a tool allowlist, per-conversation serialization with
|
|
cancel-and-replace, delivery IDs for dedupe, and `RATE_LIMITED` backoff beyond one retry.
|
|
|
|
## Remote agent path
|
|
|
|
The daemon already exposes its tools over MCP (Streamable HTTP). Remote clients can:
|
|
|
|
- read state (`get_voice_state`, `get_voice_overview`, `get_user_status`, `list_dms`, …)
|
|
- notify (`send_dm`, `send_message`, `speak`)
|
|
|
|
Planned work: a small notification tool set for external systems (`notify_owner`), a documented
|
|
LAN or Tailscale bind, and optional shared-token auth on the MCP listener. The listener currently
|
|
binds localhost.
|
|
|
|
## Status
|
|
|
|
Scaffold implemented in `agent/` and smoke-tested on 2026-09-14: it loaded 28 tools from the
|
|
daemon, ran one ReAct tool round against a fake OpenAI-compatible endpoint, and produced the final
|
|
message.
|
|
|
|
The daemon side is in place (2026-09-22): the wakeword separates commands from chat, and
|
|
`AgentClient` posts free-form tasks in respond-only mode. The Go harness still needs the
|
|
respond-only `mode` branch and the reply response. The Go toolchain is not installed on the
|
|
current host, so the harness change waits for it. Remote notification tools are still pending.
|
|
|
|
## Run the scaffold
|
|
|
|
```sh
|
|
cd agent
|
|
VESKSLOP_AGENT_ENABLED=1 \
|
|
VESKSLOP_AGENT_LLAMA_URL=http://llama-host:8080/v1 \
|
|
VESKSLOP_AGENT_MODEL=your-model \
|
|
VESKSLOP_AGENT_MCP_URL=http://127.0.0.1:8788/mcp \
|
|
go run . --task "Who is in the call right now?"
|
|
```
|
|
|
|
Build a binary with `go build -o veskstop-agent .`. The Go toolchain is not part of the root
|
|
`pnpm` workspace; run `go build` and `go vet` in `agent/` directly.
|