zib/Veskslop-t
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Record the incremental scrape checkpoint

Browse Source
This commit is contained in:
Fisto
2026-09-21 10:46:35 +02:00
parent 816939288c
commit 41732bd96f
1 changed files with 8 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CURRENT.md
+8
View File
@@ -40,6 +40,14 @@ mobile draft at 390px.
- [x] C11 preview caching: Discord-hosted image/GIF previews (WebP 320px, hash-addressed, 5 MiB
cap, one attempt) run after every walk in both modes. Live run `...-20260916090722`:
`previewed: 2`; thumbs served over `/api/media/<scrape>/<item>/thumb` (200).
- [x] C60 Incremental re-scrapes and archive cleanup. A new scrape of a channel that already
has a completed walk now starts after that walk's newest message (the best completed scrape by
message count), so it only covers the newer gap instead of re-fetching history; the previous
newest is stored as `afterId` and shown as the from-date. Live-verified: a re-scrape of
`559354230444654605` started after `999789325183090769` from the 2026-09-17 scrape. Deleted the
four pre-pagination-fix test scrapes (superseded by full walks) and the truncated DM scrape
with no complete counterpart; `pnpm verify:scrapes` now reports 10 complete, 0 superseded,
0 incomplete.
- [x] C59 Scrape names and the completeness check. Scrape listings (History, Clean media,
Bot-tab job/lane cards) now lead with the first two words of the channel or group-chat name
plus the scrape id. New `poc/scripts/verify-scrapes.ts` (`pnpm verify:scrapes`) checks every