Watchdog

Watchdog answers one question every day: is your app actually working right now? It fuses three signals — recent code changes, real-user errors, and your feature inventory — to decide what to probe, then drives the product in a real browser (Playwright/Chromium) to verify it, recording the whole check as one video. Every failure it files has been reproduced on screen. It's recommend-only: it never changes your product and never opens a PR.

Prerequisites

• Turn it on. Watchdog is off until you enable it — one toggle on the Watchdog tab. It then wakes daily on its own cadence; the Find now button runs it on demand.
• A browsable target. Watchdog needs a URL to point the browser at. If none is configured it skips with a one-line reason.
• Better with a connected GitHub repo and a synced feature inventory — that's how it risk-ranks your actual flows. Without them it still runs, probing the flows it can find on the site itself (sign-up, sign-in, the core action) and noting that connecting GitHub will sharpen future runs.

How it works

1. Orient + risk-rank. Watchdog shallow-clones your repo read-only, reads recent commits (roughly the last 36 hours, or the last 20) to see what changed, and pulls the full feature inventory plus the top open real-user errors.

2. Choose a bounded probe set — a handful of real user flows, not pages (about 8 at most), in priority order:

   • Starred stories you pinned — always included.
   • Onboarding / sign-up — the highest-value check, and it needs no login (it creates a fresh throwaway account).
   • Sign-in and the core action — the main thing users come to do, plus anything that saves user data.
   • Billing / checkout — the money flows.
   • At risk — stories whose source files changed recently, or that map to a live error.
   • Backfill — a couple of never-verified stories to drain the backlog over days.

   It deliberately skips static/informational pages — terms, privacy, changelog, marketing, blog. A 200 on /terms tells a founder nothing.

3. Write and run one spec. It writes a single Playwright test — one continuous video — signing in once up front when a flow needs an account. Each probe is wrapped so a mismatch is recorded as a finding, never a crash, so the video always finishes and every probe runs. It instruments console errors, page errors, dead clicks, and rough timing.

4. File findings. One result per probed item: pass (works as expected), fail (broken — doesn't complete, errors, or dead-ends), or friction (works but with notable measured friction). A pass clears any prior ticket for that thing; a reproduced fail or notable friction files a draft ticket — or updates the existing one, since dedup is automatic — with the recording attached.

What you get

• A draft ticket per reproduced problem on the Tasks board, each with the screen recording of the failure and an observed-vs-expected summary. Because failures dedup against open tickets, a recurring break updates its ticket instead of spawning duplicates.
• A run summary — e.g. "Probed 5 stories; sign-in broken, checkout OK" — and cleared tickets for anything that now passes.
• Nothing when all is well. No noise on a healthy day; the value is that the silence is trustworthy.

Notes & limits

• Recommend-only. Watchdog observes and reports. It never edits code, opens a PR, or changes your product — you decide what to do with each finding (comment on the ticket to hand it to a coding loop).
• Read-only on production. When the target is your live site it probes non-destructively only — no deletes, purchases, or sign-ups that email real users. Destructive checks (like a real sign-up) run only against a staging target. Point Watchdog at staging with a throwaway account to get the fullest coverage.
• A slice per run, not everything. It probes a handful of the highest-stakes flows each day rather than the whole inventory, rotating through never-verified stories over time.
• It reads the inventory to aim. The richer and more current your feature inventory, the better Watchdog targets what matters — keep GitHub connected and the map synced.