Skip to content

Session handover — 2026-07-20

The corpus is in. Run 3 scored 41/42 and calibration/feature-matrix.json now carries 38 sites — the board the calibration fit has been waiting on since Slice 6. Getting there took disproving the story this session opened with: run 2 did not die to the box, it died to a quick-crawl bug, and the box was healthy the whole time. Two fixes (db1bff8) took the same 42-site run from 2/42 to 41/42. The corpus's headline finding is that a single veto, chromeDisagree, gates half the portfolio — and it needs adjudicating before any renderer work.

Where to look

  • known-issues — the corpus block is rewritten: run-3 result, the chromeDisagree 33/38 measurement (sole veto on 16; only 2 sites ship clean today; clearing it → 18 of 38) with the instrument-artifact theory explicitly marked UNTESTED HYPOTHESIS plus its adjudication trigger, the null-axis composite bug's second instance (cassidymonumentals), the new transform-seed-images URL-mangling bug, and mesh prevalence recorded as still-unmeasured after a failed attempt. The 07-19 partial-scores block is marked superseded, with a note that the 12-site sample ranked the fix list backwards.
  • known-patterns — "A batch whose failure MODE migrates upstream…" carries a dated correction: the detection rule is not sufficient on its own, because run 2 produced the identical signature from code. The correction adds the one-grep discriminator — did any other network-dependent stage succeed in the same minutes? (images ✓ (28.9s) downloading from wixstatic, interleaved with the discover ✗ wall, proves egress was fine.) Run it before --list-boots.
  • scope-2026-07-18-cms-batch-runner — EC2 runbook gains the instance identity (the long-open chore): Lightsail Cathal-Dev3, large_3_0 8 GB, eu-west-1a, full ARN + account 371676573654, local creds Lightsail read-only. Plus the resize-drops-firewall-rules trap and its port-based diagnostic, the WordPress co-tenancy (cathaldev1/2/3.fcrweb.ie, cert expires 2026-09-01) and its contention surface, and the OOM finding.
  • Commits: db1bff8 (both fixes + runbook), plus this session-end. Run 3 log: ~/corpus-run-0720.log on the box.

What shipped

  • quick-crawl sitemap race fixed. The sitemap is now fetched with context.request.get() instead of page.goto(). Navigating left a failed navigation in flight on sites Wix redirects, which then collided with the nav-crawl fallback. Deterministic on sitemap presence — which is why it read as environmental. All 27 of run 2's discover failures now pass. It also finds sitemaps the old path missed: navigating to XML corrupted the parse, so an unknown number of "No sitemap, falling back to nav crawl" lines were false and those sites were nav-crawled while holding a usable sitemap.
  • cms-build timeout raised 15 → 30 min. All 12 of run 2's build failures were spawnSync npm ETIMEDOUT at ~931 s against that ceiling.
  • The 8 GB resize is the real fix for the build cluster, and it is much bigger than the timeout change: builds landed at 86–276 s (never touching even the old ceiling), crawl at 55–104 s against a recorded ~7.5 min, and capture-live at 10–26 s against ~5.4 min. The old 2 GB box was swap-thrashing every stage, so every duration in the pre-07-20 runbook was measured in a degraded state.
  • Corpus aggregated. batch-scorecard → 38 sites, 4 skipped, and the null-axis loud-skip verified working (bluestars excluded from the fit, not silently folded in) — the specific check 07-19 asked for.

The honest parts

  • chromeDisagree being an instrument artifact is a THEORY, not a finding. The 33/38 count is measured; the explanation is not. It is written up as an untested hypothesis with priors (87% uniform prevalence is an odd defect; the 07-19 session already called header-bg "a measurement, not a scoped fix"; the top-bar-missing precedent where the verifier's crop was the bug). Do not scope a renderer slice against it — measure the captures first.
  • Mesh prevalence is still unmeasured, and this session failed at it. Three section-count approaches were tried off the seed JSON and each failed its control (brianlarkinsolicitor, a healthy 87.9 SHIP, counted 0 under two of them). No number was produced and none was recorded rather than banking a figure derived from a metric that had just failed. It needs the pipeline's own definition of a section, not seed-schema guesswork.
  • The self-heal watchdog has still never fired. The cron survived the rebuild but /var/log/networkd-selfheal.log has never been created. Since no network failure recurred, that is consistent with "nothing to heal" — it is untested, not proven working. Separately its probe target (169.254.169.254, link-local metadata) is a weaker signal than real internet egress and may not fail when public routing does.
  • This session's ETAs were wrong repeatedly and in both directions — 12 h, then "about an hour", against an actual 5 h 46 m. Both errors came from extrapolating a whole-corpus rate from an unrepresentative sample (first the degraded box's timings, then the cached sites that skip the expensive stages). Per-site cost has real spread: one build was 276 s against a typical 86 s.
  • Box/git drift is open. The box is 5 commits behind and carries the two patched files as a surgical scp copy rather than a bundle sync — chosen deliberately to avoid a merge against its dirty tree of corpus artifacts mid-run, but it should be reconciled before the next slice.

Next

  1. Adjudicate chromeDisagree — the highest-leverage action in the lane. Pull live-vs-ours captures for qconsulting (96.7), johnrochford (95.9), charlietaxiservices (94.1) — highest scorers, sole veto — and decide artifact-vs-defect from the images before any code moves. Worth 16 ships.
  2. Cathal labels ship / needs-polish / hold per site. Worth holding until (1) lands: a verdict either way reclassifies 16 of the sites being labelled.
  3. fit-calibration → review → promote only if fitted:true + precision.
  4. Then the residual veto load, now correctly ranked by the full board: criticalSectionDropped (7), contentDestroyed (6), deadAssetRef (4).
  5. Smaller: the transform-seed-images URL-mangling bug (1 site — recover with --force images --site sreenanandcompany.ie), the null-axis composite derivation (should refuse to emit from a single surviving axis), and re-measuring mesh prevalence with the right metric.
  6. Reconcile the box's git drift; re-time a build to replace the degraded-era figures in the runbook.