Near-match scorecard contract (ADR-0009 §6, Slices 4 & 6)¶
The metric that operationalises ADR-0009's promise — a replatform "reads as the same site" — into a reproducible, unsupervised ship / hold / manual-polish verdict. It is the ship/hold gate §6 calls for and the precondition for Slice 4 (GenericSection + structural metrics as the gate) and Slice 6 (calibrate the pass rate on a 30–50-site batch).
Designed 2026-07-07 from two independent, blind parallel studies (deep-reasoner + Codex, briefed identically) that converged on ~90% of the design; this contract is the synthesis. Step 0 (the S axis) is shipped + validated on the two deployed sites — see Status.
Sibling contracts: the site-level aesthetic token contract and the per-section section-style contract define what we emit; this one defines how we measure whether it worked.
The crux call (read this first)¶
Three judgment calls settle the design; each was independently reached by both studies (except where noted):
-
A measured-feature scorecard, NOT a literal screenshot diff. ADR-0009 §6 says "screenshot-diff scorecard"; taken literally that is the one thing that cannot work. A perfect replatform still diffs non-trivially against live because fonts are legally substituted (Wix private webfonts can't ship) and decorative parastorage CSS is stripped (Wix IP) — so raw pixel distance has a large, per-site-variable irreducible floor that fails good builds and passes bad ones, and it can't localise a fix to an owner. Pixels survive only as a masked, structure-first guardrail (
G_det), never the primary score. -
Deterministic-first; the LLM never gates. A ship/hold gate must be reproducible.
verify-design.js-style Gemini scoring drifts run-to-run, so it is confined to triage (ranking the manual queue, sampling deterministic/LLM disagreements, assisting batch labelling) and is excluded from the composite and the gate. -
Fidelity requires an INDEPENDENT re-measurement of OUR output — the load-bearing insight (deep-reasoner). The per-section records in
section-backgrounds.jsonare the input to our render. Comparing them to what we stamped is an identity (imageSide→imagePositionby construction) and measures nothing. So we run the same capture probe against our deployed page to get a symmetricsection-backgrounds.ours.jsonand diff live-vs-ours. A tone/imageSide/columns agreement then proves the prop reached the rendered DOM and the CSS actually painted — not merely that the seed carried the value. (Codex's design compared live-capture to stored seed/assembly metadata, which cannot see render-path failures; this contract adopts the re-capture.) Step 0 immediately proved the value: it caughttone:muted→defaulton the garvanbay CMS home — bands that were stamped but do not paint — which a compare-to-stamp check would have scored as perfect.
1. The metric¶
A 0–100 Near-Match Score (100 = indistinguishable within the bounded, legally/editably allowed skeleton — not pixel-identical). Composed of three deterministic axes plus one advisory:
| Axis | Weight (seed) | What it measures | Determinism |
|---|---|---|---|
| S structural | 0.50 | section presence, order, band/tone sequence, image side, columns, alignment, density, GenericSection/matcher-gap rate | deterministic |
| T theme | 0.25 | h1/h2/h3 size + weight + rule-applied; primary/accent/foreground/background placement; brand source/confidence; fallback flags | deterministic |
| G_det masked perceptual | 0.25 | font/decoration-robust structural image diff (block-occupancy grid + low-freq hue map), per section crop | deterministic (no model) |
| G_llm gestalt | excluded from gate | one rubric-locked structured call per critical page: {yes\|needs-polish\|no} + reason |
advisory (triage only) |
Weights are calibration parameters (seed values above), re-fit on the batch (§4). The composite orders the queue; the gate does not trust it alone — see the hurdle+veto model in §4.
Section-weighting (§6)¶
Every section carries a role weight so a broken hero can't hide behind a dozen
fine body sections. Roles are cheap: on our side from the canonical block
type; on live from position + heading + the chrome header/footer bands.
| Role | Weight (seed) |
|---|---|
| hero (first content band) | 1.6 |
| header / primary CTA | 1.3–1.4 |
| footer | 1.2 |
| content / service-grid / gallery | 1.0 |
| FAQ | 0.75 |
| blog tail / ribbon strip | 0.5 |
Unit & aggregation¶
section → page(viewport) → site. Desktop (1440×900) and mobile (390×844) are scored independently; the gate takes the worse viewport (mobile collapse is a real failure, not something to average away). Page = a weighted blend of its sections + page/site-global T. Site = worst-of on critical pages (home + one exemplar per distinct template signature) plus fraction-of-pages-passing, with homepage weighted 2×.
2. Per-component computation (evolve what exists, don't greenfield)¶
The symmetric-capture spine — S (built; cheapest; highest signal)¶
Run the existing lib/capture-section-backgrounds.js against both URLs:
- LIVE Wix →
section-backgrounds.json(today's artifact). - OURS deployed →
section-backgrounds.ours.json, via the new--base <url> --out <path>flags (additive; default behaviour unchanged).
lib/structural-diff.js (pure) aligns the two and emits S. No LLM,
reproducible, runs today.
Deterministic axes — build on what exists¶
| Sub-score | Source | Status |
|---|---|---|
| S (structural sequence + per-dim agreement) | new lib/structural-diff.js over the symmetric capture |
shipped (Step 0) |
| T typography (clamp-max px vs live, weight, rule-applied) | lib/fidelity-harness.js#buildTypographyScorecard → lib/theme-fidelity.js |
shipped (Step 1) |
T palette (primary/accent/fg/bg placement, brand source/confidence, fallbackNavy, #000000 remnant) |
lib/fidelity-harness.js#buildPaletteScorecard → lib/theme-fidelity.js |
shipped (Step 1) |
| G_det masked perceptual | upgrade lib/fidelity-harness.js#buildVisualDelta |
scaffolded (raw greyscale MAD) → replace |
G_det upgrade: the scaffolded delta (resize 320px, greyscale, mean-abs-diff) is too font/decoration-sensitive to threshold. Replace with a structure-first representation per section crop: (a) a coarse block-occupancy grid (luminance-variance cells → where is there stuff) and (b) a low-frequency hue/lightness map (palette-on-which-element). Diff = occupancy distance (catches layout collapse / dropped-moved blocks) + hue-placement (catches palette-on-wrong-element). Glyph shapes and decorative texture wash out at this resolution — exactly the legitimately-divergent surface we must not penalise.
Bounded LLM — G_llm (buy, capped, advisory)¶
One structured Gemini call per critical page (not per region — that is
verify-design.js's cost model and its drift surface). Forced output
{verdict, confidence, primaryReason, note}; the rubric hard-codes the
legitimate-divergence carve-outs ("ignore font substitution, minor spacing,
Wix-only decoration; judge whether it reads as the same business's site").
Used only for queue ranking, disagreement sampling, and labelling assistance —
never the gate. Cost-bounded to the critical page set and preferentially the
composite "gray band" near τ (≈ one call per site).
3. Structural sub-metrics + alignment¶
Sub-metrics (per matched section unless noted)¶
sectionCount · sectionOrder · bandTone (5-enum sequence) · imageSide ·
columns · align · density (when the classifier ships) · header/topbar
(site-level, from chrome) · genericFallbackRate (fraction rendered by
the measured GenericSection; a bounded escape hatch but every use is matcher
debt — shipped Slice 4b-ii: computed Σgeneric/Σblocks from a build-time
{section→matcher} sidecar (section-matchers.json) because the capture can't
see block identity; null when no sidecar — unknown, never a bogus 0.
GenericSection on a role-weight ≥ 1.3 region also trips the
genericFallbackCritical gate veto) ·
matcherGap (unmatched-live drops, unmatched-ours inserts, props withheld
on identity-unknown matches — shipped Slice 4a: a reported projection of
debt already in S, not a second penalty. It role-weights every drop/insert and
records each ours-missing dim, so debt is explicit and portfolio-rollup-able —
the §5 "ceiling events are a deliverable" model extended to matcher debt).
Per-dimension agreement is omit-aware: a dim the live classifier omitted (out-of-gamut/ambiguous) is excluded from the denominator and logged as a ceiling event (§5), never a penalty.
Alignment algorithm¶
The live section list is noisy; ours is clean. Three stages:
- Junk filter (both sides). The live capture's
buildSectionsenumeration (built for tone-seeding, where over-capture is harmless) pulls in Wix widget/chrome pseudo-sections a replatform correctly omits — a<style>block leaking CSS intotextHead, social-share bars, video-player strips. These are not content the build "dropped"; counting them as drops tanks S. Filtered from both lists (conservative: social/video noise only when the section has no real heading). Step 0 filtered 6/31/9 such sections on WCP's three pages. - Canonicalise (both sides). Wix emits each section as a near-duplicate
pair (responsive/layer doubling): identical normalised heading + tone +
style tuple, adjacent, ~50px height delta. Our clean output has no doubling,
so without collapse ours looks like it dropped half its sections (garvanbay
home: 6 dup pairs). Collapse adjacent same-tone-same-style pairs, including
the empty-heading form (an image-based hero h1 has no text; its heading
appears in the sibling welcome's
textHead) — this is why the hero/welcome pair must merge rather than orphan the hero. Ribbon/spacer strips (short, heading-less) are tagged low-weight, not dropped. - Heading-anchored Needleman–Wunsch global alignment. Content survives
replatform 1:1, so heading text is a strong, cheap anchor that sidesteps the
id/index mismatch and the Wix duplication. Substitution cost is dominated by
heading similarity (Levenshtein ratio), with a
textHead+ style-tuple fallback when a heading is absent. Gap (insert/delete) cost is a constant; role-weighting is applied in scoring. Output: matched pairs (→ per-dim agreement), unmatched-live = dropped (role-weighted penalty — the expensive structural error), unmatched-ours = inserted (smaller penalty).
S = 100 · Σ(w_role · matchQuality) / Σ(w_role over matched+dropped+inserted),
where matchQuality = 0.5·presence + 0.5·dimAgreement (half credit for being in
the right place, half for looking right — a seed split, calibratable).
A known capture-noise guard baked into the diff: columns over-fires on
live non-grid sections (an image|text split reports cols=2; see known-issues
"columns capture over-fires"). It is a bounded prop only on
service-grid/team-grid/gallery, so ours is the grid-carrier authority: if
ours omits columns, the live value is capture noise → ceiling, not a miss.
(The same over-fire affects imageSide on media-backed heroes; Step 0 leaves
that as a penalty — the conservative direction, which favours manual review over
false-ship — pending block-type awareness. See follow-ups.)
4. The gate + calibration¶
Hurdle + veto (not a single averaged threshold)¶
verdict(site):
if any HARD-FAIL veto -> HOLD (never auto-ship)
elif composite ≥ τ_ship AND S≥floorS AND T≥floorT AND G_det≥floorG
-> SHIP (unsupervised)
elif composite ≥ τ_hold -> MANUAL-POLISH queue
else -> HOLD (degrade to house-style)
Vetoes (catastrophic regardless of other axes — averaging hides exactly the
crux failure): hero/header/primary-CTA dropped or unmatched; fallbackNavy
(brand identification failed); transparentBlackRemnant; GenericSection on a
role-weight-≥1.3 region; mobile structural collapse.
Calibration (Slice 6) — thresholds are fit, never hand-picked¶
- Labels: on the 30–50-site batch, a human 3-way verdict per site (and per critical page) — ship / needs-polish / hold for "reads as the same site" — plus a one-word reason on "no" (colour / type / layout / missing) to confirm the axes fire on the right cause.
- Fit: a low-capacity logistic regression on
{S, T, G_det}(3 features — deliberately tiny to avoid overfitting ~40 labels) predicts P(human = ship).τ_ship= the cut where precision(ship) ≥ ~0.98 on held-back folds;τ_holdseparates needs-polish from no. Vetoes/floors are fit separately by precision/recall (they encode "catastrophic regardless," which a linear model smooths away). - Operating point: the errors are asymmetric — a false hold costs ~30–60
min of human time; a false ship costs a client seeing a wrong site + brand
damage. So calibrate for high ship-precision, accept lower ship-recall
(borderline → humans). As confidence grows,
τ_shipcan be lowered on re-calibration to shrink the manual budget. - Persist the fitted
{weights, τ_ship, τ_hold, floors, vetoes}in a versionedcalibration.json, stamped into every scorecard (calibrationVersion), so verdicts are reproducible. - The manual budget falls out: predicted manual-band fraction × 30–60 min = the staffing number Slice 6 must produce.
Sanity anchor: garvanbay + WCP are already eyeball-validated as good. They are the fixed points the metric must rank high before any threshold is trusted — Step 0 confirms garvanbay at S=96 and WCP at S=69.6, a sensible ship-vs-needs-polish spread. If the known-good sites don't rank high, the metric is miscalibrated, not the sites.
5. Legitimate-divergence handling¶
The reconciliation is structural, not a fudge factor: we compare bounded render-intent features, not pixels. Legitimately-divergent surfaces are simply not in the feature set; infidelity surfaces are.
| Divergence | Why it doesn't penalise |
|---|---|
| Substituted fonts | T compares size/weight/scale + hierarchy, never font family or glyph; G_det blurs text regions; the LLM rubric ignores font substitution. |
| Stripped decorative Wix CSS | S ignores decoration; G_det operates on occupancy + low-freq hue; the rubric ignores flourishes. |
| The house-style skeleton | We only compare captured live facts quantised into the same bounded enums the skeleton renders; out-of-gamut live values are omitted → ceiling event, not a failure. |
| Wix widget/chrome sections | The junk filter (§3) removes social bars, video players, <style>-leak pseudo-sections from both sides. |
Still caught: wrong colour placement (G_det hue-map + T fallbackNavy +
tone-sequence in S), wrong order (S transposition), dropped section (S
role-weighted drop), flipped image side / wrong column count / wrong alignment
(S per-dim disagreement).
Ceiling events are a deliverable, not an error. Omitted/out-of-gamut dimensions roll up per-site and portfolio-wide, giving the owner the ADR-0009 §3 decision input directly: a dimension that omits on many sites and correlates with human "needs-polish" is an add-a-dimension candidate; one that omits rarely is accept-the-gap; a high-value site that plateaus is manual-polish. This is how the metric "tells us where the skeleton plateaus."
6. Output artifact¶
builds/<domain>/structural-diff.json today (S axis); it grows into
fidelity-scorecard.json (all axes + verdict). Shape:
{
"domain": "...", "calibrationVersion": "...", "verdict": "ship|manual-polish|hold",
"composite": 86.4, "axes": { "S": 88, "T": 95, "G_det": 79 },
"vetoes": [], "ceilingEvents": [ { "dim": "density", "count": 4, "reason": "not-yet-shipped" } ],
"structural": { "matcherGap": { "droppedLive": {"count":3,"weight":2}, "insertedOurs": {"count":5,"weight":2.5},
"propsWithheld": {"count":4}, "byRole": {"content":{"dropped":1,"inserted":5},"ribbon":{"dropped":2,"inserted":0}},
"genericFallbackRate": null } }, // Slice 4a; genericFallbackRate deferred → 4b
"llm": { "verdict": "yes", "confidence": 0.8 }, // advisory
"thresholds": { "tau_ship": 0.72, "floors": { "S": 70, "T": 80, "G_det": 55 } },
"pages": [ { "slug": "home", "viewports": { "desktop": { "score": 87,
"sections": [ { "role": "hero", "weight": 1.6, "matchStatus": "matched",
"dims": { "tone": {"live":"brand","ours":"brand","agree":true}, "imageSide": {…} } },
{ "role": "content", "matchStatus": "dropped-live", "penalty": 1.0, "owner": "lib/cms/transformer.js" } ] },
"mobile": { … } } } ],
"manualQueue": [ { "page": "home", "section": "…", "issue": "tone-not-painting",
"owner": "…", "impact": 1.6, "thumbs": { "live": "…", "ours": "…" } } ]
}
Drives the gate via verdict+vetoes; drives the manual queue via
manualQueue, sorted by role-weight × disagreement, each row carrying the
likely owner (reuse verify-design.js's owner map) and before/after section
thumbnails — so a human spends 30–60 min on the highest-impact fixes.
7. Build order¶
| Step | Adds | Reuses | Status |
|---|---|---|---|
| 0 | symmetric re-capture (--base/--out) + lib/structural-diff.js → S |
capture probe | shipped 2026-07-07 |
| 1 | T axis (lib/theme-fidelity.js) + combined lib/scorecard.js emitter (S+T, partial composite) |
fidelity-harness.js |
shipped 2026-07-07 |
| 2 | G_det axis (lib/perceptual-diff.js): full-page --shots + rect.y, section-crop structure-first diff (coarse colour+luma grid), wired into scorecard.js |
probe shots + structural-diff#alignPage |
shipped 2026-07-07 |
| 3 | full composite + hurdle/veto GATE (verdict); enforce the collected veto flags |
lib/scorecard.js (extend) |
shipped 2026-07-07 |
| 4a (Slice 4a) | matcherGap roll-up into S (drops/inserts/props-withheld, role-weighted, per-site + per-role rollup) — the scorecard now measures matcher debt explicitly |
structural-diff.js ops |
shipped 2026-07-08 |
| 4b-i (Slice 4b-i) | measured GenericSection renderer (both lanes) + closes the CMS-lane silent-drop |
matcher loud-report | shipped 2026-07-08 |
| 4b-ii (Slice 4b-ii) | genericFallbackRate sub-metric (build {section→matcher} sidecar → Σgeneric/Σblocks) + genericFallbackCritical gate veto |
structural-diff.js rows + scorecard-gate.js |
shipped 2026-07-08 |
| 5 (Slice 6) | batch 30–50 sites; human labels; fit calibration.json; bounded LLM triage; set manual budget |
all above | planned |
Both lanes share the probe + structural-diff.js (lane-neutral), so the static
lane is a drop-in once it consumes capture (its known deferral) — MVP on CMS, no
rework.
Status¶
Step 0 shipped 2026-07-07. lib/capture-section-backgrounds.js gained
additive --base/--out (default crawl unchanged; base stamped into output).
lib/structural-diff.js (pure) implements junk-filter → canonicalise →
heading-anchored NW alignment → omit-aware per-dim agreement → role-weighted S.
First cross-site numbers, both against the live deployed CMS workers:
- garvanbay (
emdash.dcathal.org) home: S = 96 — 9/9 sections matched (6 Wix dup-pairs collapsed, 0 dropped), the only deductions two realtone:muted→defaultmisses. - waterfordcountypainters (
wcp.dcathal.org) index/gallery/commercial-painting: S = 69.6 — real findings surfaced: CTAtone:brand→default,imageSidenot rendered on several content sections, animageSide:right→leftflip, a couple genuine content drops.
Step 2 shipped 2026-07-07. lib/perceptual-diff.js implements the G_det
axis: the probe gained --shots <dir> (full-page screenshot per page) +
rect.y (document-absolute section top); G_det reuses structural-diff#alignPage
(identical junk-filter → canonicalise → NW alignment), crops each matched
section from both full-page shots by rect.y/h, and compares them on a coarse
20×28 grid (per-cell RGB colour-placement 0.6 + luma-structure 0.4). The coarse
grid is the mask — glyphs/decoration wash out (substituted fonts don't
penalise), band placement + gross layout survive. scorecard.js now emits all
three axes + a weight-re-normalised partial composite (still not a verdict —
the gate is Step 3). Three-axis numbers (same-version re-capture with shots):
| Site | S | T | G_det | partial composite |
|---|---|---|---|---|
| garvanbay (home) | 96 | 100 | 80.8 | 93.2 |
| WCP (index/gallery/commercial-painting) | 59.9 | 100 | 70.9 | 72.7 |
G_det adds orthogonal signal: it flags the garvanbay hero (33% cell-sim) and
WCP's commercial-painting FAQ (25%) as the largest visual divergences — gaps
S can't see. WCP's S fell 69.6→59.9 vs Step 0 because the fresh same-version
re-capture is more complete: it now measures align/columns on WCP's real
service-grids (which garvanbay lacks), surfacing genuine columns:3→4 and align
gaps — the first time the columns dimension is exercised against real grids.
Step 3 shipped 2026-07-07. lib/scorecard-gate.js (pure) implements the §4
hurdle+veto ladder — computeVerdict({axes, vetoes, calibration}) →
verdict: ship|manual-polish|hold + reasons — and
collectStructuralVetoes(pages) (a live hero/primary-CTA dropLive at
role-weight ≥ criticalRoleWeight → criticalSectionDropped → HOLD).
lib/scorecard.js enforces the collected theme flags (fallbackNavy,
transparentBlackRemnant) + structural vetoes and stamps
verdict/composite/thresholds/calibrationVersion into
fidelity-scorecard.json. Thresholds live in versioned
lib/calibration.json (calibrationVersion: seed-2026-07-07,
fitted: false) — SEED values on the 0–100 composite, provisional until the
Slice-6 logistic fit replaces τ_ship with a P(ship) cut: τ_ship 85,
τ_hold 60, floors S≥70 / T≥80 / G_det≥55. A missing axis can never
ship (the floor check fails on null → manual-polish/hold at best). Verdict
contract locked by lib/scorecard-gate.validate.mjs (19 fixture checks:
veto-beats-composite, floor demotion, partial-axes cap, boundary ≥
semantics, structural-veto collection). Anchor verdicts under seed thresholds:
garvanbay SHIP (93.2, floors met), WCP MANUAL-POLISH (72.7, S=59.9
fails the S-floor) — the expected ship-vs-needs-polish spread. Note: an S-floor
failure with composite ≥ τ_ship is arithmetically impossible under the seed
weights (T,G ≤ 100 ⇒ composite ≥ 85 requires S ≥ 70) — the S-floor becomes
load-bearing only if the weights are re-fit. Also: fidelity-harness.js's own
standalone report now writes fidelity-harness.json (was clobbering
fidelity-scorecard.json with a differently-shaped artifact).
Step 4a shipped 2026-07-08 — matcherGap roll-up into S.
lib/structural-diff.js now emits a matcherGap field per page (diffPage)
and site (diffSite), surfaced through lib/scorecard.js into the structural
block of fidelity-scorecard.json. It is a reported projection of debt
already in S (S is byte-identical before/after — the change is purely additive),
NOT a second penalty: droppedLive and insertedOurs roll up the same
role-weighted alignment ops S already scores, and propsWithheld counts each
matched section's ours-missing dim (a live-present prop ours failed to carry —
columns-na/live-omit ceilings excluded). byRole gives the per-role
drop/insert breakdown (the add-a-matcher signal); genericFallbackRate stays
null (deferred → Slice 4b). Locked by lib/structural-diff.validate.mjs (now
53 checks, +11). Anchor numbers stable — garvanbay SHIP (S=96, matcherGap
0/0/0, clean) · WCP MANUAL-POLISH (S=60.9, matcherGap 3 dropped w=2 / 5
inserted / 4 props-withheld, byRole content 1d·5i, ribbon 2d). The
criticalSectionDropped veto (Step 3) reads the same drop rows and is unchanged
— no drop lands at role-weight ≥ 1.3, so WCP stays manual-polish not hold,
coherent with its debt being 1 content + 2 ribbon strips. Slice 4a is
metric-only (no render, no deploy); the measured GenericSection that
reduces this debt is Slice 4b.
Step 4b-ii shipped 2026-07-08 — genericFallbackRate + the GenericSection
gate veto. A build-time {section→matcher} sidecar
(builds/<domain>/section-matchers.json, {order,heading,matcher,type} per
emitted block) is written by lib/cms/transformer.js (returns a manifest) +
lib/cms/seed-from-build.js (collects per page, writes alongside seed.json) —
the only record of which matcher produced each rendered block, since the
capture sees painted DOM, not block identity. lib/structural-diff.js computes
genericFallbackRate = Σgeneric/Σblocks straight from the build counts (the
ground truth of what was emitted, not re-derived from the capture; null when no
sidecar — unknown, never a bogus 0) and tags each ours row with oursMatcher via
a heading-first / content-order join (makeMatcherLookup), bridged across the
home/index/homepage slug aliasing (manifestForSlug). S is byte-identical —
the tag and the rate are additive, so this is a projection like matcherGap, not
a new penalty. lib/scorecard-gate.js#collectGenericFallbackVetoes flags a
GenericSection block at role-weight ≥ criticalRoleWeight (1.3) →
genericFallbackCritical → HOLD, mirroring criticalSectionDropped on the same
rows; wired through lib/scorecard.js. Locked by structural-diff.validate.mjs
(rate/join/rollup/alias + additive-S) and scorecard-gate.validate.mjs (veto
collection). Anchors stable: garvanbay SHIP gfr=null (no sidecar — its build
is on EC2), WCP MANUAL-POLISH gfr=0 (sidecar present, 0 fallback used),
22 ours rows tagged (join fired incl. the homepage after the alias fix). The
nonzero rate + veto were proven end-to-end on the real WCP capture by forcing a
synthetic GenericSection hero (gfr=0.031, genericFallbackCritical fires on
index/hero/w=1.6), then reverting. Both genericFallbackRate and matcherGap
are now live; the S axis is feature-complete for the seed batch. See
known-patterns "genericFallbackRate is a build fact, not a capture measurement".
Step 1 shipped 2026-07-07. lib/theme-fidelity.js folds
fidelity-harness.js's typography + palette reports into a graded 0–100 T
axis (size-proximity + weight + rule-applied for h1/h2/h3; palette penalised for
fallbackNavy/low-confidence/#000000 remnant, which also emit gate veto
flags). lib/scorecard.js is the combined emitter → fidelity-scorecard.json
with axes {S, T} + a clearly-labelled partial composite (S+T
re-normalised; not a ship/hold verdict — the gate is Step 3) + collected veto
flags. Both sites: T = 100 (garvanbay surface/high, WCP logo/high — no veto
flags), partial composite garvanbay 97.3, WCP 79.7. T is saturated on
these two well-built sites by design; it discriminates on the wider batch via
the fallbackNavy veto.
Globals (chrome) reporting projection shipped 2026-07-12 pt2.
lib/globals-diff.js compares the LIVE chrome vs an independent re-measurement
of OUR deployed chrome (header/topbar/footer background + headerLayout /
headerCta) → an additive globals block in fidelity-scorecard.json
(S/T/G_det byte-identical; lib/globals-diff.validate.mjs 36 checks). It closes
the "header/topbar/footer are excluded from every axis" blind spot as a
REPORTING projection (like variantMismatch / the asset-manifest), realising
the §3 "header/topbar (site-level, from chrome)" sub-metric in reporting mode.
Every dim is null-safe (agree true|false|null; null = not compared), so on a
pre-header-slice v2 capture headerLayout/headerCta report n/a and only
the background/presence dims compare. On the v3 re-capture (2026-07-12 pt3) they
carry real numbers: WCP headerLayout agree (split-center = split-center,
our deployed header matches live); headerCta mismatch (live null — the
SERVICE AREAS pill is pinned outside <header>, fast-follow #2 — vs ours
SERVICE AREAS). Gate vetoes + thresholds → Slice 6.
Findings the scorecard surfaced (site-fidelity, not metric bugs)¶
- garvanbay CMS home renders
defaultwhere live ismutedon the MANAGEMENT ACCOUNTING + Accounts Preparation service teasers — the alternating grey bands were stamped but do not paint on the deploy. (Caught only because we re-measure the deployed DOM, not the stamp.) - WCP CTA/brand bands render
defaultand several WCP content sections don't render their side image (imageSideomitted vs live) — WCP was deployed as a separate worker; its D1 content likely predates/omits the measured tone/imageSide stamping.
These are inputs to the manual-polish queue, not chased in Step 0 (Step 0 is the metric, not the fixes).
Follow-ups¶
- Text-contrast probe (proposed 2026-07-09, from the operator-caught hero
defect). The black-subtext-on-dark-scrim hero shipped under a SHIP verdict
because no axis converts "readable text over hero media" into a gate signal:
G_det's coarse colour+luma grid deliberately washes glyphs out (a text line
flipping white→black barely moves it), and T compares emitted tokens, not the
deployed DOM's computed styles. Cheap deterministic addition: the capture
already records per-section heading/para computed colour (
_styleRaw .textColor, 2026-07-09) on BOTH live and ours captures — diff the classified textTone live-vs-ours per media section (a mismatch = flag), and/or a WCAG-style contrast ratio of text colour vs the section crop's mean colour under it (a sub-threshold ratio on a role-weight ≥1.3 section = veto candidate). Fold into Slice 6 calibration so thresholds are fitted, not hand-set. imageSideover-fires on media-backed heroes (same class as the gatedcolumnsover-fire). Left as a penalty for now — the conservative direction — pending block-type awareness in the diff so N/A can be distinguished from a genuinely dropped image.- G_det outliers triaged 2026-07-08 — mixed: one genuine, two alignment
mis-pairings. Cropped the flagged sections from the on-disk
live/oursshots (offline, no re-crawl) and eyeballed paired live|ours crops against healthy controls (garvanbay PAYROLL sim=93 is a clean like-for-like — the crop path itself is sound). Findings: - WCP commercial-painting FAQ (25%) — GENUINE. Ours renders the FAQ on a dark navy background (dark-on-dark, barely legible) where live is white — a real wrong-tone/colour-placement defect (same class as the WCP tone gaps already logged; D1 content predates the tone stamping). A "Loading…" spinner in the crop hints the accordion is lazy/JS-rendered, so a minor timing component isn't excluded, but the navy-vs-white background is the dominant, genuine signal. G_det is doing its job here.
- garvanbay hero (33%) — ALIGNMENT ARTIFACT (+ a real gap S can't see). The hero/welcome heading-dup-pair collapses to one canonical section, but the collapse keeps the welcome rect (liveH=536, intro paragraph + 3 photos) to represent role=hero — so G_det compares live's welcome band against ours' hero band (a 168px dark-blue "GARVANBAY ACCOUNTING" strip): apples-to-oranges → 33%. Underneath, there IS a genuine divergence (ours has no welcome intro+images band at all), but S scores it 1↔1 matched because the pair collapsed — so the low G is right for the wrong reason.
- WCP content #1 (37.8) — ALIGNMENT ARTIFACT. A second live "Commercial Painting" hero-dup instance is NW-aligned to ours' partner-logo strip (two unrelated sections paired because headings/positions line up) → low sim on a w=1 role.
Determination / gate-readiness. G_det penalises real divergence correctly
(FAQ), but it is more sensitive to alignPage mis-pairings than S is — S
compares quantized style tokens (which can coincidentally agree on a mis-paired
match) while G_det compares pixels (which won't). So enabling G_det as a gate
surfaces alignment-quality problems S tolerated, and they land on
high-weight roles (hero w=1.6, content w=1), materially dragging the
composite. Root cause is in structural-diff.js alignPage/canonicalise
(shared with S), NOT in perceptual-diff.js#cropGrid (the crop is faithful to
what alignPage matched). Two mechanisms: (1) dup-pair collapse keeps the wrong
rect (garvanbay: welcome rect represents hero); (2) NW alignment pairs a
leftover live dup against an unrelated ours section (WCP hero-dup ↔ logo-strip).
Before G_det's floor gates unsupervised, fix upstream in alignPage: prefer
the hero-band rect on a hero/welcome collapse; reject/flag matches with a large
height-ratio + heading dissimilarity; optionally emit a per-match confidence so
G_det can N/A low-confidence pairs (a perceptual-diff-side height-ratio/confidence
gate is a cheaper fallback but only masks the symptom). This dovetails with the
still-missing structural-diff.js .validate.mjs below — lock the collapse/align
behaviour with fixtures while fixing it. Triage script: scratchpad
gdet-triage.js (reuses alignPage + comparePair, writes side-by-side crops).
RESOLVED 2026-07-08 — both alignment artifacts fixed at the root
(canonicalise). Two changes in lib/structural-diff.js#canonicalise: (1)
a bothEmptyDup collapse form (adjacent, same tone/style, identical normalised
textHead, both headings empty) — Wix doubles image-heroes and brand ribbons
with empty headings, which the prior heading-keyed detector missed, letting a
leftover twin get NW-force-matched to an unrelated ours section (WCP hero-dup ↔
logo-strip); (2) anchor/visual-rect decoupling — a collapsed unit keeps its
heading anchor from the heading-bearing member but its rect from the earlier
(visually-first) member, so role=hero crops the hero band, not the welcome
band (garvanbay). Results: garvanbay hero G 33.4→63.6 (honest hero-vs-hero;
S unchanged 96, composite 93.2→94.7, still SHIP); WCP spurious match gone (ours
logo-strip now a clean insertOurs), twins single-penalised (commercial-painting
S 68.3→72.3, site S 59.9→60.9, composite 72.7→73.7, still MANUAL-POLISH); WCP
FAQ stays 24.9 (genuine, untouched). The aligner's cost model (GAP/subCost)
was deliberately not touched — the pre-pass collapse is what removes the
force-match. Locked by lib/structural-diff.validate.mjs (42 checks). See
known-patterns "A section-alignment collapse serves TWO axes with different
needs". G_det is now gate-ready on artifact grounds; its floor still awaits
the Slice-6 calibration like the other axes. Residual (separate follow-up): the
hero/welcome collapse still hides a genuine intra-pair drop from BOTH S and G
(garvanbay renders no welcome intro+images band; S counts it 1↔1 matched) —
catching intra-collapse content drops is a distinct S improvement, not chased here.
- Junk-filter misses a "Watch Now" video-chrome strip on WCP index (surfaced
by Slice 4a's matcherGap, which made every S drop legible). isJunk only tags a
heading-less video strip on Now Playing|Play Video; WCP's "Watch Now" strip
has a heading, so it survives as a ribbon-role dropLive (w=0.5). Pre-existing
and low-impact (S already counted it; matcherGap only named it), but the
junk-filter regex should widen to the "Watch Now" form (still heading-gated to
stay conservative). Not chased in 4a (metric-only). Same class as the deferred
GenericSection decision: distinguish Wix chrome to drop from content to render.
- G_det is colour+luma grid, not yet an explicit occupancy/edge term. The
coarse downscale approximates the contract's "block-occupancy grid" (variance
washes into cell luma) but doesn't compute a dedicated edge-density map. Adequate
for MVP; add an explicit gradient/occupancy term if colour+luma proves too
forgiving of layout collapse in calibration. Also no cookie-banner/fixed-chrome
masking yet (the probe dismisses the consent banner and excludes header/footer,
so low-risk).
- T re-measures emission, not the deployed DOM. theme-fidelity.js compares
our emitted theme (from theme.json) against live _raw — it checks the
derive+emit pipeline, not the deployed page's computed h1/h2/h3 sizes+colours
(unlike S, which re-measures the deployed output). A deployed-computed-style
probe would make T consistent with the S philosophy; deferred (Step 2+).
- .validate.mjs: structural-diff.js done (2026-07-08), theme-fidelity.js
still missing. lib/structural-diff.validate.mjs (42 checks) now locks
canonicalise (all three dup forms + anchor/visual-rect decoupling + non-mutation),
align (match/drop/insert + the force-match cost model), scoreMatch
(agree/disagree/ours-missing + live-omit + columns-na ceilings), isJunk, and
roleOf — per the repo convention (known-patterns "Lock current behaviour as
expected behaviour"). theme-fidelity.js still has no validate script; add one
before its T-floor is trusted unsupervised.
- Ours-side slug coverage: Step 0 captured only home/index (+2 WCP pages).
The CMS deploys serve a subset of live slugs; per-site the scorecard should
capture the intersection and report live-only slugs (already surfaced).