EVAL-RUN: measuring the Phase-2 sweep

Work package EVAL-RUN of the SGH programme (operations/research/sgh-program-20260908/PLAN.md), 8 September 2026. CPU only, research/sgh-synthetic-histopathology/.venv/bin/python, --jobs 12. Nothing was committed to git. No GPU was used.

Engineering evidence only. Every instrument here measures shape, spacing, texture, colour, or one pathology encoder. None of them diagnoses, grades or stages anything, and no number below says an image is diagnostically correct. No pathologist has seen any of this; Yash has not reviewed a sheet. Every "better" or "worse" names its instrument and its n.

0. What was run

cd [local]/sgh-program-20260908
P=../../../research/sgh-synthetic-histopathology/.venv/bin/python

$P packages/sweep-v1/evaluate_sweep.py all \
     --records packages/sweep-v1/retrieved/records.jsonl \
     --out     packages/sweep-v1/eval \
     --jobs 12 --probes

--ncc-refs was left at its default targeted (own donor field plus up to two phikon nearest-neighbour reference fields per output), as EVAL_BUILD_RESULT.md section 5 documents. No other flag was added. Wall clock: features 142 s, a1 <1 s, d1 9 s, arms+probes 267 s, screens 782 s (embed 168 s + 417 NCC pairs 614 s), sheets 55 s, select 3 s, report <1 s — about 22 minutes, plus two stage reruns after the harness fixes in section 1.

Deliverables, all under packages/sweep-v1/eval/:

Instrument self-check. The harness rescored the 22 real held-out fields from scratch and reproduced C1's morphometrics/scores.csv Mahalanobis distances to max |delta| = 0.0000. It is the same instrument.

Hand-check of the harness's arithmetic. Every one of the 41 sweep rows and 8 reference rows of a1-table.csv was recomputed independently with pandas (system python3 3.13 / pandas 3.0.2, a different interpreter from the harness venv) straight from features.csv:

and the reference rows reproduce PLAN.md's recorded baseline exactly: fullset_pass2 mean ring 0.7256 normal / 0.4753 gastritis / 0.9170 IM / 0.7260 mixed, real held-out 0.8538 / 0.7428 / 0.7101 / 0.7261. The tables in this document are safe to quote.

1. Two things went wrong, and what was done about them

1.1 One of the 407 sweep outputs is a zero-byte file (a DATA fault, not a harness fault)

packages/sweep-v1/retrieved/slot0/out/a1/pass2/intestinal_metaplasia/intestinal_metaplasia_s14_oblique.png is 0 bytes. Its record (a1_p2_intestinal_metaplasia_s14_oblique) carries output_sha256 = 1779fb07b5f8dda2878a5291c8e864e407c676202be764ecf9ed1a0bd042dbb4, i.e. the generator wrote and hashed a real 4096x2048 PNG. The VM's own SHA256SUMS already records the file as e3b0c442... — the SHA-256 of the empty string — so the file was already empty on the VM before retrieval, and SWEEP_RESULT.md's "SHA256SUMS verified 143 files OK" verified an already-empty file transferring correctly. Slot a is the slot that was preempted at 15:38:39 UTC; the most likely cause is that the power-off dropped the page cache for a file that had been written but not yet flushed. SWEEP_RESULT.md's coverage check counted PNG existence, not size, so it reported 143/143.

I verified the rest: 406 of 407 outputs hash exactly to their record's output_sha256 (full re-hash of all 5.5 GB, n_mismatch = 1). The sweep is otherwise byte-intact.

Consequence: intestinal_metaplasia oblique pass2-si6 is n = 4, not 5, and IM pass2-si6 pooled is n = 14. Every table below carries the real n. The image is excluded everywhere and is not recoverable without regenerating on GPU.

A second bookkeeping point found while checking: 8 pairs of sweep records are byte-identical (same output_sha256) — every d1 donorK arm whose donor happens to be the a1 donor for that seed duplicates the corresponding a1 cell. EVAL_BUILD_RESULT.md limitation 11 predicted this for donor0; it happens for donor0, donor1, donor2 and donor3 depending on the seed. So the 407 records are 398 distinct images. Two of the IM top-8 candidates (ranks 6 and 7) are the same file under two arm names.

1.2 Two harness bugs, both fixed in packages/sweep-v1/evaluate_sweep.py

(a) --probes aborted on the zero-byte PNG. run_probes.py has no per-image error handling, so one bad PNG in an --add-set records file killed the whole probe run (exit 1, PIL.UnidentifiedImageError). The harness's own features stage handles this fine; only the probe subprocess did not. Fixed by adding unreadable_outputs() and filtering probe_specs(): any record whose feat_status is not ok, or whose output is missing or zero-length, is dropped from the probe input, logged, and written to eval/probes-input/excluded.json and into stage-arms.json. Rerun: arms --probes exit 0 in 267 s, one record excluded.

(b) the a2 own-source figure was structurally zero. _library_origins_for() read p.get("library_origin") or p.get("library_file"), but in the real records library_origin is the pixel coordinate of the window inside the donor field ([0, 512]), not a path. Path(str([0, 512])).name is the string "[0, 512]", which never matches a reference basename, so every library arm reported library_source_fraction = 0.000 — silently, and it looked like a real finding ("a2 tiles never land on a conditioning field"). Fixed to prefer library_file and to ignore library_origin unless it is a string. After the fix a2's figure is 0.756 mean / 0.875 median (range 0.25-1.00), which is the number question (d) asks for. The screens stage was rerun with --reuse-screen (phikon reused, 417 NCC pairs recomputed, 577 s), then report.

No other harness change was made, and no data was touched.

2. Extra measurements this package added

The harness answers the questions it was built for. Four of the brief's questions needed numbers it does not produce, so these were measured with the harness's own functions (evaluate_sweep._d1_one, same masks, same config) so they are directly comparable:

3. Answers

(a) A1 per category: topology and envelope, versus real and versus the old baseline

Medians per category, pooled over the three cuts (n = 15 per arm except where marked). env pct is the percentage of that category's real held-out fields at or below this image's envelope distance, so 50 is the real median and 100 means outside the entire real held-out spread. D is the C1 MinCovDet Mahalanobis. lap is C2's mean absolute Laplacian (real held-out 20.4-23.6).

(The bolded old-baseline ring means are exactly PLAN.md's 0.726 / 0.475 / 0.917 / 0.726, and the bolded D values are morphometrics/scores.csv set fullset_pass2 medians 8.64 / 6.32 / 5.28 / 6.74.)

Did the A1 cartoons trade topology for realism? Yes, and the trade is real but partial.

• On topology, the new A1 cartoons are worse than the old baseline in three of four categories: ring mean 0.591 vs 0.726 (normal), 0.394 vs 0.475 (gastritis), 0.530 vs 0.726 (mixed) at the best new arm (si3, n = 15 each vs n = 5 baseline). Only IM improves, and it improves by moving down toward real: 0.645 vs 0.917, against a real held-out mean of 0.710. The old baseline's IM number was above real, not below it.
• On the envelope, the new arms are better in all four: D 7.10 vs 8.64 (normal), 5.77 vs 6.32 (gastritis), 4.67 vs 5.28 (IM), 5.78 vs 6.74 (mixed). Median improvement 0.55-1.54 D.
• On detail — the axis C2 called "the largest single lever on the tell" — the improvement is the largest of the three. si3 closes the Laplacian gap to real by 47% (normal), 67% (gastritis), 52% (IM), 68% (mixed) relative to the old two-pass baseline (medians, n = 15 vs n = 5 vs n = 4-6 real). This is visible: sheets/detail-1to1.png at 1:1 pixels shows the old two-pass field as a flat watercolour wash with smooth featureless nuclei, and the new si3/si6 fields with markedly more contrast, granular cytoplasm and some intranuclear texture. They are still short of real, whose nuclei carry speckled chromatin and whose cytoplasm is visibly granular.
• Envelope percentile is a saturated instrument here. 41 of 249 pass-2/pass-3 outputs are inside their category's held-out band at all; the median of every single a1 cell except three gastritis cells sits at pct = 100. Where the brief asks for "envelope percentile", the honest reading is "at or beyond the top of the real spread", and the distance D is the number with resolution.

Which cut keeps lumens best? oblique, at every pass-2 depth (a1, pooled over categories, n = 20 per cut per arm):

across is the worst at pass 2 si6 (0.095) and along is the worst at pass 1 (0.091, a geometric consequence — A1_RESULT.md gap 5: an along plane spans the whole mucosal depth and shows tapering tubes rather than ring profiles).

Is any arm inside the real range on BOTH instruments? Yes, but only 23 of 249 pass-2/pass-3 images (9.2%), and they are concentrated:

So the hit rate is 23/249 = 9.2% for the new sweep against 4/20 = 20% for the old baseline, but the new sweep's best single arm, a1 pass2-si3, is 14/60 = 23%. By category the 23 are IM 14, gastritis 8, normal 1, mixed 0 — no mixed output anywhere in the sweep is inside both bands, and only one normal output is (a1_p2_normal_s12_oblique_si3, which is rank 2 on the normal candidate list).

Normal and gastritis specifically, against the old baseline:

• normal — worse on topology (ring mean 0.591 vs 0.726, real 0.854; ring-in-range 33% vs 40%), better on the envelope (D 7.10 vs 8.64, real 3.33), better on detail (lap 18.12 vs 14.00, real 22.78). Only 1 of 35 normal pass-2 images is inside both bands, against 2 of 5 for the baseline's ring and 1 of 5 for its envelope. Normal is the weakest category in the sweep on every count.
• gastritis — worse on topology at si6/si9 (ring mean 0.062 / 0.110 vs 0.475) and roughly level at si3 (0.394 vs 0.475, real 0.743); better on the envelope at si3 (D 5.77 vs 6.32, real 4.19, and 100% in band vs 100% in band — note the gastritis band is the wide one, C1 limitation 2); better on detail (18.60 vs 14.95, real 20.40). The pass-2-makes-gastritis-worse effect recorded in PLAN.md is not just still present, it is much stronger on the new cartoons: pass 1 0.371 -> si6 0.062.

Sheets: sheets/a1-<category>-<cut>.png (12). Looking at them: the A1 cartoons in column 1 are a dense field of small purple ovals on pink with a few white slots — a nuclear scatter, not a gland drawing. Pass 1 is a blurred version of exactly that and looks like nothing histological. Pass 2 at si6 and si3 is the first column that reads as gastric mucosa: columnar epithelium, foveolar pits, mucin-filled cells, capillaries with red cells. The real held-out column is visibly more organised than any of them — one or two large glands with an open lumen and a regular basal palisade, where the generated fields show many small irregular lumina and less orderly nuclear rows.

(b) Does pass 3 (si15) raise detail toward real without moving topology?

No. It lowers detail, and it also lowers topology. n = 10 pairs (5 gastritis + 5 IM, each pass-3 output against its own pass-2 parent):

Real held-out Laplacian is 22.77; pass 2 sits at 16.9 (IM) / 17.5 (gastritis) and pass 3 moves it away, to 15.84 / 16.76. Topology "held" only in the sense that the median change is zero — but the pass-3 arm's own ring median is 0.000 for both categories, because its pass-2 parents were already at 0.000-0.091. Pass 3 is a third light img2img (si15, 5 steps) on top of an already-repainted canvas; every step of it removes detail rather than adding it. On this evidence the si15 pass-3 arm should be dropped.

(c) D1: is layout set by the cartoon and independent of the donor?

One-sentence verdict: at pass 1 the cartoon sets the layout almost completely and the donor sets nothing; by pass 2 (si6) that has reversed — the cartoon is gone and the donor decides what the canvas looks like.

The harness pools the two passes and returns "YES, 3 of 5 checks". That pooled answer is wrong, and the sheet says so at a glance. Split by pass (n = 25 outputs per pass, 5 cartoons x 4 donors + 5 rot180):

The sheets/d1-layout-factorial.png grid (rows = 5 cartoons, columns = 4 donors, last column rot180, all pass 2) is unambiguous: going down a column — same donor, five different cartoons — the five images are near-copies of each other; the donor2 column is the same pale mauve field with the same dark diagonal streak in the upper right, five times. Going across a row — same cartoon, four different donors — the four images have nothing in common. That is what SSIM 0.870 vs 0.505 means in pixels.

The donor NCC number does not contradict this. 0.328 is well below the native positive control (0.409 generate, 0.642 img2img), so the pass-2 canvas is not a spatial re-registration of the donor field. What the donor transfers is the tissue's content and appearance — which cell types, which stain, which texture — not its coordinates. That is the same structure-inheritance effect C2 measured; probe B here puts 77.2% of a1 tiles nearest their own donor field (old two-pass 89.4%, so the new recipe reduced it, but not by much).

The a1 ladder, measured the same way on all 60 a1 cartoons against their own label maps and against a per-(category, cut) ceiling, shows the collapse is monotone in denoising depth:

This is the most consequential finding in the package, because it inverts the reading of (a): the arm that scores best on the envelope, pass2-si3, is the arm that has thrown the most of the cartoon away. The programme is currently buying realism by abandoning the imposed geometry, so "the cartoon controls the layout" is only true of the pass-1 output, which is also the output that looks least like tissue (D 7.8-11.5, lap 11.9-15.5).

Caveat on the label maps used. SWEEP_RESULT.md section 6 established that the VM-rendered cartoons are not byte-identical to the local ones (numpy/scipy/Pillow skew; the difference is confined to stromal-nuclei scatter, with lumen labels essentially identical). The VM cartoons were not retrieved — packages/sweep-v1/retrieved/aux/cartoons/{a1,b1,d1,e2}/ contains only MANIFEST.sha256.json, no PNGs — and I confirmed the mismatch directly (cartoons/d1/intestinal_metaplasia_s11_across_cartoon.png hashes to e7a6eeef…; the record's reference_sha256 is ac6876dd…). Every IoU in this section therefore uses the LOCAL Mac label maps. Since 99.8% of the label difference is the stroma/stroma-nucleus pair and label 1 (lumen) is essentially identical, the lumen IoU is the right metric to have used and the numbers should be very close to the VM-cartoon values — but they are not measured against the exact bytes the generator saw, and the epithelium IoU (labels 2,3,4) is slightly less safe. Retrieving the VM cartoons would remove this caveat for about 300 MB of transfer.

(d) A2, composition-matched token library, versus a1 on the same cartoons

20 pass-2 canvases (4 categories x 5 across seeds), each paired with the a1 pass2-si6 cell on the identical cartoon. Medians of the paired deltas:

• Topology: a small gain, +0.094 ring pooled, driven by IM (+0.227) and gastritis (+0.121); mixed loses 0.051. 5 of 20 land inside the real ring range.
• Envelope: a wash pooled (-0.10 D), better for normal and mixed, clearly worse for gastritis (+1.57 D). 0 of 20 a2 canvases are inside any envelope band.
• Detail: a clear cost, -3.48 Laplacian pooled and -5.76 for gastritis. This is the largest single detail regression of any arm except d2 at high alpha.
• Seams: not measurable with the instruments in this harness. The library picks tokens per 1024 px window (median 2 distinct donor fields per canvas, range 1-3), so a seam would be a window-boundary discontinuity; nothing here computes one, and at the 768 px sheet crop none is visible by eye. Recorded as not measured, not as absent.
• Own-source figure (question as asked): a1's tiles land on its single donor field 86.7% (si6) / 91.7% (si3) of the time; a2's tiles land on any field the library drew from 75.6% mean, 87.5% median (range 25-100%) of the time. So the library spreads the provenance, but only from ~90% one field to ~76% of two or three. Probe D agrees and is the cleaner statement: a1 sits at the 96.6th percentile of real held-out manifold distance once its donor slide is removed from the bank, while a2 — which has no single donor slide to remove — sits at 76.1. That is a genuine reduction in donor dependence.

Did it help gastritis? No. It lifted the ring fraction off the floor (0.000 -> 0.131) but cost +1.57 D on the envelope and -5.76 Laplacian, and left 0 of 5 inside the band. On sheets/a2a3-hpylori_gastritis.png the five a2 canvases are visibly the same crowded field of purple cells with clear vacuoles, five times over, where the five a1 cells are five different fields. Which brings up the thing the harness does not measure:

A2 costs image diversity. Mean pairwise SSIM within an arm (this package's within-arm-diversity.json; same coarse-grey SSIM the d1 stage uses):

Real fields are essentially uncorrelated with each other (0.035). Every generated arm is an order of magnitude more self-similar, and a2/a3/b1 are close to producing one image repeatedly.

(e) A3, mean and nearest-to-mean tokens: coherent tissue or mush?

Both, split by arm — and both are one image repeated.

• mean (grand mean of the category's tokens) is mush, and the sheet is why I use that word: on sheets/a2a3-intestinal_metaplasia.png column 3 every image is the same pale, low-contrast field of loosely-packed cells with evenly-sized round nuclei — no gland, no lumen, no palisade, no goblet cell, no vessel. It is a cell suspension, not tissue. D 8.77-11.13, +2.62 D worse than its a1 partner (n = 10).
• nearmean (the real token grid nearest the mean) is coherent, and for IM it is the best-scoring tissue in the entire sweep: D 4.15-4.67 against a real held-out IM median of 2.58 and max 4.50, ring 0.52-0.66 against a real median of 0.774, 4 of 5 inside the envelope band. Column 4 of that sheet shows clusters of clear goblet-like vacuoles between nuclear strands with pink stroma and occasional red cells — it reads as intestinal metaplasia.
• The caveat that kills it as a result: the token grid is fixed for the whole arm, so the five IM nearmean canvases have mean pairwise SSIM 0.877. It is one image, generated five times from five different cartoons that had no effect. Its four "inside both bands" images are one image counted four times. And for gastritis the same arm produces ring 0.000 on all five and D 9.78-10.50, so the coherence does not transfer between categories.
• A3 mean has the second-largest copy-screen margin of any arm (max cosine 0.701, min margin 0.249, behind d2 alpha=1's 0.260), which is consistent: it is the furthest from any specific real field.

(f) D2, self-conditioned pass 2 by alpha

IM across, 5 cartoons per alpha, alpha = 0 is the a1 pass2-si6 cell on the same cartoon. Every trend is monotone:

Topology rises with alpha, envelope and detail fall, and the envelope/detail loss is much the bigger effect. From alpha 0 to alpha 1: ring +0.167, D +3.72 (a 67% increase in distance), Laplacian -5.90 (a 35% loss of the detail statistic that C2 identified as the biggest component of the tell), HF energy -0.034 (a 77% loss).

Does alpha = 1 (no real donor at pass 2) hold up? No. sheets/d2-ladder.png is a clean visual gradient left to right: the same structures stay in the same places while the rendering gets progressively smoother and flatter, until at alpha = 1 the nuclei are uniform featureless ovals in a flat cytoplasm — the "stamped nuclei, flat wash" appearance C2's probe-B sheet described, in its purest form. The real donor's UNI tokens are doing substantial work at pass 2 and cannot be removed.

The one thing alpha = 1 buys is provenance: own-source tiles fall 0.867 -> 0.025 and max phikon cosine 0.888 -> 0.690 (min copy margin 0.260, the largest minimum margin of any arm in the sweep). If the goal were minimum donor dependence at any quality cost, alpha = 1 is the arm; on realism it is the worst pass-2 variant except a3 mean and b1.

(g) E2, cartoon fidelity ladder 0/1/2/3

IM across, 5 seeds per rung; level 2 is the shipped renderer, i.e. the a1 cell of the same seed.

There is no optimum, because there is no trend. The envelope distance spread across the whole ladder is 0.51 D (5.32-5.83) on n = 5 per rung, which is smaller than the within-rung spread; the paired deltas against the a1 f2 cell are -0.21 / +0.23 / +0.17 D for f0 / f1 / f3. If anything the crudest cartoon (f0, flat colours, no texture at all) is marginally the best on both ring (0.267, the highest of the four) and D (5.32, the lowest) — and it is also the cheapest to render.

sheets/e2-ladder.png shows why: the four cartoons in each row are visibly different (f0 flat purple ellipses on flat pink; f3 with a greenish optical haze and softer edges), and the four generated outputs in the same row are visibly the same image. The row-3 example even carries the same dark diagonal streak in all four outputs. Combined with the layout-IoU numbers (e2 pass-2 IoU 0.020-0.063, i.e. the cartoon is gone by pass 2 whatever its fidelity) the conclusion is that cartoon photorealism is not a lever on this pipeline at all; what survives the repaint is the coarse geometry, and even that only through pass 1.

(h) B1, gastritis surface-compartment donors

Every b1 pass-2 canvas is much worse than a1 gastritis, and none of them shows anything organism-like.

The paired delta against the a1 group median is +5.80 D (pair_kind = group, since a1 never rendered the b1 seeds 21-25). The b1 arm has the worst envelope distance of any arm in the sweep.

What the images actually are. sheets/b1-overview.png shows the five whole canvases downsampled next to three a1 gastritis oblique canvases and two b1 cartoons. The five b1 outputs are not gastric mucosa: the upper 80% of each canvas is a mass of horizontal wispy eosinophilic strands — reading as loose fibrin, mucus strands or shredded collagen — in a nearly empty pale field, with isolated elongated dark nuclei, and only a thin strip of actual glandular mucosa along the bottom edge. All five seeds produced the same thing (mean pairwise SSIM 0.864). The a1 gastritis canvases in the same sheet show proper foveolar pits, glands, columnar epithelium and lamina propria.

The mechanism is legible and is the same donor-dominance effect as D1: the twelve organism-library windows B1 selected are surface / mucus-interface windows (B1_RESULT.md's table: "foveolar pit lumen with apical mucin caps", "surface mucinous exudate", "surface mucus and fibrin band"), so conditioning a whole 4096x2048 canvas on tokens from one of them paints mucus and fibrin over the entire field. Conditioning on a donor whose content is one compartment gives you that compartment everywhere.

Organism-like rods: none. I looked at 1:1 pixels (2x) on sheets/b1-zoom-pass2.png (384 px native crops, 1 px = 0.25 um, across the surface band of all five canvases) and on sheets/b1n-zoom.png and sheets/b1n-real-vs-generated.png (the 12 native 1024 px organism-donor windows next to the real library windows they came from). I see no clusters of faint, uniform, gently curved 2-4 um rods lying free in luminal or mucus space, in any b1 or b1-native output. What is present in the pale space is occasional 1-2 px pink/magenta specks that read as stain speckle, and elongated nuclei attached to cells. This is the expected result and it matches B1's own finding (0 of 60 real candidates graded likely organisms): the donors do not demonstrably contain organisms, so the generator has nothing to reproduce. The b1 arm is a gastritis-surface-compartment-donor experiment and must never be described as an organism experiment.

The b1-native arm (12 windows) is a clean structure-inheritance demonstration: the generated 1024 window is recognisably a redrawing of its real donor window — same cell types, same arrangement, different pixels. Its detail statistics (Laplacian median 19.11) are not comparable to the canvas numbers: at 1024x1024 the analysis raster is 0.25 um/px, not the 0.6667 um/px the envelope is calibrated on, so the harness correctly excludes all 12 from every envelope table.

(i) Copy screens on all pass-2 / pass-3 outputs

Nothing fails. No output needs to be excluded on copy grounds.

phikon-v2 embedding screen, 249 candidates (all pass-2 and pass-3 outputs), 1992 candidate tiles against 118 real reference fields (96 train + 22 held out), fail line 0.95:

The overall maximum, 0.9049, is on a1_p2_pass2-si3__normal_s11_across_si3.png; it is 0.045 below the 0.95 fail line, which is the tightest margin in the sweep. For reference, the old two-pass fullset run reached 0.8859 on this identical screen and C2 calibrated a pixel-level copy at 0.987 against 0.66 for unrelated real fields. The screen passed: true.

Pixel NCC search (pixcell_patch_search.py, 64 px window, stride 256, threshold 0.92, targeted references: each output against its own donor field plus up to two phikon nearest-neighbour reference fields): 417 pairs, max NCC 0.8635, median 0.533, 0 flagged windows, 417/417 NO_COPIED_PATCHES, 0 failures. Highest per arm: a1 si3 0.864, b1 0.775, d2 self0.75 0.741, everything else ≤ 0.73.

Two honest limits on that clean result. First, the NCC is a targeted screen, not a proof — 29,500 exhaustive pairs would be about 13 hours (EVAL_BUILD_RESULT.md section 5). Second, and more important, the copy screen is the wrong screen for the risk that is actually present here: probe A separates every one of the nine sweep sets from real tissue at 100% of fields called generated, and the real risk this sweep exposes is not copying but donor dependence (own-source tiles 0.87-0.92 for a1, probe D 96.6th percentile once the donor slide is removed) and within-arm duplication (section d). Neither is a copy, and neither is caught by a copy screen.

(j) Candidates

eval/candidates.csv, top-8 per category from morphometrics/pareto_select.py on (|ring − real held-out median|, envelope distance, copy margin = 0.95 − max cosine). All 32 are on the Pareto front.

My honest view of sheets/candidates.png.

• IM and mixed have images I would put in front of a pathologist, for the narrow question "does this read as gastric tissue of this type". IM ranks 1-2 (intestinal_metaplasia_s12_across_si3, _s12_oblique_si3, D 3.17 / 3.21 against a real held-out median of 2.58 and max 4.50 — inside the real band) show gland profiles with pale vacuolated cytoplasm and a recognisable epithelial border; mixed ranks 1-3 show a surface epithelium with a columnar band and vessels carrying red cells. IM is also where the sweep concentrates its "inside both bands" images (14 of 23). Mixed is a split verdict: it looks the second-best of the four on the sheets, but 0 of its 35 pass-2/pass-3 outputs is inside the envelope band (its best is D 4.46 against a held-out max of 4.38), so the visual impression and the envelope disagree and only the visual half is favourable.
• Normal is not there. Ranks 3-5 are three views of the same seed at ring 0.977 and D 5.0-5.4, and what they show is scattered cells with bright red-orange globules in a loose vacuolated matrix — not the parallel, tightly-packed straight gland profiles of oxyntic mucosa that the real held-out column shows. The a1 normal sheets confirm it: pass 2 on normal cartoons produces sheets of cells with variable amounts of blood rather than an ordered glandular field.
• Gastritis is the category that is still clearly not there, and it is the one where the selection objective is being gamed. Ranks 1-3 have ring 0.964-1.000, right on the real median of 0.807, so the Pareto front loves them — but they achieve it with 27-29 detected rings against a real median of 43.5 and a pale tissue fraction of 0.354-0.373 against a real 0.229, and on the sheet they are a washed-out hypocellular honeycomb. This is exactly the failure mode C1 demonstrated in its section 7 (fading a real field 40% toward white raises its ring fraction from 0.805 to 0.973 while pushing it out of the envelope). The more plausible-looking gastritis candidates are ranks 4-6 (one si3 and two si9), which show nucleated columnar epithelium and which the ranking puts below the pale ones. Separately, the whole category is missing what defines it: stromal nuclear density is 1060-1300/mm2 across the nine a1 gastritis pass-2 cells against a real held-out 4588/mm2, i.e. the inflammatory infiltrate is largely absent (C1 failure mode 3, unchanged by this sweep).
• Two structural problems with the list itself: IM ranks 6 and 7 are byte-identical (section 1.1), and gastritis rank 8 is a b1 canvas at D 11.32 that is on the front only because its copy margin (0.239) is the best in the category — the third objective can promote a visibly broken image. A results page quoting this list should drop the duplicate and say why rank 8 is there.

4. What I would change next, on this evidence

1. The cartoon does not survive pass 2. Any further work on cartoon realism (E2's whole premise) is wasted; the ladder is flat and the outputs are indistinguishable across it. The lever is the pass-2 conditioning, not the cartoon. F2's structure adapter, which conditions on the label map directly rather than on a rendered RGB reference, is the arm that actually attacks this.
2. si3 (17 steps) beats si6 on topology, envelope and detail in all four categories, and si9 and pass-3-si15 should be dropped (si9 is worse than si6 on the envelope in all three gastritis cuts, and pass 3 is negative on every axis). But si3's advantage comes from abandoning the cartoon (layout IoU 9% of ceiling), so it is a better image generator and a worse layout generator. Pick which one the programme wants before optimising further.
3. Diversity needs to become a first-class metric. Real held-out fields are mutually uncorrelated (SSIM 0.035); every arm here is 0.48-0.89. The Pareto selection has no diversity objective and will happily return five copies of one image (a3 nearmean IM is exactly that).
4. The gastritis selection objective is being gamed and needs a cellularity or pale-fraction guard before any gastritis candidate goes to review.
5. Retrieve the VM cartoons (~300 MB) so the layout IoUs are measured against the exact bytes the generator saw.

5. Caveats carried into every number above

1. Not clinical, not a review. No pathologist; Yash has not reviewed a sheet.
2. Envelope percentile is saturated: 208 of 249 pass-2/pass-3 outputs sit at pct = 100. Use D.
3. The envelope calibration is thin (C1 limitation 2): 4-6 real held-out fields per category from 2-3 slides, so percentiles quantise to 25% (normal) / 16.7% (the rest), and the gastritis band is wide because one held-out field is atypical — "100% of gastritis si3 in band" is a weaker statement than the same phrase for IM.
4. ring_with_lumen_fraction does not separate the four categories on real tissue (C1 section 6, eta2 = 0.007, p = 0.89). It is a plausibility gate, never a target — and section 3(j) shows what happens when a selection objective treats it as one.
5. The pixel NCC is targeted, not exhaustive, and is a screen rather than a proof.
6. d1's layout IoU uses the LOCAL Mac cartoons, not the VM cartoons the sweep consumed (section 3c). The lumen labels are essentially identical between the two renders; the epithelium labels are less safe.
7. str_nuc_density_per_mm2 is a noisy residual mask (C1 limitation 5) — comparable across arms because every image is measured identically, not a real stroma segmentation.
8. Detail statistics cannot certify 0.25 um/px content (C1 section 7): a half-resolution round trip moves neither the ring metric nor the envelope. A higher Laplacian means "more of something", not "genuine sub-micron detail".
9. b1's 12 native windows analyse at 0.25 um/px, not the envelope's 0.6667, and are excluded from every envelope table; their detail statistics are not comparable with the canvas numbers.
10. B1 is not organisms (B1_RESULT.md: 0 of 60 candidates graded likely organisms).
11. Category and donor stay confounded for a1, d1, e2 and d2 by design; only a2 and a3 break it, and probe B cannot attribute a donor for those arms (0.000 by construction), so their structure-inheritance row is blank rather than zero.
12. Small n throughout: 5 per (category, cut, arm) cell, 5-6 real held-out fields per category, 5 per d2/e2 rung. There is no significance testing anywhere in this package and there is not enough data for any.
13. packages/sweep-v1/eval-standin*/ are plumbing fixtures with fabricated provenance and must never be quoted; only packages/sweep-v1/eval/ is this run.

Download public Markdown export