# A1-V2 result Work package A1-V2 of the SGH synthetic gastric histology programme (`operations/research/sgh-program-20260908/PLAN.md`), 8 September 2026. CPU only. Nothing was committed to git. Engineering evidence only. Nothing here is a clinical claim, and no number says a cartoon or a generated field is diagnostically valid. Every morphometric below is an image-only measurement produced by `code/cartoon_metrics.py` -- the same code path, at full 4096 x 2048 / 0.25 um/px, for real HiESD fields and for cartoons. No half-scale readings were used at any point (`ring_topology` reads `micrometres_per_pixel` from `sydney_qa_v1.json`, not from the image, so a 0.5 um/px render is scored with 4x-too-small pixel areas; A1_RESULT.md section 2 records that trap). **Headline.** `cartoons/a1v2/` (60 cartoons) and `cartoons/d1v2/` (10) replace A1's pit-sized rosettes with gland-calibre profiles cut through the gland zone. Scored on the same six metrics against the same real 10-90% bands, **68 of 72 (category x cut x metric) cells are inside the band against a1's 50**, and on A1's own five-metric per-cartoon score the set moves from 3.20/5 (deviation 1.643) to **4.60/5 (deviation 0.083)**. `ring_with_lumen_fraction` on the cartoon is **0.84-0.86 on `across` and 0.81-0.91 on `oblique`** in all four categories (target 0.8; a1 was 0.48-0.88 on `across`), and `along` is 0.58-0.71, which misses it. `stroma_fraction` is now in band for every category and cut, where a1's normal was 0.30-0.31 against a real 0.065-0.244. The `across` and `oblique` ring density and median ring area cells that A1 listed as its top two gaps are all inside the band. Whether the generator keeps the lumens is a GPU question this package could not answer; `jobs/a1v2-pass1.json`, `jobs/a1v2-pass2.json` and `jobs/a1v2-pass2-si3.json` are the experiment that settles it. --- ## 1. The problem this package was given A1 put the cartoons inside the real morphometric envelope on nuclear density, stromal nuclear density and stroma fraction, and the first two-pass output from them (`packages/sweep-v1/early/`, intestinal metaplasia seed 11 `across`) is measurably closer to real tissue than the old set (morphometric distance 5.6 against 6.5-8.6). But its `ring_with_lumen_fraction` collapsed to **0.20** in pass 2 (0.55 after pass 1; the old set reaches 0.83-0.98 and real held-out IM sits at 0.71). The cartoon it came from is `packages/sweep-v1/early/early-sheet.jpg`, top strip: many small rosettes about 25-35 um across -- pit calibre -- each fragmented into radial mucin wedges, at 227-252 rings/mm2 against a real p90 of 143-174. Real IM gland profiles are 74-122 um in outer diameter with 15-50 um lumens. The generator did not read those rosettes as glands, so the lumens vanished in pass 2. A1_RESULT.md already named the residual gap (its section 7, items 1-3): `across` and `oblique` ring density above band in every category, `oblique` median ring area 669-1019 um2 against a real p90 of 480-582, and `stroma_fraction` 0.30-0.31 for normal against a real 0.065-0.244. ## 2. Why A1's cartoons could not have both gland calibre and an in-band ring area `ring_topology` thresholds hematoxylin at the 72nd percentile **within the image**, closes that mask with a 6-um disk, and calls every enclosed gap a ring. What decides the ring is therefore not the gland but where the darkest 28% of the tissue sits. In a real field the epithelial cytoplasm carries chromatin/RNA granularity at a few micrometres, so the 6-um closing seals the whole gland wall and the only surviving hole is the pale lumen. Measured on the same code path: a real IM held-out field has a smooth hematoxylin distribution (p50 0.194, p72 = the threshold = 0.300, p90 0.657) and its closed band covers essentially the whole field, leaving small isolated holes. An A1 cartoon of the same category is bimodal -- p50 0.311, threshold 0.449, then a jump to 0.99 at p80 -- so the threshold falls in the gap and the mask is the geometric nuclei and nothing else. In an A1 cartoon the wall is smooth, so the hole is *everything inside the basal nuclear palisade*: lumen plus the entire cytoplasmic wall. Hole radius is then `R - nuc_basal_frac x wall - nucleus semi-axis`. Solving that for the real median ring area of about 400 um2 (hole radius 11.3 um) with a basal nucleus gives an outer gland diameter of about 33 um. **That is the whole story of A1's pit-sized rosettes**: with a smooth cytoplasm, an in-band `median_ring_area_um2` and a gland-sized profile are mutually exclusive, and A1 chose the metric. A1-V2 breaks the coupling by giving the epithelial cytoplasm dark granules at 5.8 um spacing (`style="v2"`, see section 3.2). The closing bridges them, the wall joins the band, the ring becomes the lumen, and gland calibre and ring area become independent knobs. ## 3. What was built Everything is **additive**. `cartoons/a1`, `cartoons/d1`, `cartoons/e2` and `cartoons/b1` render byte-for-byte what A1 shipped; the checks are in section 7. ### 3.1 `code/tissue3d.py` -- new `TissueParams` fields, all defaulting to A1's behaviour | field | default (= A1) | what it does | |---|---|---| | `wall_max_frac` | 0.62 | cap on wall thickness as a fraction of the outer radius; A1 hard-coded 0.62, which forbids a wide gland with a narrow lumen (an oxyntic gland is exactly that). v2 uses 0.78 | | `across_depth_frac` | (0.42, 0.60) | centre of the `deep` (`across`) plane, as a fraction of `slab_depth`. A1 hard-coded one range for every category, so a preset whose pit reaches 0.62 of the mucosa (gastritis) was cut **through the pit zone** | | `oblique_frame` / `oblique_depth_frac` | `"upper"` / (0.20, 0.30) | new frame policy `"mid"`: the top edge is pinned inside the mucosa instead of just above the surface, so an oblique cut runs neck-to-base with no surface and no background in frame | | `across_tilt_deg` / `oblique_tilt_deg` / `along_tilt_deg` | (4, 16) / (38, 56) / (82, 88.5) | per-cut tilt ranges, previously hard-coded in `plane_basis` | | `mucin_min_gap` | 0.0 | minimum radial clearance between a goblet vacuole and the lumen. A1's vacuole centre sat at `rl + 0.38 x wall` with a 6-um radius and `rl` about 7 um, so every vacuole punched into the lumen and every profile came out as a rosette | | `stroma_lattice_jitter` | 0.40 | jitter of the 3-D lattice the lamina-propria nuclei sit on | `slice()` gained a `depth_frac=(lo, hi)` override and a `frame_for(cut, frame)` helper. `TissueParams.to_json()` omits any of these fields while it still holds its A1-equivalent default (`V2_FIELD_DEFAULTS`), so an a1/d1/e2/b1 sidecar stays byte-identical and `render_set.py --verify` is clean; a v2 preset changes them, so they are recorded exactly where they do something. ### 3.2 `code/render_cartoon.py` -- `render(..., style="v1")` `style` selects a block from `STYLES`; `"v1"` is A1's renderer unchanged and is the default, so every existing caller gets identical bytes. The shipped `"v2"` block: | knob | v1 | v2 | why | |---|---|---|---| | `granule_spacing_um` / `granule_radius_um` / `granule_gain` | off | 5.8 / 0.92 / 1.00 | dark chromatin granules on a jittered lattice inside epithelial cytoplasm only (never lumen or goblet mucin). Colour (60, 31, 102), h = 2.1, above the 0.63-1.31 threshold these fields actually produce. Radius 0.92 um keeps a granule under `minimum_nucleus_area_um2` = 5 um2, so granules are not counted as nuclei | | `granule_apical_bias` | - | 0.45 | thins the granules towards the basal pole, where the nuclear palisade already closes the wall | | `granule_warp` | - | 0.85 | smooth warp of the lattice; without it the dots read as a halftone screen across the whole field | | `apical_pale_frac` | 0 | 0.26 | pale apical mucin cap on foveolar/neck cells. Free for the ring metric (the granules keep that band inside the closed band, so the *hole* is still the lumen) but it is what lifts `lumen_tissue_ratio` off the gland lumens alone | | `basophilia_gain` | 0.34 | 0.40 | | | `speckle_gain_epi` / `speckle_gain_stroma` | 0.46 / 0.50 | 0.22 / 0.30 | with the granules doing the closing, the broad speckle no longer contributes to the band (it sits at h 0.38, far below the threshold) and only greyed the image | | `basement_gain` | 0.45 | 0.80 | stronger basement-membrane rim, so the gland boundary is legible | | `fibre_gain` / `fibre_dark` | 0.46 / 0 | 0.80 / 1.0 | darker collagen strokes in the lamina propria | | `granule_lighten` | 0 | 0 | tried and rejected; see section 6 | The 9-label vocabulary is unchanged. Granules, the apical pale cap and the fibre strokes are **colour only**: they never write a label, so the label PNG a structure adapter (F2) reads is the same 9 classes A1 emits. ### 3.3 `code/render_cartoon.py` -- the `*_v2` presets `CATEGORY_PARAMS` gains `normal_body_v2`, `normal_antrum_v2`, `hpylori_gastritis_v2`, `intestinal_metaplasia_v2` and `mixed_v2`. `resolve_category` maps `normal_v2` to body/antrum by the same seed parity and reports the canonical category, so jobs, donors and output directories stay on PLAN.md's four categories. `CANONICAL_V2` lists the family. | parameter | normal_body | normal_antrum | hpylori_gastritis | intestinal_metaplasia | mixed | |---|---|---|---|---|---| | `pitch` | 72 | 78 | 72 | 92 | 88 | | `outer_diam` (um) | 44-60 | 46-66 | 52-74 | 58-84 | 56-80 | | `wall_thickness` (um) | 14-22 | 16-24 | 18-25 | 20-29 | 19-28 | | `wall_max_frac` | 0.78 | 0.78 | 0.78 | 0.78 | 0.78 | | `pit_depth_frac` | 0.22 | 0.42 | 0.50 | 0.28 | 0.38 | | `pit_lumen_scale` | 1.55 | 1.30 | 1.30 | 1.25 | 1.28 | | `gland_lumen_scale` | 1.0 | 1.0 | 0.80 | 0.86 | 0.86 | | `base_lumen_pinch` | 0.90 | 0.88 | 0.90 | 0.88 | 0.88 | | `radius_wave` | 0.05 | 0.22 | 0.26 | 0.28 | 0.26 | | `across_depth_frac` | 0.38-0.62 | 0.52-0.72 | 0.58-0.72 | 0.40-0.62 | 0.48-0.68 | | `oblique_depth_frac` | 0.20-0.32 | 0.20-0.32 | 0.20-0.32 | 0.20-0.32 | 0.20-0.32 | | `goblet_frac` | 0 | 0 | 0 | 0.36 | 0.30 | | `mucin_radius` / `mucin_min_gap` (um) | - | - | - | 3.7-5.1 / 4.5 | 3.7-5.1 / 4.5 | | `nuc_semi_radial` | 3.00 | 3.00 | 3.00 | 3.10 | 3.05 | | `nuc_basal_frac` | 0.30 | 0.26 | 0.26 | 0.22 | 0.24 | | `stroma_nuclei_per_mm2` | 12500 | 10500 | 17000 | 10500 | 15000 | | `stroma_nuc_radius` (um) | 2.35-3.10 | 2.25-2.95 | 2.45-3.20 | 2.25-2.95 | 2.45-3.20 | | `follicles_per_mm2` / `follicle_radius` | - | - | 0.9 / 60-130 | - | 0.8 / 60-125 | Shared (`_COMMON_V2`): `across_tilt_deg` (11, 20), `oblique_tilt_deg` (24, 36), `along_tilt_deg` (72, 84), `oblique_frame` `"mid"`, `stroma_lattice_jitter` 0.08, `mm_nuc_radius` (2.3, 3.1), `mm_nuclei_per_mm2` 19000, `pit_depth_jitter` (0.80, 1.16). Surface band, follicles, infiltrate, muscularis, cell kinds, the fidelity ladder and the 9-label vocabulary are all carried over from A1 unchanged. ### 3.4 `code/render_set.py` Two new sets, `a1v2` and `d1v2`, added to `SETS`; `build()` gained a `style` keyword and only writes `render_style` into the sidecar when it is not `"v1"`. ## 4. Tuning: every iteration Each iteration is a full re-render at 4096 x 2048 / 0.25 um/px of the five v2 presets x the listed seeds x three cuts, scored against the category's real 10-90% band on the five tuned metrics (`ring_density_per_mm2`, `median_ring_area_um2`, `lumen_tissue_ratio`, `nuclear_density_per_mm2`, `stroma_fraction`). "deviation" is the total distance outside the band, in band-widths, summed over the five metrics; 0 means every metric is inside. | iter | seeds | change | inside | deviation | |---|---|---|---|---| | 1 | 11 | first v2 presets: gland calibres 44-84 um, pitch 60-92, per-preset `across_depth_frac` below each preset's own pit, `oblique_frame="mid"`, granules (6.6 um / r 1.20 / colour 116,74,156), `mucin_min_gap` 1.6 | 3.67 / 5 | 3.403 | | 2 | 11 | granules darker (78,40,122) and smaller (6.4 / 0.98); normal pitch 60/66 -> 76/78 | 3.73 / 5 | 2.171 | | 3 | 11 | granules at full strength (no basal bias), 8.0 / 1.35; `mucin_min_gap` 4.5 | 3.87 / 5 | 1.747 | | 4 | 11 | **the granule block moved to after the speckle, apical shading, cell borders and basement rim.** Each of those blends a lighter colour over the cytoplasm and was washing the granule from h 1.83 to about 1.2, below the threshold of a nucleus-dense field | 4.00 / 5 | 0.484 | | 5 | 11 | stromal nuclei bigger; granules finer (5.8 / 0.92) | 4.00 / 5 | 0.514 | | 6 | 11, 12 | `stroma_lattice_jitter` 0.40 -> 0.14; pale apical mucin cap; `along_tilt_deg` (72, 84); `radius_wave` up; calibre trim | 4.37 / 5 | 0.187 | | 7 | 11, 12 | granules darker still (60,31,102) so they outrank stromal nuclei whatever the preset's cellularity; `apical_pale_frac` 0.26; warped granule lattice. (A preset edit in this iteration failed its assertion, so the run is iteration 6's presets with the render changes) | 4.43 / 5 | 0.236 | | 8 | 11-13 | v1 presets restored after an earlier non-unique string replace had edited them (section 8); v2 follicle / muscularis / stromal nucleus sizes, `stroma_lattice_jitter` 0.08, `oblique_tilt_deg` (26, 40), `gland_lumen_scale` | 4.67 / 5 | 0.147 | | 9 | 11, 12 | visual pass: bigger epithelial nuclei, fewer granules (6.6 / 0.86 + 14% dropout), softer speckle, stronger basement rim, narrower stromal nuclei | **3.37 / 5** | **1.393** | | 10 | 11-13 | reverted the two changes that broke the seal (granule thinning, narrow stromal nuclei); kept the bigger epithelial nuclei, the softer speckle, the basement rim and the warp | 4.42 / 5 | 0.132 | | 11 | 11-13 | dark collagen strokes in the lamina propria; stromal nuclei roughly halved (12500-17000/mm2, `stroma_nucleus` label fraction 0.30 -> 0.22); `oblique_tilt_deg` (24, 36); final calibre trim | 4.67 / 5 | 0.070 | | 12 (shipped) | 11-15 | goblet vacuoles smaller and pushed further from the lumen; `goblet_frac` 0.42 -> 0.36 (IM), 0.36 -> 0.30 (mixed) | **4.60 / 5** (60 cartoons) | **0.083** | The shipped row is the whole a1v2 set (60 cartoons), so it is scored on 4-5x more material than rows 1-11 and is not exactly comparable to them. The A1 set on the same scoring is 3.20 / 5 at deviation 1.643. ### The four changes that did the work 1. **Epithelial chromatin granules, applied last.** Dark dots at 5.8 um in epithelial cytoplasm only. The 6-um closing bridges them, so the wall joins the closed band and the ring becomes the lumen. Iteration 4 -- moving the block after every later lightening pass -- is the single largest step in the package (deviation 1.747 -> 0.484); before that the granules were being washed out and only the sparse-stroma presets sealed. 2. **Cut planes placed below each preset's own pit zone.** `across_depth_frac` per preset and `oblique_frame="mid"`. A1 cut `across` at 0.42-0.60 of the mucosa for every category; gastritis' pit reaches 0.47-0.81 of it (`pit_depth_frac` 0.62 x `pit_depth_jitter` 0.75-1.30), so gastritis `across` was a pit-zone cut in every seed. As a side effect the v2 `oblique` frame carries no background at all (`background_fraction` 0.000-0.002 against A1's 0.036-0.045, and real 0.000). 3. **`mucin_min_gap`.** Goblet vacuoles are discrete pale ovals inside the wall instead of radial spokes fused into the lumen, so each profile has one connected pale interior. 4. **A more regular lamina propria** (`stroma_lattice_jitter` 0.40 -> 0.08) rather than a denser one. The 3-D lattice is thinned by the section, so at jitter 0.40 it leaves 200-500 um2 voids between glands that the ring metric counts as **non-pale** rings. That was the whole of the residual `ring_with_lumen_fraction` deficit: at iteration 4, 39 of 96 rings in a `normal_body across` field were lamina-propria voids. Regularising the lattice removed them, and let the stromal nuclear density come back down from 15000-21000 to 10500-17000 per mm2. ### Two things that were tried and reverted * **Lightening the granules** (`granule_lighten`, kept in the code at 0 with the measured result in a comment). A lighter granule would leave the epithelial nuclei visibly the darkest thing in the wall, which is what the "row of beads" reading needs. At `granule_lighten=1.0` the wall stops closing: `normal_body across` `ring_with_lumen_fraction` falls 0.85 -> 0.63 and `along` to 0.23, and mixed `oblique` `median_ring_area_um2` rises to 700. Rejected. * **Fewer, rounder stromal nuclei** (iteration 9). Elongated nuclei cover more ground per object; narrowing `stroma_nuc_aspect` to (1.7, 2.8) re-opened the lamina-propria voids (`normal_body across` ring density 152 -> 225, `ring_with_lumen_fraction` 0.89 -> 0.40). Reverted. --- ## 5. Where the shipped cartoons land All tables below: median over the 5 seeds of that (category, cut) cell; **bold** = outside the real 10-90% band. Real bands are the combined train + held-out envelope (`|all`, 28-30 fields per category) from `morphometrics/real-envelope.json`, the same band A1 tuned against. Per-cartoon rows: `morphometrics/a1v2-metrics.csv`. ### Real envelope: median (10-90%), train + held-out | metric | normal | hpylori_gastritis | intestinal_metaplasia | mixed | |---|---|---|---|---| | `ring_density_per_mm2` | 87 (39-160) | 93 (49-161) | 120 (77-174) | 115 (76-143) | | `median_ring_area_um2` | 392 (313-520) | 399 (320-489) | 389 (315-481) | 417 (315-582) | | `lumen_tissue_ratio` | 0.124 (0.036-0.261) | 0.145 (0.056-0.293) | 0.183 (0.057-0.373) | 0.131 (0.052-0.319) | | `ring_with_lumen_fraction` | 0.876 (0.380-1.000) | 0.765 (0.294-1.000) | 0.745 (0.289-1.000) | 0.775 (0.182-1.000) | | `nuclear_density_per_mm2` | 12293 (9318-18854) | 10569 (9016-14673) | 11679 (9132-14990) | 11012 (8755-13757) | | `stroma_fraction` | 0.129 (0.065-0.244) | 0.172 (0.087-0.293) | 0.197 (0.118-0.332) | 0.161 (0.098-0.311) | ### `ring_density_per_mm2` -- median over 5 seeds; **bold** = outside the real 10-90% band | category | cut | real 10-90% | a1 | a1v2 | |---|---|---|---|---| | normal | across | 39-160 (med 87) | **315** | 135 | | normal | along | 39-160 (med 87) | 98 | 82 | | normal | oblique | 39-160 (med 87) | **230** | 138 | | hpylori_gastritis | across | 49-161 (med 93) | **247** | 147 | | hpylori_gastritis | along | 49-161 (med 93) | 93 | 74 | | hpylori_gastritis | oblique | 49-161 (med 93) | **172** | 133 | | intestinal_metaplasia | across | 77-174 (med 120) | **250** | 141 | | intestinal_metaplasia | along | 77-174 (med 120) | 105 | **74** | | intestinal_metaplasia | oblique | 77-174 (med 120) | 170 | 135 | | mixed | across | 76-143 (med 115) | **246** | 125 | | mixed | along | 76-143 (med 115) | 85 | **72** | | mixed | oblique | 76-143 (med 115) | **176** | 126 | ### `median_ring_area_um2` -- median over 5 seeds; **bold** = outside the real 10-90% band | category | cut | real 10-90% | a1 | a1v2 | |---|---|---|---|---| | normal | across | 313-520 (med 392) | **644** | 373 | | normal | along | 313-520 (med 392) | 377 | 434 | | normal | oblique | 313-520 (med 392) | **831** | 408 | | hpylori_gastritis | across | 320-489 (med 399) | **558** | 394 | | hpylori_gastritis | along | 320-489 (med 399) | **718** | **583** | | hpylori_gastritis | oblique | 320-489 (med 399) | **669** | **503** | | intestinal_metaplasia | across | 315-481 (med 389) | **787** | 396 | | intestinal_metaplasia | along | 315-481 (med 389) | 381 | 351 | | intestinal_metaplasia | oblique | 315-481 (med 389) | **1019** | 392 | | mixed | across | 315-582 (med 417) | **792** | 455 | | mixed | along | 315-582 (med 417) | **597** | 411 | | mixed | oblique | 315-582 (med 417) | **947** | 479 | ### `lumen_tissue_ratio` -- median over 5 seeds; **bold** = outside the real 10-90% band | category | cut | real 10-90% | a1 | a1v2 | |---|---|---|---|---| | normal | across | 0.036-0.261 (med 0.124) | 0.088 | 0.049 | | normal | along | 0.036-0.261 (med 0.124) | 0.107 | 0.074 | | normal | oblique | 0.036-0.261 (med 0.124) | 0.138 | 0.054 | | hpylori_gastritis | across | 0.056-0.293 (med 0.145) | 0.077 | 0.056 | | hpylori_gastritis | along | 0.056-0.293 (med 0.145) | 0.094 | 0.111 | | hpylori_gastritis | oblique | 0.056-0.293 (med 0.145) | 0.129 | 0.078 | | intestinal_metaplasia | across | 0.057-0.373 (med 0.183) | 0.127 | 0.064 | | intestinal_metaplasia | along | 0.057-0.373 (med 0.183) | 0.107 | 0.077 | | intestinal_metaplasia | oblique | 0.057-0.373 (med 0.183) | 0.140 | 0.065 | | mixed | across | 0.052-0.319 (med 0.131) | 0.098 | 0.058 | | mixed | along | 0.052-0.319 (med 0.131) | 0.098 | 0.076 | | mixed | oblique | 0.052-0.319 (med 0.131) | 0.137 | 0.062 | ### `ring_with_lumen_fraction` -- median over 5 seeds; **bold** = outside the real 10-90% band | category | cut | real 10-90% | a1 | a1v2 | |---|---|---|---|---| | normal | across | 0.380-1.000 (med 0.876) | 0.477 | 0.844 | | normal | along | 0.380-1.000 (med 0.876) | **0.232** | 0.622 | | normal | oblique | 0.380-1.000 (med 0.876) | 0.688 | 0.889 | | hpylori_gastritis | across | 0.294-1.000 (med 0.765) | 0.736 | 0.855 | | hpylori_gastritis | along | 0.294-1.000 (med 0.765) | 0.600 | 0.714 | | hpylori_gastritis | oblique | 0.294-1.000 (med 0.765) | 0.812 | 0.914 | | intestinal_metaplasia | across | 0.289-1.000 (med 0.745) | 0.881 | 0.836 | | intestinal_metaplasia | along | 0.289-1.000 (med 0.745) | **0.222** | 0.583 | | intestinal_metaplasia | oblique | 0.289-1.000 (med 0.745) | 0.913 | 0.826 | | mixed | across | 0.182-1.000 (med 0.775) | 0.689 | 0.862 | | mixed | along | 0.182-1.000 (med 0.775) | 0.412 | 0.618 | | mixed | oblique | 0.182-1.000 (med 0.775) | 0.824 | 0.812 | ### `nuclear_density_per_mm2` -- median over 5 seeds; **bold** = outside the real 10-90% band | category | cut | real 10-90% | a1 | a1v2 | |---|---|---|---|---| | normal | across | 9318-18854 (med 12293) | 10034 | 9617 | | normal | along | 9318-18854 (med 12293) | 11812 | 11709 | | normal | oblique | 9318-18854 (med 12293) | 10101 | 9957 | | hpylori_gastritis | across | 9016-14673 (med 10569) | 12754 | 10029 | | hpylori_gastritis | along | 9016-14673 (med 10569) | 14228 | 11718 | | hpylori_gastritis | oblique | 9016-14673 (med 10569) | 10807 | 10448 | | intestinal_metaplasia | across | 9132-14990 (med 11679) | 10053 | 9532 | | intestinal_metaplasia | along | 9132-14990 (med 11679) | 12660 | 11436 | | intestinal_metaplasia | oblique | 9132-14990 (med 11679) | 10395 | 9446 | | mixed | across | 8755-13757 (med 11012) | 10744 | 9680 | | mixed | along | 8755-13757 (med 11012) | 13735 | 11469 | | mixed | oblique | 8755-13757 (med 11012) | 10155 | 9906 | ### `stroma_fraction` -- median over 5 seeds; **bold** = outside the real 10-90% band | category | cut | real 10-90% | a1 | a1v2 | |---|---|---|---|---| | normal | across | 0.065-0.244 (med 0.129) | **0.302** | 0.103 | | normal | along | 0.065-0.244 (med 0.129) | **0.308** | 0.132 | | normal | oblique | 0.065-0.244 (med 0.129) | **0.313** | 0.116 | | hpylori_gastritis | across | 0.087-0.293 (med 0.172) | 0.197 | 0.110 | | hpylori_gastritis | along | 0.087-0.293 (med 0.172) | 0.217 | 0.152 | | hpylori_gastritis | oblique | 0.087-0.293 (med 0.172) | 0.255 | 0.125 | | intestinal_metaplasia | across | 0.118-0.332 (med 0.197) | 0.300 | 0.122 | | intestinal_metaplasia | along | 0.118-0.332 (med 0.197) | 0.310 | 0.131 | | intestinal_metaplasia | oblique | 0.118-0.332 (med 0.197) | 0.304 | 0.121 | | mixed | across | 0.098-0.311 (med 0.161) | 0.281 | 0.117 | | mixed | along | 0.098-0.311 (med 0.161) | 0.280 | 0.126 | | mixed | oblique | 0.098-0.311 (med 0.161) | 0.308 | 0.123 | ### Cells inside the band, per set | set | cells inside / 72 (4 categories x 3 cuts x 6 metrics) | |---|---| | a1 | 50 / 72 | | **a1v2** | **68 / 72** | The four cells a1v2 misses are: `intestinal_metaplasia` and `mixed` `along` ring density (74 and 72 against p10 77 and 76), and `hpylori_gastritis` `along` and `oblique` median ring area (583 and 503 against p90 489). Section 6 says why. Scored per cartoon on A1's five tuned metrics, the way A1_RESULT.md section 6 reports it: | | n | inside | deviation (band-widths outside, summed over 5 metrics) | |---|---|---|---| | a1 | 60 | 3.20 / 5 | 1.643 | | **a1v2** | **60** | **4.60 / 5** | **0.083** | | a1v2 normal (body + antrum) | 15 | 4.80 / 5 | 0.019 | | a1v2 hpylori_gastritis | 15 | 4.40 / 5 | 0.265 | | a1v2 intestinal_metaplasia | 15 | 4.53 / 5 | 0.012 | | a1v2 mixed | 15 | 4.67 / 5 | 0.038 | | a1v2 `across` | 20 | 4.65 / 5 | 0.009 | | a1v2 `along` | 20 | 4.45 / 5 | 0.186 | | a1v2 `oblique` | 20 | 4.70 / 5 | 0.055 | ### The ring-size distribution, which is the point Pooled over 5-6 fields per row, same code path, full scale. This is what "the ring is now the lumen" buys: a1's rings are one big hole per gland and nothing small; a1v2's distribution is close to real, which is a mixture of small inter-nuclear gaps and a tail of gland lumens. | ring area | real IM held-out (n=365) | a1 IM across (n=639) | a1v2 IM across (n=362) | |---|---|---|---| | 200-300 um2 | 34.5% | 3.9% | 32.9% | | 300-500 um2 | 27.4% | 17.2% | 32.9% | | 500-1000 um2 | 23.8% | 48.0% | 22.9% | | 1000-2000 um2 | 9.9% | 24.1% | 10.8% | | 2000-5000 um2 | 3.3% | 6.6% | 0.6% | | > 5000 um2 | 1.1% | 0.2% | 0.0% | | **median** | **408 um2** | **791 um2** | **397 um2** | | pooled `ring_with_lumen_fraction` | 0.732 | 0.862 | 0.831 | ### Geometry and label composition of the shipped set Medians over 5 seeds. `background_fraction` is 0.000-0.002 on `across` and `oblique` (A1: 0.001 on `across`, 0.036-0.045 on `oblique`) because the v2 `oblique` frame is inside the mucosa; `along` still carries the surface and 0.031-0.040 of background by design. | category | cut | tilt (deg) | depth z (um) | tube profiles | lumen | epi cyt | mucin | epi nuc | stroma | stroma nuc | |---|---|---|---|---|---|---|---|---|---|---| | normal | across | 12-19 | 186..388 | 85 | 0.038 | 0.293 | 0.000 | 0.041 | 0.377 | 0.246 | | normal | along | 72-80 | -26..484 | 18 | 0.053 | 0.216 | 0.008 | 0.038 | 0.348 | 0.299 | | normal | oblique | 25-36 | 108..409 | 94 | 0.047 | 0.260 | 0.000 | 0.038 | 0.391 | 0.275 | | hpylori_gastritis | across | 12-19 | 205..392 | 63 | 0.046 | 0.359 | 0.000 | 0.050 | 0.300 | 0.238 | | hpylori_gastritis | along | 72-80 | -36..473 | 32 | 0.058 | 0.223 | 0.027 | 0.040 | 0.310 | 0.296 | | hpylori_gastritis | oblique | 25-36 | 108..409 | 65 | 0.069 | 0.276 | 0.000 | 0.044 | 0.336 | 0.276 | | intestinal_metaplasia | across | 12-19 | 146..342 | 66 | 0.040 | 0.311 | 0.043 | 0.035 | 0.350 | 0.222 | | intestinal_metaplasia | along | 72-80 | -26..483 | 12 | 0.041 | 0.219 | 0.034 | 0.030 | 0.354 | 0.291 | | intestinal_metaplasia | oblique | 25-36 | 108..409 | 53 | 0.040 | 0.300 | 0.041 | 0.035 | 0.355 | 0.225 | | mixed | across | 12-19 | 176..370 | 59 | 0.034 | 0.295 | 0.031 | 0.036 | 0.339 | 0.255 | | mixed | along | 72-80 | -31..478 | 13 | 0.039 | 0.203 | 0.032 | 0.032 | 0.341 | 0.311 | | mixed | oblique | 25-36 | 108..409 | 56 | 0.045 | 0.260 | 0.027 | 0.035 | 0.351 | 0.269 | For comparison, F2's segmentation of the real held-out fields (`pairs/real_label_fractions.json`, one field per category): lumen 0.041-0.094, epithelium_cytoplasm 0.143-0.229, goblet_mucin 0.004-0.013, epithelial_nucleus 0.029-0.055, stroma 0.373-0.657, stroma_nucleus 0.080-0.119. The two composition gaps that remain are recorded in section 6. --- ## 6. Looking at it: `cartoons/a1v2-contact-sheet.png` Per category: real held-out crop | a1 across | a1v2 across | a1v2 along | a1v2 oblique. Every cell is the identical 1024 x 512 px window (256 x 128 um at 0.25 um/px) from the centre of the field, downscaled 2x, so the scale is matched across all five columns. A second figure, `cartoons/a1v2-gland-detail.png`, shows real | a1 across | a1v2 across at 1:1 (300 x 170 um, no downscaling), because the thing that changed is 5-um detail and the contact sheet cannot resolve it. **Do the v2 rings read as glands -- one row of beads around one connected pale interior?** Honestly: **partly, and much more than a1's do.** What is clearly better, and is the defect the brief named: * Each v2 profile has **one** connected pale interior. The a1 rosettes are a central pale blob with radial mucin wedges cut into it; at 1:1 the v2 intestinal-metaplasia profile is a single pale oval lumen, then a ring of discrete pale goblet vacuoles sitting **in** the wall (not fused with the lumen), then the wall, then the basement rim. That is what `mucin_min_gap` bought and it is visible without measuring anything. * The lumens are gland-sized and there are far fewer of them: 59-94 tube profiles in the field on `across` against a1's 122-172, at outer diameters of 44-84 um rather than 31-50. Side by side in `a1v2-gland-detail.png`, the a1 panel is a scatter of 10-20 um white specks with no gland outline; the a1v2 panel has discrete round profiles you can point at. * The gland boundary is legible: the basement rim plus the granulated wall against the paler lamina propria draws an outline that a1 does not have. What is **not** there yet: * At the contact-sheet scale the epithelial nuclei do not read as a distinct row of beads. They are present -- at 1:1 the palisade is a ring of purple ellipses outside the goblet ring -- but the granules are darker than the nuclei (that is what makes the wall close), so the wall reads as an evenly stippled halo rather than as a row of nuclei. Lightening the granules fixes the look and breaks the metric (section 4). * The lamina propria is far more cellular than real tissue: `stroma_nucleus` is 0.22-0.31 of the field against F2's 0.080-0.119 on real held-out fields. It reads as a dense field of dark rods, and it competes with the epithelium for attention. Reducing it below the densities in section 3.3 re-opens the lamina-propria voids and the ring metric collapses (one seed, `hpylori_gastritis across`: `ring_with_lumen_fraction` 0.91 at 17000/mm2 falls to 0.56 at 13000/mm2, and `stroma_fraction` rises 0.11 -> 0.24). This is the sharpest unresolved trade-off in the package. * The granule lattice is warped but still slightly periodic in the flattest regions of a wall, and the epithelial cytoplasm as a whole is more purple than real gastric epithelium. Compared against the real column, the v2 cartoons remain obviously cartoons -- as they should be; they are conditioning inputs, not outputs. The question this package had to answer is whether a generator can find a gland in them, and on the visual evidence a v2 `across` profile is a much more findable gland than an a1 one. **Whether that survives the two-pass generator is a GPU question this package could not run** (CPU only): the `ring_with_lumen_fraction` 0.20 collapse is a property of the pass-2 output, not of the cartoon, and only `jobs/a1v2-pass2.json` can settle it. ## 7. What could not be reconciled 1. **`ring_with_lumen_fraction` on `along` cuts is 0.58-0.71, below the 0.8 the brief asks for.** `across` (0.84-0.86) and `oblique` (0.81-0.91) meet it in every category; a1 was 0.48-0.88 on `across` and 0.22-0.60 on `along`. An `along` frame is 55-65% lamina propria with only 12-32 gland profiles in it, so the non-pale lamina-propria voids that section 4 describes are a much larger share of the ring count there. Every lever that closes them further (denser or larger stromal nuclei) pushes `nuclear_density_per_mm2` and the stromal nucleus label fraction further from real. 2. **`hpylori_gastritis` median ring area is above band on `along` (583) and `oblique` (503) against a p90 of 489.** The gastritis preset has the widest lumens (`pit_lumen_scale` 1.30 over a 0.50 pit) and an `along` cut turns a lumen into a long channel. Shrinking it further (`gland_lumen_scale` 0.80 -> 0.74) brings `oblique` to 508 but takes `lumen_tissue_ratio` to 0.051, below the real p10 of 0.056. Both cannot be satisfied at once with a single lumen-calibre knob. 3. **`intestinal_metaplasia` and `mixed` ring density on `along` is 74 and 72 against p10s of 77 and 76** -- 4-5% below band. Those two presets have the widest pitch (92 and 88 um), so an `along` frame crosses only 11-12 tubes and cannot produce many rings. 4. **The lamina propria is too cellular** (item in section 6). `stroma_nucleus` 0.22-0.31 of the field against a real 0.08-0.12, and `epithelium_cytoplasm` 0.20-0.36 against a real 0.14-0.23: the v2 field is more epithelium and more nuclei, and less loose stroma, than a real one. `stroma_fraction` as `cartoon_metrics.py` defines it (tissue outside the closed band) is nevertheless in band everywhere, because the closed band now covers the glands. 5. **`lumen_tissue_ratio` is in band but low.** 0.034-0.111 against real medians of 0.124-0.183. Real fields carry pale area the cartoon does not model -- oedema, clefts, cytoplasmic pallor, retraction artefact. The pale apical mucin cap closed part of this gap; the rest is not modelled. 6. **The 5-um granule lattice is a rendering trick, not tissue.** It reproduces the metric consequence of real chromatin/RNA granularity (the closing seals the wall) without reproducing its cause. A structure adapter trained on the label map will never see it; a diffusion model conditioned on the RGB will. 7. **`nuclear_density_per_mm2` is now in band on every cut for every category** (a1's gastritis `across` 12754 and `along` 14228 were the two nearest the ceiling). It sits in the lower half of every band, 9446-11718, against real medians of 10569-12293. 8. **No pathologist has seen any of this**, and Yash has not yet reviewed the contact sheet. Categories are named after the HiESD labels of the fields they were tuned against; nothing here says a cartoon is diagnostically correct. 9. **Tuning used the combined train + held-out envelope** (`|all`), as A1 did, so the band comes from 28-30 fields rather than 4-6. Held-out fields were used for nothing except that envelope and the left-hand column of the two figures. 10. **The GPU sweep on a1/d1/e2/b1 was running throughout.** Nothing under `packages/sweep-v1/` was touched, and `cartoons/a1`, `cartoons/d1`, `cartoons/e2`, `cartoons/b1` and every pre-existing `jobs/*.json` are byte-identical (section 8). ## 8. Reproducibility and the checks that the existing sets are untouched Determinism is unchanged from A1: every draw comes from `numpy.random.default_rng` seeded from the cartoon seed (model `seed`, cut plane `seed*7919 + cut index`, renderer `seed*104729 + 17`), and the granule lattice is a pure hash of the pixel grid and the seed with its own generator for the warp, so it cannot shift any other random stream. Versions on this machine: numpy 2.4.4, scipy 1.17.1, Pillow 12.1.0 (the VM must match for byte identity, as A1 recorded). | check | result | |---|---| | `render_set.py --set b1 --out cartoons/b1 --verify` | 15 reference files, **0 mismatched, 0 missing** | | `render_set.py --set d1 --out cartoons/d1 --verify` | 30 reference files, **0 mismatched, 0 missing** | | `render_set.py --set a1 --out cartoons/a1 --verify` | 180 reference files, **0 mismatched, 0 missing** | | `render_set.py --set e2 --out cartoons/e2 --verify` | 60 reference files, **0 mismatched, 0 missing** | | `render_set.py --set a1v2 --out cartoons/a1v2 --verify` | 180 reference files, **0 mismatched, 0 missing** | | `render_set.py --set d1v2 --out cartoons/d1v2 --verify` | 30 reference files, **0 mismatched, 0 missing** | | d1v2 rot180 arrays are exactly `a1v2[::-1, ::-1]`, cartoon and label map | **true** for both | | d1v2 base cartoons are byte-identical to their a1v2 `across` counterparts | **true** | | `make_jobs.py --root . --set-suffix v2` re-run, SHA-256 of every pre-existing `jobs/*.json` | **all 17 identical**, five new files added | | `jobs/README.md` | the only pre-existing file whose bytes changed: it now lists the five new manifests and their two arms (and does not when `--set-suffix` is omitted) | Three byte-identity bugs were found and fixed by these checks rather than by inspection, and all three are worth recording because they are the kind of thing that silently invalidates a running sweep: 1. Adding fields to `TissueParams` changed every sidecar JSON, because `to_json()` dumps the whole dataclass. Fixed with `V2_FIELD_DEFAULTS` (section 3.1). 2. Adding `render_style` to `build()`'s `extra` dict changed the sidecars twice: first by existing at all for v1 renders, then -- after that was fixed with a conditional assignment after the literal -- by *key order*, because `json.dumps` preserves insertion order. It is now inserted in place inside the dict literal, and `cartoons/d1v2` was re-rendered so its sidecars match the shipped code (its 20 PNGs were unaffected). 3. Editing the `*_v2` presets with `str.replace(old, new, 1)` on patterns that also occur in the **v1** presets silently rewrote six lines of `normal_antrum`, `hpylori_gastritis`, `intestinal_metaplasia` and `mixed` -- the A1 presets the running sweep depends on. It was caught by the a1 spot check two iterations later, and the whole v1 `CATEGORY_PARAMS` block was restored verbatim from a backup taken before any edit; a1's PNGs then matched again. Every later preset edit was applied only to the slice of the file after `CATEGORY_PARAMS.update({`. The first two never changed a PNG: in both cases all 120 image files matched and only the JSON sidecars differed. The third would have, which is why the byte check has to be run after every edit and not once at the end. ## 9. Job manifests `code/make_jobs.py` gained `--set-suffix`, which turns the a1 and d1 builders into `_build_a1(donors, suffix)` / `_build_d1(donors, suffix)`. The suffix goes on the experiment id, the cartoon set and the output directory only; the grid, the seeds, the cuts, the donor rotation, the pass-1/pass-2 seeds and start indices, and every `extra` field are identical to a1/d1, so an a1v2 job is the same job against a v2 cartoon and is directly comparable. | manifest | jobs | pass | arms | notes | |---|---|---|---|---| | `jobs/a1v2-pass1.json` | 60 | 1 | `base` | img2img, reference `cartoons/a1v2/_cartoon.png`, `start_index` 12, seed 51001 | | `jobs/a1v2-pass2.json` | 60 | 2 | `base` | reference `PASS1/a1v2/pass1/...`, `start_index` 6, seed 51002 | | `jobs/a1v2-pass2-si3.json` | 60 | 2 | `si3` | `start_index` 3, seed 51003, output `_si3.png` -- the same arm `code/add_si3_arm.py` derived for a1, generated here instead | | `jobs/d1v2-pass1.json` | 25 | 1 | `donor0..donor3`, `rot180` | | | `jobs/d1v2-pass2.json` | 25 | 2 | `donor0..donor3`, `rot180` | | a1v2 deliberately has **no** `si9` arm: that arm exists in a1 because gastritis lost ring fidelity at `start_index` 6, and the brief for this package lists pass1 / pass2 / pass2-si3 only. `code/worker.sh` needs no change: it derives the cartoon sets to render from each job's `reference` prefix (`cartoons//`), so a slot given `a1v2-pass1` renders `cartoons/a1v2` with `code/render_set.py --set a1v2` before it starts. The cartoons are therefore rendered on the VM as PLAN.md requires, and no PNG crosses the wire. `code/split_manifests.py` `DEFAULT_ORDER` gains the five names **after** every existing entry, so an already-packed slot layout is unchanged when they are absent. Re-split check with all of them present: `--slots 4` places 635/635 jobs, 0 broken pass-1 -> pass-2 chains. Resolution, local, with the cartoons in place: | manifest | jobs | runnable | dropped | |---|---|---|---| | `a1v2-pass1` | 60 | 60 | 0 | | `a1v2-pass2` | 60 | 60 | 0 | | `a1v2-pass2-si3` | 60 | 60 | 0 | | `d1v2-pass1` | 25 | 25 | 0 | | `d1v2-pass2` | 25 | 25 | 0 | Two caveats on how that was run, both of which apply identically to `jobs/a1-pass1.json`: * `token_reference` is the absolute on-VM donor path (`[local]/...`), which does not exist on this Mac, so `resolve_jobs.py` drops every job of *any* pass-1 manifest here, a1 included. The table above was produced against manifest copies whose donor prefix was rewritten to the local copy of the **same donor files** (`research/sgh-synthetic-histopathology/.artifacts/pixcell-fields-v1/wide/train`). Every cartoon `reference` and `labels` path resolves against the repo as shipped. * the pass-2 manifests resolve against a run root with the pass-1 outputs staged (empty files at the paths pass 1 will write), which is what `worker.sh` produces at run time. ## 10. Exact commands ```sh cd [local]/sgh-program-20260908 # 1. the real envelope this package tuned against (already present; ~2 min, 118 fields) python3 code/cartoon_metrics.py --real --split both \ --out morphometrics/real-metrics.csv --summary morphometrics/real-envelope.json # 2. render the two new sets (~15 s per cartoon on this Mac: 15 min + 1.5 min) python3 code/render_set.py --set a1v2 --out cartoons/a1v2 python3 code/render_set.py --set d1v2 --out cartoons/d1v2 # 3. byte-for-byte reproducibility, new sets and old python3 code/render_set.py --set a1v2 --out cartoons/a1v2 --verify python3 code/render_set.py --set d1v2 --out cartoons/d1v2 --verify python3 code/render_set.py --set a1 --out cartoons/a1 --verify python3 code/render_set.py --set d1 --out cartoons/d1 --verify python3 code/render_set.py --set e2 --out cartoons/e2 --verify python3 code/render_set.py --set b1 --out cartoons/b1 --verify rm -rf cartoons/*.verify # 4. measure both new sets (same code path, full scale) python3 code/cartoon_metrics.py --cartoons cartoons/a1v2 \ --out morphometrics/a1v2-metrics.csv --summary morphometrics/a1v2-envelope.json python3 code/cartoon_metrics.py --cartoons cartoons/d1v2 \ --out morphometrics/d1v2-metrics.csv # 5. the two figures and the tables in this file python3 code/contact_sheet_v2.py --out cartoons/a1v2-contact-sheet.png python3 code/report_tables_v2.py # 6. job manifests (the v2 ones only appear with --set-suffix) python3 code/make_jobs.py --root . # unchanged output for a1..b1 python3 code/make_jobs.py --root . --set-suffix v2 # + a1v2-* and d1v2-* python3 code/split_manifests.py --slots 4 --out /tmp/split-check # 7. resolve a manifest (see section 9 for the donor-path caveat) python3 code/resolve_jobs.py --manifest jobs/a1v2-pass1.json --run-root . --out /tmp/x.json # single v2 cartoon, any category / cut python3 code/render_cartoon.py --category intestinal_metaplasia_v2 --seed 11 --cut across --out /tmp/x # the tuning loop this package was driven by (scratchpad script, not shipped): # render 5 presets x seeds x 3 cuts in parallel and score each against |all ``` ## 11. Files | path | what | |---|---| | `code/tissue3d.py` | + `wall_max_frac`, `across_depth_frac`, `oblique_depth_frac`, `oblique_frame`, per-cut tilt ranges, `mucin_min_gap`, `stroma_lattice_jitter`; frame policy `"mid"`; `slice(depth_frac=...)`; `V2_FIELD_DEFAULTS` so old sidecars are unchanged | | `code/render_cartoon.py` | + `render(..., style=...)`, `STYLES`/`STYLE_DEFAULTS`, `_granule_weight`, `GRANULE_COLOR`, `APICAL_PALE`, `FIBRE_DARK`; + the five `*_v2` presets, `CANONICAL_V2`, `normal_v2` resolution | | `code/render_set.py` | + sets `a1v2` and `d1v2`; `build(..., style=...)` | | `code/make_jobs.py` | + `--set-suffix`; `build_a1`/`build_d1` refactored to `_build_a1`/`_build_d1` with a suffix; `build_v2` builder; DEPENDENCIES and README rows | | `code/split_manifests.py` | + the five v2 manifests at the end of `DEFAULT_ORDER` | | `code/contact_sheet_v2.py` | the A1-V2 contact sheet (new file; `code/contact_sheet.py` untouched) | | `code/report_tables_v2.py` | the real / a1 / a1v2 tables in section 5 (new file) | | `cartoons/a1v2/` | 60 cartoons, 180 files, 734 MB, with `MANIFEST.sha256.json` | | `cartoons/d1v2/` | 10 cartoons (5 base + 5 rot180), 30 files, 122 MB, with `MANIFEST.sha256.json` | | `cartoons/a1v2-contact-sheet.png` | real held-out \| a1 across \| a1v2 across \| a1v2 along \| a1v2 oblique, matched scale | | `cartoons/a1v2-gland-detail.png` | the same comparison at 1:1, 300 x 170 um, where the 5-um detail is visible | | `morphometrics/a1v2-metrics.csv`, `morphometrics/a1v2-envelope.json` | per-cartoon rows and per-(category, cut) envelopes for the shipped set | | `morphometrics/d1v2-metrics.csv` | the same for d1v2 | | `jobs/a1v2-pass1.json`, `jobs/a1v2-pass2.json`, `jobs/a1v2-pass2-si3.json`, `jobs/d1v2-pass1.json`, `jobs/d1v2-pass2.json` | 230 jobs | Unchanged and verified unchanged: `cartoons/{a1,d1,e2,b1}/`, every pre-existing `jobs/*.json`, `morphometrics/{real,cartoon}-*.csv|json`, `code/contact_sheet.py`, `code/report_tables.py`, and everything under `packages/sweep-v1/`.