C1-ENVELOPE: a classical morphometric realism instrument, calibrated on real gastric tissue

Engineering evidence only. Nothing below diagnoses, grades, or stages anything, and no number here is clinical validation. Read it as: how far does this image sit from the spread of real HiESD fields, measured with classical morphology.

Status: built, calibrated, and applied to every generated set that exists locally (292 images, 0 extraction failures). Nothing was committed to git.

1. Why

Before this package the programme had exactly one quantitative number, ring_with_lumen_fraction from ring_topology.py. It counts gland holes. It says nothing about nuclei, stroma, stain, or texture, so a pale, washed-out, smooth image can score at or above real tissue. That is not a hypothetical: fading a real held-out field 40% toward white raises its ring fraction from 0.805 to 0.973 (section 7). The instrument below closes that gap and, unlike the ring metric, is calibrated: a generated image is reported as a distance and as a percentile of what real held-out tissue scores.

2. The instrument

operations/research/sgh-program-20260908/morphometrics/morphometrics.py

Analysis raster. Every image is resampled with sydney_qa.resize_for_analysis using the same analysis_max_dimension (1536) rule that ring_topology.ring_topology uses, so the ring metrics and every new feature are computed on the identical pixels. For the 4096x2048 canvases used throughout the programme that is 1536x768, i.e. an effective 0.6667 um/px. effective_um_per_px is emitted per image and the CLI warns if a batch mixes resolutions (it did not here: all 292 images are 4096x2048).

Stain separation. Ruifrok-style fixed-vector deconvolution reusing sydney_qa's reference vectors (H = [0.650, 0.704, 0.286], E = [0.072, 0.990, 0.105]) and its optical density -log((rgb+1)/256). Fixed vectors, not per-image Macenko, are deliberate: a per-image basis would absorb exactly the stain drift the envelope must notice.

Masks, all from config/sydney_qa_v1.json so they match ring_topology:

Features (flat dict, 54 keys; the 33 listed in ENVELOPE_FEATURES define the envelope):

• ring_* - all nine metrics returned by ring_topology.ring_topology, unmodified. Verified identical to the existing pixcell-fullset-20260908/ring-summary.json to 4 decimal places for all ten reported cells (section 10).
• nuc_* - nuclear density per mm2 of tissue, area median and IQR (um2), eccentricity and solidity medians (skimage regionprops), mean hematoxylin OD inside nuclei, nuclear area fraction. Shape statistics use a deterministic subsample above 1500 objects.
• epi_* - the "beads on a string" check. Nuclear centroids falling inside the epithelium band, their nearest-neighbour distances in um (median and coefficient of variation, via a KD-tree), their density per mm2 of band, band area fraction.
• str_* - stroma texture: structure-tensor orientation coherence (l_max - l_min)/(l_max + l_min) on total OD with sigma = 2 um (0 = isotropic mesh, 1 = perfectly combed), total structure-tensor energy, eosin OD variance and mean, stromal nuclear density, stroma area fraction.
• col_* - tissue-masked CIE-LAB mean and std per channel, hematoxylin and eosin OD means, fraction of tissue pixels that are pale (> 220), fraction clipped (max >= 250 or min <= 5), mean HSV-style saturation.
• glob_* - tissue fraction, enclosed-lumen-to-tissue ratio, background fraction, lumen count per mm2, median lumen area.

Distances. Per category, on the 24 real TRAIN fields:

• per feature, z = (x - median) / (1.4826 * MAD); a feature whose MAD is zero falls back to the standard deviation and is dropped if that is zero too (none were dropped);
• robust_z_rms = RMS of z over all 33 envelope features (diagonal, always defined, directly interpretable feature by feature);
• mahalanobis = the MinCovDet (sklearn 1.9.0, support_fraction=0.75, random_state=0) robust-covariance distance in the space of 10 core features, with a ridge of 0.10 x mean(diag) added for stability. Ten dimensions, not 33: 24 training fields cannot support a 33-dimensional covariance. MinCovDet fitted cleanly for all four categories; location, covariance and its inverse are stored in envelope.json so scores are reproducible without sklearn.

Core features: ring_with_lumen_fraction, ring_density_per_mm2, nuc_density_per_mm2, nuc_area_median_um2, epi_nn_distance_median_um, epi_nn_distance_cv, str_orientation_coherence, str_eosin_od_var, col_h_od_mean, col_e_od_mean.

glob_background_fraction is measured and reported but deliberately excluded from the envelope: for these tissue-dense crops it is exactly 1 - glob_tissue_fraction with an identical MAD, so including it would count the same evidence twice.

Cost: about 2 s per 4096x2048 canvas on one CPU core; 292 images in 3 min 33 s on 12 cores.

3. Calibration on real held-out fields

The envelope is fitted on TRAIN only (24 fields per category, 8-10 slides per category). The held-out fields are slide-disjoint (3 slides: 0d55cc29, 0dddc3e2, 1a203f16; 2 fields per slide per category, except normal, which has fields from only 2 of the 3) and are scored out-of-sample. That held-out distribution is the calibration.

Diagonal distance for the same held-out fields (robust_z_rms): normal 0.41 / 0.55 / 1.56 (min / p50 / max), gastritis 0.87 / 1.03 / 1.80, IM 0.53 / 0.72 / 1.20, mixed 0.58 / 0.97 / 1.83.

Reading of this table: out-of-sample real tissue scores about 1.0-1.6x the in-sample training median, and the whole real held-out spread lives below D = 4.5 for three categories. hpylori_gastritis is the loose one (max 7.97): its six held-out fields include a genuinely atypical field, so the gastritis band is wide and a generated gastritis image can land "inside the band" while still being visibly wrong. Reported percentiles for gastritis, and for normal (n = 4, so percentiles quantise to 0/25/50/75/100), should be read as coarse.

scores.csv carries, per image: mahalanobis, robust_z_rms, robust_z_median_abs, pct_vs_heldout (percent of real held-out fields of that category at or below this distance), ratio_to_heldout_median, ratio_to_heldout_max, inside_heldout_band, ring_with_lumen_fraction, ring_gap_to_real_median, and the three worst features with their z.

4. Ranking of every existing set

Pooled over the categories each set covers; D50 = median MCD Mahalanobis; "x real" = median of each image's distance divided by its category's held-out median; "in band" = percent at or below its category's held-out maximum.

Both distances agree on the ordering, so this is not an artefact of the covariance model. Per (set, category) rows are in morphometrics/set-summary.csv; per-arm rows (each lock arm, each start_index) are in morphometrics/arm-summary.csv.

What this says.

1. The sanity check passes. Cartoons sit at 3.1x the real held-out median, and 0 of 20 are inside any category's band - even though their ring fraction (0.80-0.88) is indistinguishable from real. The instrument sees what a human sees; the ring metric alone does not.
2. fullset_pass1 is worse than the cartoons it came from (D50 14.70 vs 10.24). Pass 1 is start_index 12, i.e. only 8 of 20 denoising steps: it smooths the cartoon without rebuilding tissue, so it inherits the cartoon's geometry and loses the cartoon's contrast. Pass 2 (start_index 6, 14 steps) more than halves it in both categories where both passes exist: gastritis 16.76 -> 6.33, mixed 14.36 -> 6.74.
3. The dominant knob is how far the repaint runs from the cartoon, and the lock is not doing much. Deduplicating the 54 lock records to their 42 distinct PNGs and grouping by start_index (intestinal metaplasia only, so the column is internally comparable):

The distance falls monotonically as start_index falls: 11.6 at si12 down to 4.6-5.3 at si0, a factor of 2.4. Locked and unlocked arms are within 0.1-0.6 D of each other at every rung, i.e. the lumen lock costs and buys almost nothing on these measurements. The LoRA arms improve si12 by about 0.4-1.0 D and remain far outside the band. A second pass from si6 (7.10) lands exactly where a single si6 pass lands (7.09/7.11), so the second pass is not adding anything the first pass at that start_index did not.

The ring metric does not reproduce this ordering: si0 (ring 0.58-0.68) and si9 (ring 0.89-0.92) are about equally far from the real IM median of 0.774, while the envelope separates them by a factor of two. That is the clearest single demonstration in this package of why the ring fraction is not enough to steer on. 4. native_i2i025 is the closest thing to real we have - and that is a warning, not a win. It is a light img2img (start_index 12) on a real training field, so it is a repaint of real tissue rather than a construction. Low envelope distance and high copy risk are the same fact here. This is exactly why pareto_select.py takes a copy-screen margin as a third objective. 5. The ten finals are mid-table (D50 6.02, 50% in band) and their ring fraction is far below real (0.20-0.43 vs 0.77-0.88): they were selected before this instrument existed, on copy-screen and reviewer criteria.

5. What the generated images actually fail on

This is the actionable part. Numbers are the median across each set's images of the ratio to that category's real TRAIN median (so 1.00 = real); the robust z is in brackets. Full table: morphometrics/feature-deviations.csv; plot: morphometrics/discriminative-strips.png.

Stated the way a reviewer would say it, in priority order:

1. Nuclei are too few, too large and too variable in size, and spaced too far apart. Every single generated set is on the same side of all four features. Cartoon-conditioned arms are the extreme (nuclear density 0.70x, median nuclear area 1.80-1.91x, spacing 1.23-1.30x, i.e. z of +5 to +7 on a 0.30 um MAD); even the best set, native img2img, is 1.07-1.08x on area and IQR. The instrument cannot distinguish "genuinely larger nuclei" from "adjacent nuclei that are not resolved as separate objects" - both read as loss of nuclear crispness, which is what a human calls "mushy". Either way this is the single biggest, most consistent defect, and it is the thing to fix.
2. The stroma is too combed. Orientation coherence 1.09-1.28x real everywhere. Generated stroma has a directional grain; real lamina propria is a more isotropic mesh.
3. The stroma is under-populated. Stromal nuclear density 0.31-0.68x real for every set except native img2img. Concretely: the inflammatory infiltrate that defines gastritis is largely missing. (Caveat in section 8: this "stroma" is a small residual mask, so the density is noisy.)
4. Cartoon-conditioned arms leave background on the canvas. Tissue fraction 0.92-0.93x real (z = -4 to -5). Real fields are tissue-dense crops by construction; pass-1 and si12 outputs keep the cartoon's white margins.
5. Too many closed epithelial contours (ring density 1.14-2.70x) with a thinner epithelial band (0.64-0.89x): the palisade is fragmented, which manufactures holes. This is why ring_with_lumen_fraction can look healthy while the tissue is not - the denominator is inflated too.
6. Stain balance drifts in the cartoon-conditioned arms: pass 1 and the cartoons are blue-heavy (hematoxylin 1.37x and 1.29x real) and every cartoon-conditioned arm is pink-poor (eosin 0.56-0.59x for pass 1, lock_base and the cartoons; 0.73x for pass 2). native_gen025 is the other way round: hematoxylin 1.38x with eosin at 1.03x.

Not a failure mode, contrary to the brief's hypothesis: epithelial spacing CV is essentially correct everywhere (0.88-1.05x real). The palisade is not too regular; it is too coarse. The regularity of the spacing is fine, the scale is wrong.

6. Which features carry disease signal

One-way effect sizes over the 96 real TRAIN fields (envelope.json > category_separation_real_train), sorted by eta squared:

Two results worth carrying forward:

• ring_with_lumen_fraction is the least category-discriminative of all 33 features. On real tissue it does not separate normal from gastritis from IM from mixed at all (eta2 = 0.007, p = 0.89). It is a useful plausibility check on gland architecture; it is not a disease signal and should not be used as the programme's headline target.
• Only 2 of 33 features survive Bonferroni at 0.05. With 24 fields per category, this classical feature set separates the four categories only weakly. The features that do carry signal are stromal texture and colour dispersion, not gland topology. Treat this as an argument for the learned probes (C2-PROBES), not against them.

7. Does the instrument catch what the ring metric misses?

perturbation_check.py degrades real held-out fields (one per held-out slide per category: 11 fields, since normal has only two held-out slides) in four ways and rescores them. Medians over the 11 fields:

The second row is the point of this package: fading a real field toward white improves its ring score by 0.12 while the envelope pushes it out of the real band. The stain push is invisible to the ring metric (-0.04) and obvious to the envelope (1.58x, 73% out).

The last row is an honest failure: throwing away half the spatial resolution and interpolating back moves neither metric. At the 1536-max-dimension analysis raster (0.6667 um/px) there is nothing left to see. This instrument cannot certify that a canvas carries genuine 0.25 um/px detail. If that matters, it needs a separate high-resolution check.

8. Limitations

1. Not clinical. No pathologist has annotated anything. This measures shape, spacing, texture and colour, not disease. "Inside the real held-out band" means "not obviously different on 33 classical measurements", not "diagnostically valid".
2. The calibration is thin. 4-6 held-out fields per category from 2-3 slides. Percentiles quantise to 25% steps for normal and 16.7% for the rest. The gastritis band is wide because one held-out field is atypical. Treat percentiles as ordinal, not exact.
3. One dataset, one stain. All real fields are HiESD. The envelope encodes HiESD staining and HiESD's tissue-dense field selection. A different lab's slides would score as outliers on the colour features without being any less real.
4. Analysis resolution. All features are computed at 0.6667 um/px because that is the raster ring_topology uses. Nuclear morphometry at that scale is coarse: a 6 um nucleus is 9 px across, so "bigger nuclei" and "unresolved adjacent nuclei" are the same measurement (section 5.1), and sub-micrometre defects are invisible (section 7).
5. epithelium is not epithelium. It is ring_topology's hematoxylin-rich band after a 6 um closing, which covers 82-90% of tissue in real fields. stroma is therefore a small residual (5-10% of tissue), which makes str_nuc_density_per_mm2 and str_area_fraction noisy and inflates stromal density in normal mucosa. They are still comparable across sets because every image is measured the same way, but do not read them as a real epithelium/stroma segmentation. F2-PAIRS' segmentation would give a better one.
6. A robust covariance from 24 fields in 10 dimensions is a coarse instrument. The ridge (0.10 x mean diagonal) matters; absolute distances would shift under a different ridge. Relative ordering of sets is stable across both distance definitions, which is what the ranking rests on.
7. Low distance is not a goal in itself. A near-copy of a real field scores best (native_i2i025). Envelope distance must always be read next to a copy screen.
8. Sets are not balanced. The lock experiment is intestinal-metaplasia only, so its arms are only comparable to the IM column. fullset_pass1 was retrieved only for gastritis and mixed (10 of the 20 pass-1 outputs are missing locally, listed in the run log as missing .../pass1/normal/* and .../pass1/intestinal_metaplasia/*).
9. lock_release is not 14 samples. Its 14 PNGs are only 6 distinct files: the three lumen-release levels (0.25 / 0.4 / 0.6) emitted byte-identical images at both si0 and si3, and those four images are byte-identical to the corresponding lock_ladder files as well (morphometrics/duplicates.csv, 292 files -> 280 distinct). Its row in the ranking is therefore four images counted three times each plus two layout controls, and the release sweep produced no evidence at all. Every other set is duplicate-free. set-summary.csv and arm-summary.csv carry an n_distinct_files column so this is visible wherever the numbers are read.
10. native_gen025 is a 10-per-category sample of 48/48/48/34 available canvases, taken at even spacing over the sorted record order (deterministic, recorded in manifest.csv).

9. Deliverables

All under operations/research/sgh-program-20260908/:

Note: morphometrics/real-metrics.csv, real-envelope.json, real-run.log and baseline-cartoon-metrics.csv in the same directory were written by a different Phase-1 package running in parallel (the cartoon/tissue-model work). They are not C1 outputs and were not touched.

10. Exact commands

Python is research/sgh-synthetic-histopathology/.venv/bin/python throughout. matplotlib 3.11.1 and scikit-learn 1.9.0 were installed into that venv (nothing else); numpy 2.5.2, scipy 1.18.1, scikit-image 0.26.0, Pillow 12.3.0 were already there.

cd [local]/morphometrics
P=[local]/python

# one-shot: manifest -> features -> envelope -> scores -> plots  (3 min 33 s on 12 cores)
$P calibrate.py --jobs 12
# re-fit only (reuses features.csv)
$P calibrate.py --jobs 12 --reuse-features

# score an arbitrary batch with the extractor alone
$P morphometrics.py --images /path/to/*.png --set my_set --category intestinal_metaplasia \
    --out /tmp/my-features.csv --jobs 12

# sensitivity check (11 held-out fields x 5 variants, ~2 min)
$P perturbation_check.py --per-category 3

# selection: two objectives (ring fraction near the real median, low envelope distance)
$P pareto_select.py --features scores.csv --category intestinal_metaplasia --n 5

# selection: three objectives, adding the copy-screen margin (bigger margin = safer)
$P pareto_select.py --features scores.csv --category normal --n 5 --sets fullset_pass2 \
    --join copy-screen-pass2.csv --third copy_margin --third-direction max \
    --out /tmp/normal-top5.csv

Verification that the ring half of the instrument is unchanged: the per-category means of ring_with_lumen_fraction recomputed here match operations/research/pixcell-fullset-20260908/ring-summary.json to 4 decimal places on all ten cells it reports (pass 1 gastritis 0.6284, pass 1 mixed 0.6652, pass 2 normal 0.7256, pass 2 gastritis 0.4753, pass 2 IM 0.9170, pass 2 mixed 0.7260, real held-out 0.8538 / 0.7428 / 0.7101 / 0.7261).

11. What failed or was skipped

• 10 of the 40 pixcell-fullset-20260908 pass-1 outputs (normal, intestinal metaplasia) are recorded in records.jsonl but not present locally, so fullset_pass1 covers only gastritis and mixed. Logged by calibrate.py as manifest notes: missing ....
• 14 pixcell-lock-20260907 records point at release/... while the files live in release-lockonly/...; calibrate.py rewrites that prefix. All 54 lock records therefore resolve to the 40 PNGs that exist.
• operations/research/pixcell-cartoons-v2/ is byte-identical to pixcell-fullset-20260908/cartoons/ (md5 over all 20 cartoon PNGs in each directory agrees), so the cartoons were scored once, from the latter.
• The lock experiment's lumen-release sweep produced no distinct outputs. All three release levels at si0 and at si3 wrote byte-identical PNGs, and each is also identical to the matching ladder/K_*_lock_si*.png. 12 of the 14 release-lockonly files are therefore copies of 4 images. Worth checking whether the release parameter reached the sampler at all before that arm is repeated.
• No GPU work; nothing was uploaded or generated. 0 extraction failures across 292 images.

Download public Markdown export