# C1-ENVELOPE: a classical morphometric realism instrument, calibrated on real gastric tissue Engineering evidence only. Nothing below diagnoses, grades, or stages anything, and no number here is clinical validation. Read it as: *how far does this image sit from the spread of real HiESD fields, measured with classical morphology.* Status: built, calibrated, and applied to every generated set that exists locally (292 images, 0 extraction failures). Nothing was committed to git. --- ## 1. Why Before this package the programme had exactly one quantitative number, `ring_with_lumen_fraction` from `ring_topology.py`. It counts gland holes. It says nothing about nuclei, stroma, stain, or texture, so a pale, washed-out, smooth image can score at or above real tissue. That is not a hypothetical: fading a real held-out field 40% toward white **raises** its ring fraction from 0.805 to 0.973 (section 7). The instrument below closes that gap and, unlike the ring metric, is calibrated: a generated image is reported as a distance and as a percentile of what *real held-out tissue* scores. --- ## 2. The instrument `operations/research/sgh-program-20260908/morphometrics/morphometrics.py` **Analysis raster.** Every image is resampled with `sydney_qa.resize_for_analysis` using the same `analysis_max_dimension` (1536) rule that `ring_topology.ring_topology` uses, so the ring metrics and every new feature are computed on the identical pixels. For the 4096x2048 canvases used throughout the programme that is 1536x768, i.e. an effective **0.6667 um/px**. `effective_um_per_px` is emitted per image and the CLI warns if a batch mixes resolutions (it did not here: all 292 images are 4096x2048). **Stain separation.** Ruifrok-style fixed-vector deconvolution reusing `sydney_qa`'s reference vectors (H = [0.650, 0.704, 0.286], E = [0.072, 0.990, 0.105]) and its optical density `-log((rgb+1)/256)`. Fixed vectors, not per-image Macenko, are deliberate: a per-image basis would absorb exactly the stain drift the envelope must notice. **Masks**, all from `config/sydney_qa_v1.json` so they match `ring_topology`: | mask | definition | |---|---| | `tissue` | OD sum > 0.15 and mean RGB < 245 | | `nuclear_raw` | tissue pixels with hematoxylin at or above the 72nd percentile of h over tissue | | `epithelium` | `nuclear_raw` closed with a disk of radius `CLOSING_RADIUS_UM` = 6 um (imported from `ring_topology`), components < 100 um2 dropped - byte-for-byte the same construction `ring_topology` uses | | `pale` | mean RGB > 220 (`lumen_brightness_floor`) | | `stroma` | tissue AND NOT epithelium AND NOT pale | | `nuclei` | `nuclear_raw` opened by 1 px, split by a distance-transform watershed (Gaussian sigma 1 px, seed separation 3 um), objects kept when area is in [5, 180] um2 | **Features** (flat dict, 54 keys; the 33 listed in `ENVELOPE_FEATURES` define the envelope): - `ring_*` - all nine metrics returned by `ring_topology.ring_topology`, unmodified. Verified identical to the existing `pixcell-fullset-20260908/ring-summary.json` to 4 decimal places for all ten reported cells (section 10). - `nuc_*` - nuclear density per mm2 of tissue, area median and IQR (um2), eccentricity and solidity medians (skimage regionprops), mean hematoxylin OD inside nuclei, nuclear area fraction. Shape statistics use a deterministic subsample above 1500 objects. - `epi_*` - the "beads on a string" check. Nuclear centroids falling inside the epithelium band, their nearest-neighbour distances in um (median and coefficient of variation, via a KD-tree), their density per mm2 of band, band area fraction. - `str_*` - stroma texture: structure-tensor orientation coherence `(l_max - l_min)/(l_max + l_min)` on total OD with sigma = 2 um (0 = isotropic mesh, 1 = perfectly combed), total structure-tensor energy, eosin OD variance and mean, stromal nuclear density, stroma area fraction. - `col_*` - tissue-masked CIE-LAB mean and std per channel, hematoxylin and eosin OD means, fraction of tissue pixels that are pale (> 220), fraction clipped (max >= 250 or min <= 5), mean HSV-style saturation. - `glob_*` - tissue fraction, enclosed-lumen-to-tissue ratio, background fraction, lumen count per mm2, median lumen area. **Distances.** Per category, on the 24 real TRAIN fields: - per feature, `z = (x - median) / (1.4826 * MAD)`; a feature whose MAD is zero falls back to the standard deviation and is dropped if that is zero too (none were dropped); - `robust_z_rms` = RMS of z over all 33 envelope features (diagonal, always defined, directly interpretable feature by feature); - `mahalanobis` = the MinCovDet (sklearn 1.9.0, `support_fraction=0.75`, `random_state=0`) robust-covariance distance in the space of **10 core features**, with a ridge of 0.10 x mean(diag) added for stability. Ten dimensions, not 33: 24 training fields cannot support a 33-dimensional covariance. MinCovDet fitted cleanly for all four categories; location, covariance and its inverse are stored in `envelope.json` so scores are reproducible without sklearn. Core features: `ring_with_lumen_fraction`, `ring_density_per_mm2`, `nuc_density_per_mm2`, `nuc_area_median_um2`, `epi_nn_distance_median_um`, `epi_nn_distance_cv`, `str_orientation_coherence`, `str_eosin_od_var`, `col_h_od_mean`, `col_e_od_mean`. `glob_background_fraction` is measured and reported but deliberately excluded from the envelope: for these tissue-dense crops it is exactly `1 - glob_tissue_fraction` with an identical MAD, so including it would count the same evidence twice. Cost: about 2 s per 4096x2048 canvas on one CPU core; 292 images in 3 min 33 s on 12 cores. --- ## 3. Calibration on real held-out fields The envelope is fitted on TRAIN only (24 fields per category, 8-10 slides per category). The held-out fields are slide-disjoint (3 slides: `0d55cc29`, `0dddc3e2`, `1a203f16`; 2 fields per slide per category, except normal, which has fields from only 2 of the 3) and are scored out-of-sample. **That held-out distribution is the calibration.** | category | n train | n held-out | held-out D min | D p50 | D p90 | D max | train in-sample D p50 | held-out ring p50 | |---|---|---|---|---|---|---|---|---| | normal | 24 | 4 | 2.61 | 3.33 | 4.07 | 4.34 | 2.56 | 0.884 | | hpylori_gastritis | 24 | 6 | 3.19 | 4.19 | 6.84 | 7.97 | 2.57 | 0.807 | | intestinal_metaplasia | 24 | 6 | 1.50 | 2.58 | 4.18 | 4.50 | 2.43 | 0.774 | | mixed | 24 | 6 | 3.12 | 3.66 | 4.18 | 4.38 | 2.49 | 0.774 | Diagonal distance for the same held-out fields (`robust_z_rms`): normal 0.41 / 0.55 / 1.56 (min / p50 / max), gastritis 0.87 / 1.03 / 1.80, IM 0.53 / 0.72 / 1.20, mixed 0.58 / 0.97 / 1.83. Reading of this table: out-of-sample real tissue scores about 1.0-1.6x the in-sample training median, and the whole real held-out spread lives below D = 4.5 for three categories. `hpylori_gastritis` is the loose one (max 7.97): its six held-out fields include a genuinely atypical field, so the gastritis band is wide and a generated gastritis image can land "inside the band" while still being visibly wrong. Reported percentiles for gastritis, and for normal (n = 4, so percentiles quantise to 0/25/50/75/100), should be read as coarse. `scores.csv` carries, per image: `mahalanobis`, `robust_z_rms`, `robust_z_median_abs`, `pct_vs_heldout` (percent of real held-out fields of that category at or below this distance), `ratio_to_heldout_median`, `ratio_to_heldout_max`, `inside_heldout_band`, `ring_with_lumen_fraction`, `ring_gap_to_real_median`, and the three worst features with their z. --- ## 4. Ranking of every existing set Pooled over the categories each set covers; D50 = median MCD Mahalanobis; "x real" = median of each image's distance divided by its category's held-out median; "in band" = percent at or below its category's held-out maximum. | rank | set | files | distinct | categories | D50 | x real | RZ50 | in band | |---|---|---|---|---|---|---|---|---| | - | **real_train** (in-sample, optimistic) | 96 | 96 | all 4 | 2.48 | 0.76 | 1.00 | 91.7% | | - | **real_heldout** (the reference) | 22 | 22 | all 4 | 3.51 | 1.00 | 0.88 | 100% | | 1 | native_i2i025 | 20 | 20 | IM, mixed | 3.24 | 1.23 | 1.04 | 75.0% | | 2 | lock_release | 14 | **6** | IM | 5.30 | 2.06 | 1.82 | 0% | | 3 | lock_ladder | 12 | 12 | IM | 5.32 | 2.07 | 1.65 | 0% | | 4 | native_gen025 | 40 | 40 | all 4 | 5.33 | 1.57 | 1.46 | 42.5% | | 5 | finals (the ten) | 10 | 10 | all 4 | 6.02 | 1.65 | 1.43 | 50.0% | | 6 | fullset_pass2 | 20 | 20 | all 4 | 6.50 | 1.87 | 1.40 | 35.0% | | 7 | lock_pass2 | 9 | 9 | IM | 7.24 | 2.81 | 1.70 | 0% | | 8 | cartoons | 20 | 20 | all 4 | 10.24 | 3.09 | 2.60 | 0% | | 9 | lock_base | 13 | 13 | IM | 10.35 | 4.02 | 2.28 | 0% | | 10 | lock_lora | 6 | 6 | IM | 10.66 | 4.14 | 2.26 | 0% | | 11 | fullset_pass1 | 10 | 10 | gastritis, mixed | 14.70 | 3.94 | 2.56 | 0% | Both distances agree on the ordering, so this is not an artefact of the covariance model. Per (set, category) rows are in `morphometrics/set-summary.csv`; per-arm rows (each lock arm, each start_index) are in `morphometrics/arm-summary.csv`. **What this says.** 1. **The sanity check passes.** Cartoons sit at 3.1x the real held-out median, and 0 of 20 are inside any category's band - even though their ring fraction (0.80-0.88) is *indistinguishable from real*. The instrument sees what a human sees; the ring metric alone does not. 2. **`fullset_pass1` is worse than the cartoons it came from** (D50 14.70 vs 10.24). Pass 1 is `start_index` 12, i.e. only 8 of 20 denoising steps: it smooths the cartoon without rebuilding tissue, so it inherits the cartoon's geometry and loses the cartoon's contrast. Pass 2 (`start_index` 6, 14 steps) more than halves it in both categories where both passes exist: gastritis 16.76 -> 6.33, mixed 14.36 -> 6.74. 3. **The dominant knob is how far the repaint runs from the cartoon, and the lock is not doing much.** Deduplicating the 54 lock records to their 42 distinct PNGs and grouping by `start_index` (intestinal metaplasia only, so the column is internally comparable): | start_index | denoise steps | D50 lock | D50 no lock | D50 LoRA | ring lock / no lock | |---|---|---|---|---|---| | 0 | 20 | 5.25 (n=4) | 4.64 (n=2) | - | 0.681 / 0.579 | | 3 | 17 | 5.55 (n=2) | 5.01 (n=2) | - | 0.729 / 0.659 | | 6 | 14 | 7.09 (n=1) | 7.11 (n=2) | - | 0.851 / 0.860 | | 9 | 11 | 9.60 (n=4) | 9.56 (n=2) | - | 0.894 / 0.922 | | 12 | 8 | 11.59 (n=5) | 11.04 (n=3) | 10.72 / 10.60 (n=3 each) | 0.750 / 0.758 | | pass 2 from si6 | - | 7.10 (n=6) | - | - | 0.863 | | pass 2 from si9 | - | 9.33 (n=3) | - | - | 0.871 | The distance falls monotonically as `start_index` falls: 11.6 at si12 down to 4.6-5.3 at si0, a factor of 2.4. Locked and unlocked arms are within 0.1-0.6 D of each other at every rung, i.e. the lumen lock costs and buys almost nothing on these measurements. The LoRA arms improve si12 by about 0.4-1.0 D and remain far outside the band. A second pass from si6 (7.10) lands exactly where a single si6 pass lands (7.09/7.11), so the second pass is not adding anything the first pass at that `start_index` did not. The ring metric does **not** reproduce this ordering: si0 (ring 0.58-0.68) and si9 (ring 0.89-0.92) are about equally far from the real IM median of 0.774, while the envelope separates them by a factor of two. That is the clearest single demonstration in this package of why the ring fraction is not enough to steer on. 4. **`native_i2i025` is the closest thing to real we have - and that is a warning, not a win.** It is a light img2img (`start_index` 12) *on a real training field*, so it is a repaint of real tissue rather than a construction. Low envelope distance and high copy risk are the same fact here. This is exactly why `pareto_select.py` takes a copy-screen margin as a third objective. 5. **The ten finals are mid-table** (D50 6.02, 50% in band) and their ring fraction is far below real (0.20-0.43 vs 0.77-0.88): they were selected before this instrument existed, on copy-screen and reviewer criteria. --- ## 5. What the generated images actually fail on This is the actionable part. Numbers are the median across each set's images of the ratio to that category's real TRAIN median (so 1.00 = real); the robust z is in brackets. Full table: `morphometrics/feature-deviations.csv`; plot: `morphometrics/discriminative-strips.png`. | failure | real held-out | native_i2i | native_gen | finals | pass2 | pass1 | lock_base | cartoons | |---|---|---|---|---|---|---|---|---| | nuclear density (per mm2) | 0.99 | 1.00 | 0.95 | 0.91 | 0.82 [-1.3] | 0.72 [-2.0] | 0.70 [-2.0] | 0.84 | | nuclear median area | 0.95 | 1.08 | 1.12 | 1.31 [+1.4] | 1.29 [+1.6] | 1.91 [+4.6] | 1.80 [+4.9] | 1.55 | | nuclear area IQR | 1.01 | 1.07 | 1.09 | 1.32 | 1.50 [+2.0] | 1.86 [+3.3] | 1.78 [+2.6] | 1.56 | | epithelial NN spacing | 1.02 | 1.02 | 1.04 | 1.08 | 1.08 [+1.7] | 1.30 [+7.0] | 1.23 [+5.2] | 1.15 | | stroma orientation coherence | 1.07 | 1.09 | 1.14 | 1.15 [+2.2] | 1.15 [+1.8] | 1.28 [+4.5] | 1.22 [+3.0] | 1.24 | | stromal nuclear density | 1.34 | 1.03 | 0.55 [-0.6] | 0.36 [-0.9] | 0.68 | 0.31 [-0.9] | 0.41 | 0.41 | | ring density per mm2 | 0.94 | 1.01 | 1.14 | 1.30 | 1.45 [+1.4] | 2.70 [+4.9] | 1.75 [+2.8] | 1.87 | | epithelial band fraction | 0.99 | 0.95 | 0.89 | 0.84 | 0.89 | 0.64 [-2.5] | 0.64 [-2.7] | 0.64 | | tissue fraction | 0.98 | 1.01 | 1.01 | 1.01 | 1.01 | 0.92 [-5.4] | 0.93 [-4.1] | 0.92 | | hematoxylin OD (tissue mean) | 1.02 | 1.04 | 1.38 [+1.7] | 1.21 | 1.05 | 1.37 [+1.7] | 1.10 | 1.29 | | eosin OD (tissue mean) | 1.09 | 0.93 | 1.03 | 1.38 | 0.73 | 0.59 [-1.0] | 0.56 [-1.0] | 0.59 | Stated the way a reviewer would say it, in priority order: 1. **Nuclei are too few, too large and too variable in size, and spaced too far apart.** Every single generated set is on the same side of all four features. Cartoon-conditioned arms are the extreme (nuclear density 0.70x, median nuclear area 1.80-1.91x, spacing 1.23-1.30x, i.e. z of +5 to +7 on a 0.30 um MAD); even the best set, native img2img, is 1.07-1.08x on area and IQR. **The instrument cannot distinguish "genuinely larger nuclei" from "adjacent nuclei that are not resolved as separate objects"** - both read as loss of nuclear crispness, which is what a human calls "mushy". Either way this is the single biggest, most consistent defect, and it is the thing to fix. 2. **The stroma is too combed.** Orientation coherence 1.09-1.28x real everywhere. Generated stroma has a directional grain; real lamina propria is a more isotropic mesh. 3. **The stroma is under-populated.** Stromal nuclear density 0.31-0.68x real for every set except native img2img. Concretely: the inflammatory infiltrate that defines gastritis is largely missing. (Caveat in section 8: this "stroma" is a small residual mask, so the density is noisy.) 4. **Cartoon-conditioned arms leave background on the canvas.** Tissue fraction 0.92-0.93x real (z = -4 to -5). Real fields are tissue-dense crops by construction; pass-1 and si12 outputs keep the cartoon's white margins. 5. **Too many closed epithelial contours** (ring density 1.14-2.70x) with a thinner epithelial band (0.64-0.89x): the palisade is fragmented, which manufactures holes. This is why `ring_with_lumen_fraction` can look healthy while the tissue is not - the denominator is inflated too. 6. **Stain balance drifts** in the cartoon-conditioned arms: pass 1 and the cartoons are blue-heavy (hematoxylin 1.37x and 1.29x real) and every cartoon-conditioned arm is pink-poor (eosin 0.56-0.59x for pass 1, lock_base and the cartoons; 0.73x for pass 2). `native_gen025` is the other way round: hematoxylin 1.38x with eosin at 1.03x. **Not** a failure mode, contrary to the brief's hypothesis: epithelial spacing CV is essentially correct everywhere (0.88-1.05x real). The palisade is not too regular; it is too coarse. The regularity of the *spacing* is fine, the *scale* is wrong. --- ## 6. Which features carry disease signal One-way effect sizes over the 96 real TRAIN fields (`envelope.json` > `category_separation_real_train`), sorted by eta squared: | feature | eta2 | F | p | largest pair | |---|---|---|---|---| | str_structure_energy | 0.164 | 5.99 | 8.9e-04 | normal vs gastritis | | col_lab_a_std | 0.163 | 5.98 | 9.0e-04 | gastritis vs IM | | str_eosin_od_var | 0.135 | 4.80 | 3.8e-03 | gastritis vs IM | | nuc_eccentricity_median | 0.126 | 4.40 | 6.1e-03 | normal vs mixed | | epi_nuc_density_per_mm2 | 0.096 | 3.24 | 2.6e-02 | normal vs mixed | | ring_density_per_mm2 | 0.087 | 2.94 | 3.7e-02 | normal vs IM | | ... | | | | | | **ring_with_lumen_fraction** | **0.007** | **0.21** | **0.89** | normal vs mixed | Two results worth carrying forward: - **`ring_with_lumen_fraction` is the least category-discriminative of all 33 features.** On real tissue it does not separate normal from gastritis from IM from mixed at all (eta2 = 0.007, p = 0.89). It is a useful *plausibility* check on gland architecture; it is not a disease signal and should not be used as the programme's headline target. - Only 2 of 33 features survive Bonferroni at 0.05. With 24 fields per category, this classical feature set separates the four categories only weakly. The features that do carry signal are stromal texture and colour dispersion, not gland topology. Treat this as an argument for the learned probes (C2-PROBES), not against them. --- ## 7. Does the instrument catch what the ring metric misses? `perturbation_check.py` degrades real held-out fields (one per held-out slide per category: 11 fields, since normal has only two held-out slides) in four ways and rescores them. Medians over the 11 fields: | degradation | ring fraction | change in ring | envelope D | x original | left the real band | |---|---|---|---|---|---| | original | 0.805 | - | 3.43 | 1.00 | 0% | | blur, sigma 2 px | 0.538 | **-0.254** | 5.37 | 1.71 | 73% | | fade 40% to white | 0.973 | **+0.122** | 5.37 | 1.62 | 64% | | hematoxylin x 1.4 | 0.776 | -0.043 | 6.13 | 1.58 | 73% | | half-resolution round trip | 0.776 | -0.019 | 3.57 | 0.98 | **0%** | The second row is the point of this package: fading a real field toward white **improves** its ring score by 0.12 while the envelope pushes it out of the real band. The stain push is invisible to the ring metric (-0.04) and obvious to the envelope (1.58x, 73% out). The last row is an honest failure: throwing away half the spatial resolution and interpolating back moves neither metric. At the 1536-max-dimension analysis raster (0.6667 um/px) there is nothing left to see. **This instrument cannot certify that a canvas carries genuine 0.25 um/px detail.** If that matters, it needs a separate high-resolution check. --- ## 8. Limitations 1. **Not clinical.** No pathologist has annotated anything. This measures shape, spacing, texture and colour, not disease. "Inside the real held-out band" means "not obviously different on 33 classical measurements", not "diagnostically valid". 2. **The calibration is thin.** 4-6 held-out fields per category from 2-3 slides. Percentiles quantise to 25% steps for normal and 16.7% for the rest. The gastritis band is wide because one held-out field is atypical. Treat percentiles as ordinal, not exact. 3. **One dataset, one stain.** All real fields are HiESD. The envelope encodes HiESD staining and HiESD's tissue-dense field selection. A different lab's slides would score as outliers on the colour features without being any less real. 4. **Analysis resolution.** All features are computed at 0.6667 um/px because that is the raster `ring_topology` uses. Nuclear morphometry at that scale is coarse: a 6 um nucleus is 9 px across, so "bigger nuclei" and "unresolved adjacent nuclei" are the same measurement (section 5.1), and sub-micrometre defects are invisible (section 7). 5. **`epithelium` is not epithelium.** It is `ring_topology`'s hematoxylin-rich band after a 6 um closing, which covers 82-90% of tissue in real fields. `stroma` is therefore a small residual (5-10% of tissue), which makes `str_nuc_density_per_mm2` and `str_area_fraction` noisy and inflates stromal density in normal mucosa. They are still comparable across sets because every image is measured the same way, but do not read them as a real epithelium/stroma segmentation. F2-PAIRS' segmentation would give a better one. 6. **A robust covariance from 24 fields in 10 dimensions is a coarse instrument.** The ridge (0.10 x mean diagonal) matters; absolute distances would shift under a different ridge. Relative ordering of sets is stable across both distance definitions, which is what the ranking rests on. 7. **Low distance is not a goal in itself.** A near-copy of a real field scores best (`native_i2i025`). Envelope distance must always be read next to a copy screen. 8. **Sets are not balanced.** The lock experiment is intestinal-metaplasia only, so its arms are only comparable to the IM column. `fullset_pass1` was retrieved only for gastritis and mixed (10 of the 20 pass-1 outputs are missing locally, listed in the run log as `missing .../pass1/normal/*` and `.../pass1/intestinal_metaplasia/*`). 9. **`lock_release` is not 14 samples.** Its 14 PNGs are only 6 distinct files: the three lumen-release levels (0.25 / 0.4 / 0.6) emitted byte-identical images at both si0 and si3, and those four images are byte-identical to the corresponding `lock_ladder` files as well (`morphometrics/duplicates.csv`, 292 files -> 280 distinct). Its row in the ranking is therefore four images counted three times each plus two layout controls, and the release sweep produced no evidence at all. Every other set is duplicate-free. `set-summary.csv` and `arm-summary.csv` carry an `n_distinct_files` column so this is visible wherever the numbers are read. 10. **`native_gen025` is a 10-per-category sample** of 48/48/48/34 available canvases, taken at even spacing over the sorted record order (deterministic, recorded in `manifest.csv`). --- ## 9. Deliverables All under `operations/research/sgh-program-20260908/`: | file | what | |---|---| | `morphometrics/morphometrics.py` | the feature extractor + CLI (`--images` / `--manifest`, multiprocessing) | | `morphometrics/calibrate.py` | manifest build, extraction, envelope fit, scoring, plots | | `morphometrics/pareto_select.py` | Pareto front / top-K selection | | `morphometrics/perturbation_check.py` | the section-7 sensitivity check | | `morphometrics/manifest.csv` | 292 images with set, category, provenance and md5 | | `morphometrics/features.csv` | 54 features per image | | `morphometrics/envelope.json` | per-category median/MAD, MinCovDet location + covariance + inverse, held-out calibration, category separation | | `morphometrics/scores.csv` | distance, percentile vs real held-out, ring gap, worst features, per image | | `morphometrics/feature-deviations.csv` | median robust z and ratio-to-real per (set, category, feature) | | `morphometrics/set-summary.csv` | per (set, category) headline numbers | | `morphometrics/arm-summary.csv` | per generator arm (start_index, lock, LoRA), with `n_distinct_files` | | `morphometrics/duplicates.csv` | byte-identical PNGs found in the manifest | | `morphometrics/top-deviating-features.json` | the ranking behind the strip plot | | `morphometrics/perturbation-check.csv` | section 7 raw numbers | | `morphometrics/copy-screen-pass2.csv` | pass-2 copy margins, derived from the existing phikon-v2 screen, as a worked join for `pareto_select.py` | | `morphometrics/topology-vs-distance.png` | ring fraction (x) vs envelope distance (y, log), one marker per image, coloured by set, real held-out band shaded, one panel per category | | `morphometrics/discriminative-strips.png` | robust z by set for the 8 features on which generated sets deviate most, coloured by category | Note: `morphometrics/real-metrics.csv`, `real-envelope.json`, `real-run.log` and `baseline-cartoon-metrics.csv` in the same directory were written by a **different** Phase-1 package running in parallel (the cartoon/tissue-model work). They are not C1 outputs and were not touched. --- ## 10. Exact commands Python is `research/sgh-synthetic-histopathology/.venv/bin/python` throughout. `matplotlib 3.11.1` and `scikit-learn 1.9.0` were installed into that venv (nothing else); numpy 2.5.2, scipy 1.18.1, scikit-image 0.26.0, Pillow 12.3.0 were already there. ```bash cd [local]/morphometrics P=[local]/python # one-shot: manifest -> features -> envelope -> scores -> plots (3 min 33 s on 12 cores) $P calibrate.py --jobs 12 # re-fit only (reuses features.csv) $P calibrate.py --jobs 12 --reuse-features # score an arbitrary batch with the extractor alone $P morphometrics.py --images /path/to/*.png --set my_set --category intestinal_metaplasia \ --out /tmp/my-features.csv --jobs 12 # sensitivity check (11 held-out fields x 5 variants, ~2 min) $P perturbation_check.py --per-category 3 # selection: two objectives (ring fraction near the real median, low envelope distance) $P pareto_select.py --features scores.csv --category intestinal_metaplasia --n 5 # selection: three objectives, adding the copy-screen margin (bigger margin = safer) $P pareto_select.py --features scores.csv --category normal --n 5 --sets fullset_pass2 \ --join copy-screen-pass2.csv --third copy_margin --third-direction max \ --out /tmp/normal-top5.csv ``` Verification that the ring half of the instrument is unchanged: the per-category means of `ring_with_lumen_fraction` recomputed here match `operations/research/pixcell-fullset-20260908/ring-summary.json` to 4 decimal places on all ten cells it reports (pass 1 gastritis 0.6284, pass 1 mixed 0.6652, pass 2 normal 0.7256, pass 2 gastritis 0.4753, pass 2 IM 0.9170, pass 2 mixed 0.7260, real held-out 0.8538 / 0.7428 / 0.7101 / 0.7261). --- ## 11. What failed or was skipped - 10 of the 40 `pixcell-fullset-20260908` pass-1 outputs (normal, intestinal metaplasia) are recorded in `records.jsonl` but not present locally, so `fullset_pass1` covers only gastritis and mixed. Logged by `calibrate.py` as `manifest notes: missing ...`. - 14 `pixcell-lock-20260907` records point at `release/...` while the files live in `release-lockonly/...`; `calibrate.py` rewrites that prefix. All 54 lock records therefore resolve to the 40 PNGs that exist. - `operations/research/pixcell-cartoons-v2/` is byte-identical to `pixcell-fullset-20260908/cartoons/` (md5 over all 20 cartoon PNGs in each directory agrees), so the cartoons were scored once, from the latter. - **The lock experiment's lumen-release sweep produced no distinct outputs.** All three release levels at si0 and at si3 wrote byte-identical PNGs, and each is also identical to the matching `ladder/K_*_lock_si*.png`. 12 of the 14 `release-lockonly` files are therefore copies of 4 images. Worth checking whether the release parameter reached the sampler at all before that arm is repeated. - No GPU work; nothing was uploaded or generated. 0 extraction failures across 292 images.