F3 — adapter-guided repaint, and a coarse-label adapter
Work package F3 of the SGH programme (operations/research/sgh-program-20260908/PLAN.md), executed
2026-09-08 on spot A100 slot d (sgh-a100-slot-d, us-central1-f). Slots a, b and c were running other
packages and were never touched. The VM was stopped at the end and is TERMINATED. Nothing was
committed to git.
Engineering evidence only. Everything below measures geometry, pixel statistics, throughput and one pathology encoder. No clinical claim is made or implied, no pathologist has seen any of these images, and nothing here says a synthetic field is diagnostically usable.
1. Headline
Two follow-ups to F2's structure adapter, on the 20 a1 across cartoons (4 categories x seeds 11-15).
| question | answer |
|---|---|
| (1) Does an adapter-guided img2img repaint keep the layout while reaching two-pass-si3 realism? | Layout yes, realism no. Layout IoU is higher than F2's from-noise (lumen 0.478-0.523 vs 0.466; ring 0.318-0.336 vs 0.257) against a two-pass si3 of 0.051/0.047, but the envelope distance D stays 0.2-2.8 worse than two-pass si3. The repaint improves D over F2 from-noise by 0.1-1.9 and no further |
(2) Does the --scale 0 ablation show the adapter is doing the holding? |
Yes, decisively and paired. Same cartoon, same seed, same start image, same donor, same 17 steps: with the adapter the layout IoU is +0.271 (ring) and +0.434 (lumen) on 20 of 20 images; without it the repaint lands on 0.045/0.051 — chance — and on the two-pass envelope. The cost of holding is +1.30 D on 16/20, +3400 nuclei per mm2 on 20/20 and -10.2 um2 of median nucleus area on 20/20 |
| (3) Does the coarse (5-channel, no nucleus channels) adapter fix nuclear count and size? | Yes. Paired against F2's 9-channel adapter on the same cartoons: nuclear density 6655 vs 10070 per mm2 against a real 6376; median nucleus area 24.2 vs 17.6 um2 against a real 23.1. From +58% and -24% error down to +4% and +5%. It costs 15-21% of the layout IoU (lumen 0.395 vs 0.466, ring 0.202 vs 0.257) |
| (4) Is any single configuration inside the real range on topology, within ~1 D of the two-pass-si3 envelope and above 0.25 layout IoU? | Yes — the coarse adapter, from noise. 11 of its 20 images clear all three on the iou_lumen instrument the brief names (3/5 normal, 3/5 gastritis, 2/5 IM, 3/5 mixed). No other arm measured here clears more than 7/20, and the three two-pass arms measured on the same cartoons (pass 1, si6, si3) clear 0/20 each. The other sweep-v1 experiments (a2, a3, d1, d2, e2, b1) were not put on this bar |
The single most useful sentence: F2's adapter was over-conditioned. The 9-channel one-hot at 2 um per latent cell told it where each nucleus goes, and it obeyed — which is why it controlled layout and why it made 1.4-2.1x too many nuclei, each 20-40% too small. Deleting the two nucleus channels (and the mucin/vessel/red-cell detail) removes the defect and keeps most of the control.
Everything on one line, medians over the 20 cartoons
| arm | ring | D | IoU lumen | IoU ring | nuc/mm2 | median nuc um2 | Laplacian | rings/mm2 |
|---|---|---|---|---|---|---|---|---|
| real held-out (n=22) | 0.841 | 3.51 | — | — | 6376 | 23.1 | 22.8 | 95 |
two-pass si3 (sweep-v1) |
0.481 | 5.77 | 0.051 | 0.047 | 5935 | 29.6 | 19.2 | 119 |
| F2 adapter, 9-ch, from noise | 0.899 | 9.03 | 0.466 | 0.257 | 10070 | 17.6 | 19.3 | 204 |
| A si6 donor | 0.897 | 7.51 | 0.510 | 0.319 | 9173 | 18.0 | 19.3 | 204 |
| A si3 donor | 0.953 | 7.53 | 0.478 | 0.318 | 9311 | 17.8 | 18.7 | 201 |
| A si3 shuffled tokens | 0.946 | 7.75 | 0.523 | 0.336 | 9394 | 17.6 | 18.7 | 199 |
| A si3 adapter OFF | 0.568 | 5.44 | 0.051 | 0.045 | 6054 | 28.0 | 20.6 | 122 |
| B coarse, from noise | 0.784 | 4.60 | 0.395 | 0.202 | 6655 | 24.2 | 19.9 | 166 |
| B coarse, si3 repaint | 0.764 | 4.40 | 0.357 | 0.188 | 6351 | 25.8 | 18.2 | 171 |
2. What was built
Everything is under packages/f3/. Total upload 0.4 MB of code: the F2 run root was still on slot
d's disk, so the 60 label maps, the segmented data/, F2's final checkpoint and F2's 64 from-noise PNGs
were read in place and nothing heavy moved in either direction.
2.1 adapter-train/infer_adapter.py — --start-image and --token-shuffle
--start-image <png> --start-index k. The image is VAE-encoded throughpixcell_generate.encode_source(unchanged: tiling above 2 Mpx,((mode - shift) * scale), cast fp16), noised to schedule stepkwithscheduler.add_noise(..., timesteps[:1])afterset_begin_index(k), and denoised fromtimesteps[k:]— the same constructiondenoise_canvasalready used for--init cartoon, now able to start from any canvas-sized PNG. The adapter keeps injecting the label window at every one of the remaining steps, which is the whole point of the experiment and is asserted by the smoke test (adapter_calls == denoiser_calls, per injection point).--token-shuffle.shuffle_tokens(tokens, seed)permutes the 16 UNI tokens inside each window withnumpy.random.default_rng(seed * 1000 + window_index)— byte-for-byte the streampixcell_generate.patchbag_rng/shuffle_token_gridsuses, so an A4-style shuffle and this one are the same draw. The permutations are written into the provenance JSON.- Provenance now carries
adapter_sha256,start_image,start_image_sha256,start_index,token_shuffle,token_shuffle_seed_base,token_shuffle_permutations,label_groupsandoutput_sha256. - From-noise behaviour is unchanged when the flags are absent — the smoke test asserts the latent is bit-identical to the pre-F3 code at the same seed.
2.2 pair_dataset.py / train_pairs_adapter.py — --label-groups
parse_label_groups accepts a preset (coarse5) or an explicit partition (0|1|2,3,4|5,6|7,8) and
validates that it is a partition of the 9 tissue3d ids, so the conditioning channels still sum to 1 in
every latent cell. The merge happens inside label_one_hot_latent when the one-hot is area-pooled to
the 128x128 latent grid, so no label PNG on disk is rewritten and groups=None reproduces F2's
9-channel tensor exactly (asserted). coarse5 is
| channel | tissue3d ids | name |
|---|---|---|
| 0 | 0 | background above surface |
| 1 | 1 | lumen |
| 2 | 2, 3, 4 | epithelium (cytoplasm + goblet mucin + epithelial nucleus) |
| 3 | 5, 6 | stroma (+ stromal nucleus) |
| 4 | 7, 8 | vessel + red cell |
The grouping is written into the checkpoint (label_groups, label_groups_spec,
label_group_names, cond_channels), and infer_adapter.py / infer_jobs.py rebuild the identical
conditioning from the checkpoint rather than from a flag — so an adapter cannot be run against the wrong
conditioning by mistake (a channel-count mismatch is a hard error).
2.3 adapter-train/infer_jobs.py — one process, many generations (new)
F2-TRAIN's planner spawned one infer_adapter.py per image and paid 79 s wall for 46.3 s of
generation — about 35 minutes of A100 spent re-reading weights. This driver loads PixCell, the SD3 VAE
and UNI2-h once and loops, and caches
- adapter checkpoints by path,
- UNI tokens by donor field — one 4096x2048 donor is 21 windows x 16 crops and costs 8.4-8.6 s (measured here, matching SWEEP's 8.3 s), and part A reuses each donor across four arms,
- the per-window label conditioning by (label map, rotate180, grouping).
It writes the same <out>.png + <out>.png.json pair, one record per job in a --records jsonl, and
skips a job whose output PNG already exists and is non-empty, so a preemption resumes where it
stopped. The model is loaded lazily, on the first job that actually has to run, so a fully-resumed run
loads nothing. Measured effect: part A cost 48.1 s per image all-in against F2's 79 s — 41 minutes
and about USD 1.4 saved on this package alone.
2.4 plan_jobs.py and worker.sh
plan_jobs.py writes the three job manifests on the VM (every path absolute there), resolves the donor
as REFS_ROOT/<category>/<file> with the A1 rotation (seed - 11) mod 5, and exits 2 with
MISSING_INPUTS if any label map, donor or start image is absent — the FLEET gotcha-8 failure (a step
that "succeeds" having done nothing) made explicit. worker.sh runs GPU check -> input check ->
scipy guard -> CPU smoke -> plan A -> part A -> part B training -> plan B -> part B from-noise -> part B
si3, with F2's evidence pattern (steps.log, exit_code.txt, SHA256SUMS, WORKER_DONE) and the
20-minute self-stop under unit sgh-fleet-stop with the stop / reset-failed / re-arm dance from FLEET
gotcha 4 and SWEEP fix 3. F2's data/ is reached through a symlink because the manifest stores label
paths relative to its run root.
2.5 CPU smoke test — 56 checks, 0 failures, on the Mac and again on the VM
run_cpu_smoke.py keeps F2's 27 checks (so a regression in the shared code is still caught) and adds 29.
It ran 56 passed / 0 failed in 5 s on the Mac (Python 3.14) before staging and 56 passed in 32 s
on the VM (Python 3.12) before any GPU work. The new sections:
6 coarse label groups
[PASS] coarse5 is a 5-way partition of the 9 tissue3d labels
[PASS] a non-partition grouping is rejected
[PASS] coarse conditioning is (5,128,128)
[PASS] each coarse channel is the sum of its fine channels -- max |d| 0.000e+00
[PASS] coarse channels are still a partition (sum == 1 per latent cell)
[PASS] the nucleus channels are gone (4 and 6 folded into epithelium/stroma) -- 5 vs 9
[PASS] groups=None reproduces the F2 9-channel one-hot exactly
[PASS] a coarse-conditioned adapter trains -- {"steps": 2, "cond_channels": 5}
[PASS] the checkpoint records the grouping -- [[0],[1],[2,3,4],[5,6],[7,8]]
[PASS] the reloaded adapter takes 5 mask channels
[PASS] infer rebuilds the SAME conditioning from the checkpoint's grouping
7 --start-image, --token-shuffle, --scale 0
[PASS] the start image encodes to the canvas latent -- (1, 16, 128, 192)
[PASS] start_index 6 of 20 runs 14 steps -- t0=699
[PASS] start_index 3 of 20 runs 17 steps (heavier repaint)
[PASS] the adapter is injected at EVERY step of the repaint, not only the first -- [14,14] vs 14
[PASS] an img2img start changes the result vs from noise
[PASS] from-noise is bit-identical to before the F3 change (same seed, no new flags)
[PASS] the shuffle stream is default_rng(seed*1000 + window_index)
[PASS] every window gets its OWN permutation
[PASS] shuffling permutes the 16 tokens and invents none
[PASS] shuffled tokens change the output
[PASS] --scale 0 is a real ablation: no residual is added at all -- [0, 0]
[PASS] --scale 0 changes the image (so arm 4 is not arm 2 under another name)
8 infer_jobs single-process driver
[PASS] the driver ran every job
[PASS] the driver loaded the model ONCE and the donor tokens once
[PASS] provenance carries the adapter sha, start-image sha, start_index and permutation
[PASS] provenance records the coarse grouping the adapter was trained with
[PASS] the scale-0 arm is recorded as scale 0 and differs from the scale-1 arm
[PASS] a job whose PNG already exists is skipped, not regenerated
--scale 0 is a genuine ablation, not a small residual: StructureAdapterAttachment._residual returns
None when scale == 0.0, so the injection hook adds nothing at all and the reported injection counts
are [0, 0]. Measured on the GPU, it also runs 0.28 s faster per 4096x2048 canvas (38.92 s vs 39.20 s).
3. Run timeline (UTC, 2026-09-08)
| time | event |
|---|---|
| 18:56:19 | fleet.sh up d — spot A100 acquired first attempt, guest deadline +32400 s |
| 18:57:45 | ssh ready, NVIDIA A100-SXM4-40GB, 40960 MiB, driver 580.173.02 (86 s from start) |
| 18:58:16-18:58:48 | fleet.sh stage d ../packages/f3 — 5 items, 12 files, 32 s (0.4 MB of code) |
| 18:59:01 | launch (returned in 8 s) |
| 18:59:11 | inputs verified: 64 F2 from-noise PNGs, 60 label maps, data/manifest.json, latest.txt, refs |
| 18:59:11 | scipy 1.17.1 skimage 0.26.0 numpy 1.26.4 — F2-TRAIN's repair survived on this disk, so the guard installed nothing |
| 18:59:11-18:59:43 | CPU smoke on the VM: 56 passed, 0 failed, 32 s |
| 18:59:43 | part A planned: 80 jobs, all inputs present |
| 19:00:06-19:01:17 | PixCell + UNI2-h loaded once; first image |
| 20:03:50 | part A done: jobs=80 ok=80 skipped=0 failed=0 seconds=3845 |
| 20:03:52 | part B training starts (coarse 5-channel adapter) |
| 20:04:14 | 64 train / 32 val fields, slides held out ['hiesd-324885ef','hiesd-9ded7e41','hiesd-da8667dc'] — the same split F2 used |
| 20:04:43 | validation loss @ step 0: 0.1641 — bit-identical to F2's step 0 (see 5.1) |
| 22:20:53 | step 3000, validation loss 0.1612, checkpoint step_003000 |
| 22:20:54 | done: 3000 steps in 8198s (2 h 17 m, 2.73 s/step) |
| 22:20:55-22:45:10 | part B from-noise: jobs=24 ok=24 skipped=0 failed=0 seconds=1453 |
| 22:45:11-23:04:22 | part B si3 repaint: jobs=20 ok=20 skipped=0 failed=0 seconds=1149 |
| 23:04:23 | WORKER_DONE, exit_code.txt = 0, 131 PNGs, out/SHA256SUMS (149 files) |
| 23:05:37 | the 20-minute self-stop timer stopped by hand so it could not race a 2 GB retrieve |
| 23:05:42-23:20:53 | fleet.sh retrieve d …/packages/f3/retrieved — 2.0 GB, 911 s, SHA256SUMS OK (149 files), 131 PNGs |
| 23:26:38-23:27:34 | fleet.sh down d; fleet.sh status confirms TERMINATED |
Preemptions: none in 4 h 30 m, so the resume path was again not exercised by this package. Failures: none. 124 of 124 planned generations landed, no job was skipped and no job failed.
One optimisation worth recording: at 20:08, while part B was training, the 80 part-A PNGs and the
evidence directory were pulled with a targeted tar-over-ssh (1.1 GB in 4 min, GPU untouched), so the
whole part-A evaluation ran during the 2 h 17 m training window instead of after it. It cost about
USD 0.13 of duplicated egress.
Cost and powered time
From fleet/state/d/powered.log (UP 1788893822 -> DOWN 1788910054).
| powered window | 2026-09-08 18:57:02Z -> 23:27:34Z |
| powered minutes | 270.5 min (4.509 h), one session, zero preemptions |
| spot A100 (a2-highgpu-1g) @ USD 2.12/h | USD 9.56 |
| egress, 3.1 GB to the Mac @ ~USD 0.12/GB (2.0 GB retrieve + the 1.1 GB early part-A pull) | ~USD 0.37 |
| total attributable to this package | ~USD 9.93 |
Where the 270 minutes went: 3 boot/stage/smoke, 64 part A, 137 part B training, 41 part B
generation, 15 retrieve, 1 shutdown, ~9 of orchestration slack. GPU-busy fraction 89%. VRAM peaked at
6.86 GB of 40 GB during training (nvidia-smi 7658 MiB), so batch 16 would still have fitted; the run
was encoder-bound, as F2's was.
4. What was generated
Part A — adapter-guided pass 2. The 20 a1 across cartoons, F2's final checkpoint
(structure_adapter.safetensors sha d3fc7a86f6…), start image = F2's own from-noise output for that
cartoon (all 80 records' start_image_sha256 verified equal to the sha of the retrieved F2 PNG),
donor = that category's donor (seed - 11) mod 5 from donors.json resolved under REFS_ROOT, which is
the A1 sweep's own rotation — so these outputs are donor-matched with sweep-v1's a1 arms on the same
cartoons, which F2-TRAIN's comparison was not.
| arm | start_index | steps run | seed | tokens | adapter | s/image (median) |
|---|---|---|---|---|---|---|
A_si6_donor |
6 | 14 (84 denoiser calls) | 51002 | donor | scale 1.0 | 32.27 |
A_si3_donor |
3 | 17 (102 calls) | 51003 | donor | scale 1.0 | 39.20 |
A_si3_shuffled |
3 | 17 | 51003 | donor, 16 tokens permuted per window | scale 1.0 | 39.20 |
A_si3_adapter_off |
3 | 17 | 51003 | donor | scale 0.0 | 38.92 |
Part B — coarse-label adapter. cond=labels, label_groups=coarse5, everything else exactly F2's
config (3000 steps, batch 4 at 1024, lr 1e-4 cosine + 100 warmup, wd 0.01, clip 1.0, CFG dropout 0.10,
conditioning dropout 0.10, bf16, gradient checkpointing, checkpoint every 500, validate every 250,
seed 0, val_split by-slide). Adapter width=320 blocks=4 injections=2 at layers [0, 14] of 28 =
8.20M trainable parameters (F2's 9-channel one was 8.25M; the difference is the 4 fewer input
channels). UNI check: max abs diff 0.000e+00 against pixcell_generate.uni_tokens, as in F2.
Then, with checkpoint sha e49b6e1f02…: from-noise on the same 20 cartoons plus one --rotate180 per
category (seed 51001, the seed F2 used, 20 steps / 120 calls, 45.84 s median), and the adapter-guided
si3 repaint of those 20 outputs (seed 51003, start_index 3, 102 calls, 38.99 s median).
5. Part B training
5.1 An instrument check that came free
The validation loss at step 0 is 0.1641 — the same four decimals F2 recorded. It has to be: at step 0
the zero-init heads contribute nothing, so the conditioning cannot matter, and the crops, the noise and
the timesteps are fixed (linspace(50, 950), manual_seed(1234 + batch)). Getting the same number
confirms the by-slide split picked the same three slides, the same 32 val fields and the same 128 fixed
crops, so the two runs' validation curves are directly comparable.
5.2 The curve
| step | F2, 9 channels | B, 5 channels |
|---|---|---|
| 0 | 0.1641 | 0.1641 |
| 250 | 0.1611 | 0.1619 |
| 500 | 0.1604 | 0.1616 |
| 750 | 0.1601 | 0.1617 |
| 1000 | 0.1598 | 0.1615 |
| 1250 | 0.1598 | 0.1614 |
| 1500 | 0.1596 | 0.1613 |
| 1750 | 0.1595 | 0.1614 |
| 2000 | 0.1595 | 0.1612 |
| 2250 | 0.1595 | 0.1612 |
| 2500 | 0.1595 | 0.1612 |
| 2750 | 0.1594 | 0.1612 |
| 3000 | 0.1594 (-2.86%) | 0.1612 (-1.77%) |
grad_norm median 0.0117, min 0.0029, max 0.2483 over all 3000 steps — never zero, so failure criterion 3 does not apply here
either. Throughput 2.73 s/step steady state, 8198 s for 3000 steps.
Read honestly: the coarse adapter learns about 62% of what the 9-channel adapter learned on the epsilon-MSE, and it plateaus earlier (flat from step ~1000, against F2's ~1500). The four channels that were removed — epithelial nucleus, stromal nucleus, goblet mucin and red cell — carry 38% of the total learnable validation gain. That is exactly what you would expect if most of what the fine adapter learned was where to put nuclei, and it is the loss-curve counterpart of section 7's morphometry: the fine adapter's extra "knowledge" was the defect.
6. Part A: does an adapter-guided repaint hold the layout?
All numbers are medians over the 20 cartoons unless a table says otherwise; the full tables are
packages/f3/eval/tables.md and the per-image rows are features.csv / layout-iou.csv.
6.1 Two layout instruments, both reported
The programme has been quoting two different numbers under the name "layout IoU" and F3 has to be comparable with both, so every arm carries both:
iou_lumen— EVAL-RUN'sa1-layout-iou:evaluate_sweep.analysis_masks(...)["lumen"](pale components at least the sydney minimum lumen area that are not border-connected background) againstlabel_mask(..., 1). Reference ladder on these cartoons: pass 1 0.493, si6 0.026, si3 0.051; the cartoon's own self-IoU (the ceiling) is 0.709.iou_ring— F2-TRAIN's T3/T4 instrument:f2_eval.ring_mask(the accepted ring interiors — holes of the closed hematoxylin band inside the sydney ring-area gate) against the same label lumen mask. Reference: F2 adapter 0.26-0.56, two-pass 0.10-0.13, chance 0.03-0.06. On these 20 cartoons the cartoon's own self-IoU is only 0.211, so this instrument is not bounded by the cartoon and a value above 0.21 is possible (A gastritis reaches 0.464).
Instrument self-check. Recomputing F2-TRAIN's T4 rotation table with this code reproduces its published numbers to four decimals — gastritis 0.4599 / 0.0417, IM 0.2173 / 0.0397, mixed 0.2634 / 0.0389, normal 0.3494 / 0.0283, and the unrotated column 0.5170 / 0.1982 / 0.2566 / 0.3519. The morphometric side reproduces C1's held-out Mahalanobis distances to max |delta| 4.9e-06. Nothing was re-fitted.
A caveat EVAL-RUN had to carry does not apply here. EVAL-RUN measured layout IoU against the Mac
label maps while the sweep's generator saw VM-rendered cartoons. F2 uploaded its label maps, so F2 and
F3 were conditioned on exactly the bytes measured here — verified by sha256 (normal_s11_across_labels.png
7da4ac55… on the Mac and on the VM). The two-pass reference rows still carry the original caveat.
6.2 Layout, per category
iou_lumen, mean per category, with two independent chance levels (the same map rotated 180, and a
different cartoon of the same category):
| arm | normal | gastritis | IM | mixed |
|---|---|---|---|---|
| a1 cartoon itself (ceiling) | 0.858 | 0.841 | 0.409 | 0.549 |
| two-pass pass 1 (si12) | 0.628 | 0.659 | 0.347 | 0.462 |
| two-pass si3 | 0.054 | 0.057 | 0.048 | 0.053 |
| F2 adapter from-noise | 0.703 | 0.466 | 0.371 | 0.468 |
| A si6 donor | 0.602 | 0.621 | 0.306 | 0.461 |
| A si3 donor | 0.553 | 0.540 | 0.362 | 0.486 |
| A si3 shuffled | 0.602 | 0.569 | 0.385 | 0.511 |
| A si3 adapter OFF | 0.054 | 0.062 | 0.045 | 0.051 |
| chance (rot / other), all arms | 0.02-0.06 | 0.00-0.06 | 0.02-0.04 | 0.02-0.05 |
iou_ring:
| arm | normal | gastritis | IM | mixed |
|---|---|---|---|---|
| a1 cartoon itself | 0.264 | 0.368 | 0.188 | 0.210 |
| two-pass si3 | 0.052 | 0.047 | 0.042 | 0.051 |
| F2 adapter from-noise | 0.319 | 0.495 | 0.212 | 0.252 |
| A si6 donor | 0.345 | 0.464 | 0.285 | 0.286 |
| A si3 donor | 0.342 | 0.453 | 0.275 | 0.283 |
| A si3 shuffled | 0.356 | 0.478 | 0.300 | 0.298 |
| A si3 adapter OFF | 0.047 | 0.044 | 0.039 | 0.048 |
A heavy repaint with the adapter active keeps and in three of four categories slightly improves the
layout F2's from-noise pass established: on iou_ring the A arms are 0.28-0.48 against F2's 0.21-0.50,
with IM going 0.212 -> 0.275-0.300 and mixed 0.252 -> 0.283-0.298. Only gastritis loses a little
(0.495 -> 0.453-0.478). The same repaint with the adapter switched off lands on chance.
6.3 The ablation, fully paired
Same cartoon, same seed 51003, same start image, same donor, same 17 steps — the only difference is
--scale 1 vs --scale 0. Paired over all 20 cartoons:
| quantity | adapter ON | adapter OFF | median paired delta | sign consistency |
|---|---|---|---|---|
iou_lumen vs own label map |
0.478 | 0.051 | +0.434 | 20/20 |
iou_ring vs own label map |
0.318 | 0.045 | +0.271 | 20/20 |
| envelope distance D | 7.53 | 5.44 | +1.297 | 16/20 |
| nuclei per mm2 | 9311 | 6054 | +3400 | 20/20 |
| median nucleus area um2 | 17.8 | 28.0 | -10.2 | 20/20 |
| mean absolute Laplacian | 18.7 | 20.6 | -1.93 | 20/20 |
| within-arm pairwise SSIM | 0.26-0.33 | 0.50-0.58 | +0.22 | all 4 categories |
Three independent lines say the same thing. (a) The layout is entirely the adapter's doing: without it a 17-step repaint of an adapter-made canvas erases the geometry back to chance, and back to exactly the two-pass si3 number (0.051 / 0.045 vs 0.051 / 0.047). (b) The adapter-off arm also lands on the two-pass si3 envelope (paired delta -0.02 to -0.51 D, 3-4 of 5 closer per category), so "adapter off" is not a broken arm — it is the two-pass recipe reproduced from a different pass-1, which is the strongest available check that the ablation changed one thing only. (c) The adapter also preserves diversity: five different cartoons stay distinct with it on (SSIM 0.26-0.33, matching F2's 0.25-0.33) and collapse toward one donor-determined appearance with it off (0.50-0.58, matching two-pass si3's 0.49-0.57), which is EVAL-RUN's finding (c) — "by pass 2 the cartoon is gone and the donor decides the canvas" — reproduced and then defeated.
And the cost is in the same table: holding the layout costs +1.3 D, +3400 nuclei per mm2 and -10 um2 of nucleus size. The adapter is not merely failing to fix the nuclei during the repaint; it is actively re-stamping them at every one of the 17 steps.
6.4 The two nulls in part A
- Token shuffling does nothing. Permuting each window's 16 UNI tokens changes D by +0.24 (median),
iou_ringby +0.02, Laplacian by +0.03 and the ring fraction by 0.000. Whatever the 16 tokens per window encode about where appearance goes inside that window, this pipeline does not use it — which is consistent with the tokens entering as an unorderedencoder_hidden_statessequence for cross-attention. - Repaint depth barely matters once the adapter is on. si6 (14 steps) vs si3 (17 steps): D -0.15,
iou_ring+0.005, Laplacian +0.42. The adapter re-imposes its conditioning at every step, so more steps do not buy the repaint more freedom. (The one place they differ is the ring fraction for IM and mixed, where si6 is 0.518 / 0.619 and si3 is 0.896 / 0.859.)
6.5 Realism: the repaint helps, and stops well short
Paired on the same cartoon, median delta in D (negative = closer to the real envelope):
| arm | vs F2 from-noise | vs two-pass si3 |
|---|---|---|
| A si6 donor | -0.12 / -1.80 / -0.46 / -1.87 | +1.20 / +2.27 / +0.27 / +1.64 |
| A si3 donor | -0.70 / -1.42 / -0.60 / -1.76 | +0.63 / +2.65 / +0.97 / +2.84 |
| A si3 shuffled | -1.13 / -1.66 / -0.36 / -1.51 | +0.20 / +2.74 / +1.07 / +2.74 |
| A si3 adapter OFF | -0.87 / -4.74 / -0.78 / -3.00 | -0.02 / -0.51 / -0.11 / -0.04 |
(normal / gastritis / IM / mixed.) So the answer to question (1) is: layout yes, realism no. The repaint moves the adapter's fields toward the real envelope in every category — by up to 1.9 D — but leaves them 0.2 to 2.8 D short of the two-pass si3 arm, and the shortfall is entirely the nuclei (T5: 8506-10674 per mm2 against a real 5996-6762, median area 14.2-19.6 um2 against a real 21.8-25.1). Ring fraction also overshoots harder after the repaint than before (0.86-0.98 against a real 0.71-0.85), so "in the real range" is satisfied for gastritis and mixed and not for normal.
7. Part B: does dropping the nucleus channels fix the nuclei?
Paired against F2's 9-channel adapter, both from noise, on the same 20 cartoons:
| quantity | B, 5 channels | F2, 9 channels | real held-out | B error | F2 error |
|---|---|---|---|---|---|
| nuclei per mm2 | 6655 | 10070 | 6376 | +4% | +58% |
| median nucleus area um2 | 24.2 | 17.6 | 23.1 | +5% | -24% |
| epithelial nuclei per mm2 | 8076 | 12645 | 7302 | +11% | +73% |
| rings per mm2 | 166 | 204 | 95 | +74% | +115% |
| ring_with_lumen_fraction | 0.784 | 0.899 | 0.841 | -0.06 | +0.06 |
| envelope distance D | 4.60 | 9.03 | 3.51 | +1.09 | +5.52 |
| mean absolute Laplacian | 19.9 | 19.3 | 22.8 | -2.9 | -3.5 |
iou_lumen |
0.395 | 0.466 | — | ||
iou_ring |
0.202 | 0.257 | — |
Yes — the coarse adapter fixes nuclear count and size, and the fix is large: the single systematic defect F2-TRAIN identified ("1.4x to 2.1x too many nuclei, each 20-40% too small") is gone. The envelope distance halves, from a median 9.03 to 4.60 against a real 3.51, and it does so in every category (T2: normal 8.51 -> 4.89, gastritis 10.21 -> 4.34, IM 5.28 -> 3.66, mixed 9.22 -> 5.04). Against the two-pass si3 arm — the realism reference — the coarse adapter is now closer to the real envelope in all four categories (paired median -1.64 / -1.57 / -0.57 / -0.85, 4-5 of 5 cartoons per category), which is the condition F2's section 8 asked for and got only for IM.
What it costs: 15-21% of the layout signal (iou_lumen 0.395 vs 0.466, iou_ring 0.202 vs
0.257 — still 4-8x the two-pass arm's 0.05, and 6-8x chance), and a loss of within-arm diversity
(mean pairwise SSIM 0.45-0.51 against F2's 0.25-0.33 and a real floor of 0.03-0.05) — note the B arm's
five images per category use five different donors and are still more alike than F2's five, which all
share one donor, so this is the coarse conditioning being less discriminative between cartoons, not a
donor artefact. Ring density is improved but still 1.7x real.
Rotation test, coarse adapter (one cartoon per category, the conditioning rotated 180 and nothing else):
| category | instrument | vs rotated map | vs unrotated map (chance) | ratio |
|---|---|---|---|---|
| normal | iou_lumen |
0.679 | 0.044 | 15.5x |
| normal | iou_ring |
0.239 | 0.028 | 8.4x |
| gastritis | iou_lumen |
0.330 | 0.065 | 5.0x |
| gastritis | iou_ring |
0.356 | 0.031 | 11.5x |
| IM | iou_lumen |
0.309 | 0.040 | 7.7x |
| IM | iou_ring |
0.106 | 0.039 | 2.7x |
| mixed | iou_lumen |
0.332 | 0.040 | 8.2x |
| mixed | iou_ring |
0.144 | 0.046 | 3.1x |
The output rotates with the conditioning in all four categories (2.7x-15.5x separation, against F2's 5.5x-17.8x on the same measurement). H04's failure mode does not reproduce with the coarse conditioning either, though IM and mixed are visibly weaker on the ring instrument.
The si3 repaint on top of the coarse adapter is a wash (paired, 20 cartoons): D +0.05, nuclei -307
per mm2, median nucleus area +1.3 um2, Laplacian -1.75, iou_lumen -0.01, iou_ring -0.01. It buys
nothing the coarse from-noise pass did not already have. On the medians it edges ahead on D
(4.40 vs 4.60) and behind on detail; there is no reason to pay 39 s per image for it.
8. Copy screens
phikon-v2 embedding screen (1024 px tiles, fail line 0.95) against all 118 real fields, plus the
pixel NCC search (64 px windows, stride 256, threshold 0.92) against each output's own donor and its
phikon-nearest reference fields — the harness's targeted policy, which is what EVAL-RUN used.
| set / arm | n | max cosine | median max cosine | own-source frac | tiles >= 0.95 | NCC pairs | max NCC | windows >= 0.92 | COPIED |
|---|---|---|---|---|---|---|---|---|---|
| two-pass si3 | 20 | 0.905 | 0.867 | 1.000 | 0 | 28 | 0.825 | 0 | 0 |
| F2 from-noise | 20 | 0.833 | 0.798 | 0.375 | 0 | 44 | 0.599 | 0 | 0 |
| A si6 donor | 20 | 0.857 | 0.807 | 0.500 | 0 | 44 | 0.655 | 0 | 0 |
| A si3 donor | 20 | 0.855 | 0.805 | 0.375 | 0 | 47 | 0.655 | 0 | 0 |
| A si3 shuffled | 20 | 0.824 | 0.794 | 0.250 | 0 | 49 | 0.710 | 0 | 0 |
| A si3 adapter OFF | 20 | 0.922 | 0.877 | 1.000 | 0 | 26 | 0.815 | 0 | 0 |
| B coarse from-noise | 20 | 0.928 | 0.896 | 1.000 | 0 | 29 | 0.741 | 0 | 0 |
| B coarse si3 | 20 | 0.937 | 0.898 | 1.000 | 0 | 28 | 0.836 | 0 | 0 |
Nothing is flagged: 0 tiles at or above the 0.95 embedding fail line and 0 NCC windows at or above 0.92, in 200 candidates and 372 pairs. The calibration for reference is: a recoloured copy scores 0.987 (embed) / 0.9995 (NCC), a generated field against its own reference 0.83 / 0.85, unrelated real fields 0.66 / 0.88.
Two honest readings. First, the arms whose appearance is donor-dominated — two-pass si3, A adapter-off, and both B arms — have own-source fraction 1.000: every 1024 tile's nearest neighbour among 118 real fields is its own token donor. For B this is worth stating precisely, because B is generated from noise and never saw the donor's pixels, only its 16 UNI tokens per window; the tokens are the donor's appearance, so "nearest to its own donor" is the expected behaviour, not evidence of copying, and the pixel NCC (0.741) is far below the copy level. Second, the margin to the fail line is smallest for the B arms (0.937 max, margin 0.013) and largest for the A/F2 adapter arms (0.824-0.857). The coarse adapter buys its realism partly by leaning harder on the donor's appearance statistics, and a future run should watch that margin.
9. Contact sheets — read with the Read tool and described honestly
packages/f3/eval/sheets/f3-<category>.png — one row per cartoon seed (s11, s12, s13), ten columns:
cartoon | F2 from-noise | A si6 donor | A si3 donor | A si3 shuffled | A si3 adapter-off | B from-noise |
B si3 | two-pass si3 | real held-out. Every cell is the same 768 px window of its 4096x2048 canvas
(evaluate_sweep.CROP_768 = (1664, 640)-(2432, 1408)), so a difference between two cells is a difference
between images, not between crops; the real held-out column is a different field, so it is the same
rectangle of a different slide. Each cell is labelled ring / D / lap / IoU_lumen / IoU_ring.
packages/f3/eval/sheets/f3-zoom-<category>.png is the same nine sources plus real at 1:1 pixels,
512 px = 128 um, because a 768 px crop scaled into a sheet cell cannot show whether a nucleus has
chromatin. sheets-small/ is the field-scale sheet at 230 px cells, which is the readable one for
layout.
What I see.
At field scale the layout claim is visible without any measurement. Across a row — same cartoon,
different arm — F2's from-noise column and the three adapter-on A columns put their pale spaces in the
same places, and those places track the cartoon's white slots; the adapter-off column and the two-pass
si3 column are a completely different field with no relation to the cartoon, and (in mixed rows 2 and
3) they resemble each other far more than either resembles the cartoon. The B columns sit in between:
they carry the cartoon's gland arrangement — in mixed s11 and s13 there is a long tall-columnar
epithelial band with an open lumen where the cartoon put one — while looking much more like tissue than
the A columns do.
At 1:1 pixels the trade is unmistakable. The A and F2 crops are a dense, fairly uniform scatter of
small round soft-edged nuclei on washed pink cytoplasm with little internal texture; the epithelium is
not organised into a polarised band and the pale holes are walled by a crowd of small nuclei rather than
a palisade. That is F2-TRAIN's description and the repaint does not change it. The adapter-off and
two-pass si3 crops are the other extreme: large elongated nuclei with visible chromatin, an organised
epithelial band, dense eosinophilic stroma — convincing tissue that ignores the label map. The B
coarse crops are the first column in this programme that looks like both: in gastritis s11 the B
from-noise crop has an epithelial surface with basally-oriented elongated nuclei and a red blood cell,
at layout IoU 0.348 against the adapter-off crop's 0.064; in IM s11 the B crops have large well-formed
nuclei with chromatin and a dark epithelial cluster, at IoU 0.332 / 0.312 against two-pass si3's 0.041.
They are still short of the real held-out column, which has speckled chromatin, crisper nuclear
membranes, red cells in vessels and a stronger stain contrast, and whose Laplacian is 22.8 against B's
19.9. In normal the A arms look plausible as oxyntic mucosa (pale parietal-like cells, small round
nuclei) and that is the category where their D penalty is smallest.
10. Verdicts
(1) Does the adapter-guided repaint keep the layout while reaching two-pass-si3 realism?
Half. Layout is kept and slightly improved: iou_ring 0.318-0.336 and iou_lumen 0.478-0.523,
against F2's from-noise 0.257 / 0.466 and the two-pass si3's 0.047 / 0.051, with two independent chance
levels at 0.03-0.06. Realism is not reached: D stays 0.2-2.8 above the two-pass si3 arm in every
category, and the whole gap is the nuclei. The repaint moves the adapter's fields 0.1-1.9 D toward real
and then stops, because the adapter re-imposes its own conditioning at every step of the repaint.
(2) Does the ablation show the adapter is doing the holding?
Yes, unambiguously. Paired on 20 images with only --scale changed: +0.434 iou_lumen and +0.271
iou_ring, 20/20 both; with the adapter off the repaint lands on chance and on the two-pass envelope
simultaneously. The ablation also localises the cost: +1.30 D (16/20), +3400 nuclei per mm2 (20/20),
-10.2 um2 median nucleus area (20/20). This is the --scale 0 vs --scale 1 control F2-TRAIN could not
afford to run, and it is now run.
(3) Does the coarse adapter fix nuclear count and size? Yes. Nuclear density 6655 per mm2 against a real 6376 (F2: 10070); median nucleus area 24.2 um2 against a real 23.1 (F2: 17.6); epithelial nuclear density 8076 against 7302 (F2: 12645). Envelope distance halves from 9.03 to 4.60 against a real 3.51, in all four categories, and it beats the two-pass si3 arm on the same cartoons in all four. The price is about a quarter of the layout IoU, a loss of within-arm diversity (SSIM 0.45-0.51 vs F2's 0.25-0.33), a ring density still 1.7x real, and an embedding-screen margin that narrows to 0.013.
(4) Which single configuration, if any, is inside the real range on topology, within ~1 D of the
two-pass-si3 envelope and above 0.25 layout IoU?
The coarse adapter run from noise (B_fromnoise, seed 51001, adapter
out/adapter-b/checkpoints/step_003000, sha e49b6e1f02…, donor = the A1 seed rotation). On the
iou_lumen instrument the brief names it clears all three conditions on 11 of 20 images, spread over
all four categories (normal 3/5, gastritis 3/5, IM 2/5, mixed 3/5); its coarse-si3 repaint clears the
same 11; F2's 9-channel adapter clears 7/20 (5 of them IM); the A repaint arms clear 3-5/20 and the adapter-off arm 0/20; and the three two-pass arms measured on these cartoons (pass 1, si6, si3) clear 0/20 each. On the stricter iou_ring > 0.25 reading the same arm clears 6/20 (normal
3/5, gastritis 3/5, IM 0/5, mixed 0/5), because B's ring-mask IoU for IM (0.149) and mixed (0.173) sits
below the bar — so the answer is instrument-dependent for IM and mixed and holds for normal and
gastritis either way.
11. What this package did not do
- No pathologist review and no clinical claim of any kind. Ring fraction, envelope distance, mask IoU, Laplacian, SSIM and one encoder's cosine are engineering measures of geometry and pixel statistics.
- The coarse grouping was not swept.
coarse5is one point. The obvious next probes are 6 channels (mucin split back out) and 4 (vessel folded into stroma), and a nucleus channel at lower spatial resolution rather than none. - One training configuration per adapter. No sweep over width, injection depth, step count or
conditioning dropout; no
cond=cartoonorlabels+cartoonarm. acrosscut only, 20 cartoons. F2 measured 60 cartoons over three cuts. EVAL-RUN foundobliquekeeps lumens best andacrossworst at pass 2, so this package measured the harder cut; the numbers are not directly comparable with F2-TRAIN's all-cut tables and the report quotes theacrosssubset of those tables wherever a comparison is made.- The preemption/resume path was not exercised — no preemption in 4 h 30 m.
infer_jobs.py's resume is covered by the smoke test only. - The mucin/goblet channel still has no dedicated measurement, as F2_RESULT section 8 asked; the coarse grouping folds it into epithelium and the sheets are the only evidence about it.
- B's within-arm diversity was not chased down. It is measurably lower than F2's and the cause is not established.
- Nothing was committed to git.
12. Files
packages/f3/worker.sh part A -> part B training -> part B generation
packages/f3/plan_jobs.py the three job manifests, built on the VM, with MISSING_INPUTS
packages/f3/STAGE_LIST code only (0.4 MB); everything heavy is F2's, read in place
packages/f3/code/pixcell_generate.py the canonical generator, byte-identical to F2's (sha 64d6103a…)
packages/f3/adapter-train/infer_adapter.py + --start-image / --start-index / --token-shuffle
packages/f3/adapter-train/infer_jobs.py NEW: one process, model + tokens + conditioning cached
packages/f3/adapter-train/pair_dataset.py + parse_label_groups / label_group_names, grouped one-hot
packages/f3/adapter-train/train_pairs_adapter.py + --label-groups, recorded in the checkpoint
packages/f3/adapter-train/config_coarse.json F2's config + "label_groups": "coarse5"
packages/f3/adapter-train/run_cpu_smoke.py 56 checks (F2's 27 + 29 new)
packages/f3/adapter-train/structure_adapter.py unedited copy (sha b85d2dba…)
packages/f3/retrieved/out/f3a/<arm>/<cat>/<stem>.png(+.json) 80 part-A canvases (1.1 GB)
packages/f3/retrieved/out/f3b/{fromnoise,si3_donor}/… 44 part-B canvases (638 MB)
packages/f3/retrieved/out/f3a-records.jsonl 80 records
packages/f3/retrieved/out/f3b-{fromnoise,si3}-records.jsonl 24 + 20 records
packages/f3/retrieved/out/adapter-b/ loss_log.jsonl, val_loss.json, summary.json,
train_config.json, checkpoints/step_{000500..003000}
packages/f3/retrieved/out/SHA256SUMS 149 files, verified after transfer
packages/f3/retrieved/evidence/ steps.log, exit_code.txt (0), runtime.txt,
smoke.log, plan_a.log, plan_b.log, infer_a.log,
infer_b1.log, infer_b2.log, train_b.log, WORKER_DONE
packages/f3/eval/f3_features.py features.csv via evaluate_sweep._feat_one + C1's envelope
packages/f3/eval/f3_layout.py both layout instruments + the rotation test, one pass
packages/f3/eval/f3_screens.py phikon-v2 + pixel NCC, the harness's own conventions
packages/f3/eval/f3_sheets.py contact sheets and the 1:1 zoom grids
packages/f3/eval/f3_tables.py every table above
packages/f3/eval/run_eval.sh the whole evaluation, in order
packages/f3/eval/{features,layout-iou,screen-embed,screen-ncc}.csv
packages/f3/eval/{layout-iou,stage-features,stage-screens,within-arm-diversity}.json
packages/f3/eval/tables.md the generated tables
packages/f3/eval/sheets/, sheets-small/ 8 + 4 contact sheets
fleet/logs/{d.log,f3-watch.out,f3-retrieve.out}, fleet/state/d/powered.log
Reproduce with:
cd operations/research/sgh-program-20260908/fleet
./fleet.sh up d && ./fleet.sh stage d ../packages/f3 && ./fleet.sh launch d ../packages/f3
./fleet.sh watch d
./fleet.sh ssh d 'sudo -n systemctl stop sgh-fleet-stop.timer'
./fleet.sh retrieve d "$PWD/../packages/f3/retrieved" && ./fleet.sh down d && ./fleet.sh status
cd .. && bash packages/f3/eval/run_eval.sh packages/f3/retrieved 10