Grail Computer · research record
Dated experiment record. Statements describe this run, not current submission readiness. Historical H. pylori category names do not establish infection; copy screens are bounded tests. Claims that a source does not exist mean none was identified in that recorded search, not proof of absence. Licensing interpretations in the notes remain unconfirmed. Local access details have been omitted. Current limitations and remaining work.

REAL-SOURCES — result

Work package: REAL-SOURCES, SGH synthetic gastric histology programme. Date: 2026-09-09. Deadline: 11 September 2026, 18:00 SGT. No git commits made.


Headline

The 70 real images can be delivered, from one source, at native resolution, under a clean CC BY 4.0 licence — except for the H. pylori organism label, which no public source can supply.


What was produced

operations/research/sgh-selection-70/real-sources/
├── INVENTORY.md              what Yash downloaded, what is on the workbench, per-source verdict
├── SOURCES.md                the public-source survey, the H. pylori assessment, the 70 composition
├── REAL_SOURCES_RESULT.md    this file
├── hiesd_field_yield.csv     per-slide category capacity for all 104 HiESD slides
├── hiesd_field_yield.py      the survey script (runs on the workbench, masks only, no WSI reads)
├── measure_pixel_scale.py    the LoG scale-space um/px estimator used on the Commons images
├── licence-evidence/         dated API snapshots of the figshare licence fields + README
└── candidates/
    ├── hiesd/                4 verification fields, 4096x2048 @0.25 um/px + provenance.csv
    │                         + CONTACT_SHEET.png
    └── wikimedia-commons/    14 CC BY-SA micrographs + provenance.csv (assessed, NOT recommended)

On the IOFlood workbench:


What to submit as the "source(s) of the real images" text

The 70 real images are 4096 x 2048 px fields cut from HiESD — A Fully Annotated Pathology Slide Dataset for Early Gastric Cancer and Precancerous Lesions (Ge et al., Scientific Data, 2025; DOI 10.1038/s41597-025-05679-1), deposited on figshare under CC BY 4.0 at DOI 10.6084/m9.figshare.28919840. The slides are H&E-stained gastric endoscopic submucosal dissection specimens scanned at 40x objective; each reports openslide.mpp-x = 0.2458 um/px. Each field was read at 4165 x 2083 level-0 pixels — the exact extent of 1024 x 512 um — and resampled down by a factor of 1.017 to 4096 x 2048 px at 0.25 um/px. No image was upscaled. Category assignment follows the pathologist region annotations distributed with the dataset (Annotation_Category_Label_Description.txt): complete and incomplete intestinal metaplasia, chronic gastritis, chronic atrophic gastritis, and normal glands. Any field containing a carcinoma annotation was excluded.

Note on the H. pylori categories: HiESD does not record Helicobacter pylori organism status. Images assigned to categories (i) and (iii) are assigned on the annotated chronic gastritis pattern — the histological context in which H. pylori infection presents — and not on an observed organism. No publicly available, redistributable H&E dataset was found that provides verified H. pylori organisms at 40x with a field of view sufficient for a 4096 x 2048 image at 0.25 um/px.

The last paragraph must not be dropped, softened, or moved to a footnote.


What remains impossible

1. A real image with a verified H. pylori organism, at 4096 x 2048 and 0.25 um/px, that we are allowed to redistribute. This does not exist in the public record as of 9 September 2026.

The search was thorough and the failure is structural, not incidental:

Candidate Why it fails
DeepHP (Zenodo 10.5281/zenodo.8117177) — 394,926 images, 111k HP-positive Embargoed to 2030-12-31, plus DUA and application. And at 2776 x 2080 it would still be 1.5x short.
NEU H. pylori dataset (github.com/ailhan-NEU/...) — 204 H&E images, Sydney HP density grades in the filenames CC BY-NC-ND 4.0. ND forbids distributing adapted material; any crop or resample is adapted material. Separately, 1280 x 960 is a 3.2x upscale short.
IEEE DataPort HP set (10.21227/ryks-j091) — actual organism point coordinates The annotations are on immunohistochemistry, not H&E. Also behind a subscription.
Girona/Barcelona HP work (arXiv:2412.13857) IHC, and never publicly released.
Wikimedia Commons HP micrographs Organisms genuinely visible, but they are 400x-1000x oil-immersion camera frames: measured at 0.039-0.16 um/px, covering only 74-684 um of tissue against the 1024 x 512 um required. The images that show the organisms best have the smallest fields. All CC BY-SA (ShareAlike).
Paper supplementary figures 1000-2000 px, JPEG-compressed. Upscaling 2-4x would destroy the 2.5-5 um curved-rod morphology that is the entire point of the category.
HiESD itself Right resolution, right field, right licence — no H. pylori class. Our own detector sweep (B1_RESULT.md) found 0 likely organisms in 60 hand-graded candidates across 753 mucosal-surface windows on 13 slides, with the negative arm scoring like the positive arm.

Zenodo returns 8 records for "Helicobacter pylori" AND (histopatholog* OR histolog* OR "whole slide" OR biopsy) restricted to datasets; seven are clinical tables with no images and the eighth is the embargoed DeepHP. There is no Kaggle equivalent. There is no gastric CAMELYON/PANDA. The gap is real.

2. A guarantee of 70 distinct patients. HiESD is 104 slides from 44 patients but publishes no slide-to-patient map. We can guarantee 70 fields from distinct slides with 300 um minimum spacing; we cannot claim 70 distinct patients and must not.

3. A pathologist's confirmation of any category assignment. Every label in this package traces to the source dataset's own pathologist annotations. Nothing here has been re-read by a GI pathologist.


Two things to resolve before upload

HiESD licence conflict — 10 minutes of work. The figshare data record returns {"value": 52, "name": "CC BY", "url": "https://creativecommons.org/licenses/by/4.0/"} through the API — captured first-hand today in licence-evidence/hiesd-figshare-record-20260909.json, along with the file manifest (114 files, 104 .svs, 58.65 GB). The Scientific Data article page renders a CC BY-NC-ND notice. These govern different objects and the data record is the one that covers the .svs files — but a reviewer will ask. Screenshot the figshare page in a browser on the day of submission and file it in licence-evidence/. The API snapshot is already there; the browser view is what a reviewer will compare against.

GasHisSDB licence conflict — relevant only if it is cited anywhere. figshare says CC BY 4.0; the arXiv paper says the database is published "for non-commercial purposes"; the Gitee mirror declares no licence at all. Since GasHisSDB contributes nothing to the 70, the simplest fix is not to cite it as a source of submitted images.


Method notes, for anyone who needs to re-run or challenge this

The whole-collection capacity survey never touched a whole-slide image. HiESD ships ESD_40X_annotation_downsample64 masks for all 104 slides, and those are already on the workbench. hiesd_field_yield.py slides a 4165 x 2083 level-0 window (65 x 33 mask pixels at downsample 64) across each mask with integral images, and greedily selects disjoint windows meeting each category rule, excluding any window containing a carcinoma label. That is how a 104-slide capacity table was produced without downloading 58 GB. Results: hiesd_field_yield.csv.

The Commons pixel-scale measurement is calibrated, not guessed. No Commons file states a pixel size, so measure_pixel_scale.py estimates it: the scale-normalised Laplacian-of-Gaussian response over nuclear texture peaks at a sigma proportional to 1/(um per px). Calibrated on HiESD, where openslide.mpp-x is known: K = sigma_peak * mpp = 1.61, stable to within 1.5% across the native image and its 2x and 3x downsamples. Where the peak pinned at the edge of the scale-space range the provenance row records "not measurable" rather than a fabricated number.

The four verification fields in candidates/hiesd/ are a geometry proof, not the production extraction. The HIESD-EXTRACT package owns the extraction. These four exist to demonstrate concretely that each category is reachable and that the resample is a downsample — every row of candidates/hiesd/provenance.csv records resample_factor 0.9832 and operation: downsample (Lanczos), alongside the slide id, level-0 read coordinates, source mpp and SHA-256. CONTACT_SHEET.png shows all four: goblet cells in the IM field, dense lamina propria infiltrate in the gastritis field, foveolar epithelium with inflammation beside a metaplastic gland in the co-occurrence field, and clean oxyntic mucosa in the normal field.


Recommendation

Cut all 70 from HiESD, per the composition table in SOURCES.md §6. Ship the source statement above verbatim, including the H. pylori caveat. Do not include CC BY-SA material, do not include anything from GasHisSDB or the NEU dataset, and do not upscale anything.

Send the author requests to DeepHP, GAGL and GasHisSDB anyway — none will land before 11 September, but the H. pylori gap will still be there for the next round.

Download public Markdown export