REAL-SOURCES — result
Work package: REAL-SOURCES, SGH synthetic gastric histology programme. Date: 2026-09-09. Deadline: 11 September 2026, 18:00 SGT. No git commits made.
Headline
The 70 real images can be delivered, from one source, at native resolution, under a clean CC BY 4.0 licence — except for the H. pylori organism label, which no public source can supply.
- HiESD (figshare
10.6084/m9.figshare.28919840, CC BY 4.0, 104 whole slides, 40x,openslide.mpp-x= 0.2458) is the only public gastric source that passes both the sampling test and the field-of-view test. Every 4096 x 2048 @ 0.25 um/px field cut from it is a 1.017x downsample of level 0. Nothing is upscaled. - Supply is not the constraint. The 22 slides now on the workbench yield 316 IM-only, 275 IM+gastritis, 951 gastritis-only and 403 normal disjoint fields, against a requirement of 14 / 14 / 21 / 21.
- Nothing Yash downloaded can be used. GasHisSDB is 160 x 160 px patches — a 25.6x linear upscale short of the target — with no H. pylori label, no IM label and no published um/px.
- No public dataset annotates H. pylori organisms in H&E at a usable resolution and field size. The two that come closest are blocked: DeepHP is embargoed to 31 December 2030, and the NEU H. pylori dataset is 1280 x 960 px under CC BY-NC-ND 4.0, where ND forbids the derivative that any crop would create.
What was produced
operations/research/sgh-selection-70/real-sources/
├── INVENTORY.md what Yash downloaded, what is on the workbench, per-source verdict
├── SOURCES.md the public-source survey, the H. pylori assessment, the 70 composition
├── REAL_SOURCES_RESULT.md this file
├── hiesd_field_yield.csv per-slide category capacity for all 104 HiESD slides
├── hiesd_field_yield.py the survey script (runs on the workbench, masks only, no WSI reads)
├── measure_pixel_scale.py the LoG scale-space um/px estimator used on the Commons images
├── licence-evidence/ dated API snapshots of the figshare licence fields + README
└── candidates/
├── hiesd/ 4 verification fields, 4096x2048 @0.25 um/px + provenance.csv
│ + CONTACT_SHEET.png
└── wikimedia-commons/ 14 CC BY-SA micrographs + provenance.csv (assessed, NOT recommended)
On the IOFlood workbench:
[local]/hiesd-wsi— 6 additional HiESD slides downloaded (4.83 GB; no single file above 2 GB), every one md5-verified against figshare's publishedsupplied_md5. Ledger:[local]/dl.log.[local]/real-sources— survey and proof scripts,hiesd_yield.json,proof.json.
What to submit as the "source(s) of the real images" text
The 70 real images are 4096 x 2048 px fields cut from HiESD — A Fully Annotated Pathology Slide Dataset for Early Gastric Cancer and Precancerous Lesions (Ge et al., Scientific Data, 2025; DOI 10.1038/s41597-025-05679-1), deposited on figshare under CC BY 4.0 at DOI 10.6084/m9.figshare.28919840. The slides are H&E-stained gastric endoscopic submucosal dissection specimens scanned at 40x objective; each reports
openslide.mpp-x= 0.2458 um/px. Each field was read at 4165 x 2083 level-0 pixels — the exact extent of 1024 x 512 um — and resampled down by a factor of 1.017 to 4096 x 2048 px at 0.25 um/px. No image was upscaled. Category assignment follows the pathologist region annotations distributed with the dataset (Annotation_Category_Label_Description.txt): complete and incomplete intestinal metaplasia, chronic gastritis, chronic atrophic gastritis, and normal glands. Any field containing a carcinoma annotation was excluded.Note on the H. pylori categories: HiESD does not record Helicobacter pylori organism status. Images assigned to categories (i) and (iii) are assigned on the annotated chronic gastritis pattern — the histological context in which H. pylori infection presents — and not on an observed organism. No publicly available, redistributable H&E dataset was found that provides verified H. pylori organisms at 40x with a field of view sufficient for a 4096 x 2048 image at 0.25 um/px.
The last paragraph must not be dropped, softened, or moved to a footnote.
What remains impossible
1. A real image with a verified H. pylori organism, at 4096 x 2048 and 0.25 um/px, that we are allowed to redistribute. This does not exist in the public record as of 9 September 2026.
The search was thorough and the failure is structural, not incidental:
| Candidate | Why it fails |
|---|---|
DeepHP (Zenodo 10.5281/zenodo.8117177) — 394,926 images, 111k HP-positive |
Embargoed to 2030-12-31, plus DUA and application. And at 2776 x 2080 it would still be 1.5x short. |
NEU H. pylori dataset (github.com/ailhan-NEU/...) — 204 H&E images, Sydney HP density grades in the filenames |
CC BY-NC-ND 4.0. ND forbids distributing adapted material; any crop or resample is adapted material. Separately, 1280 x 960 is a 3.2x upscale short. |
IEEE DataPort HP set (10.21227/ryks-j091) — actual organism point coordinates |
The annotations are on immunohistochemistry, not H&E. Also behind a subscription. |
Girona/Barcelona HP work (arXiv:2412.13857) |
IHC, and never publicly released. |
| Wikimedia Commons HP micrographs | Organisms genuinely visible, but they are 400x-1000x oil-immersion camera frames: measured at 0.039-0.16 um/px, covering only 74-684 um of tissue against the 1024 x 512 um required. The images that show the organisms best have the smallest fields. All CC BY-SA (ShareAlike). |
| Paper supplementary figures | 1000-2000 px, JPEG-compressed. Upscaling 2-4x would destroy the 2.5-5 um curved-rod morphology that is the entire point of the category. |
| HiESD itself | Right resolution, right field, right licence — no H. pylori class. Our own detector sweep (B1_RESULT.md) found 0 likely organisms in 60 hand-graded candidates across 753 mucosal-surface windows on 13 slides, with the negative arm scoring like the positive arm. |
Zenodo returns 8 records for "Helicobacter pylori" AND (histopatholog* OR histolog* OR
"whole slide" OR biopsy) restricted to datasets; seven are clinical tables with no images and the
eighth is the embargoed DeepHP. There is no Kaggle equivalent. There is no gastric
CAMELYON/PANDA. The gap is real.
2. A guarantee of 70 distinct patients. HiESD is 104 slides from 44 patients but publishes no slide-to-patient map. We can guarantee 70 fields from distinct slides with 300 um minimum spacing; we cannot claim 70 distinct patients and must not.
3. A pathologist's confirmation of any category assignment. Every label in this package traces to the source dataset's own pathologist annotations. Nothing here has been re-read by a GI pathologist.
Two things to resolve before upload
HiESD licence conflict — 10 minutes of work. The figshare data record returns
{"value": 52, "name": "CC BY", "url": "https://creativecommons.org/licenses/by/4.0/"} through the
API — captured first-hand today in licence-evidence/hiesd-figshare-record-20260909.json, along with
the file manifest (114 files, 104 .svs, 58.65 GB). The
Scientific Data article page renders a CC BY-NC-ND notice. These govern different objects and
the data record is the one that covers the .svs files — but a reviewer will ask.
Screenshot the figshare page in a browser on the day of submission and file it in
licence-evidence/. The API snapshot is already there; the browser view is what a reviewer will
compare against.
GasHisSDB licence conflict — relevant only if it is cited anywhere. figshare says CC BY 4.0; the arXiv paper says the database is published "for non-commercial purposes"; the Gitee mirror declares no licence at all. Since GasHisSDB contributes nothing to the 70, the simplest fix is not to cite it as a source of submitted images.
Method notes, for anyone who needs to re-run or challenge this
The whole-collection capacity survey never touched a whole-slide image. HiESD ships
ESD_40X_annotation_downsample64 masks for all 104 slides, and those are already on the workbench.
hiesd_field_yield.py slides a 4165 x 2083 level-0 window (65 x 33 mask pixels at downsample 64)
across each mask with integral images, and greedily selects disjoint windows meeting each
category rule, excluding any window containing a carcinoma label. That is how a 104-slide capacity
table was produced without downloading 58 GB. Results: hiesd_field_yield.csv.
The Commons pixel-scale measurement is calibrated, not guessed. No Commons file states a pixel
size, so measure_pixel_scale.py estimates it: the scale-normalised Laplacian-of-Gaussian response
over nuclear texture peaks at a sigma proportional to 1/(um per px). Calibrated on HiESD, where
openslide.mpp-x is known: K = sigma_peak * mpp = 1.61, stable to within 1.5% across the native
image and its 2x and 3x downsamples. Where the peak pinned at the edge of the scale-space range the
provenance row records "not measurable" rather than a fabricated number.
The four verification fields in candidates/hiesd/ are a geometry proof, not the production
extraction. The HIESD-EXTRACT package owns the extraction. These four exist to demonstrate
concretely that each category is reachable and that the resample is a downsample — every row of
candidates/hiesd/provenance.csv records resample_factor 0.9832 and
operation: downsample (Lanczos), alongside the slide id, level-0 read coordinates, source mpp and
SHA-256. CONTACT_SHEET.png shows all four: goblet cells in the IM field, dense lamina propria
infiltrate in the gastritis field, foveolar epithelium with inflammation beside a metaplastic gland
in the co-occurrence field, and clean oxyntic mucosa in the normal field.
Recommendation
Cut all 70 from HiESD, per the composition table in SOURCES.md §6. Ship the source statement above
verbatim, including the H. pylori caveat. Do not include CC BY-SA material, do not include anything
from GasHisSDB or the NEU dataset, and do not upscale anything.
Send the author requests to DeepHP, GAGL and GasHisSDB anyway — none will land before 11 September, but the H. pylori gap will still be there for the next round.