# REAL-SOURCES — result Work package: REAL-SOURCES, SGH synthetic gastric histology programme. Date: 2026-09-09. Deadline: 11 September 2026, 18:00 SGT. No git commits made. --- ## Headline **The 70 real images can be delivered, from one source, at native resolution, under a clean CC BY 4.0 licence — except for the H. pylori organism label, which no public source can supply.** - **HiESD** (figshare `10.6084/m9.figshare.28919840`, CC BY 4.0, 104 whole slides, 40x, `openslide.mpp-x` = 0.2458) is the only public gastric source that passes both the sampling test and the field-of-view test. Every 4096 x 2048 @ 0.25 um/px field cut from it is a **1.017x downsample** of level 0. Nothing is upscaled. - Supply is not the constraint. The 22 slides now on the workbench yield **316 IM-only**, **275 IM+gastritis**, **951 gastritis-only** and **403 normal** disjoint fields, against a requirement of 14 / 14 / 21 / 21. - **Nothing Yash downloaded can be used.** GasHisSDB is 160 x 160 px patches — a 25.6x linear upscale short of the target — with no H. pylori label, no IM label and no published um/px. - **No public dataset annotates H. pylori organisms in H&E at a usable resolution and field size.** The two that come closest are blocked: DeepHP is embargoed to 31 December 2030, and the NEU H. pylori dataset is 1280 x 960 px under **CC BY-NC-ND 4.0**, where ND forbids the derivative that any crop would create. --- ## What was produced ``` operations/research/sgh-selection-70/real-sources/ ├── INVENTORY.md what Yash downloaded, what is on the workbench, per-source verdict ├── SOURCES.md the public-source survey, the H. pylori assessment, the 70 composition ├── REAL_SOURCES_RESULT.md this file ├── hiesd_field_yield.csv per-slide category capacity for all 104 HiESD slides ├── hiesd_field_yield.py the survey script (runs on the workbench, masks only, no WSI reads) ├── measure_pixel_scale.py the LoG scale-space um/px estimator used on the Commons images ├── licence-evidence/ dated API snapshots of the figshare licence fields + README └── candidates/ ├── hiesd/ 4 verification fields, 4096x2048 @0.25 um/px + provenance.csv │ + CONTACT_SHEET.png └── wikimedia-commons/ 14 CC BY-SA micrographs + provenance.csv (assessed, NOT recommended) ``` On the IOFlood workbench: - `[local]/hiesd-wsi` — **6 additional HiESD slides downloaded** (4.83 GB; no single file above 2 GB), every one md5-verified against figshare's published `supplied_md5`. Ledger: `[local]/dl.log`. - `[local]/real-sources` — survey and proof scripts, `hiesd_yield.json`, `proof.json`. --- ## What to submit as the "source(s) of the real images" text > The 70 real images are 4096 x 2048 px fields cut from **HiESD — A Fully Annotated Pathology Slide > Dataset for Early Gastric Cancer and Precancerous Lesions** (Ge et al., *Scientific Data*, 2025; > DOI 10.1038/s41597-025-05679-1), deposited on figshare under **CC BY 4.0** at DOI > 10.6084/m9.figshare.28919840. The slides are H&E-stained gastric endoscopic submucosal dissection > specimens scanned at 40x objective; each reports `openslide.mpp-x` = 0.2458 um/px. Each field was > read at 4165 x 2083 level-0 pixels — the exact extent of 1024 x 512 um — and resampled **down** by > a factor of 1.017 to 4096 x 2048 px at 0.25 um/px. **No image was upscaled.** Category assignment > follows the pathologist region annotations distributed with the dataset > (`Annotation_Category_Label_Description.txt`): complete and incomplete intestinal metaplasia, > chronic gastritis, chronic atrophic gastritis, and normal glands. Any field containing a carcinoma > annotation was excluded. > > **Note on the H. pylori categories:** HiESD does not record *Helicobacter pylori* organism status. > Images assigned to categories (i) and (iii) are assigned on the annotated chronic gastritis > pattern — the histological context in which H. pylori infection presents — and not on an observed > organism. No publicly available, redistributable H&E dataset was found that provides verified > H. pylori organisms at 40x with a field of view sufficient for a 4096 x 2048 image at 0.25 um/px. The last paragraph must not be dropped, softened, or moved to a footnote. --- ## What remains impossible **1. A real image with a verified H. pylori organism, at 4096 x 2048 and 0.25 um/px, that we are allowed to redistribute. This does not exist in the public record as of 9 September 2026.** The search was thorough and the failure is structural, not incidental: | Candidate | Why it fails | |---|---| | **DeepHP** (Zenodo `10.5281/zenodo.8117177`) — 394,926 images, 111k HP-positive | **Embargoed to 2030-12-31**, plus DUA and application. And at 2776 x 2080 it would still be 1.5x short. | | **NEU H. pylori dataset** (`github.com/ailhan-NEU/...`) — 204 H&E images, Sydney HP density grades in the filenames | **CC BY-NC-ND 4.0.** ND forbids distributing adapted material; any crop or resample is adapted material. Separately, 1280 x 960 is a 3.2x upscale short. | | **IEEE DataPort HP set** (`10.21227/ryks-j091`) — actual organism point coordinates | The annotations are on **immunohistochemistry, not H&E**. Also behind a subscription. | | **Girona/Barcelona HP work** (`arXiv:2412.13857`) | IHC, and never publicly released. | | **Wikimedia Commons HP micrographs** | Organisms genuinely visible, but they are 400x-1000x oil-immersion camera frames: measured at 0.039-0.16 um/px, covering only **74-684 um** of tissue against the 1024 x 512 um required. The images that show the organisms best have the smallest fields. All CC BY-SA (ShareAlike). | | **Paper supplementary figures** | 1000-2000 px, JPEG-compressed. Upscaling 2-4x would destroy the 2.5-5 um curved-rod morphology that is the entire point of the category. | | **HiESD itself** | Right resolution, right field, right licence — **no H. pylori class**. Our own detector sweep (`B1_RESULT.md`) found 0 likely organisms in 60 hand-graded candidates across 753 mucosal-surface windows on 13 slides, with the negative arm scoring like the positive arm. | Zenodo returns **8 records** for `"Helicobacter pylori" AND (histopatholog* OR histolog* OR "whole slide" OR biopsy)` restricted to datasets; seven are clinical tables with no images and the eighth is the embargoed DeepHP. There is no Kaggle equivalent. There is no gastric CAMELYON/PANDA. **The gap is real.** **2. A guarantee of 70 distinct patients.** HiESD is 104 slides from 44 patients but publishes **no slide-to-patient map**. We can guarantee 70 fields from distinct slides with 300 um minimum spacing; we cannot claim 70 distinct patients and must not. **3. A pathologist's confirmation of any category assignment.** Every label in this package traces to the source dataset's own pathologist annotations. Nothing here has been re-read by a GI pathologist. --- ## Two things to resolve before upload **HiESD licence conflict — 10 minutes of work.** The figshare **data record** returns `{"value": 52, "name": "CC BY", "url": "https://creativecommons.org/licenses/by/4.0/"}` through the API — captured first-hand today in `licence-evidence/hiesd-figshare-record-20260909.json`, along with the file manifest (114 files, 104 `.svs`, 58.65 GB). The *Scientific Data* **article** page renders a CC BY-NC-ND notice. These govern different objects and the data record is the one that covers the `.svs` files — but a reviewer will ask. **Screenshot the figshare page in a browser on the day of submission and file it in `licence-evidence/`.** The API snapshot is already there; the browser view is what a reviewer will compare against. **GasHisSDB licence conflict — relevant only if it is cited anywhere.** figshare says CC BY 4.0; the arXiv paper says the database is published "for non-commercial purposes"; the Gitee mirror declares no licence at all. Since GasHisSDB contributes nothing to the 70, the simplest fix is not to cite it as a source of submitted images. --- ## Method notes, for anyone who needs to re-run or challenge this **The whole-collection capacity survey never touched a whole-slide image.** HiESD ships `ESD_40X_annotation_downsample64` masks for all 104 slides, and those are already on the workbench. `hiesd_field_yield.py` slides a 4165 x 2083 level-0 window (65 x 33 mask pixels at downsample 64) across each mask with integral images, and greedily selects **disjoint** windows meeting each category rule, excluding any window containing a carcinoma label. That is how a 104-slide capacity table was produced without downloading 58 GB. Results: `hiesd_field_yield.csv`. **The Commons pixel-scale measurement is calibrated, not guessed.** No Commons file states a pixel size, so `measure_pixel_scale.py` estimates it: the scale-normalised Laplacian-of-Gaussian response over nuclear texture peaks at a sigma proportional to 1/(um per px). Calibrated on HiESD, where `openslide.mpp-x` is known: `K = sigma_peak * mpp = 1.61`, stable to within 1.5% across the native image and its 2x and 3x downsamples. Where the peak pinned at the edge of the scale-space range the provenance row records "not measurable" rather than a fabricated number. **The four verification fields in `candidates/hiesd/` are a geometry proof, not the production extraction.** The HIESD-EXTRACT package owns the extraction. These four exist to demonstrate concretely that each category is reachable and that the resample is a **downsample** — every row of `candidates/hiesd/provenance.csv` records `resample_factor` 0.9832 and `operation: downsample (Lanczos)`, alongside the slide id, level-0 read coordinates, source mpp and SHA-256. `CONTACT_SHEET.png` shows all four: goblet cells in the IM field, dense lamina propria infiltrate in the gastritis field, foveolar epithelium with inflammation beside a metaplastic gland in the co-occurrence field, and clean oxyntic mucosa in the normal field. --- ## Recommendation Cut all 70 from HiESD, per the composition table in `SOURCES.md` §6. Ship the source statement above verbatim, including the H. pylori caveat. Do not include CC BY-SA material, do not include anything from GasHisSDB or the NEU dataset, and do not upscale anything. Send the author requests to DeepHP, GAGL and GasHisSDB anyway — none will land before 11 September, but the H. pylori gap will still be there for the next round.