# INVENTORY — what has actually been downloaded, and whether it can serve the 70 real images

Work package: REAL-SOURCES. Date: 2026-09-09. Requirement being tested against:
**4096 x 2048 px PNG, gastric H&E, ~0.25 um/px (40x-objective equivalent), four categories.**

The single test applied to every source below is: **can it yield a 4096 x 2048 field at ~0.25 um/px
without upscaling?** A 4096 x 2048 field at 0.25 um/px covers a physical area of
**1024 um x 512 um**. A source therefore needs *both*:

1. a native sampling of about 0.25 um/px or finer, **and**
2. a field of view at least 1024 x 512 um wide.

Failing either one means the image can only reach 4096 x 2048 by interpolation, which invents
detail that was never recorded. Nothing in this package is presented as native unless both tests pass.

---

## A. Downloads found on Yash's Mac (from the Codex thread)

Source transcript read:
`[local]/rollout-2026-09-09T16-49-01-01a08636-eba7-7880-9173-546506ce7198.jsonl`
(589 JSONL records; user turns, assistant turns and tool calls extracted).

The thread covers four things: a hunt for the free full text of two papers, the GasHisSDB download and
inspection, a broad dataset-landscape review, and a discussion of scraping Google Images / Wikipedia
for H. pylori H&E with magnification captions. **Only one image dataset was actually written to disk.**

| # | Item | Path on disk | Size | Format / native px | Stated scale | Licence | Category labels | Verdict |
|---|---|---|---|---|---|---|---|---|
| A1 | **GasHisSDB** (archive) | `[local]/GasHisSDB.rar` | 4,379,656,835 B (4.38 GB), md5 `4b1f2fb48532bbe3998833a9e591a8a9` — matches figshare's `supplied_md5` exactly | RAR | — | **CC BY 4.0** (figshare record 15066147) | Normal / Abnormal only | **UNUSABLE for the 70.** See A2. |
| A2 | **GasHisSDB** (extracted) | `[local]/{80,120,160}/{Normal,Abnormal}` | 4.5 GB, 245,196 PNGs | PNG, **80x80, 120x120 and 160x160 px** | none published | CC BY 4.0 | Normal 148,120 / Abnormal 97,076 | **UNUSABLE.** The largest patch is 160x160 = 25,600 px. The target is 8,388,608 px — **328x more pixels**. Reaching 4096x2048 would be a 25.6x linear upscale. It also has no H. pylori label, no IM label, no um/px, and no slide/patient map. |
| A3 | `selection-intestinal_metaplasia.json` | `[local]/Downloads` | 175 B | JSON | — | internal | — | Not image data — a pick list (`IN42`, `IN24`) exported from our own SGH candidate-selection page. |
| A4 | `Gastrointestinal Endoscopy_20260909.zip` | `[local]/Downloads` | 3.15 MB | 20 journal PDFs | — | publisher | — | Not image data. Journal issue front matter and endoscopy papers; no histology image set. |
| A5 | `ijms-23-14581.pdf` | `[local]/Downloads` | 2.54 MB | PDF | — | MDPI (CC BY) | — | Not image data. |

### GasHisSDB counts, as verified locally in the prior package

| Native dimensions | Normal | Abnormal | Total |
|---|---:|---:|---:|
| 80 x 80 | 87,500 | 59,151 | 146,651 |
| 120 x 120 | 40,460 | 24,801 | 65,261 |
| 160 x 160 | 20,160 | 13,124 | 33,284 |
| **Total** | **148,120** | **97,076** | **245,196** |

(15 byte-identical duplicate files in the 80 px Normal group. Full inspection:
`operations/research/sgh-dataset-landscape-20260909/GASHISSDB-INSPECTION.md`.)

### The Google Images / Wikipedia route discussed in the thread

The thread's `File:Stomach_helicobacter_he.JPG` example was checked against the Commons API:

- **848 x 640 px**, 77,439 bytes, CC BY-SA 3.0, by Commons user *Alex_brollo*, 2006.
- Its own description reads: *"Gastric biopsy, H&E, 400x. Helicobacter pylori is barely visible inside mucus."*

Reaching 4096 x 2048 from 848 x 640 is a **4.8x linear upscale**, and the author states the organisms
are barely visible even at native size. **No files from this route were saved to disk** — the thread
discussed it and moved on. Nothing from Google Images exists locally to inventory.

**Net result for section A: nothing Yash downloaded can supply a single one of the 70 images.**

---

## B. Material already staged on the IOFlood workbench

Container `sgh-histopathology/workbench`, under `[local]/raw`.
Access: `cd research/sgh-synthetic-histopathology && ./ioflood exec '<cmd>'`.

| # | Source | Path | Size | Format / native px | Scale | Licence | Verdict |
|---|---|---|---|---|---|---|---|
| B1 | **HiESD** WSIs | `raw/hiesd-wsi/*.svs` | 16 slides on arrival (14.3 GB); **22 slides now** after this package's 6 downloads (19.1 GB) | Aperio `.svs` pyramids, level-0 up to ~113,593 x 78,923 px | **`openslide.mpp-x` = 0.2458 um/px**, scanned at 40x | **CC BY 4.0**, figshare DOI `10.6084/m9.figshare.28919840` | **USABLE — this is the workhorse.** See SOURCES.md. |
| B2 | **HiESD** annotations | `raw/hiesd/` | 327 MB | region masks (downsample 64), quality masks (GrandQC, 0.9995 um/px), XML, thumbnails, `ESD104-Summary-new.csv` | 64x downsampled from level 0 | CC BY 4.0 | **USABLE.** Ten pixel-coded tissue classes, including `Chronic Gastritis`, `Chronic atrophic gastritis`, `Complete intestinal metaplasia`, `Incomplete intestinal metaplasia`, `Normal Gland`, plus three carcinoma classes to exclude. **Masks for all 104 slides are present locally**, even for slides whose `.svs` has not been downloaded — this is what made the whole-collection yield survey below possible without any further downloads. |
| B3 | **GAGL** | `raw/gagl/files/*.png` | 176 MB, 9 PNGs | PNG, 2400x2880 to 4800x2016 | Record says *"9 **downsampled** WSI"* — **no um/px published** | **CC BY 4.0**, Zenodo `10.5281/zenodo.7032067` | **UNUSABLE at 4096x2048.** Two independent failures: (a) **no file is >= 4096 x 2048** — the closest, `GA1.png` at 4800x2016, is **32 px short in height**; (b) they are explicitly downsampled whole slides, so their sampling is far coarser than 0.25 um/px and the pixels to reach the target simply do not exist. Labels are useful (3 normal / 3 gastric atrophy / 3 IM), and the underlying study has 85 WSIs from 20 patients — worth an author request, not a download. |
| B4 | **NEU H. pylori dataset** (`ailhan-NEU/Helicobacter-Pylori-Dataset`) | `raw/hp-neu/extracted/Dataset/{Positive,Negative}` | 267 MB | JPEG. **Positive: 103 files** — 100 at 1280x960, 3 at 2880x2048. **Negative: 101 files**, all 1280x960 | **not stated**; paper says 400x total on a Nikon Eclipse Ni-U + DS-L3 camera — no um/px anywhere | **CC BY-NC-ND 4.0** (repo `LICENSE.md`) | **UNUSABLE, on two independent grounds.** See below. |
| B5 | Non-gastric pretraining corpora | `raw/public/flux-general-he-v1/` | 26 GB | CAMELYON17 `.tif`, Kather 2016 | — | per-source | Not gastric. Out of scope for the 70 real images. |

### B4 in detail — the only H. pylori-labelled H&E set we hold

(The directory is named `hp-neu` on the workbench. There is no dataset called "HP-Neu"; the `NEU` in
the GitHub handle is *Near East University, Nicosia*. Correct name: the **NEU H. pylori dataset**.)

This is the dataset behind Ibrahim, Dirilenoğlu, Hacisalihoğlu, Ilhan & Mirzaei (2024),
*Classification of H. pylori Infection from Histopathological Images Using Deep Learning*,
Journal of Imaging Informatics in Medicine (DOI `10.1007/s10278-024-01021-0`, open access via
PMC11169399). 204 H&E images from **50 cases** — so images within a case are correlated, as the
authors themselves note. Filenames carry Sydney-system density grades
(`B129-19 HP (++).JPG`, `B198-19 HP (+++).JPG`) — i.e. **a pathologist did assert organism presence
and density**, which is exactly the label HiESD lacks.

It still cannot be used:

1. **Resolution.** 1280 x 960 is a **3.2x linear upscale** short of 4096 x 2048. Even the three
   largest files at 2880 x 2048 are 1.42x short in width. There is no crop of any file that is
   natively 4096 x 2048.
2. **Licence.** `CC BY-NC-ND 4.0`. **ND** forbids distributing adapted material, and any crop,
   resample or format conversion is an adaptation. **NC** forbids commercial use. Submitting a
   derived image to a challenge is exactly the redistribution of adapted material that ND blocks.
   This is a hard stop independent of the resolution problem.

Its remaining value is as a *reference for what H. pylori looks like in H&E* and as an argument for
writing to the authors — not as submission material.

---

## C. New material staged by this package

### C.1 Additional HiESD whole slides (downloaded to the workbench)

Chosen from the whole-collection yield survey (see SOURCES.md) to add slides that are strong on the
two scarce categories (IM-only and IM-with-gastritis) and to widen the specimen pool beyond the 16
slides already held. Every file is under 2 GB; the whole batch is 4.83 GB.

Destination `[local]/hiesd-wsi`. Each was verified against figshare's
published `supplied_md5` after download.

| slide_id | figshare download URL | bytes (achieved) | md5 (published + achieved) | IM-only fields | IM+gastritis fields | gastritis fields | normal fields |
|---|---|---:|---|---:|---:|---:|---:|
| `23cb29c8-6582-464a-be20-212b51e5348c` | https://ndownloader.figshare.com/files/54166370 | 619,320,860 (md5 verified) | `c913810f257e7c9698dba1506e71aabe` | 20 | 23 | 40 | 16 |
| `31bea597-ee13-4734-8798-ce7d0270ca57` | https://ndownloader.figshare.com/files/54166391 | 1,022,570,812 (md5 verified) | `101fe5fbd73c7c26ffcb6e6cf939f729` | 20 | 20 | 58 | 2 |
| `5daa08f1-24d4-4aa7-bf41-066870a05325` | https://ndownloader.figshare.com/files/54163259 | 523,302,125 (md5 verified) | `fbf6dc8c4bbdd7e205590ea2151daabf` | 21 | 19 | 28 | 2 |
| `b122600e-40b3-4eae-9ec5-a6baf2360487` | https://ndownloader.figshare.com/files/54167264 | 933,822,976 (md5 verified) | `4f0c223b5fc2fc4cb12985223ef5e2a8` | 14 | 16 | 62 | 4 |
| `9a222dd1-6b65-4dae-8c8b-0dd72288beb4` | https://ndownloader.figshare.com/files/54166355 | 792,632,311 (md5 verified) | `430a42e7139349ea7b146821239a9342` | 12 | 18 | 21 | 0 |
| `d8fec178-e4d2-4951-83f9-2cceea41c706` | https://ndownloader.figshare.com/files/54167345 | 940,325,708 (md5 verified) | `7880a6f3427bbdbc9b1b112d175a6b90` | 11 | 13 | 58 | 1 |

Download ledger with the achieved md5 and byte count per file:
`[local]/dl.log` on the workbench.

Reason recorded for downloading (per the package's own >2 GB rule — no *single* file exceeds 2 GB,
and the batch total of 4.83 GB is recorded here for completeness): the 16 slides already held can
supply the required field counts, but they are a small slice of the 104-slide collection and HiESD
publishes no patient map, so specimen diversity in a 70-image submission would otherwise rest on an
unverifiable assumption. These six slides roughly double the IM and IM+gastritis pool.

### C.2 Wikimedia Commons micrographs (downloaded to the Mac, staged as candidates)

`candidates/wikimedia-commons/` with `provenance.csv`. **Fourteen files, all CC BY-SA 3.0 or 4.0.**
These are the best-resolution publicly licensed gastric H&E images that actually show H. pylori
organisms. They are staged because they are the *only* real material we hold in which organisms are
visible, and because the assessment of them is a required deliverable — **not** because they clear
the resolution bar. Most do not. See SOURCES.md section 3 for the measurement and the verdict.

---

## D. Summary verdict table

| Source | Redistributable? | Native >= 0.25 um/px? | Field >= 1024 x 512 um? | Can supply 4096x2048 @0.25 without upscaling? |
|---|---|---|---|---|
| GasHisSDB | Yes (CC BY 4.0) | unknown, unpublished | **No** — 160 x 160 px total | **No** (25.6x upscale) |
| Wikimedia `Stomach_helicobacter_he.JPG` | Yes (CC BY-SA 3.0) | probably | **No** — 848 x 640 px | **No** (4.8x upscale) |
| GAGL (Zenodo 7032067) | Yes (CC BY 4.0) | **No** — downsampled WSIs | borderline | **No** — and no file even reaches 4096 x 2048 px |
| NEU H. pylori dataset | **No** (CC BY-NC-**ND**) | not stated (400x total) | **No** — 1280 x 960 px | **No** (3.2x upscale) |
| Wikimedia Nephron/CoRus13 gastric set | Yes, but **ShareAlike** | varies, mostly finer than 0.25 | **mostly No** | **Rarely** — see SOURCES.md §3 |
| **HiESD** | **Yes (CC BY 4.0)** | **Yes — 0.2458 um/px** | **Yes — whole slides** | **Yes** |

**One source clears the bar for the geometry: HiESD. It does not carry an H. pylori label.**
That single sentence is the whole problem, and SOURCES.md sets out what to do about it.