This dataset is released for research use. Access is reviewed and granted manually by the maintainers. Please state your name, affiliation, and intended use.

Roles

Roles: canon repo — annot is the source label, kept machine-parseable as the gold for verification and reward parsing; there is no reasoning column and this repo is not itself a training view. Derived repos (-annotated, -grounding, -region, -mcq) each state their own regime on their own card. Geometry for every record lives in metadata.geometry (below).

D23

Defect detection & classification (bbox; polygon GT kept in metadata). Category B, task T-B2, in the unified Smart-Manufacturing SFT schema.

The repository name is an internal task code. See Provenance below for the underlying dataset.

Records

1,894 records (train=880 · validation=1014).

Unified SFT schema

field	type	meaning
`query`	str	the question / instruction (model input)
`image`	Image	the input image (bytes embedded); for multi-image rows, a preview of the first view
`images`	list[Image]	(multi-image rows) all input views / modalities for the row, bytes embedded
`annot`	str	the answer — for this dataset: one line per defect, plain text, in ONE of two declared grammars: `class,[x, y, width, height]` on the 11 subsets whose class set has two or more members; `[x, y, width, height]` — no class token — on `Capacitor`, `Electronics` and `Screw`, whose only "class" is a placeholder (`0`, `damage`, `defect`) — see Single-type subsets below. The COCO polygon segmentation is preserved as ground truth in `metadata.objects` (not asked in the query) — see Task, segmentation & split below
`reasoning`	null	no native CoT in these datasets
`cate`	"B"	SFT category
`task`	"T-xx"	unified task id
`metadata`	str (JSON)	split, provenance, `image_path`, `image_sha256` (dedup key)
`mask`	Image \| null	(T-B1/T-B2 only) the pixel ground-truth mask, bytes embedded
`masks`	list[Image]	(multi-image T-B1 / D21) per-view masks aligned with `images` (None where a view has no defect), or multi-region masks

Task, segmentation & split

What this is. VISION (Bai et al., arXiv:2306.07890, 2023) — a vision-based industrial inspection benchmark: 14 subsets, 44 defect types, with COCO instance-segmentation annotations (bounding boxes + polygons). Every image is defective; the underlying goal is to find, classify, and outline each defect instance.

Query & answer (this repo's SFT task). query is our own instruction template (the raw dataset ships no natural-language question — only COCO json). It names the subset and asks the model to detect each defect. On the eleven subsets whose class set has two or more members it lists that subset's classes and asks for each defect's class and bounding box, one line per defect as class,[x, y, width, height] — exactly what annot holds. On Capacitor, Electronics and Screw no class is asked and each defect is one [x, y, width, height] line — see Single-type subsets below.

Two declared answer grammars, by subset. Which grammar a record carries is decidable from its query's directive (and stamped as metadata.query_pool): D23/box_classes → class,[x, y, width, height] on Cable, Casting, Console, Cylinder, Groove, Hemisphere, Lens, PCB_1, PCB_2, Ring, Wood (1,630 records); D23/box_only → [x, y, width, height] on Capacitor, Electronics, Screw (264 records). A consumer's parser needs a rule for each: the first splits a line at its first comma into class and box; the second has no class and the whole line is the box. One grammar per repo is not a design rule; an undeclared second grammar would be.

Why bbox, not the polygon. VISION's segmentation is a real COCO polygon (text coordinates), but the polygons are often very detailed (median ~40 vertices, up to ~2680), which a text-output model cannot realistically reproduce. So the SFT task here is detection + classification (class + bbox). The full COCO instances — including the polygon segmentation — are preserved as ground truth in metadata.objects (each with category, bbox, segmentation, area, iscrowd) for pixel-precise / segmentation-model evaluation; the polygon is simply not asked of the model.

Class names. 10 of the 14 subsets have meaningful defect names (e.g. break, Scratch, Porosity, mouse_bite, open_circuit); 4 subsets ship generic placeholder names (Capacitor = 0, Hemisphere = Defect-A..D, PCB_2 = defect1..7, Screw = defect). All of them are kept as released in metadata.categories, metadata.objects[].category and metadata.geometry.instances[].class; whether a placeholder appears in the answer is decided by the rule below. Hemisphere and PCB_2 are placeholder-named but carry 4 and 7 classes respectively, so they are outside the single-type rule and unchanged by this revision.

Single-type subsets: no class in the answer (2026-09-17 ruling, the 2026-09-16 anonymous-family form). Capacitor, Electronics and Screw each have exactly one "class" in the source — 0, damage, defect — and each is a placeholder, not a name: it restates "a defect is here" and says nothing the box does not (a real name states a kind, a place or a mechanism, as Porosity or mouse_bite do). A closed-set class question on a set of one hands the answer over in the prompt — measured on the previous revision, the class token was named in the query on 230 of the 264 records — and an open naming of a placeholder can never be matched. So on those 264 records the query asks for boxes alone and each answer line is [x, y, width, height]: the class token is dropped from every line, never blanked, and it never appears in the answer. The tokens stay in metadata on every record that carried them. Type accuracy is not defined on these subsets, because there is no class to score.

⚠ The answer column changed at this revision. annot moved on 264 records — all of Capacitor (77: train 35, validation 42), Electronics (67: train 36, validation 31) and Screw (120: train 57, validation 63) — from class,[x, y, width, height] lines to [x, y, width, height] lines with the class token removed, e.g. 0,[515, 130, 11, 21] → [515, 130, 11, 21]. The set of boxes on every record is unchanged and equals its metadata.geometry instances. No other record's annot moved. Stated here because a changed query is visible to a consumer (the card renders the query text and its pools) but a changed annot is not — nothing else on the card would differ — so the card is the only place a consumer can learn of it.

Split. train + validation (per-subset COCO annotations). The inference split ships images without ground truth (official eval only) and is not included. See Records for counts.

Query text — pooled paraphrases (v2)

This repository ships 2 question forms over the same images, and each draws from its own pool in common/vision_query_pools.json (metadata.query_template is the index within that form's pool; metadata.query_pool says which form a record is):

D23/box_classes — 1,630 records, 37 gate-verified paraphrases (37 in use, top share 3.1%); template 0 is v1's wording byte for byte.
D23/box_only — 264 records, 36 gate-verified paraphrases (36 in use, top share 6.1%); this form has never been published before, so it has no earlier wording to reproduce and every template in its pool was gated as new.

Both pools clear the 30-variant floor on their own (37 and 36 gate-verified paraphrases), so neither form's diversity rests on the other's count. The two are separate index spaces: metadata.query_pool is stamped on every record because a template index alone does not say which pool it indexes — 13 of the re-drawn records land on the same index NUMBER in the new pool as they carried in the old.

Template ↔ gold independence on this build: 1,894 records, 73 templates, worst template p = 0.0149, alpha 1.4e-04, 0 flagged → PASS.

Frame-size floor (common/lazy_floors.py, the standing (width, height)-only row): MAE 1.671 vs 1.362 blind median, 230 distinct frame sizes — no signal (5-fold cross-validation: the repository ships one split).

image, mask, annot, reasoning, cate, task and the split are byte-identical to the previous revision — this revision was issued from the published parquet itself (tools/requery_repool.py re-draws the text, tools/requery_stream.py --push carries every other column out of the live shard). What moved: query on 264 of 1,894 records, and metadata on 1,894 (the added query_pool key; query_template on 251). The image identities in §8 were carried from the previous pass and re-measured from the metadata.pixel_sha256 this repository already ships — no image was decoded again, because none was touched.

Provenance

Underlying dataset: VISION. Upstream license: CC BY-NC 4.0 (this card is license: other; respect the upstream terms). Converted read-only from the raw source into the unified schema; conversion code under D23/ (with publish/push_to_hf.py) in AI4Manufacturing/forge_model.

Overlap / de-duplication (§8)

D15 (DefectSpectrum) shares 641 of this repo's 1,894 images (33.8%), byte-identical (sha-verified 2026-07-14): DefectSpectrum re-annotated 6 of VISION's 14 subsets by copying the image files verbatim.

subset	shared / D23 records	subset	shared / D23 records
Capacitor	77 / 77	Groove	98 / 99
Screw	120 / 120	Console	180 / 192
Wood	102 / 102	Ring	64 / 87

The other 8 subsets (Cable, Casting, Cylinder, Electronics, Hemisphere, Lens, PCB_1, PCB_2) share nothing. All 641 sit in D15's train split, while here 315 are train and 326 are validation — so D23-validation is contaminated as an eval set for any model trained on the D15 family. (D15 also holds 76 VISION images from the unlabeled inference split, which are not in this repo at all. MMAD/D05 shares no VISION imagery — the previous vague note here was wrong.)

The two repos' labels materially disagree on shared images — different upstream annotation policies, not noise: VISION deliberately preserves the original sparse crowd-sourced defect decisions, while DefectSpectrum re-annotated exhaustively with fine masks and its own class names. Measured on all 641: DS masks confirm 88.5% of VISION boxes (718/811); a median 68% of DS defect pixels fall inside the VISION-box union, and on 259/641 images it is under half (worst: Groove — e.g. the same photo carries 1 gold defect here and 8 in D15-grounding).

Rule: build mixtures with each shared image from exactly one repo — prefer the D15 family for detection-shaped supervision (finer, exhaustive lineage); this repo's copies are the VISION-policy alternative, not additional supervision. The machine-readable key list is overlap_with_D15.json (641 sha256 keys + stats; generated by annotate/D23/overlap_sidecar.py in forge_model), joinable to every record's metadata.image_sha256:

overlap = set(json.load(open("overlap_with_D15.json"))["sha256"])
keep = [r for r in d23 if json.loads(r["metadata"])["image_sha256"] not in overlap]

Two identities, and they answer different questions. metadata.image_sha256 hashes the file bytes: it finds byte-identical copies and is blind to a re-encode. metadata.pixel_sha256 hashes the decoded image (mode | size | pixels): it finds the same photograph saved twice. Only the second one settles whether an image is duplicated.

Decoded at this revision, and re-derived from the shipped stamps (2026-09-21). Every image was decoded again for this revision — 1,894 images, 1,894 distinct — and the counts below ARE that measurement, re-derived here from this repository's own metadata.pixel_sha256 stamps and asserted equal; they also reproduce the identity block this repository published previously (evidence forge_model/evidence/identity_chains/identity_D23.json, sha16 01b7bc8b10d1eab3, tracked in forge_model 8f10bb4ae6fc — fetch the exact bytes from git rather than trusting a digest with no history). Corrected 2026-09-21: an earlier revision of this card denied that any image had been decoded for it and named the decoding pass as revision None, which recorded nothing.. A disagreement aborts the build and names the offending records:


images checked	1,894
distinct by decoded pixels	1,894
images carrying more than one record	0
images on both sides of the split	0

Cross-family evaluation lock — metadata.eval_lock (stamped 2026-09-20; manifest revision fe6e286912b0, generated 2026-09-08). Every record of this repository, locked or not, carries metadata.eval_lock, computed by forge_model/common/overlap.py::Overlap.stamp_for against common/overlap_manifest.json at that revision — so within this repository the absence of the key cannot occur. Shape: {"locked": bool, "against": [{"repo": …, "split": …}, …], "own_split": …, "manifest_revision": …, "manifest_generated": …}. locked is true when the image is evaluation material anywhere in the corpus; against names every repository and split in which it is (sorted; [] when not locked; it includes the record's own family where that is so); own_split marks a record locked by its own split. The per-record field is the authority — the count here is quoted once, at this revision, and a later manifest may change it: 1,014 of 1,894 records (1,014 distinct images) are locked — by column: 326 by the cross-family manifest, 1,014 by their own split, 326 both ways and counted once; counterparts (records per counterpart; a record can appear under several): D23 validation 326; 1,014 locked by their own split: validation. In words: 1,014 of the 1,894 records in this repository are evaluation material by their own metadata.split (validation: 1,014) and sit inside the HF split named train / validation — under the uniform-split convention the HF split name is a container name, and metadata.split together with metadata.eval_lock carries the truth; a train pool must exclude them. A stamp whose manifest_revision differs from the current manifest is stale, not wrong — recompute it (Overlap.stamp_is_current); a record with no stamp has not been checked against the corpus as it now is. Overlap.partition / assert_train_pool_clean read the field: a train pool built from this repository must exclude every locked record.

Geometry (`metadata.geometry`)

Every record carries a geometry block inside the existing metadata JSON string, so that its gold can be re-derived at any render size. No schema column changed; existing loaders are unaffected.

Coordinates are native pixels of the image in that record (coords_frame: "record_image"). scale is 1.0 throughout — this repo publishes at source resolution, nothing was downscaled at publish time.

"geometry": {
  "image_wh":  [W, H],        // dims of the image in THIS record
  "source_wh": [W, H],        // dims of the original source image
  "scale": 1.0,               // image_wh / source_wh; < 1.0 would disclose a publish-time downscale
  "n_instances": 2,
  "instances": [
    { "instance_id": 1, "bbox_xywh": [x, y, w, h], "min_side_px": 65, "class": null }
  ],
  "n_dropped_subminimum": 0,  // components removed by the filters below
  "union_box_fallback": false,// true => boxes are per-class unions, NOT real instances
  "conventions": { ... }      // see table
}

instances is present even when empty. [] means the record genuinely has no defects; an absent block would mean geometry could not be recovered. Those are different states and are never conflated.

Conventions used to derive it

There is no universal definition of "one defect instance" — it depends on the mask the source shipped. This repo's is stated, not implied:

field	value
`algorithm`	`source_annotation`
`binarisation`	`n/a`
`connectivity`	`4`
`merge`	`none`
`min_area_px`	`0`
`max_instances`	`None`
`artifact`	`fine`
`fill_floor`	`None`
`legibility_floor_px`	`None`
`min_side_floor_px`	`None`
`spec_sha`	`a98e8ed866091f30`

Provenance and verification


records	1,894
carrying a geometry block	1,894 / 1,894
instances per record	`1`: 1,155, `2`: 291, `3`: 120, `4`: 54, `5+`: 274
total instances	4,473
image dimensions	1600×1200 (481), 1280×1024 (186), 3840×2748 (159)
`scale` values present	[1.0]

Computed from this repo's own masks and verified against this repo's own published answers before it was written — a recomputation that disagreed with the shipped gold would have aborted the update rather than overwritten it.

⚠ The 16px floor applies at the RENDER, not at native

min_side_px is in native pixels. The model does not see native: Qwen2-VL caps by megapixels AND snaps each dimension to a multiple of 28. So min_side_px >= 16 is the floor tested in the wrong frame. Measured on this repo:


native → rendered (qwen2_vl @ 2.36MP)	217×664 → 224×672, 262×192 → 252×196, 413×212 → 420×224
shipped boxes	4,473
legible at that render (>=16px there)	3,139 (70.2%)

⚠ An earlier version of this section reported the inverse — boxes clearing 16px at native and failing at the render — and that number was misleading. It is frame-relative: publishing at a larger native size lets more boxes clear 16 in the published frame, so more can "fail", which penalises exactly the choice that helps. Measured on 179: publishing native (3024) means a box needs >=32px native to be legible at the render and 86.7% qualify; the previous 1024 publish needed >=47px native and only 69.5% qualified. The native republish improved rendered legibility by 17 points while the old metric scored it as 12.5% "broken". The figure above is the comparable one.

Nothing in the data is frame-dependent — geometry is native and complete. Use forge_model/D23/adapt.py, which applies the floor at whatever size the consumer renders.

Using it

Coordinates only stay correct if they are rescaled with the image. A patch-based VLM does not render at native size: Qwen2-VL's processor snaps both dimensions to a multiple of 28, so this repo's 217×664 is rendered 224×672 and native-pixel boxes are then wrong by a few pixels. forge_model/D23/adapt.py regenerates coordinates for a target render size, re-derives counts, and drops records whose gold no longer holds there.

Downloads last month: 140

Total file size:

451 MB

Paper for AI4Manufacturing/D23

VISION Datasets: A Benchmark for Vision-based InduStrial InspectiON

Paper • 2306.07890 • Published Jun 13, 2023 • 1

You need to agree to share your contact information to access this dataset