layerdepth-stratified

Stratified train metadata for princeton-vl/LayeredDepth-Syn.

This dataset does not duplicate LayeredDepth images (~1.9 TB). It publishes:

1. Per-scene bucket assignments (compressed layer count 1–4)
2. A round-robin batch mix manifest for balanced multilayer training
3. Preprocessing code aligned with the original LayeredDepth / SeeGroup pipeline

Use it to train models that avoid fake-layer collapse by balancing layer-count buckets each epoch.

Base dataset

Item	Value
Images & depth PNGs	princeton-vl/LayeredDepth-Syn
Train scenes	14,800
Layers per scene	4 depth maps (IDs 1, 3, 5, 7)
This repo	Metadata + sampling code only

Files

Path	Description
metadata/sample_buckets.parquet	One row per train scene: row_index, bucket, layer stats
metadata/bucket_manifest.json	Bucket → row_index lists + default batch_mix
metadata/summary.json	Build provenance and histogram
layerdepth_stratified/preprocess.py	LayeredDepth depth collapse + RGB/depth decoding
layerdepth_stratified/stratified_sampling.py	Epoch order + DDP rank split
layerdepth_stratified/dataset_loader.py	High-level iterator API

Quick start

from datasets import load_dataset
from layerdepth_stratified import iter_from_hub_metadata

# 1) Load stratified metadata from this repo
meta = load_dataset("YOUR_USERNAME/layerdepth-stratified", split="train")
print(meta[0])  # row_index, bucket, scene_max_compressed, ...

# 2) Iterate preprocessed samples in stratified epoch order
for sample in iter_from_hub_metadata(
    "YOUR_USERNAME/layerdepth-stratified",
    cache_dir="/path/to/hf/cache",
    seed=42,
    epoch=1,
):
    image = sample["image"]          # float32 HWC, [0, 1]
    depth = sample["depth"]          # float32 HWD, meters, LayeredDepth convention
    valid_mask = sample["valid_mask"]
    break

Preprocessing (matches original LayeredDepth)

1. Decode RGB PNG → float32 [0, 1]
2. Decode depth PNG → meters (/1000, clip invalid/>80m)
3. Layer collapse: invalid shallow pixels inherit deeper valid depth (standard LayeredDepth convention)
4. Optional layer subset via selected_layer_ids

See layerdepth_stratified/preprocess.py for the reference implementation.

Stratified sampling

Each train scene is assigned to bucket 1–4 by compressed layer count (ray-sort + gap events).

Default per-batch mix (batch_mix):

{"1": 0.25, "2": 0.25, "3": 0.25, "4": 0.25}

Each epoch:

1. Shuffle within each bucket
2. Round-robin across buckets using batch_mix
3. Map indices → rows in princeton-vl/LayeredDepth-Syn train split

Rebuild manifest

python -m layerdepth_stratified.build_manifest \
  --cache-dir /path/to/hf/datasets \
  --output-dir artifacts/layereddepth_stratified

Citation

If you use this stratified metadata, please cite the original LayeredDepth paper/dataset and note that bucket assignments were produced with the SeeGroup stratified sampling pipeline.

See also

• LayeredDepth-Syn
• SeeGroup docs/LAYEREDDEPTH_STRATIFIED_SAMPLING.md