AstroAgentBench
Collection
An agent benchmark on space mission design tasks: paper, RL environments, traces and an interactive space • 3 items • Updated
Dataset Preview
The full dataset viewer is not available (click to read why). Only showing a preview of the rows.
Job manager crashed while running this job (missing heartbeats).
Error code: JobManagerCrashedError
Need help to make the dataset viewer work? Make sure to review how to configure the dataset viewer, and open a discussion for direct support.
case_id string | split string | benchmark string | index_metadata dict | files list |
|---|---|---|---|---|
case_0001 | test | aeossp_standard | {"case_id":"case_0001","case_seed":1051,"celestrak_snapshot_epoch_utc":"2026-04-14T00:00:00Z","horiz(...TRUNCATED) | [{"content":"mission:\n case_id: case_0001\n horizon_start: '2026-04-14T04:00:00Z'\n horizon_end:(...TRUNCATED) |
case_0002 | test | aeossp_standard | {"case_id":"case_0002","case_seed":2060,"celestrak_snapshot_epoch_utc":"2026-04-14T00:00:00Z","horiz(...TRUNCATED) | [{"content":"mission:\n case_id: case_0002\n horizon_start: '2026-04-14T06:00:00Z'\n horizon_end:(...TRUNCATED) |
case_0003 | test | aeossp_standard | {"case_id":"case_0003","case_seed":3069,"celestrak_snapshot_epoch_utc":"2026-04-14T00:00:00Z","horiz(...TRUNCATED) | [{"content":"mission:\n case_id: case_0003\n horizon_start: '2026-04-14T08:00:00Z'\n horizon_end:(...TRUNCATED) |
case_0004 | test | aeossp_standard | {"case_id":"case_0004","case_seed":4078,"celestrak_snapshot_epoch_utc":"2026-04-14T00:00:00Z","horiz(...TRUNCATED) | [{"content":"mission:\n case_id: case_0004\n horizon_start: '2026-04-14T10:00:00Z'\n horizon_end:(...TRUNCATED) |
case_0005 | test | aeossp_standard | {"case_id":"case_0005","case_seed":5087,"celestrak_snapshot_epoch_utc":"2026-04-14T00:00:00Z","horiz(...TRUNCATED) | [{"content":"mission:\n case_id: case_0005\n horizon_start: '2026-04-14T12:00:00Z'\n horizon_end:(...TRUNCATED) |
AstroAgentBench
Harbor tasks for evaluating complete agent systems on space mission design. GitHub is the canonical source for benchmark definitions, generators, and source inputs. This release is v2.0.0, generated from Git commit 6f5b9d21b1efb2ca2ee6fcabdc338a9d077612cb.
Run a taskset
Install Harbor 0.23.0 with Python 3.13 and have Docker available. Run the seven reference checks, one per benchmark family:
harbor run --repo https://huggingface.co/datasets/kaupane/AstroAgentBench@v2.0.0 --dataset smoke@v2.0.0 --agent oracle -n 1 --jobs-dir results --job-name astroagentbench-smoke
registry.json lists one dataset per benchmark and split, such as regional_coverage-test@v2.0.0 and spot5-single_orbit@v2.0.0. Select one with --dataset and supply your Harbor agent adapter and model. Only the seven tasks in smoke include oracle solutions. Native scores have benchmark-specific units and directions; do not combine scores across families.
Contents and resources
Each directory under tasks/ contains a task brief, case inputs, environment Dockerfile, and independent verifier. The solving and verifier images share a public runtime pinned by digest. Tasks default to 8 CPUs, 16384 MiB memory, 7200 seconds of agent execution, 600 seconds of verification, public network access, and 10240 MiB requested storage. Docker does not enforce the requested storage as a disk quota. Harbor resource overrides are recorded with each run.
release-manifest.json records the Git source, runtime image, byte sizes, and SHA-256 checksums. Dataset versions match the GitHub release tags. Releases from v2.0.0 use this taskset layout; historical tags retain their original contents. The complete raw source tree remains on GitHub.
Licensing and attribution
This release combines project code under the root MIT license with third-party inputs under their own terms. The MIT license does not replace those source licenses. See the benchmark READMEs and source manifests at the release commit for provenance.
The SPOT5 .spot inputs are distributed under CC BY 4.0, from Zequn Wei and Jin-Kao Hao's Mendeley Data release, version 1 (2021), DOI 10.17632/2kbzg9nw3b.1. The bundled source files are unchanged; benchmark case files are derived from them. Retain this attribution and indicate any further modifications when redistributing those data.
- Downloads last month
- 242