agentx-mini

agentx-mini is a smaller workload for benchmarking local LLM servers, derived from semianalysisai/cc-traces-weka-062126-256k. It contains 200 sessions and 9,439 model calls, including 1,363 subagent calls.

Changes from the original dataset:

• Divide input and output lengths by 8, rounding each output up. Hash IDs stay the same; logical blocks shrink from 64 to 8 tokens.
• Divide active request time by 8 and shorten idle gaps. Replay uses a 1-second idle cap.
• Keep complete sessions with less than 10,000 requested output tokens, counting all subagents and auxiliary calls.
• Preserve request grouping, spawn/join dependencies, shared prompt content, and reference overlaps. Actual replay overlap depends on server speed.

Stats

traces:                       200
main_turns:                 8,076
subagent_groups:               76
subagent_inner_requests:    1,363
total_model_requests:       9,439
total_input_tokens:   126,413,728
total_output_tokens:      962,854

Main turns are top-level calls. Input totals include repeated prefixes; actual prefill depends on the server's prefix caching. Rendered prompts add tokenizer/chat-template overhead.

Files and reproduction

traces.jsonl contains the scaled Weka records; stats.txt contains their totals. AgentX Harness handles prompt synthesis and replay using the target model's tokenizer.

To rebuild agentx-mini using distill.py:

pip install huggingface-hub orjson
python distill.py --output rebuilt

Run agentx-mini with AgentX Harness

Requires Python 3.11–3.13. Install in a dedicated virtual environment, following the AgentX Harness README:

python3 -m venv .venv
source .venv/bin/activate
pip install agentx-harness==1.0.7

Set the block size for agentx-mini. AgentX 1.0.7 needs an adjustment for this dataset's 8-token blocks. In the activated virtual environment, locate the installed plugin configuration:

python -c 'from pathlib import Path; import aiperf.plugin; print(Path(aiperf.plugin.__file__).with_name("plugins.yaml"))'

Open that file. Under public_dataset_loader → weka_hf → metadata, change default_block_size: 64 to default_block_size: 8. This setting applies to all weka_hf datasets in this environment; reinstalling the harness can reset it.

1. Download agentx-mini into a local directory.
2. Start your local LLM server and load your model, for example using llama-server. It must expose an OpenAI-compatible API. Enable parallel request slots to exercise subagents. Allow up to 32,000 tokens per request for input plus output, with additional room for chat-template overhead. Keep the server running during the benchmark.
3. Configure AgentX. Save the configuration below as benchmark.yaml beside traces.jsonl. Replace the served model name, tokenizer, and server URL with your server's settings. Set hf_weka_dataset to the absolute path of the downloaded dataset directory (run pwd there to get it).

random_seed: 42
benchmark:
  models: [YOUR_SERVED_MODEL_NAME]
  tokenizer:
    name: YOUR_TOKENIZER
  endpoint:
    urls: [http://127.0.0.1:8000]
    streaming: true
    extra: {ignore_eos: true}
    use_legacy_max_tokens: true
  datasets:
    - name: default
      type: public
      dataset: weka_hf
      hf_weka_dataset: /absolute/path/to/agentx-mini
      sampling: sequential
      trace_idle_gap_cap_seconds: 1
  phases:
    - name: profiling
      type: concurrency
      timing_mode: agentic_replay
      concurrency: 1
      duration: 1200
      trajectory_start_min_ratio: 0
      trajectory_start_max_ratio: 0

Run from that directory:

agentx profile \
  --config benchmark.yaml \
  --output-artifact-dir ./results/agentx-mini

Automatic download: instead of downloading first, set hf_weka_dataset to SocialLocalMobile/agentx-mini and run the same command.

License

Apache 2.0, matching the source dataset published by SemiAnalysis from its Claude Code proxy traces. The dataset, script, and documentation in this repository are distributed under these terms.

Modified files: traces.jsonl is a scaled and filtered derivative of source revision 8fecd2f; stats.txt is regenerated, and this README and .gitattributes are adapted for agentx-mini. The data changes are listed above. References to SemiAnalysis identify the source and do not imply endorsement.