Benchmark Results
Reproducible performance snapshots across 6 verticals
First GPU Snapshot Published — Feb 2026
The first public ML GPU benchmark snapshot is live: RTX 4000 SFF Ada, JAX 0.4.38 CUDA, NextStat 0.1.0. Snapshot packaged for Zenodo with pinned environment, baseline manifest (incl. GPU metadata via nvidia-smi), and schema-validated artifacts. More suites and hardware configs to follow.
github.com/NextStat/nextstat-public-benchmarks →Suite Overview
The benchmark program covers 6 verticals. Each suite has a dedicated runbook documenting datasets, baselines, metrics, and the correctness contract.
| Suite | Baselines | Key Metric | Status |
|---|---|---|---|
| HEP | pyhf, ROOT/RooFit | Wall-time (fit, scan, ranking), NLL parity | Internal |
| Pharma | nlmixr2, Torsten (Stan) | NLME convergence time, parameter parity | Planned |
| Bayesian | CmdStanPy, PyMC | ESS/sec, divergence rate, R-hat | Internal |
| ML | neos (JAX), pyhf | Gradient throughput, compile vs execute | Published |
| Time Series | statsmodels, filterpy | Kalman throughput, EM convergence | Internal |
| Econometrics | linearmodels, statsmodels | Panel FE wall-time, DiD/IV parity | Internal |
Existing Internal Results (Preview)
These numbers are from internal CI runs. Public reproducible snapshots with pinned environments and downloadable artifacts will follow.
| Workload | NextStat | Baseline | Speedup |
|---|---|---|---|
| MLE fit (2-channel, 23 params) | 0.8 ms | pyhf: 30 ms | 37× |
| Profile scan (30 points) | 22 ms | ROOT: 19.3 s | 880× |
| Batch toys 10k (CUDA) | 7.1 ms | pyhf: ~10 s | ~1400× |
| Ranking (23 systematics) | 18 ms | ROOT: 5.2 s | 289× |
Published Snapshots
ML GPU Snapshot — 2026-02-09
| Snapshot ID | snapshot-ml-gpu-nextstat-20260209 |
| GPU | NVIDIA RTX 4000 SFF Ada Generation |
| OS / Python | Ubuntu 24.04 / Python 3.12.3 |
| JAX | jax 0.4.38 + jax-cuda12-plugin + CUDA 12.6 (ptxas) |
| NextStat | 0.1.0 (cp312, manylinux_2_34_x86_64) |
| Wheel SHA-256 | 6c1126becb02ab1582c04c1399a09d928… |
| Zenodo SHA-256 | 09e1f929170071712d0f3603e0b7ce81… |
Includes: baseline_manifest.json (GPU metadata via nvidia-smi), snapshot_index.json, per-case results, README_snippet_ml.md. Packaged as .tar.gz for Zenodo deposit.
All-Suites GPU Snapshot (CI) — 2026-02-09
GitHub Actions Run →| Snapshot ID | snapshot-all-gpu-hetzner-rtx4000-2026-02-09 |
| Suites | HEP + Pharma + Bayesian + ML |
| GPU | NVIDIA RTX 4000 SFF Ada (driver 580.95.05) |
| Harness commit | 43b5869a58e4da0baf38c89ffa065b5ec114b307 |
| Wheel SHA-256 | 6c1126becb02ab1582c04c1399a09d928… |
| Archive SHA-256 | f8cdf20d5d71cd55e925385ea2d12951ac… |
First CI-produced snapshot from the public repo via publish_gpu.yml on the self-hosted Hetzner runner. Includes all 4 suites, nextstat_wheel.whl, baseline_manifest.json, snapshot_index.json, and per-suite README snippets. Archive: 6.6 MB.
Replication Bundle — 2026-02-09
Independent replication artifacts for the published snapshot DOI 10.5281/zenodo.18542624. Includes rerun outputs, snapshot_comparison.json, and a signed report template.
Archive SHA-256: 228287839063bbcdc2e411370cf8addbb2eef58a…
Publication Plan
- Separate public repo: nextstat-benchmarks with pinned environments
- Each suite tagged with DOI via Zenodo + CITATION.cff
- CI artifacts: raw JSON results, summary tables, baseline manifests
- Third-party replication: external reruns with GPG/Sigstore signed reports
- Blog coverage for each suite launch