ReproQuorum : Signed, Scope Deterministic Pipelines and Benchmark Quorums for Verifiable ML Reproducibility

Abstract

Reproducing machine learning results across teams and hardware often fails due to hidden randomness, drifting datasets, and unstable environments that alter metrics without visible changes to code or intent. ReproQuorum virtualizes randomness with scoped, counter‑based generators, derives hash‑locked master seeds from commit, dataset fingerprints, and canonicalized configuration, emits signed JSON‑LD provenance receipts, and packages a Nix‑pinned OCI capsule for deterministic re‑execution. Repository continuous integration targets a benchmark quorum of ≥ 2 public datasets per task; the reported experiments enforce ≥ 1 dataset per task. Across named entity recognition (WNUT 2017) and extractive question answering (SQuAD v1.1), replication attains Tier‑A on matched architectures (byte‑identical) and Tier‑B across CPU↔CUDA (metric‑identical with Δ≤10^(-6)). Performance improves by +1.1 F1 on WNUT and +0.7 F1 / +0.6 EM on SQuAD under equal budgets. Capsules and receipts enable one‑command verification and reduce ambiguity about seeds, data, and environments. Code, data manifests, checkpoints, capsules, and model cards are released with signed receipts and container digests to support independent validation.