Independent Cross-Stack Benchmark Evaluates Commercial Quantum Error Management on 156-Qubit IBM Heron Hardware

Independent Cross-Stack Benchmark Evaluates Commercial Quantum Error Management on 156-Qubit IBM Heron Hardware Provider-reported QPU seconds per Estimator job by size and configuration. An independent benchmarking study published on arXiv (arXiv:2608.05202) provides a protocol-level evaluation of commercial quantum error suppression and error mitigation software stacks. Conducted by researchers from The Catholic University of America, University of Deusto, and Universidad de los Andes, the paper compares native IBM Qiskit Runtime primitives against third-party managed pipelines in the Qiskit Functions Catalog—specifically Q-CTRL Performance Management and Qedma QESEM. All experiments were executed on ibm_pittsburgh, a 156-qubit IBM Quantum Heron r3 superconducting processor. The benchmark evaluates two execution models: sampling tasks (bitstring counts) via Sampler interfaces, and observable expectation values via Estimator interfaces. To ensure like-for-like comparisons, IBM and Q-CTRL configurations operated under matched shot budgets (215 = 32,768 shots per instance), while Qedma QESEM was configured through its native precision-and-time-budgeted interface (0.1 precision target, 600-second QPU time cap). Abstract workloads were submitted across all providers within identical hardware calibration windows. In the Sampler Track, tested across Bernstein–Vazirani (25–75 qubits), Quantum Phase Estimation (10–30 counting qubits), GHZ-state preparation (25–50 qubits), and Randomized Mirror Circuits (25–100 qubits), Q-CTRL’s autonomous suppression pipeline delivered the highest exact-output success probabilities across all 12 test cases. On 30-qubit Quantum Phase Estimation (QPE), where raw and measurement-twirled IBM runs yielded zero exact-match shots out of 32,768 (0.00% success), Q-CTRL maintained a 12.69% exact success probability. On 100-qubit Randomized Mirror Circuits, Q-CTRL retained a 76.45% exact 100-bit target string success rate, compared to 9.14% on IBM raw execution. [ Estimator Track Accuracy & QPU Overhead Benchmarks (TFIM 25–75 Qubits) ]Software Stack ConfigurationError Metrics vs. MPS Exact ReferenceError Reduction & QPU RuntimeIBM Raw Execution(SamplerV2 / EstimatorV2 Level 0)• Magnetization MAE (mX): 0.1484• Correlator MAE (cZZ): 0.0282• Overall MAE: 0.0883• 1.00× (Baseline)• Reported QPU Time: ~17.8 seconds per jobIBM TREX + Twirling(Resilience Level 1 + Gate Twirling)• Magnetization MAE (mX): 0.1026• Correlator MAE (cZZ): 0.0587• Overall MAE: 0.0807• 1.09× Error Reduction• Reported QPU Time: ~28.0–37.8 seconds per jobQ-CTRL Performance Management(Autonomous Suppression Pipeline)• Magnetization MAE (mX): 0.0376• Correlator MAE (cZZ): 0.0194• Overall MAE: 0.0285• 3.10× Error Reduction• Reported QPU Time: ~28.0 seconds per jobQedma QESEM(Characterization & Unbiased Mitigation)• Magnetization MAE (mX): 0.0278• Correlator MAE (cZZ): 0.0097• Overall MAE: 0.0188• 4.70× Error Reduction• Reported QPU Time: ~211–311 seconds per job In the Estimator Track, evaluated on an eight-layer Transverse-Field Ising Model (TFIM) circuit at 25, 50, and 75 qubits against an exact Matrix Product State (MPS) reference, Q-CTRL and Qedma QESEM reduced aggregate mean absolute error (MAE) by factors of 3.10× and 4.70× over raw execution, respectively. However, the two third-party stacks demonstrated distinct resource-accuracy profiles: Q-CTRL achieved its error reduction using 28 seconds of QPU time per job (matching IBM baseline runtime), whereas Qedma QESEM achieved the highest precision by consuming 211 to 311 seconds of QPU time per job (7.5× to 11.1× the runtime of Q-CTRL). Review the research preprint on arXiv (Quantum Error Management Benchmark) here and explore cloud primitive specifications at IBM Qiskit Runtime here and here. September 23, 2026 Mohamed Abdel-Kareem2026-09-23T21:31:12-07:00 Leave A Comment Cancel replyComment Type in the text displayed above Δ This site uses Akismet to reduce spam. Learn how your comment data is processed.