Researchers Measure Quantum Processor Speed Using New Benchmark

Andrew Wack and Riverlane Computing A performance shortfall exists for workloads involving layered, parameterised circuits repeatedly executed within classical-quantum workflows like variational algorithms and error-mitigated simulations.

The team formalises CLOPSh (Circuit Layer Operations Per Second) as a thorough speed benchmark defined over layered, hardware-aware circuits. CLOPSh measures the sustained rate at which the system executes physical layers, parallel slices of qubit-disjoint two-qubit gates separated by synchronization barriers. As each layer represents one time slice of an N-qubit circuit, this rate directly maps to the execution rate of layered N-qubit circuits connecting it to published device benchmarks. Balancing scale, quality and speed to achieve impactful quantum computation Scientists increasingly recognise three interdependent attributes governing quantum computing performance: scale, quality, and speed. Scale determines problem size representation; quality dictates circuit depth reliability; and speed defines completed work per unit time. This framework underpins the benchmarking of near-term quantum computers and becomes more pertinent as systems approach utility-scale experimentation. Recent advances in both hardware and software have enabled execution of circuits spanning over 100 qubits. Simultaneously, practical workloads shifted towards iterative layered algorithms embedded within classical optimisation or post-processing loops, including variational methods, Trotterised dynamics, and error-mitigated experiments. Consequently, even with acceptable fidelity, the overall time required for statistically meaningful results may render an experiment impractical. Error mitigation techniques trading circuit repetition for improved accuracy rapidly increase total computational cost, amplifying the importance of sustained execution throughput. Meaningful progress toward quantum advantage demands improvements not only in fidelity and scale but also systematic gains in speed at which quantum experiments can be executed. Qubit-level metrics expose fabrication quality isolating component performance under ideal conditions; furthermore, scale is non-trivial as programmable qubits differ from single physical qubits depending on architecture. Device-level metrics characterise processor scalability, for example, delivering realistic parallel operation fidelity or maximum circuits per second (MCPS) defined in Sec II B under operating conditions. Lower-level benchmarks provide guard rails for higher-level achievements: MCPS sets a hard ceiling on system throughput that software optimisation cannot exceed. Classical high-performance computing encountered similar issues; peak floating point rates overestimated achievable system throughput leading to adoption of system-level benchmarks like LINPACK measuring sustained performance across the memory and interconnect hierarchy. The analogous lesson applies to quantum systems: accurately characterising speed requires a benchmark defined at the system level inclusive of all associated costs with complete execution pipelines. A further complication arises from distinguishing latency versus throughput, as short duration benchmarks measuring small circuits are dominated by transient effects such as compilation warm up or control loading. Many practical workloads operate in steady state where pipelines are filled executing continuously over extended periods, so benchmarks not explicitly targeting this behaviour risk mischaracterising long-running experiment performance. Application levels, tied to specific computationally relevant problems akin to SPEC suites for classical computing, remain largely undeveloped for quantum systems; meaningful application benchmarks require established scientific value, known classical references and problem sizes beyond efficient simulation otherwise they cannot distinguish quantum execution from a classical solver. The QED-C suite measures result fidelity and time across algorithm families representing the most systematic attempt at benchmarking hardware at an algorithmic level but operates below utility scale without reporting overheads measuring single quality rather than sustained throughput falling short of their implied metric. Metriq aggregates metrics into composite scores excluding end-to-end overhead covering circuits well below utility scale not constituting speed benchmark in terms of delivering correct output rapidly under realistic conditions. Sample based Krylov Quantum Diagonalisation (SKQD) applied to protein ligand complexes exceeding 12,000 atoms represents domain relevant workload anchoring future application benchmarks. Currently, tractable system levels allow for principled hardware-agnostic benchmarking. These considerations motivate a need for a system-level speed benchmark defined by parallel circuit layers inclusive of all execution costs measured during steady state operation and executed under independently verified operating conditions. Early attempts including the original Circuit Layer Operations Per Second (CLOPSv) addressed this measuring how quickly systems execute parameterised circuits interacting with runtime environments using families derived from Quantum Volume experiments ensuring connection to established quality metrics. This paper reinstantiates that framework as CLOPSh defining speed over explicitly layered hardware aware circuits compatible with layer fidelity benchmarking enabling coherent interpretation within utility scale preserving holistic implementation agnosticism motivating initial proposals. Benchmarking quantum processor quality has been an active research area since early programmable system demonstrations; traditional approaches focus on characterising individual components, single- and two-qubit gate fidelities, measurement errors, coherence times using protocols like randomised benchmarking, but these component level metrics are insufficient for characterising user experience executing nontrivial algorithms. As quantum processors scale to hundreds of qubits, execution speed is a critical performance dimension alongside scale and quality. While substantial progress has been made in benchmarking circuit fidelity, existing speed metrics often fail to reflect sustained throughput experienced by users running utility-scale workloads. This shortfall is especially pronounced for layered, parameterised circuits executed repeatedly within classical-quantum workflows such as variational algorithms and error-mitigated simulations. To address this, Circuit Layer Operations Per Second (CLOPS_h) was formalised as a holistic speed benchmark defined over layered, hardware-aware circuits measuring the rate at which the system executes physical layers, parallel slices of qubit-disjoint two-qubit gates separated by synchronization barriers. CLOPSh is obtained under layer-fidelity operating conditions linking the speed measurement to an independently verified quality envelope and sharing its layer decomposition with scalable layer-fidelity benchmarks enabling coherent interpretation of both speed and quality. An EPLG covers all interactions realised as fixed sequence ordered physical sublayers, each single parallel execution slice of disjointed two-qubit gates constituting the LF notion of a “layer”: ordered sequences executed in order forming CLOPSh’s counting unit (.1). Principled speed metrics have lagged behind quality benchmarks; understanding why system level is needed requires examining existing indicators and their scope limitations. Gate per second metrics characterise raw throughput under laboratory conditions while device repetition rates measure hardware applying primitive operations but neither capture full stack costs associated with complete experiments. Achieving sustained performance with layered circuits is important as processors grow larger, demonstrated by this work reaching 74,863 circuit layers per second on a superconducting processor, a threefold improvement over prior methods measuring execution rates. This breakthrough surpasses the previously established limit for executing complex algorithms repeatedly and reliably, something impossible before due to limitations in accurately assessing true throughput under demanding conditions. By linking speed directly to independently verified quality metrics via layer fidelity benchmarking, scientists provide an unambiguous measure of device capability relevant at utility scale. Layer fidelity benchmarks averaged 98·7 percent alongside CLOPSh; this demonstrates not only speed but also reliable results with minimal error propagation through multiple circuit layers. Furthermore, a direct correlation between MCPS and CLOPSh was revealed establishing that the superconducting processor attained an upper limit of 74,863 circuits per second under sustained operation.

The team confirmed these rates were maintained across parameterised circuits used in variational algorithms like those employed for quantum simulation; no significant degradation in performance occurred even after executing thousands of consecutive layers. However, current figures represent benchmarked capability on relatively simple layered circuits and do not yet reflect execution speeds achievable when factoring in full classical control overheads or complex application-specific compilation routines needed to tackle genuinely challenging problems. Establishing a reliable measure of speed is vital as quantum computers expand beyond mere qubit counts; the formalisation of CLOPSh offers a holistic benchmark assessing sustained performance during complex calculations, a critical step towards realising practical applications like advanced materials discovery or drug design. This focus on system-level throughput deliberately sidesteps an ongoing debate concerning application benchmarks tied to demonstrable “quantum advantage”. Researchers prioritised system throughput rather than application performance for this benchmark creation providing a crucial foundation for evaluating quantum computer capabilities as they grow more complex because it establishes a standardised way to measure sustained processing speed during calculations. Scientists formalised CLOPSh, Circuit Layer Operations Per Second, to assess how quickly a system completes layers of operations rather than focusing solely on qubit numbers or individual gate speeds. The researchers demonstrated that their superconducting processor achieved a rate of 74,863 circuits per second under continuous operation using the newly defined metric, CLOPSh. This benchmark assesses sustained performance by measuring completed circuit layers, slices of two-qubit gates, rather than simply counting qubits or gate speeds. Linking this speed measurement with established methods for verifying quantum processing quality provides a clear way to evaluate overall system capabilities. The formalisation of CLOPSh offers a standardised approach to measure consistent calculation speed as quantum computers become more complex and are used in applications such as variational algorithms. 👉 More information 🗞 CLOPS: Benchmarking System Speed at Utility Scale ✍️ Andrew Wack 🧠 ArXiv: https://arxiv.org/abs/2608.18044 More like thisQuantum AlgorithmsResearchers Cut Quantum Error Rates with Optimised Atom LossQuantum AlgorithmsFire Opal and Black Opal combine for quantum finance learningQuantum Research NewsGerman scientists cut Toffoli gate count for sparse quantum statesQuantum AlgorithmsResearchers Achieve 7.44e-9 Fidelity for 200-Qubit StatesStay currentSee today’s quantum computing news on Quantum Zeitgeist for the latest breakthroughs in qubits, hardware, algorithms, and industry deals. Tags: