Quantum Canaries Detect Privacy Vulnerabilities in Models Trained on Sensitive Data

Quantum machine learning holds immense promise for accelerating computation, but training these models with sensitive data creates substantial privacy risks through the potential memorisation of individual records. Baobao Song from University of Technology Sydney, Shiva Raj Pokhrel from Deakin University, Athanasios V. Vasilakos from University of Agder, and Tianqing Zhu, alongside Gang Li, address this challenge by introducing the first framework for auditing the privacy of deployed quantum machine learning models. Their innovative approach utilises ‘quantum canaries’, strategically encoded states, to detect memorisation and accurately measure privacy leakage during training, establishing a crucial link between canary offset and quantifiable privacy loss.

This research significantly advances the field by providing a practical method for verifying privacy in quantum systems, bridging the gap between theoretical privacy guarantees and real-world performance. While theoretical frameworks like Differential Privacy exist, they often provide only upper bounds on privacy loss. Many QML applications involve closed-source models or hardware, making traditional auditing impossible without internal access.

Scientists have developed a black-box auditing framework to estimate empirical privacy leakage without needing this access. The core achievement lies in a novel approach based on Lifted Quantum Differential Privacy, allowing for more accurate estimation of privacy loss in complex scenarios. Researchers propose the use of quantum canaries, specially crafted quantum states injected into the training data. These canaries act as probes to detect if the QML model is memorizing sensitive information, and the model’s response reveals its privacy characteristics. Extensive simulations and experiments on real quantum hardware validate the framework’s effectiveness on various QML tasks, including classification, genomic data analysis, and image classification. The experiments also demonstrate a clear tradeoff between privacy and utility, showing how adding noise to achieve privacy impacts model accuracy. The methodology involves generating quantum canaries by encoding random offsets into quantum states, designed to be distinguishable from the natural data distribution. The QML model is then trained on a dataset augmented with these canaries. During inference, the model’s response to the canaries is analyzed; if the model accurately predicts their presence or characteristics, it suggests memorization of training data, potentially violating privacy. Lifted QDP is then used to quantify the privacy loss based on this response, estimating a lower bound on the privacy budget. Experiments utilize various QML models and datasets, conducted on both simulators and real quantum hardware. Key results demonstrate effective privacy estimation and detection of memorization in QML models. The experiments confirm the privacy-utility tradeoff, showing that increasing privacy reduces model accuracy. Importantly, the framework functions effectively on real quantum hardware, demonstrating its practicality. The empirical privacy estimates obtained from the framework are often more informative than theoretical upper bounds. Future research will focus on developing adaptive methods for generating quantum canaries tailored to specific models and datasets. Scientists plan to extend the framework to model multiple noise channels in real quantum hardware and address a wider range of inference threats. Combining the auditing framework with adversarial training techniques will further improve the privacy and robustness of QML models. This work represents a significant step towards practical privacy auditing in Quantum Machine Learning, paving the way for more secure and trustworthy quantum-enhanced AI systems.

Canary States Detect Quantum Model Memorization This study pioneers a novel black-box privacy auditing framework for quantum machine learning (QML), designed to detect memorization of sensitive data within trained models. Researchers engineered a system leveraging “canaries”, strategically offset-encoded quantum states, to pinpoint instances where a model inadvertently memorizes individual data records during training. This approach establishes a rigorous mathematical connection between the offset applied to these canaries and the theoretical bounds on privacy loss, allowing for empirical quantification of privacy leakage. Scientists encoded classical data into quantum states via angle encoding, mapping each feature to a rotation gate applied to an initially prepared quantum state. This creates a feature-dependent geometry within the quantum state. They then employed parameterized quantum circuits (PQCs) to transform these encoded states, integrating them into a hybrid classical-quantum architecture. Quantum measurements converted the quantum states into classical information, enabling evaluation of model predictions. A positive-operator-valued measure (POVM) modeled this process, acknowledging that finite sampling introduces statistical uncertainty. By comparing the model’s response to the original data and the strategically offset canaries, the framework precisely quantifies the extent to which the model memorizes sensitive information, providing empirical lower bounds on privacy budget consumption. Experiments were conducted using both simulated and physical quantum hardware to demonstrate the framework’s effectiveness in measuring actual privacy loss in QML models. Canaries Detect and Quantify QML Privacy Leakage This work introduces a novel black-box privacy auditing framework for quantum machine learning (QML) models, based on Lifted Differential Privacy and utilizing strategically encoded quantum states called “canaries” to detect memorization and quantify privacy leakage during training. The core achievement lies in establishing a rigorous mathematical connection between the offset applied to these canaries and the resulting bounds on trace distance, allowing for empirical lower bounds on privacy budget consumption. This bridges a critical gap between theoretical privacy guarantees and practical verification in deployed QML systems. Researchers generated canaries from the original training data and augmented the dataset, creating two versions differing only in the encoding applied to these canaries. Two QML models were then trained, one with each version of the dataset. During evaluation, the models were tested with the canaries, and the loss value, reflecting prediction error, was computed for each canary. A key metric, the loss threshold, was used to determine if a canary was “memorized” by the model. An algorithm aggregated these binary decisions across all canaries, generating an audit statistic that quantifies potential quantum memorization and privacy leakage. Statistical analysis, utilizing adaptive confidence interval estimators, then yields an empirical privacy loss bound. Comparative analysis reveals that this Lifted QDP approach reduces sample complexity compared to standard QDP auditing methods, achieving a theoretical speed-up. Further experiments validate the accuracy and efficiency of the framework, showing that it achieves more accurate bounds on the privacy parameter and reporting a significant reduction in runtime compared to baseline approaches. These results confirm the framework’s potential to provide a practical and reliable method for verifying the privacy of QML models.

Canary States Reveal Quantum Model Privacy Leakage This research introduces a novel framework for auditing privacy in quantum machine learning (QML) models, addressing a critical gap between theoretical privacy guarantees and practical verification. Scientists developed a black-box auditing method based on Lifted Differential Privacy, employing strategically designed quantum states, termed ‘canaries’, to detect memorization of sensitive data within trained models. By analyzing how these canaries are affected during model training, the framework quantifies privacy leakage and establishes a mathematical link between canary offset and the consumption of the privacy budget. Comprehensive evaluations, conducted using both simulated and physical quantum hardware, demonstrate the framework’s effectiveness in measuring actual privacy loss in QML models. 👉 More information 🗞 Black-Box Auditing of Quantum Model: Lifted Differential Privacy with Quantum Canaries 🧠 ArXiv: https://arxiv.org/abs/2512.14388 Tags: