Super Suffixes Bypass Text Generation Alignment and Guard Models, Achieving 100% Override of Security Mechanisms

The increasing use of large language models presents significant security challenges, as these systems process potentially harmful inputs and even generate executable code. Andrew Adiletta from MITRE, Kathryn Adiletta and Kemal Derya from Worcester Polytechnic Institute, along with Berk Sunar, investigate vulnerabilities in current protective measures, known as ‘guards’, designed to prevent malicious outputs. Their research introduces ‘Super Suffixes’, carefully crafted additions to prompts that simultaneously bypass multiple safety mechanisms across different language models, even those with varying methods of processing text.

The team successfully circumvents the Llama Prompt Guard 2 protection system on five distinct text generation models, demonstrating a previously unknown weakness in its design, and importantly, they also propose a novel detection method, DeltaGuard, which achieves near-perfect accuracy in identifying these attacks by analysing internal patterns within the language model itself. This breakthrough significantly enhances the robustness of language models against adversarial prompt attacks and represents a crucial step towards securing these powerful technologies. The study focuses on crafting prompts that trick LLMs into generating harmful content, despite built-in safeguards. The central idea is that while powerful, LLMs are vulnerable to subtle manipulations of input prompts, revealing fundamental weaknesses in their protective layers.

The team tested Google Gemma 2B and Llama 3 2B models against a broad range of harmful requests, including malicious code generation, chemical and biological weapon synthesis, copyright violation, misinformation, harassment, and illegal activities. They employed a technique called suffix generation, iteratively modifying prompts by adding text to the end to both reduce the harmfulness score detected by the LLM’s internal guard and maintain the original intent of the harmful request. The optimization process balanced minimizing the guard model’s harmfulness score with maximizing the likelihood of the LLM generating the desired, harmful output. Evaluation metrics included the guard score, the benign rate, and t-SNE analysis, a technique used to visualize relationships between prompts and outputs. Results using the HarmBench dataset further validated the findings, demonstrating that suffix generation successfully bypasses the safety mechanisms of both Gemma 2B and Llama 3 2B, effectively balancing the need to evade detection with the preservation of harmful intent. T-SNE analysis revealed that malicious prompts without suffixes are distinct from benign prompts, but the addition of suffixes brings them closer together, making detection more difficult.

The team also introduced “Super Suffixes,” demonstrating that more complex and optimized suffixes achieve even higher bypass rates. Further investigation into token similarity provides insights into the semantic changes introduced by the suffixes.

This research highlights the fragility of LLM safety measures and the need for more robust defenses. Red teaming, actively attempting to break the system, is crucial for identifying vulnerabilities and improving safety. While the techniques described could be misused, this work contributes to the growing body of knowledge on LLM security and provides valuable insights for developing more secure and reliable models.

Super Suffixes Attack and DeltaGuard Detection Researchers addressed growing security concerns surrounding Large Language Models (LLMs) by pioneering a novel attack vector termed “Super Suffixes” and a corresponding detection method, DeltaGuard. The study began by constructing a dataset for malicious code generation to extract a concept direction specifically associated with malicious code intent. This dataset enabled the team to map domain-specific sensitivity within the Linear Representation Hypothesis, demonstrating that high-level concepts, such as malicious code generation, are represented as distinct linear directions within the model’s embedding space. Recognizing the need for robust detection, the researchers developed DeltaGuard, a lightweight countermeasure that analyzes the changing similarity of a model’s internal state to specific concept directions during token sequence processing. Scientists demonstrated that the cosine similarity between the residual stream and these concept directions serves as a distinctive fingerprint of model intent, effectively identifying Super Suffix attacks. Experiments revealed that DeltaGuard significantly improves detection rates, achieving nearly 100% classification of non-benign prompts and enhancing the robustness of the guard model stack against adversarial prompt attacks.

Super Suffixes Bypass LLM Security Guards This work presents a breakthrough in securing large language models (LLMs) against adversarial attacks, specifically through the discovery and mitigation of “Super Suffixes.

The team achieved this by simultaneously optimizing a loss function against both the text generation model and the guard model, effectively deceiving both systems. To understand these attacks, the team analyzed the internal state of the language model, focusing on the cosine similarity between the residual stream and specific concept directions during token processing. This analysis led to the development of DeltaGuard, a countermeasure that significantly improves the detection of malicious prompts generated through Super Suffixes. DeltaGuard achieves nearly 100% accuracy in classifying non-benign prompts by tracking how the cosine similarity to a “refusal direction” evolves over time. Detailed experiments on Google Gemma demonstrate the effectiveness of this approach. Researchers visualized the cosine similarity traces for different classes of malicious code generation prompts, revealing distinct groupings based on prompt type. Specifically, the team observed that malicious prompts exhibit the highest cosine similarity to the refusal vector both before and after text generation begins, while benign prompts show significantly lower similarity. This temporal analysis, tracking changes in cosine similarity across token positions, provides a unique fingerprint of the model’s alignment state and allows for accurate detection of adversarial attacks. The methodology builds upon the Linear Representation Hypothesis, proposing that changes in relationships to concept directions encode higher-order semantics, including indicators of malicious intent.

Super Suffixes Bypass Language Model Guards This work introduces a novel approach to generating adversarial inputs, termed Super Suffixes, capable of bypassing security measures in large language models and their associated guard mechanisms. This represents a significant advancement in understanding the vulnerabilities of current guard systems, revealing that they can be circumvented by strategically crafted inputs that simultaneously misalign both the language model and its protective layer. To address this newly identified threat, the team developed DeltaGuard, a countermeasure that effectively detects Super Suffix attacks by analyzing the internal state of the language model during text processing. By monitoring the similarity between the model’s residual stream and specific concept directions. 👉 More information 🗞 Super Suffixes: Bypassing Text Generation Alignment and Guard Models Simultaneously 🧠 ArXiv: https://arxiv.org/abs/2512.11783 Tags: