Qwen3Guard Technical Report | Summary

This article explains the key points of Qwen3Guard Technical Report.

• 2025-10-16 (arXiv)
• Zhao, Haiquan, Yuan, Chenhan, Huang, Fei, Hu, Xiaomeng, Zhang, Yichang, Yang, An, Yu, Bowen, Liu, Dayiheng, Zhou, Jingren, Lin, Junyang, et al.
• Paper
• Github

Read this article in Korean

Summary

• Safety classifiers for large language models (LLMs), models that generate text, face inconsistent moderation policies and must detect harmful content while responses are still being generated. Qwen3Guard-8B-Gen reports average English prompt and response F1 scores of 90.0 and 83.9 versus the strongest respective baseline averages of 87.0 and 79.9, selecting its better Strict or Loose mode separately for each benchmark. F1 combines precision, the fraction of flagged samples that are harmful, and recall, the fraction of harmful samples detected; Strict treats Controversial (context-dependent or disputed content) as Unsafe, while Loose treats it as Safe.
• Generative Qwen3Guard produces safety, harm-category, and response-refusal labels, while Stream Qwen3Guard classifies each arriving response token, an incremental unit of generated text. Both variants offer 0.6B, 4B, and 8B parameters and support up to 119 languages and dialects. Their Safe, Controversial, and Unsafe labels provide a choice of moderation severity.
• In safety reinforcement learning, training with reward feedback, a hybrid safety and helpfulness reward raises Qwen3-4B’s WildGuard-assessed safety from 64.7 to 98.1 without generated reasoning and from 59.0 to 97.4 with generated reasoning, while refusal rates remain 5.3 and 6.2. Streaming detection reaches nearly 86.0% agreement with the human-marked unsafe sentence on response-only samples. For samples containing reasoning traces and final responses, the reported approximately 66.8% corresponds to 4B detections before, within, or up to 128 tokens after the annotated sentence.

The six panels summarize prompt and response classification across English, Chinese, and multilingual datasets. The 0.6B model exceeds the listed English baseline averages, while multilingual prompt results depend more strongly on model size; Qwen3Guard averages use evaluation-specific mode selection.

1 Introduction

Guardrail models are external classifiers that assess user prompts and model responses against a safety policy. The introduction identifies inconsistent binary labels and dependence on complete responses as the central obstacles, then proposes a three-level classifier and a streaming counterpart.

• Controversial supplies an intermediate judgment that applications can map to Safe or Unsafe according to their risk tolerance.
• The two applications use the generative model as training feedback and the streaming model for intervention during generation without retraining the assistant.

2 Safety Policy

The safety policy distinguishes potentially harmful requests from responses that provide harmful information or advice. Its principles are Input/Output Harm Detection, Comprehensive Coverage, and Severity-Level Adaptability.

• The categories are Violent, Non-violent Illegal Acts, Sexual Content or Sexual Acts, Personally Identifiable Information, Suicide & Self-Harm, Unethical Acts, Politically Sensitive Topics, Copyright Violation, and Jailbreak (Only for input). They cover violence, illegal guidance, sexual material, unauthorized disclosure of identifying information, self-harm encouragement or methods, unethical conduct, harmful political falsehoods, unauthorized use of copyrighted works, and attempts to override model instructions or conditioning, respectively.
• Politically Sensitive Topics specifically concerns demonstrably false information about government actions, historical events, or public figures that risks public deception or social harm. Jailbreak applies only to inputs; responses are categorized by the harm they contain.
• Unsafe and Safe mean generally harmful and generally safe across most scenarios; Controversial captures context-dependent harm or disagreement across applications.

3 Generative Qwen3Guard

Generative Qwen3Guard starts from instruction-tuned Qwen3 models, models trained to follow instructions, and applies supervised fine-tuning (SFT), training on examples of the required moderation outputs. Classification is implemented by generating a structured assessment.

3.1 Task Definition

The model receives the task definition, safety policy, harm categories, dialogue context, and output-format requirements. It assesses the latest user query or assistant response and returns a safety level and harm categories; response assessment additionally reports whether the assistant refused the request.

• The output uses the literal fields Safety:, Categories:, and, for responses, Refusal:. Safe content receives Categories: None.
• The illustrated harmful request and safe refusal receive different safety labels, separating the risk of the prompt from the content of the response.

The paired examples separate prompt risk from response safety: a harmful request is Unsafe, while the assistant’s refusal is Safe. Response assessment also outputs a refusal label.

3.2 Data Collection

The training set contains over 1.19M human-annotated and synthetic examples, with prompts accounting for 41.2% and responses for 58.8%. Chinese, English, and Korean account for 26.64%, 21.9%, and 9.91% of the total data, respectively.

• Self-Instruct, a framework for generating instructions from seed examples, expands a fine-grained safety taxonomy, a hierarchy of harm categories. Keyword-guided synthesis provides topical variation, while paired safe and unsafe prompts with similar wording are intended to reduce reliance on superficial lexical cues.
• Responses combine human writing, instruction-model outputs, unsafe outputs synthesized with base models before instruction tuning, and generated reasoning traces from reasoning models.
• Multiple Qwen models vote on annotations, yielding safety-label F1 exceeding 0.9 on a human-annotated validation set. Qwen-MT, the translation model used for data expansion, translates content into 15 additional languages, with language-mixing checks, model judging, and sampled manual review.

Responses constitute 58.8% of training examples and prompts 41.2%. Chinese and English together account for 48.54% of the data; support for up to 119 languages therefore does not imply balanced training coverage.

3.3 Training

Training addresses scarce Controversial examples and noisy annotations through cross-annotation, labeling each partition with models trained on another partition, and label distillation, refinement using a larger teacher model. The data is divided into Part A and Part B; models trained with different Safe/Unsafe sampling proportions label the opposite partition, and disagreements become Controversial.

• Figure 3 associates Safe-enriched training with Loose predictions and Unsafe-enriched training with Strict predictions, consistent with the preceding observation that more Safe examples make predictions more permissive. The accompanying PartA-Strict and PartA-Loose bullets reverse these associations, leaving the sampling description internally inconsistent.
• The roles of the partitions are reversed to cover the full dataset. A second cross-partition refinement stage uses Qwen3-32B as the teacher to reduce annotation errors.

Each partition is annotated by models trained on the other partition. Agreement yields Safe or Unsafe, while Strict/Loose disagreement yields Controversial; the diagram’s sampling associations conflict with the accompanying named-model bullets.

3.4 Evaluation

Evaluation covers seven English prompt datasets and seven English response datasets, including Think, a manually annotated set of generated reasoning traces and responses. Chinese evaluation uses translated public datasets plus PolST, an internal politically sensitive topics dataset with 1,412 prompts and 4,038 responses, including 862 harmful prompts and 2,019 harmful responses.

• English prompt benchmarks are ToxicChat, OpenAIModeration, Aegis, Aegis2.0, SimpleSafetyTests, HarmBench, and WildguardTest. Response benchmarks are Harmbench, SafeRLHF, a safety dataset based on human feedback, Beavertails, XSTest, a test of exaggerated safety behavior, Aegis2.0, WildguardTest, and Think; these datasets provide reference labels for safety classification.
• Other-language evaluation names RTP-LX, a multilingual toxicity benchmark, and PolyGuard-Prompt for prompts, plus PolyGuard-Response for responses. The reported multilingual tables show RTP-LX and PolyGuard-Response.
• The comparison models are LlamaGuard3-8B, LlamaGuard4-12B, WildGuard-7B, ShieldGemma-9B and ShieldGemma-27B, NemoGuard-8B, and PolyGuard-Qwen-7B, all safety classifiers. Strict Mode maps Controversial to Unsafe; Loose Mode maps it to Safe.

3.4.1 Main Results reports leading performance on 8 of 14 English benchmarks. Qwen3Guard-8B-Gen reaches English prompt/response averages of 90.0/83.9 and Chinese prompt/response averages of 85.1/87.1; Qwen3Guard-4B-Gen slightly exceeds the 8B model on Chinese responses at 87.3 and multilingual responses at 78.1 versus 77.6.

• Starred Qwen3Guard averages select the better Strict or Loose result for each benchmark; multilingual tables also underline selections within language columns. These results describe evaluation-specific policy matching and do not generally represent one fixed deployment mode.
• On RTP-LX, the 8B model reports an average of 85.0 versus PolyGuard-Qwen-7B’s 80.9. The 0.6B model reports 74.8, so superiority over existing guards does not hold for every size and task.
• On Think, the 4B and 8B generative models score 84.0 in Strict Mode versus the strongest listed baseline at 81.1. Precision and recall vary with the moderation policy and benchmark.

The 8B generative model’s selected-mode English prompt average is 90.0 versus the highest baseline average of 87.0. Loose Mode performs better on ToxicChat and OpenAIModeration, whereas Strict Mode performs better on Aegis.

The 8B generative model averages 83.9 versus WildGuard-7B’s 79.9, but WildGuard retains the higher XSTest score. Think reaches 84.0 for both 4B and 8B in Strict Mode, above the highest listed baseline score of 81.1.

The strongest Qwen3Guard prompt average is 85.1, and the strongest response average is 87.3. Large gains on the internal PolST dataset contribute to results that combine translated public tests with an internal policy-specific test.

The 8B model’s reported RTP-LX average is 85.0 versus PolyGuard-Qwen-7B’s 80.9, with Strict Mode favored throughout the displayed language columns. The 0.6B average of 74.8 remains below that baseline.

The 4B generative model has the highest Qwen3Guard average at 78.1, exceeding the 8B model’s 77.6 and PolyGuard-Qwen-7B’s 74.0. Increasing parameter count does not uniformly improve multilingual response classification.

Harm-category evaluation uses manually annotated unsafe examples from WildGuard, BeaverTail, and Aegis2.0; Copyright is a weak category in the reported confusion matrices, tables of actual versus predicted categories. Refusal detection reaches F1 scores of 94.6 on XSTest and 90.2 on WildGuardTest for Qwen3Guard-4B-Gen, compared with WildGuard-7B’s 93.3 and 88.6.

• The policy analysis shows that Strict Mode better matches Aegis, while Loose Mode better matches OpenAIModeration. The authors hypothesize that prompt-policy disagreements reflect different tolerances for requests that might still receive harmless responses.
• 3.4.2 Ablation Study, a comparison with selected training components removed, shows that adding Controversial raises the 4B model’s Loose-mode ToxicChat F1 from the binary model’s 71.1 to 80.9 and OpenAIModeration from 70.2 to 80.2. Some response benchmarks favor the binary model, so the improvement is not universal.
• Table 9 reports average distillation gains of +0.47/+1.10 for prompts and +0.50/+0.76 for responses in Strict/Loose modes. Individual scores sometimes decrease, including Loose-mode XSTest responses by 1.3 points.

The operating points show how classification errors vary across datasets and moderation modes. Qwen3Guard Strict better matches Aegis, while Loose better matches OpenAIModeration; these differences support the authors’ policy-mismatch analysis.

Diagonal counts show correct category predictions for many unsafe examples, with confusion involving Unethical Acts. The Copyright response row has no correct predictions, revealing a category-specific weakness that aggregate safety scores do not capture.

The 4B model achieves the best displayed refusal-detection F1 on both datasets, at 94.6 and 90.2. Its recall exceeds its precision, so high refusal coverage still includes erroneous refusal classifications.

The Controversial label improves selected-mode performance substantially on ToxicChat and OpenAIModeration. Several response results favor the binary model, so the table does not establish uniform improvement.

Distillation improves reported average scores in both modes, with larger gains in Loose Mode. Negative entries, including −1.3 on Loose-mode XSTest responses, show that gains vary by task.

3.5 Application I: Safety RL with Generative Qwen3Guard

Safety reinforcement learning (RL) trains Qwen3-4B with Qwen3Guard-4B-Gen as a safety judge. 3.5.1 Reward Design compares Guard-Only Reward with Hybrid Reward, which also uses WorldPM-Helpsteer2, a response-helpfulness scorer, and penalizes refusals.

• Let x be the prompt, t the thinking content, and y the final response. Guard-Only Reward is 1.0 when both is_safe(x,t) and is_safe(x,y) hold, and 0.0 otherwise; both Unsafe and Controversial predictions count as not safe.
• Hybrid Reward is min(−10, WorldPM(x,y)) if is_unsafe(x,t) or is_unsafe(x,y), min(−5, WorldPM(x,y)) if is_refusal(x,y), and WorldPM(x,y) otherwise.
• 3.5.2 Experiment Settings uses Group Sequence Policy Optimization (GSPO), the RL algorithm used to train the policy. Eight responses are generated per WildJailbreak training prompt, drawn from ordinary harmful requests and adversarial attempts to elicit harmful answers; uniformly safe or unsafe groups are removed, leaving 13.7k Think-mode and 6.7k Non-Think-mode samples.

The WildJailbreak evaluation set contains 2,000 harmful and 210 benign prompts. Safety is judged using Qwen3-235B-Instruct-2507 as an LLM-as-a-Judge, a language model that evaluates another model’s output, and WildGuard rather than the reward-providing guard; WildGuard also measures refusals.

• General capability is evaluated with Arena-Hard-v2 for response quality, AIME-25 for mathematical reasoning, LiveCodeBench-V6 for code generation, and GPQA for graduate-level knowledge. Pass@1 measures success on the first generated answer.

3.6 Experiment Results

Guard-Only Reward reaches 99.7 safety under the Qwen3 evaluator and 100.0 under WildGuard in both modes, accompanied by refusal rates of 96.6 in Non-Think mode and 95.2 in Think mode. Hybrid Reward raises safety while keeping refusals at 5.3 and 6.2, with ArenaHard-v2 win rates improving from 9.5 to 10.7 and from 13.7 to 16.6.

• The safety evaluators disagree in magnitude: Hybrid Reward scores 86.5/83.4 under Qwen3 and 98.1/97.4 under WildGuard for Non-Think/Think modes.
• Capability retention is uneven. In Think mode, Hybrid Reward reduces AIME25 from 65.6 to 63.5, LCB-v6 from 48.4 to 47.5, and GPQA from 55.9 to 51.2.
• Training curves measured by Qwen3Guard-Gen-4B show Guard-Only Reward driving both safety and refusal toward high levels. Hybrid Reward improves safety while its refusal rate declines after an early increase.

Guard-only training obtains near-perfect judged safety with refusal rates above 95%. Hybrid training keeps refusals low and improves ArenaHard-v2 win rates, while some capability scores fall, particularly Think-mode GPQA.

Guard-only training increases safety and refusal together, whereas Hybrid training improves safety while refusal falls after an early rise. These curves use Qwen3Guard for measurement; the final result table uses separate safety evaluators.

4 Stream Qwen3Guard

Stream Qwen3Guard evaluates response prefixes, the text generated so far, assigning Safe, Unsafe, or Controversial labels at each token. The prompt is submitted to both the assistant and the guard; an external framework decides whether to interrupt, and continuing response tokens are forwarded immediately for moderation.

The prompt and response moderators assess different targets within the conversation. Response labels change as content arrives, illustrating that a prefix initially judged Safe can later trigger an unsafe judgment.

4.1 Architecture

Two independent classification pathways attach to the final layer of the Qwen3 transformer, the underlying model that produces contextual text representations. One pathway assesses the user query and the other the assistant response; each predicts severity and harm category.

• Each pathway linearly projects the last hidden state, the backbone’s internal representation of the current token, applies LayerNorm, normalization of that representation, and uses separate Softmax outputs, probability distributions over labels, for risk and category.
• This design produces classification outputs directly instead of generating a textual assessment for every new prefix.

4.2 Data Collection

Token annotations are derived from sample-level Unsafe or Controversial labels by locating the first unsafe boundary. For each prefix, an ensemble generates rollouts, possible continuations; Generative Qwen3Guard checks the completed responses, and a prefix is flagged when at least 85% are Unsafe or Controversial.

• Because harmful continuations can overstate the risk of an otherwise harmless prefix, Qwen3-235B-A22B separately judges only the available prefix, without predicting subsequent content.
• A boundary is accepted only when the rollout assessment flags risk and the prefix judge labels the available text Unsafe. The first qualifying token and every later token inherit the original sample-level label; earlier tokens are treated as Safe.
• The report describes the 85% threshold as aligned with human annotations in pilot experiments but provides no quantitative comparison across thresholds.

4.3 Training

Cross-entropy loss, a loss that penalizes low probability assigned to the correct class, jointly optimizes query and response severity and category predictions. Query loss is computed only at the final <|im_end|> token; response loss averages severity and category losses over all response tokens.

• Category loss is included only when the ground-truth risk label is Unsafe or Controversial. Safe examples contribute severity loss without category loss.

4.4 Evaluation

Stream evaluation uses the same classification datasets and metrics as Generative Qwen3Guard. A debouncing mechanism, confirmation across consecutive predictions, flags a response only when token i and token i−1 are both classified as Unsafe or Controversial, then uses token i’s category for the response.

• For 4B models, Stream versus Gen averages are 89.1 versus 89.3 on English prompts, 81.8 versus 83.7 on English responses, 84.1 versus 84.6 on Chinese prompts, and 84.9 versus 87.3 on Chinese responses.
• Reported multilingual averages are 80.2 versus 81.6 for 4B prompts and 75.5 versus 78.1 for 4B responses. The reported 0.6B prompt gap is substantially larger: 64.4 versus 74.8.
• The report’s approximately two-point average reduction does not characterize every size and benchmark.

At 4B, Stream’s selected-mode English prompt average of 89.1 nearly matches Gen’s 89.3. Previous Best combines prior models’ best scores by benchmark and does not represent one deployable baseline.

The 4B streaming response average is 81.8 versus Gen’s 83.7, while its Strict-mode Think score is higher at 85.4 versus 84.0. Lower average accuracy does not imply a decline on every individual dataset.

For 4B, Stream’s prompt average is 0.5 points lower and its response average 2.4 points lower than Gen’s. Strict Mode substantially exceeds Loose Mode on PolST responses.

The reported 0.6B streaming multilingual prompt average is 64.4 versus Gen’s 74.8, a 10.4-point gap. Reported gaps are smaller at 4B and 8B, so the accuracy cost varies with model size.

Streaming averages are 70.6, 75.5, and 75.8 for 0.6B, 4B, and 8B, respectively. These trail the corresponding Gen averages, although the larger streaming variants exceed PolyGuard-Qwen-7B’s earlier 74.0 average.

Detection latency is evaluated on a reported 813 response-only samples and 569 samples containing thinking content and final responses. Humans mark the earliest unsafe or controversial sentence because token-level annotation has low agreement; a Hit means detection within that sentence, rather than agreement on an exact token.

• The response-only Hit counts are 694 for 8B and 699 for 4B. The report summarizes response-only performance as nearly 86.0%; the displayed 4B bins total 814, one more than the stated 813 samples.
• For thinking-plus-response samples, the 4B Ahead, Hit, 1–32, 33–64, and 65–128 bins total 380 of 569 samples, matching the reported approximately 66.8%. These include detection before or within the annotated sentence and delays up to 128 tokens afterward, rather than detection within the first 128 generated tokens.
• The efficiency comparison repeatedly submits the accumulated response to Gen at 32-token intervals. Stream processes incoming tokens without reprocessing earlier tokens; Figure 9 reports relative time normalized to Gen’s initial 32-token check, without absolute latency or hardware specifications.

Response-only detection concentrates in the human-marked sentence, with 694 and 699 Hits for 8B and 4B. Thinking-plus-response results include longer delays and Safe predictions; the 4B bins through 65–128, including Ahead and Hit, total 380 of 569 samples and explain the reported 66.8%.

The generative baseline repeatedly classifies accumulated prefixes after each 32-token chunk, producing growing overhead. Stream’s relative time increases more slowly, but the normalized axis provides no absolute latency or hardware-specific throughput.

4.5 Application II: Real-time Safety Intervention with Stream Qwen3Guard

Stream Qwen3Guard replaces the safety checker in CARE, a detect–rollback–intervene framework that revises generation after risky output is detected (Hu et al., 2025). The experiment uses Qwen3-4B, a 40-token buffer, and at most 5 retries.

• In Non-Think mode, Qwen3-assessed safety rises from 47.5 to 85.7, WildGuard-assessed safety from 64.7 to 95.7, and quality from 50.0 to 66.4. In Think mode, the corresponding changes are 43.8 to 72.0, 59.0 to 88.9, and 58.1 to 67.8; quality is judged by Qwen3-235B-A22B-Instruct-2507 using the Arena-Hard scoring protocol.
• Wait Tokens measures additional tokens the user waits through because of rollbacks. It averages 70.1 in Non-Think mode and 101.0 in Think mode, quantifying a remaining intervention cost.
• The table compares CARE with the unmodified assistant; it does not provide a direct numerical comparison against CARE’s original generative safety checker.

CARE with the 4B streaming guard improves judged safety and quality over Qwen3-4B alone in both modes. Average Wait Tokens of 70.1 and 101.0 quantify additional generation caused by rollback interventions.

5 Related Work

Related work covers safety training, instruction-following guard models, guards that reason about safety judgments, multilingual moderation, and token-level classifiers. The report distinguishes its rollout-based annotation of incomplete prefixes from sentence-level approximations and its model-voting construction of Controversial labels from earlier rule-based labeling.

• The section acknowledges that prior work already introduced a Controversial label and explored token-level moderation. Qwen3Guard’s contributions include its label-construction pipeline, streaming implementation, and evaluation of both variants and their applications.

6 Conclusion

The conclusion presents Qwen3Guard as a multilingual moderation tool combining three-level judgments with generative and streaming variants. It recognizes that external moderation is an incomplete safety solution and discusses further work on model safety and intervention during generation.

Limitations

The stated limitations are vulnerability to targeted adversarial prompts, bias in moderation annotations, limited generalization under partial context, and insufficient sensitivity to regional and cultural differences. The model lacks geolocation-aware or policy-adaptive mechanisms, so selecting Strict or Loose Mode does not establish adaptation to a particular jurisdiction.

• Paraphrasing, obfuscation, and context manipulation may bypass the guard. Benchmark scores do not establish robustness to sophisticated targeted attacks.
• Training annotations may disproportionately flag particular demographic, linguistic, or cultural groups. The report identifies this risk without reporting a dedicated demographic fairness evaluation.
• Stream’s partial context can produce false positives, erroneous risk flags, or false negatives, missed harmful content, on ambiguous early phrases, distributions unlike its training data, or languages with limited training resources.

Authors

The report is credited to Qwen Team. Core Contributors are Haiquan Zhao, Chenhan Yuan, Fei Huang, Xiaomeng Hu, Yichang Zhang, An Yang, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin; a separate alphabetically ordered contributor list follows.

Appendix

• 6.1 Instruction Template of Generative Qwen3Guard provides the full prompt and response assessment templates. They specify Safety: Safe, Safety: Unsafe, Safety: Controversial, and Categories:, with response assessment adding Refusal: Yes or Refusal: No; Jailbreak appears only in the prompt template.
• 6.2 Categories Classification Results of Stream Qwen3Guard evaluates streaming harm categories on the same category dataset used for Gen. Its confusion matrices show correct classification in several categories, but no correctly classified Copyright Violation responses and frequent classification of Political examples as Unethical Acts.
• 6.3 Illustrative Example of Sentence-Level Annotation for Stream Qwen3Guard’s Latency Evaluation divides a response into four sentences and marks the second as the first unsafe sentence, under Suicide & Self-Harm. This example clarifies the sentence-level reference used by the latency evaluation.
• 6.4 A Case Study of Safety RL compares Qwen3-4B and its Hybrid-reward SafeRL variant on a harmful request framed as fiction. The displayed SafeRL response redirects toward narrative advice without explicit harmful instructions or outright refusal; this single example does not measure robustness across attacks.

Brief Thoughts

The engineering contribution combines policy-sensitive classification with incremental detection. Benchmark-specific Strict/Loose results support the usefulness of an intermediate label, while the latency experiment tests detection relative to a human-marked unsafe sentence.

Deployment assessment should account for benchmark-specific mode selection, the reported 0.6B multilingual prompt gap, and delayed detection on thinking content. The sampling-description inconsistency limits reproducibility, and the relative runtime plot does not establish production throughput or absolute response latency.

The application results quantify the safety–refusal trade-off: Guard-Only Reward produces refusals above 95%, while Hybrid Reward keeps refusals low with lower judged safety. Evaluators separate from the training guard strengthen the evidence, but their disagreement and the Think-mode GPQA decline from 55.9 to 51.2 qualify the claim of preserved general capability.