Appendix: method and data

One deterministic function, truth_grade(), assigns every grade from fields already on each case row. No model and no human reviewer is consulted, so no inter-rater agreement figure exists. Source-label error is therefore a limitation, and so is error correlated with the labelling source.

Prevalence is 11.4%: 436 of 3,817. Allowing every call is right 88.6% of the time, which is why accuracy is never shown alone.

Two sources carry a restricted redistribution marker. A census of both corpora found 0 rows from either, so neither contributes a case to any figure.

Corpus s3 shares no case id with s2. Its positives are 87.3% grade A against 3.9% in s2, and its prevalence is 0.903% against 11.4%. Blocking everything there scores F1 0.0179, 11.5 times lower than on s2, purely because of prevalence. So no score on this Space is compared across the two corpora.

All 15 pairwise AUC differences among the 6 models are significant; the median minimum detectable difference is 0.0416. Positives contribute 98.7% of the AUC variance. The held-out transfer results are out of this revision by design.

For a deploy decision, a model's rank depends on which kind of destructive call dominates. Before you reuse any figure here, check which grades your own traffic resembles.

Across the 22 models, 69 of 220 buckets are empty and 51 hold fewer than 20 cases. falcon3-1b-instruct puts 1,499 of 3,817 cases into the single bucket from 0.4 to 0.5, so its threshold has little room to move.

Pick other models in the menu to compare their shapes. A shared cut-off such as 0.5 means something different for each model. Fit each threshold to your own false-block budget instead.

As shipped, deberta-v3 blocks 3,629 of 3,817 cases, 95.1%. Its one catch at the budget is not a window effect. The per-row truncated flag in the prediction files is a corpus-level request flag, set on the same 2,086 rows for every model; it is not this window.

So a longer window alone would not lift deberta-v3's catch at the budget. Look for the cause elsewhere before tuning its input length.

The prediction was that both controls land inside the chance band under length control. control-modernbert-base does, at 0.488. control-modernbert-large lands below it, at 0.389. The two controls' scores correlate at only 0.144, which points at idiosyncratic readout behaviour rather than a label leak both would see.

Even so, the controls are useful floors. As shipped, control-modernbert-base scores default-decision F1 0.250. 12 of the 19 candidates with a default decision score at or below it.

Treat the controls as the floor any candidate must clear. What a control scores is what the corpus gives away for free; a candidate at or below it shows nothing beyond that.

Pairs share 62 caught positives in all, against 15.7 expected if the models caught independently. The overlap is in-sample, like the best pair.

Low overlap is why unions add catches, but each model's catches are few. Combining them buys recall only by spending false blocks well past the budget.

The weights total 85.5 GiB. 14 models are under 3B parameters and 8 are from 3B to 6B. 8 are gated, in the Gemma, Llama 3.2 and Prompt Guard 2 acceptance groups. For granite-guardian-3.2-3b-a800m, a mixture of experts, the count is the total; about 800M are active. Qwen3 0.6B, 1.7B and 4B were dropped for provenance before any run.

1. One budget. Precision, recall, F1 and block FPR are reported at block FPR 0.00385. Each threshold is the one with the highest recall that holds it, ties broken toward the tighter threshold. The slider on the main page re-reads the same curves at other budgets.
2. Scorable cases only. Every F1, AUC, FPR and confusion count is over the 3,817 cases of grade A, B or D. The 460 grade C cases are counted nowhere.
3. Oracle figures are labelled. A best F1 found by sweeping thresholds on the scored rows is an upper bound. It appears only in its own field, never as a result.
4. Accuracy only beside all-allow accuracy, 0.886, because at 11.4% prevalence the benign class sets it.
5. One ranking variable per class structure, as in the methodology. For the 7 two-class models, 2 of them controls, P(confirm) does not exist.
6. Neither gate is durable. When the System One programme moved thresholds to a disjoint corpus, 4 of 24 zero-false-positive gates survived, 3 of them degenerate, and false-positive caps held in 8 of 48.

• The extraction for this Space reproduces all 577 figure values the previous Space published, with 0 mismatches. All 22 leaderboard rows match, and all 20 artifact digests match the files at the pinned commit.
• Each ROC curve is carried as integer (false blocks, true blocks) pairs, 6,515 points across 22 models. The area under each drawn curve equals the published raw AUC exactly, and the budget point lies on the curve.
• The previous build recomputed 2,251 figures from their primitives over 20 input files and 22 charts, aborting on any mismatch.
• A leak guard fails on any corpus case id, sensitive-text shingle, CJK codepoint or credential shape, and on any file type outside .html .css .md .json .txt or over 1 MiB.
• The monorepo build checks that every data file parses, that no page repeats an element id, and each page's word budget.

• 2026-10, this redesign. Five pages became one chaptered page plus this appendix. Every chart reads data/*.json, written from the artifacts at commit c7e264e63e. The two controls are now unranked everywhere. The budget slider re-reads every curve; the 22-panel small multiples became one overlaid ROC.
• Previous revision. Eleven pages became five. The controls' length-controlled AUCs, the DeBERTa figure and the length counter moved from the retracted unweighted estimator to the published one, and the leakage verdict was re-read from it.
• Earlier. The unweighted mean of per-quintile AUCs was retracted in favour of the pair-weighted pooled figure superseded. The claim that Prompt Guard 2 supplied an independent encoder backbone was withdrawn.

The artifacts live in benchmarks/slm_toolcall at commit c7e264e63e of cisco-ai-defense/defenseclaw. Run python3 tools/extract_slm.py --fetch and then python3 tools/build.py slm-toolcall from the monorepo root.

The laptop measurements were ad-hoc invocations never saved as a script, so they are archived rather than reproducible.