These are SkillScan's own numbers, published as measured — including the families where the rates are poor. They come from the project's README; the full methodology, counting rules and caveats are in the repo's benchmark document.
The corpus is MaliciousSkillBench — 9,740 labelled
skills: 7,505 malicious, 2,235 benign. First run 2026-11-11 with
SkillScan v0.6.0; re-run 2026-11-19 with the current
tree (76 deterministic rules) — the figures below are that run, made
with the deterministic offline mode only (no --llm).
Malicious skills were scanned as complete extracted folders; benign
skills are shipped by the benchmark as frozen text, so each was scanned
as a single SKILL.md folder. Nothing was executed — folders
were read as text only.
| Set | Skills | BLOCK | CAUTION | PASS |
|---|---|---|---|---|
| Malicious | 7,421 | 54.16% (4,019) | 28.3% (2,097) | 17.6% (1,305) |
| Benign | 2,235 | 0% false BLOCKs (0) | 24.7% (551) | 75.3% (1,684) |
Detection at BLOCK+CAUTION (flagged at any severity): 82.41% (6,116 of 7,421) of malicious skills; 24.7% of benign skills were flagged at any severity, all as CAUTION. Median scan time was ~106 ms per malicious skill (p95 ~600 ms) and ~44 ms per benign skill.
Multi-label membership; BLOCK / BLOCK+CAUTION, from the 2026-11-19 run. The weak families are left in:
| Family | Skills | BLOCK | BLOCK+CAUTION |
|---|---|---|---|
| Privilege / tool-authority abuse | 1,004 | 88.5% | 99.4% |
| Execution / code delivery | 3,205 | 83.1% | 97.9% |
| Credential access | 197 | 55.3% | 95.9% |
| Persistence / control | 249 | 68.3% | 90.0% |
| Instruction / goal manipulation | 1,560 | 72.1% | 86.2% |
| Data exfiltration / disclosure | 266 | 35.3% | 83.8% |
| Resource / availability abuse | 330 | 46.4% | 90.3% |
| Dependency / supply chain | 167 | 77.2% | 87.4% |
| Defense evasion / obfuscation | 20 | 10.0% | 100% |
| Integrity / output manipulation | 97 | 2.1% | 28.9% |
| Discovery / reconnaissance | 5 | 20.0% | 20.0% |
| (No category assigned) | 2,591 | 30.3% | 67.5% |
Twelve new deterministic BLOCK rules (the gap wave). Every shape was priced against the malicious and benign corpora before it was built — zero trips on either benign corpus was the gate. 330 malicious skills moved from CAUTION to BLOCK, every one attributed to a new rule; BLOCK+CAUTION did not move (82.41%), because every gain upgraded a skill that was already flagged. The dependency / supply-chain family's BLOCK rate rose from 14.4% to 77.2%. Benign verdicts are identical to the previous run (0 false BLOCKs; 551 CAUTION / 1,684 PASS).
Priced and not built: unpinned named-registry installs
(the shape trips 380 benign corpus skills — ordinary setup — so they
stay CAUTION), unpinned VCS dependencies, plain-http named-host
installs, custom package indexes and download-then-execute chains.
One built rule was removed after adversarial review: dd
disk-fill is syntactically identical in its honest uses (disk images,
swap files, LUKS keyfiles), so no deterministic separator exists.
One change moved the numbers: the INJECTION_EXECUTION
promotion. 297 malicious skills moved from CAUTION to BLOCK — every one
a completed staging chain (a file written to a fixed temp path and tied
to execution in the same file). That took BLOCK from 45.8% to 49.8% at
zero measured false-BLOCK cost on either benign corpus.
The same day, a fix wave tightened three over-broad rules to require the evidence their names claim. The trade was 34 explained BLOCK losses against 27 gains — net BLOCK 3,696 → 3,689 (49.7%), and benign CAUTIONs 557 → 551. Correctness over coverage: a headline rate bought with other people's false alarms is not a rate worth printing.
Eight CAUTION combinations were measured as candidate promotions to BLOCK. Only one shipped. The other seven were not promoted — each has a legitimate twin the scanner cannot tell apart at line level. Two examples, priced and closed:
Cross-file dataflow was scoped the same way: a census of the 239 code-bearing malicious skills that PASS found the chain halves absent, not unjoined — honest yield 0 — so the engine is parked, not planned.
SKILL.md with no code at all. Among pure instruction-manipulation skills the BLOCK rate is 8.8%, and pure output manipulation is ~2% — deterministic rules see behaviour shapes, not meaning.The zero-false-BLOCK figure above is exact for the corpus it names. Because that corpus ships benign skills as prose only, a second, code-bearing benign set was built: 392 hand-written benign skills with real code. There the scanner stands at 49 BLOCK / 286 CAUTION / 57 PASS. Every remaining BLOCK is a stated stance, not a defect — any persistence install blocks, disclosed or not (22), and uploads to the user's own cloud still match the exfiltration shape (16). Three defect groups the set exposed (an SSH permission fixer, plain curl downloads, a skill's own temp-folder cleanup) were fixed in the 2026-11-15 fix wave.
The deterministic figures above do not change with --llm;
the README's Modes section reports the live-model measurements
separately. On the benchmark's output-manipulation family, a
current-model judge lifts BLOCK+CAUTION from 28.9% to 54.6% with zero
false BLOCKs on 280 benign skills. Weak free-tier judges lift the
held-out malicious set materially (Gemini Flash-Lite 37.0% → 53.4%; the
default Qwen 34.1% → 43.2%). Gemini Flash-Lite also reviewed 258 of 259
held-out benign skills with 0 false BLOCKs and 0 added CAUTIONs. The
default Qwen's benign-set run was stopped by its free-tier limit after
2 calls, so that preset's benign rate at corpus scale remains
unmeasured — the README labels it that way, and so does this page.