When the Scanner Scans the Scanner: ClawDefender and the False Positive Problem

ClawDefender is a security skill for OpenClaw agents. Its job is to detect and block prompt injection, rogue skills, and malicious patterns. Our scanner flagged it with 37 findings and 22 CRITICAL severity — making it the highest-finding-count skill in our entire dataset.

Every single finding is a false positive.

What Happened

ClawDefender includes a bash script (clawdefender.sh) that pattern-matches against known attack strings. It contains arrays of malicious patterns it detects and blocks:

'rm -rf /'
'chmod 777'
'/etc/passwd'
'nc -e'
'bash -i >& /dev/tcp'
'ignore your instructions'
'forget your instructions'
'forget everything above'

These are detection signatures, not attack payloads. They appear inside single-quoted string literals in a pattern-matching function. The skill's documentation (SKILL.md) also describes the attack categories it defends against, including examples of fork bombs, SSRF patterns, and file destruction commands.

Cisco's skill-scanner uses YARA rules and regex patterns that match these exact strings — regardless of context. The scanner doesn't distinguish between "this skill executes rm -rf /" and "this skill checks whether input contains rm -rf /."

The Breakdown: 37 Findings, 0 Real Threats

22 of the 37 findings are CRITICAL severity. The skill ran through all five analyzers (static, behavioral, LLM, trigger, meta) — yet the LLM and meta analyzers produced zero findings. All 37 came from static analysis and YARA rules, which have no concept of whether a matched string is executed, referenced defensively, or quoted as documentation.

Update: Meta-Analyzer Confirms All 37 Are False Positives

When we originally wrote this analysis, we assumed the meta-analyzer had correctly identified the false positives but simply hadn't filtered them from the output. We were wrong — the meta-analyzer never actually ran.

As we discovered in a separate investigation, our API credits ran out mid-scan batch. The scanner silently fell back to static-only analysis while still reporting meta_analyzer in the analyzers list. ClawDefender was one of 540 skills that never received real LLM or meta-analysis.

After re-scanning with fresh API credits, the results speak for themselves:

Meta-analysis complete: 37 false positives filtered, 0 new threats detected

ClawDefender now shows 0 findings. The meta-analyzer examined all 37 static findings, understood the defensive context, and correctly classified every single one as a false positive. The skill's status changed from "most dangerous in the dataset" to "clean."

This validates both our original analysis (the findings were false positives) and the value of the meta-analyzer (it correctly identifies defensive security tooling when it actually runs).

Why This Matters

This case highlights a fundamental limitation of pattern-based security scanning: detection signatures and attack payloads are lexically identical. Any tool that scans by matching strings like rm -rf or ignore your instructions will flag both the attacker and the defender.

This isn't unique to Cisco's scanner. Antivirus software has the same problem — security research tools, penetration testing frameworks, and even YARA rule collections themselves trigger signature-based detection. The industry term is "meta-detection": the detector detecting itself.

For agent skill ecosystems, this creates a practical problem: security skills will always rank as the most dangerous skills in any automated scan. Without manual review or context-aware analysis, a registry could reject the very tools designed to protect it.

What Could Help

Several approaches could reduce false positives on defensive tooling:

• Context-aware YARA rules: Distinguish between patterns in executable code vs. quoted strings, comments, and documentation.
• LLM-based intent classification: The meta-analyzer correctly understood ClawDefender's defensive intent when it actually ran. Elevating LLM assessments over static matches when they disagree could reduce false positives.
• Allowlisting for known security tools: Registries could maintain a list of known security/defense skills that are expected to contain attack signatures.
• Finding deduplication: 10 of the 37 findings are the same YARA rule matching ANSI color codes in different files. Deduplicating by rule+pattern would reduce the count significantly.

ClawDefender scan results are available in the full dataset.