Static-Only vs LLM-Enhanced Scanning

Not all 22 skills ran through the same analysis pipeline. 15 skills were scanned with static + behavioral + trigger analyzers only, while 7 skills also received LLM analysis (Claude Sonnet 4.5) plus meta-validation. Comparing the two groups reveals distinct detection profiles.

Static Analysis: Pattern Matching at Scale

Static analysis excels at catching concrete, deterministic patterns: hardcoded credentials (YARA rules), suspicious URLs, environment variable harvesting, missing manifest fields, and known dangerous code patterns like while True: loops or unsanitized user input in shell commands.

The two highest-finding skills -- clickup (14 findings) and agent-browser (8 findings) -- were scanned with static analysis only. Static analysis caught everything from YARA-detected credential harvesting to behavioral cross-file exfiltration patterns. These are the kinds of findings that don't require understanding intent -- the patterns are either present or they aren't.

Static-only scans completed in under 1 second (clickup: 0.8s, agent-browser: 18ms). The speed makes them suitable for CI/CD integration.

LLM Analysis: Understanding Context

LLM-enhanced scans took 23-31 seconds per skill (average: ~26s) -- roughly 30x slower than static analysis. But they detect a fundamentally different class of issues.

The LLM analyzer found credential management anti-patterns that static analysis missed: skills instructing users to store API keys in ~/.zshenv without security guidance, incomplete trust model disclosures, and third-party data transmission risks. These are architectural and documentation-level issues, not code-level patterns.

For example, the subtitles skill was flagged for instructing users to export TRANSCRIPT_API_KEY in their shell config -- a common but risky pattern that could expose the key if the user commits their dotfiles to a public repo. Static analysis can't catch this because the instruction appears in natural language, not code.

The Overlap Gap

Interestingly, the two most dangerous skills (clickup and agent-browser) did not receive LLM analysis in this scan run. This means the current results may undercount the full risk profile of those skills -- LLM analysis might surface additional contextual findings on top of the static detections. A follow-up scan with full analyzer coverage on all 22 skills would be valuable.

Cost-Benefit

For a CI/CD gate or quick triage, static-only scanning covers the high-signal patterns in under a second. For a thorough security review -- especially of skills that handle credentials or make API calls -- the 25-second investment in LLM analysis pays off by catching design-level issues that no regex can find.