hugging-face-evaluation

SAFE max severity MEDIUMSkillSpector Do not install 100 registry huggingface last scanned 1 scan

Verdict safe, but 96 findings (92 medium, 4 low), mainly transitive trust abuse, data exfiltration, social engineering. Most recent scan 2026-02-07.

The hugging-face-evaluation skill is a legitimate tool for managing model evaluation results with MEDIUM security risk. The skill provides useful functionality for extracting evaluation tables and importing benchmark scores, but has several security concerns: (1) API key handling in example scripts could expose credentials, (2) subprocess command construction lacks input sanitization creating command injection risks, (3) markdown parsing lacks resource limits for DoS protection, (4) missing manifest metadata reduces transparency. The skill does NOT contain malicious code, data exfiltration, or prompt injection attempts. The primary risks are defensive coding issues rather than intentional threats. The scripts appropriately use HuggingFace's official APIs and follow standard patterns for evaluation workflows.

Scanner's overall assessment (LLM analyzer)
Findings96
Critical0
High0
Medium92
Low4
Analyzersstatic llm meta

Second scanner: NVIDIA SkillSpector

VerdictDo not install 100
Score100 / 100
Issues35
CategoriesSupply Chain Privilege Escalation Dangerous Code Execution
Version2.12.0 (static)

The two scanners disagree: Cisco found nothing and SkillSpector says do not install. One of them has a blind spot here; read the skill before trusting either. How the two scanners compare across the whole corpus: Two scanners, one corpus.

What this skill does

Add and manage evaluation results in Hugging Face model cards. Supports extracting eval tables from README content, importing scores from Artificial Analysis API, and running custom model evaluations with vLLM/lighteval. Works with the model-index metadata format.

Package
14 files, 126 KB
SKILL.md
2872 words, 12+ sections
Code
Python (8)

SKILL.md covers: Overview · Integration with HF Ecosystem · Version · Dependencies · Core Dependencies · Inference Provider Evaluation · vLLM Custom Model Evaluation (GPU required) · IMPORTANT: Using This Skill · ⚠️ CRITICAL: Check for Existing PRs Before Creating New Ones · Core Capabilities · 1. Inspect and Extract Evaluation Tables from README · 2. Import from Artificial Analysis

Findings

MEDIUM HTTP client library imports that enable external communication Data Exfiltrationstatic

Pattern detected: import requests

examples/artificial_analysis_to_hub.py:36

import requests
Remediation: Ensure network access is necessary and documented. Review all URLs
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("AA_API_KEY

examples/artificial_analysis_to_hub.py:42

API_KEY = os.getenv("AA_API_KEY")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

examples/artificial_analysis_to_hub.py:43

HF_TOKEN = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM HTTP client library imports that enable external communication Data Exfiltrationstatic

Pattern detected: import requests

scripts/evaluation_manager.py:62

import requests  # type: ignore
Remediation: Ensure network access is necessary and documented. Review all URLs
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/evaluation_manager.py:515

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/evaluation_manager.py:725

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("AA_API_KEY

scripts/evaluation_manager.py:865

AA_API_KEY = os.getenv("AA_API_KEY")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/evaluation_manager.py:994

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/evaluation_manager.py:1057

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/evaluation_manager.py:1105

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/run_eval_job.py:42

token = hf_token or os.getenv("HF_TOKEN") or get_token()
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/inspect_vllm_uv.py:43

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/inspect_eval_uv.py:61

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/lighteval_vllm_uv.py:39

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/run_vllm_eval_job.py:91

token = hf_token or os.getenv("HF_TOKEN") or get_token()
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/run_vllm_eval_job.py:147

token = hf_token or os.getenv("HF_TOKEN") or get_token()
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Undeclared network usage Unauthorized Tool Usestatic

Skill code uses network libraries but doesn't declare network requirement

Remediation: Declare network usage in compatibility field or remove network calls
MEDIUM Potential description-behavior mismatch Social Engineeringstatic

Skill performs actions not reflected in its description

SKILL.md

Remediation: Ensure description accurately reflects all skill capabilities
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/lighteval_vllm_uv.py:39

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/inspect_eval_uv.py:61

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/run_vllm_eval_job.py:91

token = hf_token or os.getenv("HF_TOKEN") or get_token()
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/run_vllm_eval_job.py:147

token = hf_token or os.getenv("HF_TOKEN") or get_token()
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/inspect_vllm_uv.py:43

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/run_eval_job.py:42

token = hf_token or os.getenv("HF_TOKEN") or get_token()
Remediation: Minimize access to environment variables. Document why needed
MEDIUM HTTP client library imports that enable external communication Data Exfiltrationstatic

Pattern detected: import requests

scripts/evaluation_manager.py:62

import requests  # type: ignore
Remediation: Ensure network access is necessary and documented. Review all URLs
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/evaluation_manager.py:515

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/evaluation_manager.py:725

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("AA_API_KEY

scripts/evaluation_manager.py:865

AA_API_KEY = os.getenv("AA_API_KEY")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/evaluation_manager.py:994

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/evaluation_manager.py:1057

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM Reading environment variables that may contain secrets Data Exfiltrationstatic

Pattern detected: os.getenv("HF_TOKEN

scripts/evaluation_manager.py:1105

hf_token = os.getenv("HF_TOKEN")
Remediation: Minimize access to environment variables. Document why needed
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:50

uv run scripts/evaluation_manager.py get-prs --repo-id "username/model-name"
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:71

uv run scripts/evaluation_manager.py --help
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:72

uv run scripts/evaluation_manager.py inspect-tables --help
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:73

uv run scripts/evaluation_manager.py extract-readme --help
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:115

**Benefits:** No need to use `hf_jobs()` MCP tool, can run scripts directly in terminal
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:127

uv run scripts/train_sft_example.py
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:152

uv run scripts/evaluation_manager.py inspect-tables --repo-id "username/model"
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:155

uv run scripts/evaluation_manager.py extract-readme \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:162

uv run scripts/evaluation_manager.py extract-readme \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:167

uv run scripts/evaluation_manager.py extract-readme \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:184

AA_API_KEY="your-api-key" uv run scripts/evaluation_manager.py import-aa \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:197

uv run scripts/evaluation_manager.py import-aa \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:205

uv run scripts/evaluation_manager.py import-aa \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:238

uv run scripts/run_eval_job.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:265

uv run scripts/lighteval_vllm_uv.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:270

uv run scripts/lighteval_vllm_uv.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:275

uv run scripts/lighteval_vllm_uv.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:281

uv run scripts/lighteval_vllm_uv.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:289

hf jobs uv run scripts/lighteval_vllm_uv.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:327

uv run scripts/inspect_vllm_uv.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:332

uv run scripts/inspect_vllm_uv.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:338

uv run scripts/inspect_vllm_uv.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:346

hf jobs uv run scripts/inspect_vllm_uv.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:368

uv run scripts/run_vllm_eval_job.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:374

uv run scripts/run_vllm_eval_job.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:382

uv run scripts/run_vllm_eval_job.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:401

uv run scripts/evaluation_manager.py --help
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:402

uv run scripts/evaluation_manager.py --version
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:407

uv run scripts/evaluation_manager.py inspect-tables --repo-id "username/model-name"
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:412

uv run scripts/evaluation_manager.py extract-readme \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:424

AA_API_KEY=... uv run scripts/evaluation_manager.py import-aa \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:433

uv run scripts/evaluation_manager.py show --repo-id "username/model-name"
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:434

uv run scripts/evaluation_manager.py validate --repo-id "username/model-name"
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:439

uv run scripts/evaluation_manager.py get-prs --repo-id "username/model-name"
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:445

hf jobs uv run scripts/inspect_eval_uv.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:455

uv run scripts/run_eval_job.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:464

hf jobs uv run scripts/lighteval_vllm_uv.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:471

hf jobs uv run scripts/inspect_vllm_uv.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:478

uv run scripts/run_vllm_eval_job.py \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:557

uv run scripts/evaluation_manager.py extract-readme \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:565

uv run scripts/evaluation_manager.py get-prs \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:569

uv run scripts/evaluation_manager.py extract-readme \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:582

uv run scripts/evaluation_manager.py get-prs \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

SKILL.md:586

AA_API_KEY=... uv run scripts/evaluation_manager.py import-aa \
Remediation: Review and remove transitive trust abuse pattern
MEDIUM CODE EXECUTION detected by YARA Command Injectionstatic

Detects dangerous code execution patterns in agent skills (Python/Bash): subprocess.run(

SKILL.md:636

"python", "scripts/evaluation_manager.py",
Remediation: Review and remove code execution pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:556

print(f'  uv run scripts/evaluation_manager.py inspect-tables --repo-id "{repo_id}"')
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:775

print(f'  uv run scripts/evaluation_manager.py extract-readme --repo-id "{repo_id}" --table <table-number> [--model-column-index <column-index>]')
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:1157

"Use standard Python or `uv run scripts/evaluation_manager.py ...` "
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:1165

uv run scripts/evaluation_manager.py inspect-tables --repo-id <model>
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:1167

uv run scripts/evaluation_manager.py extract-readme --repo-id <model> --table N
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:1169

uv run scripts/evaluation_manager.py extract-readme --repo-id <model> --table N --apply
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:1171

AA_API_KEY=... uv run scripts/evaluation_manager.py import-aa --creator-slug org --model-name slug --repo-id <model>
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:1194

uv run scripts/evaluation_manager.py extract-readme --repo-id username/model
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:1195

uv run scripts/evaluation_manager.py extract-readme --repo-id username/model --table 2 --model-column-index 3
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:1196

uv run scripts/evaluation_manager.py extract-readme --repo-id username/model --table 2 --model-name-override \"**Model 7B**\"  # exact header text
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:1197

uv run scripts/evaluation_manager.py extract-readme --repo-id username/model --table 2 --create-pr
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:1228

AA_API_KEY=... uv run scripts/evaluation_manager.py import-aa --creator-slug anthropic --model-name claude-sonnet-4 --repo-id username/model
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:1229

uv run scripts/evaluation_manager.py import-aa --creator-slug openai --model-name gpt-4o --repo-id username/model --create-pr
Remediation: Review and remove transitive trust abuse pattern
MEDIUM TRANSITIVE TRUST ABUSE detected by YARA Transitive Trust Abusestatic

Detects skills that delegate trust to untrusted external content: run scripts

scripts/evaluation_manager.py:1284

uv run scripts/evaluation_manager.py get-prs --repo-id "allenai/Olmo-3-32B-Think"
Remediation: Review and remove transitive trust abuse pattern
MEDIUM Hardcoded API Key Exposure Risk in Example Script Data Exfiltrationllm

The example script artificial_analysis_to_hub.py expects API keys (AA_API_KEY and HF_TOKEN) to be provided via environment variables, but the error handling raises exceptions that could expose the presence/absence of these credentials. While the script itself doesn't contain hardcoded secrets, the pattern of loading sensitive credentials without proper validation could lead to credential exposure in logs or error messages.

examples/artificial_analysis_to_hub.py

API_KEY = os.getenv("AA_API_KEY")
HF_TOKEN = os.getenv("HF_TOKEN")
...
if not API_KEY:
    raise ValueError("AA_API_KEY is not set")
if not HF_TOKEN:
    raise ValueError("HF_TOKEN is not set")
Remediation: Implement secure credential handling: (1) Use proper secret management instead of environment variables, (2) Avoid raising exceptions that reveal credential status, (3) Add warnings about not committing .env files, (4) Consider using HuggingFace's built-in token management exclusively.
MEDIUM Subprocess Command Injection Risk in Job Submission Scripts Command Injectionllm

Multiple scripts (run_eval_job.py, run_vllm_eval_job.py) construct subprocess commands using user-provided input (model_id, task names) without proper sanitization. While the scripts use list-based subprocess.run() which provides some protection, the model_id and task parameters are directly interpolated into command arguments. A malicious model_id like 'model; rm -rf /' could potentially be exploited if passed through shell interpretation layers.

scripts/run_eval_job.py

cmd = [
    "hf",
    "jobs",
    "uv",
    "run",
    str(SCRIPT_PATH),
    "--flavor",
    hardware,
    "--secrets",
    f"HF_TOKEN={token}",
    "--",
    "--model",
    model_id,  # User input directly in command
    "--task",
    task,  # User input directly in command
]
Remediation: Add input validation: (1) Validate model_id format against HuggingFace naming conventions (alphanumeric, hyphens, slashes only), (2) Whitelist allowed task names, (3) Sanitize all user inputs before subprocess calls, (4) Add explicit checks for shell metacharacters.
LOW Skill does not specify a license Policy Violationstatic

Skill manifest does not include a 'license' field. Specifying a license helps users understand usage terms.

SKILL.md

Remediation: Add 'license' field to SKILL.md frontmatter (e.g., MIT, Apache-2.0)
LOW Potential Resource Exhaustion in Table Parsing Resource Abusellm

The evaluation_manager.py script parses markdown tables from README files without size limits. A malicious README with extremely large tables or deeply nested markdown structures could cause excessive memory consumption or processing time during the parsing phase with markdown-it-py.

scripts/evaluation_manager.py

def extract_tables_from_markdown(markdown_text: str) -> List[Dict[str, Any]]:
    MarkdownIt = require_markdown_it()
    md = MarkdownIt()
    tokens = md.parse(markdown_text)  # No size limit on input
Remediation: Implement resource limits: (1) Add maximum README size check before parsing (e.g., 1MB limit), (2) Set timeout for parsing operations, (3) Limit maximum table size/rows, (4) Add early termination for excessive nesting depth.
LOW Missing allowed-tools Declaration Data Exfiltrationllm

The skill manifest does not specify the 'allowed-tools' field, which is optional but recommended for transparency. The skill executes Python and Bash scripts that perform file operations, network requests, and subprocess calls, but doesn't declare these capabilities upfront.

SKILL.md

YAML Manifest Details:
- allowed-tools: Not specified

But scripts use: Python execution, Bash commands, file I/O, network requests (Artificial Analysis API, HuggingFace Hub API)
Remediation: Add allowed-tools declaration to manifest: 'allowed-tools: [Python, Bash, Read, Write]' to clearly communicate the skill's capabilities to users and enable proper permission validation.
LOW Incomplete Metadata in Manifest Social Engineeringllm

The skill manifest is missing optional but recommended fields (license, compatibility) that help users understand the skill's terms of use and operational requirements. This reduces transparency about the skill's constraints and legal terms.

SKILL.md

- license: Not specified
- compatibility: Not specified
Remediation: Add complete metadata: (1) Specify license (e.g., 'license: MIT'), (2) Document compatibility requirements (e.g., 'compatibility: Requires GPU for vLLM evaluation, API keys for Artificial Analysis'), (3) Add version field for tracking.

What is in the package

FileKindSize
SKILL.mddoc22 KB
requirements.txtdoc465 B
examples/.env.exampleother311 B
examples/USAGE_EXAMPLES.mddoc9.3 KB
examples/artificial_analysis_to_hub.pyscript3.9 KB
examples/example_readme_tables.mddoc3.0 KB
examples/metric_mapping.jsonconfig924 B
scripts/evaluation_manager.pyscript48 KB
scripts/inspect_eval_uv.pyscript2.9 KB
scripts/inspect_vllm_uv.pyscript9.2 KB
scripts/lighteval_vllm_uv.pyscript9.2 KB
scripts/run_eval_job.pyscript2.4 KB
scripts/run_vllm_eval_job.pyscript9.3 KB
scripts/test_extraction.pyscript5.5 KB

External hosts referenced

artificialanalysis.ai huggingface.co github.com source-url.com

Hostnames that appear in the skill's files. Referencing a host is not a finding; compare against the findings above and the skill's stated purpose.

Environment variables referenced

HF_TOKEN AA_API_KEY

Package contents recorded 2026-10-22 at commit 2afeee6dc727. The scan above ran against the version current at scan time; the two can differ if the skill has since been updated.

What we checked

What we did not check

Sources

Related skills