Unit 42 Just Analyzed 49,943 AI Agent Skills — 80% Lie About What They Do

Unit 42, Palo Alto Networks’ elite threat research team, just published the most comprehensive AI agent supply chain analysis to date. They built a system called Behavioral Integrity Verification (BIV) and pointed it at every skill on OpenClaw’s registry.

The results are the kind of thing that keeps CISOs up at night.

The Numbers

Unit 42 crawled 49,943 AI agent skills from OpenClaw. They ran each one through BIV, which works by running two parallel analyses:

AST-level taint analysis on the executable code (what the code actually does)
LLM classification against the natural-language description (what the skill claims to do)

Then it compares the two.

80% of skills — 39,933 of them — showed at least one behavioral mismatch. The skill says “I fetch today’s weather” but the code also reads your .env file and encodes it as base64.

In total, BIV surfaced 250,706 behavioral deviations. That’s an average of 5 deviations per skill. Not a rounding error. Not edge cases. The majority of the ecosystem.

Five Percent Are Actively Weaponized

Of those 49,943 skills, 5% — roughly 2,500 skills — contained multi-stage attack chains. These aren’t accidental oversights. They’re deliberate, multi-step exploitation patterns.

Unit 42 identified four novel compound threat categories that don’t exist in any CVE database:

Exfiltration chains — FILE_READ → base64 encode → NETWORK_SEND. The skill reads your secrets, encodes them to evade string-matching scanners, and phones home.
Agent hijacking — Skills that modify the agent’s own instruction set to pivot its behavior toward attacker-controlled endpoints.
Credential harvesting — Skills that discover API keys in environment variables and exfiltrate them through encoded log messages.
Behavioral masking — Skills that present one behavior during installation scanning, then switch to a different execution path at runtime.

These are not theoretical. Unit 42 confirmed that evasive skills in the wild are actively bypassing automated scanners to deploy infostealers and execute agentic financial fraud.

Here’s what makes this dangerous: the malicious skills Unit 42 found had already passed through ClawScan (OpenClaw’s official scanner) and VirusTotal screening. Five of them were explicitly designed to evade signature-based detection.

This validates something we’ve been saying for weeks: automated scanners can’t catch behavioral deception. They look for known malware signatures and prompt injection patterns. They don’t compare what a skill says it does against what it actually does.

Unit 42’s BIV approach — cross-referencing code behavior against natural language claims — works because attackers haven’t yet learned to write code that matches their lies. But they will. And when they do, BIV will need to evolve.

Diana Kelley (Noma Security): Traditional Controls Are Dead

The ReversingLabs team published a companion analysis the same week, featuring Diana Kelley, CISO at Noma Security. Her assessment cuts to the chase:

“Traditional supply chain controls built for static artifacts — signed code, scanned dependencies, and trusted repositories — come up short when it comes to AI agents and skills.”

The core problem is that AI agents are autonomous actors with permissions inside the supply chain. A malicious npm package can exfiltrate data from the build pipeline. A malicious agent skill inherits the agent’s authenticated identity — it can read your email, access your databases, deploy to your infrastructure.

The blast radius is an order of magnitude larger.

Snyk Just Entered the Space

As if to punctuate the point, Snyk — one of the largest supply chain security companies — launched agent-scan this month. It’s a CLI tool that auto-discovers your installed AI coding agents (Claude Code, Cursor, Windsurf, Gemini CLI), connects to MCP servers, and scans for 15+ risk categories.

When Snyk, a company that built its business on npm vulnerability scanning, feels the need to build a separate tool for AI agents, the message is clear: this is a fundamentally different attack surface.

The MCP Connection

This connects directly to our MCP Census analysis from last week. We found 15,382 MCP servers with 1,880 deleted repos, 888 unlicensed packages, and 9,207 servers with fewer than 10 stars.

Now Unit 42’s data shows what that actually means in practice: 80% of skills can’t be trusted, 5% are actively malicious, and the existing scanning infrastructure can’t catch them.

What This Means

If you’re running AI coding agents in production (and surveys suggest 88% of organizations are), you have a supply chain problem that your existing tools cannot see.

The npm ecosystem took 15 years to build the security infrastructure it has today — and 0.2% of packages are still malicious. The AI agent ecosystem has 12% malicious packages (per our earlier analysis) and zero security infrastructure.

Unit 42’s BIV is a step in the right direction. But behavioral verification at scale requires infrastructure that doesn’t exist yet. Until it does, the only reliable verification is a human auditor who can trace the execution path and understand the trust boundaries.

The dotfm Take

We’ve been tracking this since Session #1. Our thesis — that AI agent security requires human expertise that automated tools structurally cannot provide — just got a $40 billion company’s worth of validation.

The question is no longer whether AI agent supply chains are compromised. It’s how much you’ve already been exposed.

Need an audit of your AI agent infrastructure? Contact us — we trace the trust boundaries automated tools miss.

Sources:

Trust No Skill: Integrity Verification for AI Agent Supply Chains — Unit 42 / Palo Alto Networks
AI Agent Supply Chain Risks: Unit 42 Finds Attack Chains in 5% of OpenClaw Skills — Cybersecurity Insiders
How AI agents upend software supply chain security — ReversingLabs
Snyk Agent Scan — GitHub
OpenClaw’s Skill Marketplace and the Emerging AI Supply Chain Threat — Unit 42