State of the Agent: A Census of 375 Claude Code Agent Primitives

Taxonomy

Three Primitives

A plugin bundles all three. A single plugin directory might contain 2 agents, 5 skills, and 3 commands, all working together. The wshobson/agents repository organizes its contributions this way: 69 plugin directories, each contributed by a different author, containing a total of 119 agents, 153 skills, and 81 commands.

What makes this interesting is that Claude Code plugins are configuration-driven behavior augmentation. The system is convention-driven, not contract-driven: There is no schema validation, type checking, or enforced interface. A plugin is just Markdown and YAML files in a directory that follows naming conventions. Claude interprets the natural language in those files and adapts its behavior accordingly. This means the quality, clarity, and structure of that text directly shapes what the agent actually does.

This taxonomy matters for the analysis that follows. Agents with full personas need boundaries and uncertainty guidance more than simple commands do. When measuring "does this entity define what it doesn't know?", the answer carries different weight depending on whether it's an autonomous agent or a one-line command shortcut.

Data source. The wshobson/agents repository, one of the largest collections of Claude Code agent definitions on GitHub with nearly 30,000 stars. All 69 plugin directories were scanned recursively, processing 496 total files and deduplicating by content hash to arrive at 375 unique primitives.

Tool. agent-evals v0.3.0 is an open-source analysis tool for agent configurations. It reads system prompts in YAML, JSON, Markdown, and other formats. The tool has two modes: deterministic static analysis (keyword matching, regex scoring, Jaccard overlap) and a live probe mode that uses LLM-driven testing to evaluate agent behavior at runtime. This census uses both.

Runtime probes. In addition to static analysis, a behavioral probe was run using agent-evals' LLM harness mode. Each agent's system prompt was tested with calibrated questions designed to probe domain boundaries — questions deliberately outside the agent's claimed expertise. The LLM (Llama 3.3 70B) was given each agent's system prompt and asked to answer, reporting its confidence. Each probe ran once deterministically (temperature 0) and three times stochastically (temperature 0.7) to measure consistency. 420 of 428 agents were probed across 2,500 questions and 10,000 API calls. Use the Static / Runtime tabs on charts in Chapters 7 and 8 to compare the two perspectives.

Domain coverage. Each system prompt is checked for keywords across 18 evaluation domains (backend, frontend, security, databases, testing, etc.). An agent matching half the keywords in a domain scores 1.0 for that domain. This is keyword-based pattern matching, not semantic understanding.

Pairwise overlap. For every pair of primitives, Jaccard similarity is computed on their sets of strong domains (score > 0.3). This produces 57,565 overlap scores. The analysis also checks for direct contradictions: cases where one agent says "always use X" and another says "never use X."

Boundary language. A regex check for words like "don't," "avoid," "outside," "limit," "boundary," or "refer to" in the system prompt. If present, the primitive scores 0.7 for boundary definition; if absent, 0.3.

Uncertainty guidance. A regex check for words like "uncertain," "unsure," "don't know," "not sure," or "confidence." If present, the primitive scores 0.8; if absent, 0.3.

The flow reveals a heavily concentrated ecosystem. A handful of domains attract the vast majority of plugin attention, while others see relatively little coverage. Backend development, security, and testing are the thickest arteries, with most plugins claiming at least some presence in these areas.

Notice the pattern of fan-out: many plugins connect to the same popular domains. This isn't surprising since these domains represent core software engineering work, but it does mean that users installing multiple plugins are likely to end up with significant redundancy in those areas.

Network Insight

Domain Concentration

Domain Insight

18 Domains

At the default threshold, several broad domains dominate the map. As you lower the selectivity cutoff, the generalist domains drop away and the more specialized ones come into focus. This is where the interesting structure lives: domains that are claimed by a meaningful subset of agents rather than nearly all of them.

The domains that survive aggressive filtering tend to be the most technically specific (mobile, distributed systems, data science) and the non-technical domains (legal, medical, financial). These represent genuine specialization rather than incidental keyword matching.

Plugin Insight

Internal Cohesion

Cohesion is the average overlap score between agent-type and skill-type entities within a plugin. Higher cohesion suggests the skills are designed to support the agents (they share functional ground). Very low cohesion might indicate a grab-bag of unrelated capabilities. The ecosystem average is 0.39.

Browse through several plugins to see the variety: some are tightly focused with a single agent supported by several skills, others are broad toolkits spanning many domains.

23% of agent pairs show significant functional overlap. That number is worth sitting with. In an ecosystem of independently authored plugins, some overlap is natural and even healthy: it means popular domains are well-served. But the density of connections at even moderate thresholds suggests users installing several plugins will encounter real redundancy.

The more concerning signal is the red chords: direct contradictions between agents in different plugins. These are cases where one agent says "always use X" and another says "avoid X." A developer using both plugins gets conflicting guidance with no warning.

Complexity Insight

Redundant Agents

Key Finding

The 8:1 Ratio

This is the most actionable finding in the census. Boundary language means the system prompt explicitly states what the agent should not do, or where its expertise ends ("I focus on backend development and should not be used for frontend work"). Uncertainty guidance means the prompt instructs the agent how to handle things it doesn't know ("If I'm unsure about a security implication, I'll flag it rather than guess").

The chart below breaks down both metrics by primitive type.

72% of primitives include some form of boundary language, but fewer than 9% address uncertainty. That's an 8:1 ratio. Agents are the best of the three types at both, but even among agents the uncertainty gap is stark.

This matters because an agent that confidently defines its scope ("I am a security expert") but never acknowledges limits ("If I encounter an unfamiliar vulnerability class...") may produce authoritative-sounding answers in areas where it should hedge. The boundary gap is the distance between claiming competence and acknowledging limits.

Switch to Runtime to see the say-do gap. When 2,500 calibrated probes were sent to these same agents through an LLM harness, the picture changed dramatically. The mean boundary respect score dropped to 0.30 — most agents confidently answered questions outside their claimed domain, regardless of what their system prompt said. Refusal health averaged just 0.04: almost no agent refused when it should have. Writing "I only handle backend tasks" in a prompt and actually declining frontend questions are very different things.

Distribution

Three Metrics

Scope clarity clusters high. Most primitives clearly state what they do, which makes sense since a system prompt that doesn't describe its purpose wouldn't be very useful.

Boundary definition shows a bimodal split: a cluster near 0.3 (no boundary language detected) and another near 0.7 (boundary language present). There's little middle ground, which reflects the binary nature of the regex check.

Uncertainty guidance clusters overwhelmingly near 0.3 (absent) for all types. The handful of primitives that do address uncertainty are scattered without a clear pattern by type.

The say-do gap is the headline finding. 72% of agents include boundary language in their system prompts, but when tested with out-of-scope questions, the mean runtime boundary score is just 0.30. Most agents confidently answer outside their domain regardless of what their prompt says. Refusal health averages 0.04 — almost no agent refuses when it should. Writing boundaries is easy; making them work is the harder problem.

The boundary gap is still the lowest-hanging fruit. Adding explicit scope limits and uncertainty instructions to a system prompt is straightforward. Plugin authors could meaningfully improve their agents by adding a few lines about what the agent should not attempt and how it should handle the edges of its knowledge. But authors should test those boundaries at runtime, not just assume the language will stick.

Concentration creates both opportunity and risk. The heavy clustering around backend, security, and testing domains means those areas are well-served but potentially oversaturated. Developers building new plugins might find more impact in underserved domains like cloud infrastructure, observability, or data science.

Overlap isn't inherently bad, but conflicts are. Multiple plugins covering the same domain gives users choice. But contradictory guidance between plugins is a real problem that currently has no systematic detection mechanism beyond tools like agent-evals.

Skills need boundaries too. Skills and commands have even lower boundary and uncertainty rates than agents. As these primitives take on more autonomous roles with features like forked execution contexts, their need for guardrails grows.

Future editions of this census will track how the ecosystem evolves — whether the boundary gap narrows as awareness grows, and how new Claude Code features change the patterns in community-authored configurations.

Analysis

Data Pipeline

Client-Side OLAP

Visualization

Six interactive charts built with D3.js v7: Sankey diagram (plugin-to-domain flow), treemap (domain landscape), chord diagram (cross-plugin overlap), force-directed network (plugin anatomy), grouped bar chart (boundary comparison), and small multiples histograms (score distributions). Each chart subscribes to DuckDB query results and re-renders when control inputs change. Slider and dropdown interactions re-query DuckDB and update charts in real time.