ClaudeMap

Which Tools Do Coding Agents Install? (Armature)

16,893 sessions of experiment data on how Claude Code, Codex, and Cursor actually choose third-party tools.

Tutorials & Guidesanalysisagentstool-choiceexperiment

Armature's experiment on tool selection by coding agents: 16,893 sessions, 1,163 prompt variants, 75 repos, 10 languages (first public wave: 5,292 valid sessions). Headline findings: the three agents agreed on the same tool in only 42% of scenarios; Claude Code leans on prior knowledge (~30% web search) while Codex searched 94% of the time; "being mentioned is not being chosen" — PayPal was cited 139 times and won zero selections (Stripe won 124); winners differ by language for the same need (Resend-TS, SendGrid-Python, Postmark-Go). The author discloses Armature sells dev-tools growth services — read with that in mind.

Charted

Related on the map

  • Anthropic's canonical essay on agent design — workflows vs agents, and five composable patterns.

    • official
    • agents
    • engineering
    • patterns
  • HumanLayer's 12 design principles for production-grade LLM agents — the closest thing to a 'best practices' manifesto.

    • agents
    • best-practices
    • production
    • principles