MERLIN STEIN
11 min
Abstract
Today's AI agents are built on large language models (LLMs) equipped with tools to access and modify external environments, such as corporate file systems, API-accessible platforms and websites. AI agents offer the promise of automating computer-based tasks across the economy. However, developers, researchers and governments lack an understanding of how AI agents are currently being used, and for what kinds of (consequential) tasks. To address this gap, we evaluated 177,436 agent tools created from 11/2024 to 02/2026 by monitoring public Model Context Protocol (MCP) server repositories, the current predominant standard for agent tools. We categorise tools according to their direct impact: perception tools to access and read data, reasoning tools to analyse data or concepts, and action tools to directly modify external environments, like file editing, sending emails or steering drones in the physical world. We use O*NET mapping to identify each tool's task domain and consequentiality. Software development accounts for 67% of all agent tools, and 90% of MCP server downloads. Notably, the share of 'action' tools rose from 27% to 65% of total usage over the 16-month period sampled. While most action tools support medium-stakes tasks like editing files, there are action tools for higher-stakes tasks like financial transactions. Using agentic financial transactions as an example, we demonstrate how governments and regulators can use this monitoring method to extend oversight beyond model outputs to the tool layer to monitor risks of agent deployment.
Sam: They start by looking at what each tool lets an agent do. Some tools just let it gather information, like reading a file or searching the web—think of it as eyes and ears for the agent. Others help it make sense of that info, such as summarizing text or checking if a website works properly, like a brain processing what it sees. Then there are tools that let it change things, for instance editing documents, sending emails, or even handling money transfers—that's where it goes from watching to acting. The study groups these as perception tools for gathering, reasoning tools for thinking, and action tools for modifying.
Alex: Got it—so action ones are the risky shift. But within those, they make further splits, right? Like how broad or narrow the tool is?
Sam: Yes. A narrow tool works in a tight box, say just checking one specific bank's data or sending a single type of crypto payment—limited playground. A general one opens up anywhere, like controlling a full web browser or running any code on a computer, which hands the agent a much bigger sandbox to play in or mess up. Combining that with the impact type shows action tools, especially general ones, expand what agents can reach the most.
Alex: And they tied this to real-world jobs somehow? To gauge the stakes?
Sam: Exactly—they mapped tools to job tasks using O*NET, a U.S. list of work activities ranked by how much decisions matter, from 0 for low impact like simple feedback to 100 for high like financial trades. Low-stakes might be testing software in one country; high-stakes involve worldwide money moves or managing company permissions. This scores consequentiality, highlighting tools that push agents into bigger-risk areas, with geography added too—U.S. heavy at 50 percent of use, then Europe and China.
Alex: Makes sense for spotting cascades. How solid was their sorting method?
Sam: They used language models first to classify, then humans checked a sample for agreement—reliable enough per stats. Sources were GitHub searches for MCP servers, plus registries like Smithery, covering 177,436 tools over 16 months. Downloads tracked usage trends, proxying real deployment before big issues hit.
Alex: So public repos act like an early radar. And one more thing stood out—tools made by AI itself?
Sam: Right, 28 percent of servers and 36 percent of tools showed AI help, like Claude generating code, jumping from 6 percent early on to 62 percent by the end. This self-bootstrapping speeds the ecosystem, amplifying action space growth—and risks if bad prompts steer them wrong, like mass harmful trades. The paper notes monitoring catches this evolution first.
Alex: Right, and downloads show real interest, not just uploads?
Sam: Yes—tracking installs on sites like PyPI and NPM over 16 months proxies developer testing, like counting recipe downloads to see hot kitchen trends before restaurants adopt them. It caught action tools pulling 65 percent of use by early 2026, mostly general ones for browsers or code runs in open spaces. Official servers from companies like Stripe added weight, with 45 million downloads despite fewer tools.
Alex: Huh, so even big firms lean action-heavy. Does this match other AI systems?
Sam: The paper compares to chatbots like ChatGPT or Claude, where action tools are rare or absent—mostly perception there. Public agent tools stand out with 65 percent action usage, about twice as high, and 92 percent in software tasks versus under 60 percent elsewhere. This flags agents uniquely reshaping code and finance.
Alex: Okay, so agents pull ahead on changes. But what if bad instructions hit those action tools?
Sam: Malicious prompts could steer agents to harms like mass fake emails or crypto dumps, dropping crime costs since one bad idea scales via tools. A regulator misses this if ignoring tool trends, risking agent-driven runs on banks. The study suggests watching public repos spots it early, before production cascades.
Alex: Worth keeping an eye on, sure. But breaking it down further—what kinds of tasks are these tools mostly built for, beyond just software in general?
Sam: The study sorts tools into task areas based on their descriptions. Software and IT work—like searching code or handling security—covers 67 percent of all tools published and 90 percent of downloads, since one server install counts as using all its tools. Finance and business tasks, such as trading or customer service, make up 18 percent of tools but only 5 percent of downloads. Other areas like science research or art creation are much smaller, under 5 percent each.
Alex: So software dominates downloads by a wide margin. And for those action tools especially, how do they rate on real-world impact—like, are they handling everyday stuff or bigger decisions?
Sam: To measure that, they link tools to job types using O*NET, a U.S. government database that lists thousands of occupations and surveys workers on how much their choices affect others—scored from 0 for minor things like basic checks, up to 100 for major ones like handling big money transfers. Most action tools map to medium-impact jobs, say managing computer systems day-to-day. Finance stands out, though: it has far more tools than expected for high-impact work above 75, like trades that could sway markets, even if many focus on subtasks rather than full control.
Alex: Huh, finance as the high-stakes outlier. Does that pattern hold across places, or is it more U.S.-heavy?
Sam: Downloads show heavy concentration: the U.S. takes half of action tool installs in 2025, Western Europe about 20 percent, China 5 percent, with spots like Singapore at another 5 percent. This likely ties to where developers and registries like PyPI are based, so it might undercount other regions using different channels. The software focus holds steady everywhere, but the action shift appears globally in public data.
Alex: Okay, so not just one spot. One thing earlier you noted general tools growing within actions—why does that matter for control?
Sam: General tools let agents work in open areas, like fully controlling a web browser or running code anywhere on a screen—think no fences around a playground. Narrow ones stay boxed, say pulling data from just one bank's API. Over time, general-purpose downloads rose from 41 to 50 percent overall, but 94 percent of those involve actions in loose spaces, while perception stays mostly narrow at 95 percent. This means riskier changes happen where oversight is hardest.
Alex: That lines up with the surge—actions in wilder environments. For companies putting out official tools, does the pattern match?
Sam: Even more so: commercial servers shifted action tools from 21 percent to 71 percent of downloads, pushed by general ones like browser automation or desktop scripts. It suggests businesses are leaning into agents that reshape open digital spaces, not just peek. The paper sees this as a signal to watch before private uses amplify it.
Alex: A clear pattern across data points, then. Monitoring public trends gives that early view into where agents might cascade next...
Sam: Precisely. The findings point to expanding risks from misuse—like bad prompts leading to scaled harms in emails or crypto dumps—or errors propagating in software and finance, where agents now handle medium-to-high impact tasks at speed. Yes, notable ones. Downloads track interest, not actual runs; private tools stay hidden; and general-purpose ones hide specific uses, like what a browser tool does exactly. The study calls this a lower bound, urging checks on internal systems and agent orchestration for fuller pictures.
Alex: And that crypto payments example you hinted at earlier—servers for those jumped from 47 to over 1,500 in a year. How does this monitoring help regulators spot those before they build?
Sam: It flags early trends in autonomous transactions, like crypto tools growing without oversight, which could signal agent-driven volumes stressing markets. Paired with other checks, like company announcements, it lets authorities probe deeper via interviews or data requests, anticipating issues like coordinated runs.
Alex: A solid step for understanding these shifts without overclaiming. That's our dive into how AI agent tools are evolving, per this paper. Thanks for joining ResearchPod.