Pin the exact version
We install the version you name, never a range, so the report describes one build.
Toolmark checks published agent tools at an exact version, with no network and no secrets, and shows what they did next to what they claim. MCP servers and published OpenAPI documents today; agent skills and plugins are being built.
checks output against its own schema, wrong-type input, files changed by "read-only" tools, start-up without a key
Every tool's claims sit next to what we observed in the sandbox. A mismatch is a review item with its evidence, not a grade.
The same five steps for every package. The full method
We install the version you name, never a range, so the report describes one build.
The server starts in a fresh cloud sandbox with no network, no secrets and only our synthetic data folder.
Names, input schemas and hints such as "read-only" are stored as claims, never taken as facts.
Tools that send, pay, message or publish are called last, only after a check proves the network is closed, and only with fake values. What we cannot call is marked "not tested", never held against the tool.
You get a private link. Each finding carries its evidence, its source and the date. There is no score.
These numbers are computed from a stored report when the site is built, never typed by hand.
Source: Toolmark's own example server with planted flaws, run 2026-11-18 in memory with the same checks as every package, except the network check, which needs the sandbox; without it, outward tools are not called. It is not a published package.
The same rules, the same labels and the same private report for each tool type. MCP servers can be checked today; the other two are being built.
Published on npm or PyPI and run at an exact version in a sandbox with no network. Every tool's claims next to its behaviour. Available now.
Being built. What a bundle contains versus what it claims, which scripts and hooks run and what they touch in our data folder. MCP servers inside a plugin are checked like any other.
The published OpenAPI 3.0 or 3.1 document, checked offline: its structure, operation ids, response schemas, examples against their schemas, every $ref, declared security schemes, and what changed between two versions. Available now. The API itself is not called; live calls would need a sandbox key from the provider and are not built.
A package at an exact version (npm:name@1.2.3, pypi:name==1.2.3), a name without a version, an npm, PyPI or GitHub link, or the npx or uvx line from your MCP config. Without a version we look up the latest one, and the report names the exact version that ran. Only the package is read from what you paste; keys and everything else are thrown away.
No. We observe how a tool behaves when it is called. We do not scan for vulnerabilities, and a report never says a tool is free of risk.
No score, grade or badge. Each finding is labelled verified, review item or not tested, with its evidence, source and date.
In a cloud sandbox started for one check. It is installed at the exact version, then run with no network, no secrets and only our synthetic data folder. The sandbox is thrown away afterwards.
Every tool we can fill with synthetic input. Tools that write or delete can only touch our throwaway data. Tools that send, pay, message or publish are called last, only after the sandbox proves it has no network, and only with reserved fake values, so they cannot reach anyone. Hints such as "read-only" are treated as claims.
We could not test that part, for example a tool with no description, an input we cannot fill with synthetic data, or a missing key. It is never counted against the tool.
Anyone with the private link. Reports are not listed or published, and they are deleted after 30 days.
No. Nothing is for sale today, and payment will never change a result.
The package you entered, a daily-salted hash of your network address for rate limits, and the report, for 30 days. No cookies, no JavaScript, no analytics. Privacy
A package, a link or the line from your config. You get a private link to the report.