Faintprint Buy

Notes · November 2026

What we measured, and what we could not

Faintprint reads the tool lists of MCP servers and tells you whether one is trying to trick your AI. A tool like that is only worth using if you know how often it is wrong. This note is how we found out, and where we could not.

Every figure below is a line of our claims sheet, quoted word for word with its caveat.

Two readers, one floor

Faintprint reads each server's text twice. First, fixed rules: patterns written by hand, which give the same answer every time. Then, only for the passages the rules find borderline, and only if you ask for it, an AI reviewer reads the passage and says whether it is an attack. The reviewer runs on your own Anthropic key, after you have seen the price.

Fooling the reviewer cannot make a server look safer. The fixed rules are the floor; the reviewer can only add to them. A fully fooled reviewer gives exactly the fixed-rule result.

Everything below rests on that: the reviewer can make a report stricter, never kinder.

Does it accuse harmless servers?

A scanner that cries wolf gets switched off. So the first thing we measured was how often the reviewer confirms an attack in a passage that is not one.

The AI reviewer rarely accuses a harmless server. On passages from servers believed to be harmless (chosen by fixed rules from how established and widely used they are, before anyone read them), it wrongly confirmed an attack on 5 of 1,449 checks (0.34%), all five on one passage. After one person on our side re-read the six disputed passages, the figure was 0 — but four of those labels were changed after seeing the reviewer's answer, which makes the second number weaker, so we always give both. Measured once, in September 2026, on data held back from all tuning.

Does it let known attacks through?

The opposite mistake is the dangerous one: an attack passed as harmless. Here we used attacks other people have published.

It does not wave known attacks through. On published attack examples it gave a verdict on 92% of checks and missed none of those; with the backup model used in production, 100% answered, none missed. The attacks come largely from one research paper and three templates.

Then we attacked the reviewer

An AI reviewer reads text that the server's author wrote, so that author can try to talk to it. We wrote attacks aimed at the reviewer itself and measured what they did.

We attacked the reviewer ourselves. When we wrote 24 attacks aimed at the AI reviewer itself — forged boundary markers, fake "approved by security" notes, "this is a test fixture", encoded and foreign-language instructions, the attack quoted as an example, long documentation padding, refusal bait — and put each of them on all 173 known attack passages in our evaluation set, the reviewer was talked into clearing a passage it had confirmed once in 3,272 checks. It downgraded about one in a hundred to "suspicious", almost all when the attack text claimed the passage was a quoted example; a "suspicious" passage still counts and is still shown. Two shapes made it decline to answer more often (about 29% instead of 13%); a declined check counts as no confirmation, never as a clearance. Caveats that travel with this: the attacks were written by a model of the same family as the reviewer; the passages come largely from one research paper; this is the evaluation set, not a held-out one; and the figures are for the batch path without the production fallback. Measured in October 2026; after a fix to the boundary markers shipped in 0.4.0, a re-run of the two forged-marker attacks changed none of the reviewer's answers, and it declined to answer a little more often (31 and 29 of 173 checks, against 22 and 25).

What we could not measure

Some things we have no figure for, so we do not claim them.

  • "Human-level", or any comparison with human experts. There is no human baseline.
  • "The grade is accurate." How often the fixed rules are right on real servers has not been measured. Real use is where that evidence comes from.
  • Anything about the accuracy of the rule on payment instructions to the AI. No figure exists.
  • That it resists attacks on the reviewer beyond the 24 we wrote, or that it was tested on text cut off at the length limit (not measured).

The whole list is on What we can say.

Check it yourself

You can check every finding yourself. Each one comes with the quoted text, and each passage the reviewer confirms with its reason.

A made-up server's whole report is on the example page: it shows the sentence that gave it away.

Price

One price a year, from €29

Individual €29. Companies €99, €199 or €399, by the number of people they employ.

See the prices See an example report