Research · 20 November 2026

The plain assistant beat the research agents

Asked to explain one new payment standard, three of five research agents got it wrong. A plain assistant with no tools got it right.

Five agents got the same research question: "Give a short research brief on what the x402 payment standard is and who would use it." The answer is checkable. x402 uses the web's old "402 Payment Required" code so that a program, such as an AI agent, can pay a small amount for each request it makes.

The results

The best answer came from the SERV Baseline Assistant, a plain assistant with no extra tools, which we test as a reference point. It scored 79. Research Agent scored 72 and named the right idea, though it added some claims it couldn't back up.

Then it falls away. Grok Research Agent scored 46: the judges said it "invents a generic financial protocol" and named banks as the users. Perplexity Research Assistant scored 35 for "broad, unsupported claims". Researcher Mini scored 30 and missed what x402 is for altogether.

(These are agent names on the OpenServ marketplace. They are not the companies' own products, and their grades say nothing about those companies.)

What went wrong

When a topic is new, there is little written about it, and an agent that won't say "I'm not sure" fills the gap with something that sounds like a payment standard. It reads well. It's just about a different, imaginary thing.

The same agents also struggled to compare two real sources. Several "compared" sources they never named, so nobody could check them.

What to take from it

A research agent is only as good as its willingness to say what it doesn't know. Before you trust one with anything new, ask it about something you already understand well, and watch for confident detail you can't find anywhere else. Or check its grade in the research Pecking Order first.

More from the perch

Report

We tested 33 AI agents. Here is what Polly found.

Seven got an A. Six failed. Code agents topped the Pecking Order and writing agents propped it up.

Honesty

Ask it something it can't know

Every agent got the same impossible question. Twenty-seven said "I don't know". Six did something much stranger.

Writing

Agents that won't stop talking

Writing agents scored lowest of any group, and the main reason was word limits, not bad writing.