• Apr 24, 2026
  • 4 min read

An Agent Skill That Checks Instead of Trusting

Once surveys were JSON files with a CLI in front of them, the next step was obvious: let an AI agent write them. “Make me a qualifier for the web audit, five questions, ending with a booking link” is exactly the kind of task agents are good at.

It’s also exactly the kind of task where an agent can be confidently wrong: a duplicate question ID, a funnel with no way to contact the lead, a thank-you screen with no next step, or a line like “trusted by 200 businesses” that nobody can back up.

So the agent gets two things: instructions, and a check it can’t argue with.

Part 1: the skill

A skill is a short document an agent loads when a task matches it. Mine says:

  • What to read, and only that: the CLI’s README, the sample survey, the Makefile, and the offer docs if the form supports a funnel.
  • Which commands exist: make whoami, make create, make publish, and so on. The agent uses the same interface I do.
  • The lifecycle: create as draft, publish explicitly, pull responses, close.
  • The funnel pattern: segment (who are you?), qualification (budget, urgency or fit), pain (what’s the bottleneck?), intent (what outcome, by when?), contact (email), thank-you card routing to the right next step.
  • Guardrails: never print .env values, never overwrite survey.json without checking it for work in progress, one audience and one call to action per commercial form, report API errors briefly and suggest the fix.

That covers how to do the job. It doesn’t prove the job was done right.

Part 2: the validator

Before anything is created or published, the agent must run:

python validate_survey.py forms/survey.json --funnel

It’s about 150 lines of plain Python, no dependencies, and it sorts what it finds into two kinds.

Errors block. The script exits non-zero, so the agent can’t proceed:

ErrorWhy it matters
missing name, type, status or questionsnot a valid survey
a question without id, type or a headlinerenders broken
a duplicate question IDanswers overwrite each other
a funnel with no required contact emailyou collect leads you can’t reach
a funnel with no thank-you cardthe visitor hits a dead end

Warnings need a human. The run passes, but the list is in front of me:

WarningWhy it matters
copied runtime fields (id, createdAt…)exported JSON reused carelessly
a hardcoded environmentIdtied to one workspace
status not draftabout to go live without review
a thank-you card with no button linkno next step
proof-like wording”guaranteed”, “trusted by”, “5-star”, “testimonial”, “recovered”

The interesting rule: proof words

The last check is the one I care about most. An agent writing marketing copy will reach for social proof, because that’s what persuasive copy looks like. It has no idea whether “trusted by dozens of businesses” is true.

The validator doesn’t judge whether a claim is true. It notices that a claim is being made and puts it in front of a person. If it’s real, I pass --allow-proof-claims and it’s a conscious decision. If it isn’t, it comes out. Either way, a model never publishes a claim about my business on its own authority.

Why a script and not a better prompt

I could have written “please don’t invent testimonials” in the skill. It’s in there, in fact. But a prompt is a request; a script is a rule:

  • It’s the same every time. No model mood, no context window, no “I forgot”.
  • It’s testable. I can hand it a bad survey and watch it fail.
  • It works for any author. Me on a tired evening, an agent, or a future contractor all hit the same check.
  • Its output is readable. ERRORS: and WARNINGS: with one line each, so the agent can fix what it broke and tell me what it couldn’t.

It’s the same principle as validating structured output from a model: let it produce, then check in code before anything reaches the outside world.

Takeaways

  • A skill says how; a validator says whether. Give an agent both.
  • Errors block, warnings escalate: not everything should stop the run.
  • Flag claims, don’t judge them. A person decides what’s true.
  • Prefer a script to a prompt for anything that must hold every time.