- Apr 24, 2026
- 4 min read
An Agent Skill That Checks Instead of Trusting
Once surveys were JSON files with a CLI in front of them, the next step was obvious: let an AI agent write them. “Make me a qualifier for the web audit, five questions, ending with a booking link” is exactly the kind of task agents are good at.
It’s also exactly the kind of task where an agent can be confidently wrong: a duplicate question ID, a funnel with no way to contact the lead, a thank-you screen with no next step, or a line like “trusted by 200 businesses” that nobody can back up.
So the agent gets two things: instructions, and a check it can’t argue with.
Part 1: the skill
A skill is a short document an agent loads when a task matches it. Mine says:
- What to read, and only that: the CLI’s README, the sample survey, the Makefile, and the offer docs if the form supports a funnel.
- Which commands exist:
make whoami,make create,make publish, and so on. The agent uses the same interface I do. - The lifecycle: create as draft, publish explicitly, pull responses, close.
- The funnel pattern: segment (who are you?), qualification (budget, urgency or fit), pain (what’s the bottleneck?), intent (what outcome, by when?), contact (email), thank-you card routing to the right next step.
- Guardrails: never print
.envvalues, never overwritesurvey.jsonwithout checking it for work in progress, one audience and one call to action per commercial form, report API errors briefly and suggest the fix.
That covers how to do the job. It doesn’t prove the job was done right.
Part 2: the validator
Before anything is created or published, the agent must run:
python validate_survey.py forms/survey.json --funnel
It’s about 150 lines of plain Python, no dependencies, and it sorts what it finds into two kinds.
Errors block. The script exits non-zero, so the agent can’t proceed:
| Error | Why it matters |
|---|---|
missing name, type, status or questions | not a valid survey |
a question without id, type or a headline | renders broken |
| a duplicate question ID | answers overwrite each other |
| a funnel with no required contact email | you collect leads you can’t reach |
| a funnel with no thank-you card | the visitor hits a dead end |
Warnings need a human. The run passes, but the list is in front of me:
| Warning | Why it matters |
|---|---|
copied runtime fields (id, createdAt…) | exported JSON reused carelessly |
a hardcoded environmentId | tied to one workspace |
status not draft | about to go live without review |
| a thank-you card with no button link | no next step |
| proof-like wording | ”guaranteed”, “trusted by”, “5-star”, “testimonial”, “recovered” |
The interesting rule: proof words
The last check is the one I care about most. An agent writing marketing copy will reach for social proof, because that’s what persuasive copy looks like. It has no idea whether “trusted by dozens of businesses” is true.
The validator doesn’t judge whether a claim is true. It notices that a
claim is being made and puts it in front of a person. If it’s real, I pass
--allow-proof-claims and it’s a conscious decision. If it isn’t, it comes
out. Either way, a model never publishes a claim about my business on its own
authority.
Why a script and not a better prompt
I could have written “please don’t invent testimonials” in the skill. It’s in there, in fact. But a prompt is a request; a script is a rule:
- It’s the same every time. No model mood, no context window, no “I forgot”.
- It’s testable. I can hand it a bad survey and watch it fail.
- It works for any author. Me on a tired evening, an agent, or a future contractor all hit the same check.
- Its output is readable.
ERRORS:andWARNINGS:with one line each, so the agent can fix what it broke and tell me what it couldn’t.
It’s the same principle as validating structured output from a model: let it produce, then check in code before anything reaches the outside world.
Takeaways
- A skill says how; a validator says whether. Give an agent both.
- Errors block, warnings escalate: not everything should stop the run.
- Flag claims, don’t judge them. A person decides what’s true.
- Prefer a script to a prompt for anything that must hold every time.