• Apr 30, 2026
  • 4 min read

A Public AI Chatbot Is a Public Bill

A look back at a retired client project: what I built, what a later review found, and how the fix works.

The real estate site had a small chatbot: visitors ask about the listings (something like “which flats have a balcony?”) and get answers. Its prompts and settings lived in a Markdown file, so the agent could change its tone without touching code.

Behind it was one API route, /api/chat, that sent each message to OpenAI. That makes the endpoint a meter: every request costs money, on the agency’s API key.

The first version was open

As built, /api/chat had no protection at all. Any website could embed it, any script could call it in a loop, and the bill would arrive at the end of the month. I’d already learned that lesson once with a serverless bill, and API keys in my 2024 LLM apps were another reminder that whoever can call it, spends it.

What was added

  • CORS: only requests from the site’s own origin are accepted.
  • A rate limit: 10 messages per minute per visitor, configurable by environment variables, answering 429 Too Many Requests beyond that.
  • A test script with curl: an allowed origin gets 200, a foreign origin gets 403, and 12 rapid requests end in 429s.

All three tests passed. It looked done.

What the review found

1. The origin check matched substrings.

allowedOrigins.some((allowed) => origin.includes(allowed))

https://site.example.attacker.example includes site.example, so it passed. Origins must be compared exactly.

2. No Origin header meant no check. The check only ran if (origin && …). Browsers send an Origin on cross-site requests; curl and servers don’t have to. So any script could skip it entirely by not sending the header.

3. CORS was never the real protection. CORS tells browsers which sites may read a response. It doesn’t stop anyone else from sending requests. It stops someone embedding the bot on their site; it does nothing against a script draining the key. That job belongs to the rate limit. And unlike the editor, this endpoint can’t sit behind Cloudflare Access: visitors must reach it without logging in, which is exactly why it needs limits of its own.

4. The rate limit trusted the client. It identified visitors by the first value of X-Forwarded-For, a header the client can set to anything. A new fake IP on every request meant a fresh allowance every time. Behind Cloudflare, the reliable client address is CF-Connecting-IP, which Cloudflare sets itself.

5. The limit lived in memory. A restart reset everyone’s count. Fine for a small site, but worth knowing: it limits bursts, not daily spend.

The test script passed because it tested what the code was designed to do. It didn’t test what an attacker would try.

The fix

const ALLOWED = new Set(["https://site.example"]);   // exact origins only

const origin = request.headers.get("origin");
if (!origin || !ALLOWED.has(origin)) {
  return new Response("Forbidden", { status: 403 });
}

const ip = request.headers.get("cf-connecting-ip") ?? clientAddress;
await rateLimiter.consume(ip);                        // 429 if over the limit

And the controls that cap the damage whatever happens:

  • A spending limit on the API key itself, set in the provider’s dashboard. The last line of defence is the one the attacker can’t reach.
  • A small max_tokens and a short conversation history, so each request is cheap.
  • A daily cap in persistent storage, not just a per-minute one in memory.

And the tests that were missing:

# no Origin header at all → expect 403
curl -s -o /dev/null -w "%{http_code}\n" -X POST "$API" -d '{"message":"hi"}'
# lookalike origin → expect 403
curl -s -o /dev/null -w "%{http_code}\n" -X POST "$API" \
  -H "Origin: https://site.example.attacker.example" -d '{"message":"hi"}'
# spoofed X-Forwarded-For on every request → still 429 after the limit

Takeaways

  • An LLM endpoint is a meter. Protect it like one.
  • CORS is for browsers; rate limits and spending caps are for everyone else.
  • Compare origins exactly, and reject requests without one.
  • Rate-limit by an address the client can’t set.
  • Test like an attacker, not like the developer who wrote the happy path.