- Nov 2, 2026
- 5 min read
Every Integration Behind a One-Person Lead Engine: The Map
Over the last few months this site has turned into a build log: an inbound lead engine, an outbound pipeline, and the weekly habit of reading what both return. Each article covers one piece. This one is the map: every external service and API in the system, what data it moves, and where the gaps are.
The whole picture
flowchart TB
subgraph OUT["Outbound"]
direction LR
AP[Apify<br/>maps listings] --> EN[Own-website fetch<br/>+ Companies House API]
EN --> AU[Audit engine<br/>Lighthouse]
AU --> CSV[send-ready CSV]
CSV --> ST[Sending tool<br/>CSV in, CSV out]
ST --> LK[(DuckDB funnel)]
end
subgraph IN["Inbound"]
direction LR
FM[Forms on every site] --> WK[Cloudflare Worker + KV]
CAL[Cal.com webhook] --> WK
WK --> MJ[Mailjet<br/>+ event webhook]
WK --> NT[ntfy alerts]
WK --> HL[(Homelab SQLite<br/>pull-only mirror)]
end
subgraph SEE["Visibility"]
direction LR
CWA[Cloudflare Web Analytics] --> REP[make analytics]
GSC[Search Console API] --> REP
PSI[PageSpeed Insights API]
BW[Bing Webmaster Tools]
end
OUT ~~~ IN ~~~ SEE
Outbound: from a listing to a reply
| Integration | What it does | Access | Detail |
|---|---|---|---|
| Apify (Google Maps actor) | Business listings: name, website, address, category. Reviews, photos and people switched off | API token | From Google Maps to a UK lead list |
| The business’s own website | The email address it publishes, including ones Cloudflare obfuscates | Plain HTTP, own User-Agent | Finding a business email on its own website |
| Companies House API | Active limited company? Registered name, number, office | Free API key | Matching without false positives |
| Audit engine (Lighthouse, run locally) | One measured problem per site, measured twice | None (local) | Personalised with an audit, Lighthouse variance |
| Sending tool | Warmed inboxes, sequence, A/B subject, blocklist | No API on my plan: CSV import, CSV export | Sending capacity |
| DuckDB (local file) | Bronze files → silver tables → gold counts | None (local) | The Sunday review |
The why and the rules behind it: is B2B cold email legal?, keep outbound away from inbound, the pipeline end to end.
Inbound: from a form to a follow-up
| Integration | What it does | Access | Detail |
|---|---|---|---|
| Cloudflare Workers + KV | One forms endpoint for every site, same-origin; retention as a TTL | wrangler login / API token | One forms Worker, retention as a KV TTL |
| Mailjet | Confirmation and drip emails; an event webhook for bounces | API key + secret; webhook key | Validating a sending subdomain, the event webhook |
| Mailtrap / Mailpit | A second provider behind the same adapter; local test capture | API token / none | One sender adapter, Mailpit vs Mailtrap |
| Cal.com | Bookings into the same pipeline | Signed webhook | Cal.com signed webhook |
| ntfy (self-hosted) | A push to my phone per lead; a weekly counts-only digest | Write-only publisher token, behind a tunnel | Self-hosting ntfy, weekly digest |
| Homelab mirror | KV copied to SQLite, pulled from home, no inbound port | Read-only Cloudflare API token | A pull-only homelab |
| Pages deploy hooks | Nightly rebuild so future-dated posts appear | Hook URL (a secret) | Future-dated posts |
All of it, in order: the inbound series.
Visibility: is anyone finding the sites?
| Integration | Answers | Access | Detail |
|---|---|---|---|
| Cloudflare Web Analytics | Visits, pages, referrers, countries, browsers; no cookies | GraphQL with the wrangler login | The GraphQL API |
| Google Search Console | Queries, clicks, impressions, position, sitemaps | Service account JSON | Step by step, API key vs service account |
| PageSpeed Insights | Lab speed from Google’s machines, real-user data when there is any | API key (public data only) | API key vs service account, Lighthouse variance |
| Bing Webmaster Tools | Bing’s index and traffic | Verification (DNS, meta tag or file) | Three ways to verify |
The first two sit behind one command: a monthly website check. What it led to: separate SEO titles.
Five kinds of credential
Seventeen integrations, but only five ways of proving who I am:
| Kind | Used for | Rule |
|---|---|---|
| API token / key | Apify, Companies House, Mailjet, PageSpeed, ntfy, the homelab mirror | In a config folder outside every repo; scoped to the minimum |
| OAuth login | Cloudflare via wrangler | Short-lived; refresh before scripted use |
| Service account JSON | Search Console | No project roles; added only where it’s needed, read-only scope for reports |
| Signed webhooks | Cal.com, Mailjet events | Verify the signature before reading the body |
| Public by design | Web Analytics beacon token, Bing verification code | Fine in the HTML; still not something to reuse elsewhere |
Where the data still doesn’t meet
Closing the gaps is the point of most of these articles. These are the ones still open, on purpose:
- The sending tool has no API on my plan. Results come out as CSV every Sunday, by hand. Five minutes a week is cheaper than the next plan up.
- Outbound and inbound don’t share a database, by design: different legal basis, different retention. They only meet as counts in a review.
- Bing isn’t in the monthly report yet. It will be if it keeps sending more visitors than Google.
- Outbound has no click data: the emails carry no links and no tracking. The only signal besides a reply is a visit to the outbound brand’s own site, which now has the same cookieless beacon.
Takeaways
- Write down every integration once: what it moves, how it authenticates, and where it’s explained.
- Few credential types, kept outside the repos, beat many clever ones.
- Not every gap needs closing. A weekly CSV is an integration too.