- Feb 10, 2026
- 4 min read
Everything I Built With LLMs in 2024, and What I'd Keep
Built in 2024, revisited now: what I made, and what I’d keep.
2024 was the year I built a small LLM app for almost every idea I had.
Most were Python and Streamlit, each in its own container, all self-hosted.
Looking back at the list, the interesting part isn’t any single app. It’s which ideas survived.
What I built
| App | What it did | Built with |
|---|---|---|
| Multichat | one chat UI for several providers: OpenAI, Anthropic, Groq, local models via Ollama | Streamlit, provider SDKs |
| YouTube summarizer | paste a link, get a summary | Groq (fast), Streamlit, a login screen |
| Chat with PDFs | ask questions across several documents | LangChain, FAISS vector store |
| Chat with a dataframe | ask questions of a table in plain language | PandasAI, OpenAI or Ollama |
| Scrape and ask | pull a page as Markdown, then question it | Firecrawl, OpenAI |
| Function calling tests | the model asks your code to run a function | OpenAI, Groq and Claude APIs |
| Structured outputs tests | the model must answer in a given JSON schema | OpenAI |

The multichat is on GitHub, with its container image.
Every app had a container image, built by GitHub Actions for both x86 and ARM and pushed to GHCR, so the same image ran on a laptop, a mini PC or a Pi.
One compose file brought them all up, with Caddy in front for automatic HTTPS (less configuration than NGINX, simpler than Traefik for a handful of apps).
What didn’t last
Most of the frameworks. LangChain, PandasAI, Embedchain, PhiData: each solved a real problem in 2024 and each changed its API or its focus within months. Code written against them aged fastest. The thin wrappers around a provider’s own SDK aged best.
Chat UIs as the product. A Streamlit chat box was the default interface for everything. It’s a fine demo. It’s rarely what someone actually needs, which is usually a result in a file, a field or a message, not a conversation.
RAG for everything. Chat-with-your-PDFs worked, but most of the effort was in splitting, embedding and retrieving, and the answers were only as good as the chunk the retriever happened to find. Long context windows have since removed the need for it in many small cases.
Keys in the compose file. My 2024 compose wrote API keys into a config file inside the container’s start command. It worked. It’s also exactly how keys end up in a Git repository. Secrets belong in an environment file that’s never committed.
What I’d keep
Structured outputs. This one changed how I build. When the model has to answer in a JSON schema you define, its output stops being text you hope to parse and becomes data you can check. My 2024 notes already listed two uses: generating CVs from a template and generating Markdown posts with correct front matter. Both turned out to be real projects. The pattern underneath is the one I use everywhere now: the model fills a schema, and code validates it before anything is used.
Tool (function) calling. The model doesn’t run anything. It says “call this function with these arguments”, your code decides whether to, runs it, and passes the result back. That separation is what makes it safe to let a model near real systems. In 2024 OpenAI, Groq and Claude each did it slightly differently. The idea is the same, and it’s the basis of every agent worth trusting.
Speed changes what’s possible. Groq answering in a fraction of a second made the YouTube summarizer feel like a tool instead of a wait. Latency is a feature.
Containers for every experiment. Being able to bring any old app back with one command is why I could still look at them for this post.
What I’d do differently now
- Build against the provider SDK, with my own thin layer, instead of a framework.
- Start from the output, not the chat: what file, field or message should exist at the end?
- Schema plus validator from day one, and treat anything that fails validation as missing, not “close enough”.
- One small app done properly over seven demos.
Takeaways
- Frameworks churn; API primitives last. Structured outputs and tool calls are the primitives.
- Validate model output in code. Don’t ask the model to be correct; check it.
- A chat box is a demo, not a product.
- Keep secrets out of compose files.