This site's AI assistant: RAG over my own documents
A grounded assistant that answers recruiter questions from my case studies, and refuses rather than guesses.
Live on this page. Four model providers behind it, free tiers first, so a rate limit on one never takes it down.
What it is
A FastAPI service that retrieves over my case studies, CV and project notes, and answers questions about my work. It shows which document each answer came from, and says it does not know rather than inventing a detail. It is the assistant on the Ask my AI section of this site.
The problem it solves
A portfolio asks you to read. A recruiter with forty tabs open does not read — they skim for one thing: production experience, a specific stack, or what I am weakest at. This answers that directly. The harder problem is that every public LLM demo invents things, and one invented detail about my own CV would make a recruiter discard everything else on this site.
How it works
- FastAPI backend, LangChain composing the retrieval chain and per-session history, Chroma as the vector store.
- Local embeddings rather than an API. Embeddings cannot fall back between providers — an index built with one model is meaningless to another — so moving them off the network removed that whole class of failure.
- Four chat providers in a fallback chain, free tiers first, paid last and normally never reached. If one is rate-limited or down, the next answers the same request.
- Rate limiting in three layers: per session, per IP per day, and a global daily kill switch. Every check runs before the model call, so a blocked request costs nothing.
- Input length capped in the request schema and output tokens capped on the model, so no single request can be expensive.
- Dockerised with the index and embedding model baked into the image, so a restart is instant and a deploy never depends on a third party being reachable.
- Deployed on my own VPS behind a reverse proxy with automatic TLS.
The interesting problem
Getting the model to refuse. Asked about a skill that is not in its context, a model will reach for a plausible sentence — and a plausible sentence about my own CV is a lie a recruiter will catch. The fix was not one instruction but three together: a low temperature, a prompt that treats an unanswerable question as a normal outcome with a defined response, and a corpus that states my weaknesses explicitly so there is honest material to retrieve instead of a gap to fill. The second problem was subtler: the obvious fallback design rotates every API call across providers including embeddings, which does not error — it just quietly returns irrelevant results.
What I would do differently
Stream the response. It waits for the full answer before showing anything, which on a slow provider reads as broken. I would also put the rate-limit counters in Redis from the start — in-memory is correct for one worker and silently wrong the moment there are two.