METAHEURISTICBook a call →
[Retrieval]

Hire a RAG engineer.

Retrieval that respects permissions, survives messy documents, stays fresh, and can prove it improved - because someone built a retrieval eval set before touching the chunk size.

Book a 20-minute call → See the CV
144
Stars on validex (extraction)
59
Stars on vector_lake (S3 vector DB)

Retrieval-augmented generation is easy to demo and hard to run. The tutorial version - split on 500 characters, embed, cosine similarity, top five - works beautifully on a clean corpus with no access control and no deadlines. Your corpus is not clean, your documents belong to different people, and half of them were scanned in 2019.

A RAG engineer is hired for the distance between those two situations.

What breaks in production, in order of frequency

  1. Parsing. The PDF pipeline drops table structure, so a number loses its column header and the model confidently misreads it.
  2. Chunking. Fixed-size splits cut clauses in half. Contract language, policy documents and specifications are hierarchical, and the chunker should be too.
  3. Permissions. The demo searched everything. Production must not, and post-filtering leaks.
  4. Freshness. Nobody defined how stale is too stale, so the index quietly drifts a quarter behind the source of truth.
  5. Ranking. Pure vector similarity retrieves topically related but useless passages. Hybrid search plus a reranker usually beats a better embedding model.
  6. No retrieval evals. Quality is discussed rather than measured, so every change is a guess.

What gets built

Ingestion

Layout-aware parsing, OCR where needed, table and structure extraction, deduplication, and a pipeline that is re-runnable and incremental rather than a one-off script someone ran in a notebook.

Chunking and indexing

Structure-aware splitting with overlap tuned against a real eval set. Metadata that supports filtering - source, owner, ACL, effective date, version. Hybrid BM25 plus dense retrieval, because keyword search still wins on identifiers, codes and names.

Retrieval

Pre-filtered by permission, reranked by a cross-encoder, with query rewriting for the cases where users type three words and mean a paragraph. Explicit handling of “the corpus does not cover this” - a system that refuses cleanly beats one that confabulates confidently.

Generation

Citations with spans, so a reviewer can check the claim against the source in one click. Groundedness checks that catch the answer that quietly went beyond its evidence.

Evaluation

Retrieval scored separately from generation. Recall@k and MRR on a labelled query set, groundedness and answer quality on top, both in CI.

If the system needs to take actions rather than answer questions, see hire an agentic AI engineer. If the pain is cost, latency or eval coverage at the model layer, see hire an LLM engineer.

Questions before you hire.

Our RAG demo was great and the rollout was terrible. Why? +

Almost always one of four things: the demo corpus was clean and production is not; the demo ignored permissions and production cannot; the demo had no freshness requirement; or nobody built a retrieval eval set, so quality was never measured, only felt. All four are diagnosable in days.

Which vector database should we use? +

pgvector, if you already run Postgres, until you have a measured reason to move. The database is rarely the bottleneck - parsing, chunking, permissions and reranking are. I have built an S3-backed vector store and still recommend Postgres first for most teams.

How do you handle permissions? +

Filter at retrieval, enforced by the same identity system that owns the source documents, with the filter applied before the vector search rather than after. Post-filtering leaks through result counts and latency. Permission changes have to propagate to the index on a timeline you have agreed, not eventually.

What about PDFs, scans and tables? +

This is the real work. Layout-aware parsing, table extraction, OCR for scans, and structure preserved through chunking so a number keeps its column header. validex, which I maintain, exists because of exactly this problem.

How do you measure retrieval quality separately from answer quality? +

A labelled set of queries with known-relevant documents, scored with recall@k and MRR, run independently of generation. Otherwise a generation improvement masks a retrieval regression and you optimise the wrong half of the system. I have written about going beyond the tutorial version of RAG.

Need an agentic AI engineer this quarter?

Contract, fractional, or embedded with your team. Start with a 20-minute call - you leave it with an honest read on whether your use case is buildable, whatever happens next.

Book a 20-minute call → Download CV (PDF)