Generative AI & LLM
The distance between an LLM demo and an LLM product is almost never the model. It is retrieval quality, evaluation, cost per request and the failure modes that only appear when real users ask questions you did not anticipate.
Where LLM projects actually break
A demo answers ten curated questions well. Production answers thousands, including the ambiguous, the adversarial and the ones whose answer is genuinely not in your corpus. Three things break first. Retrieval: the model is fine, but the passage it needed was never returned, so the system confidently answers from the wrong context. Evaluation: without a harness, every change is a vibe check, and nobody can tell whether last week's prompt edit helped or quietly regressed a category of question. Cost: prompts grow as teams stuff more context in to fix quality problems that were really retrieval problems, and token spend climbs faster than usage does.
The three calls that decide the outcome.
Made explicitly, with the trade-off written down, before anything gets built.
RAG, fine-tuning, or both
Retrieval handles knowledge that changes and must be attributable; fine-tuning handles form, format and task behaviour that is stable. Teams reach for fine-tuning to fix factual errors, which is usually the wrong tool — the fix is better retrieval. We use fine-tuning where output structure or domain register genuinely will not hold through prompting, and treat it as an optimisation after retrieval is working, not a substitute for it.
Chunking and retrieval strategy
Fixed-size chunking is fast to build and loses the structure that made the document answerable. Whether you need semantic chunking, hybrid keyword-plus-vector search, a reranking pass or parent-document retrieval depends on how your corpus is written and how questions are actually phrased. This is the highest-leverage tuning surface in most RAG systems and the one most often skipped in favour of swapping models.
How much autonomy an agent gets
Multi-step agents compound their own errors: a wrong decision at step two is reasoned on top of at steps three through seven. We scope autonomy explicitly — which actions require confirmation, which are reversible, where the loop terminates — and keep write access behind an approval boundary until the eval data justifies moving it. Most enterprise use cases need a well-retrieved answer, not an autonomous agent.
The capabilities we bring on day one.
Each engagement assembles from this menu, sized to your scope, paced to your calendar.
Retrieval before models
Corpus analysis, chunking strategy and hybrid retrieval tuned to how your questions are actually phrased.
Evaluation harness
A graded question set per category, run on every change, so regressions are caught before users find them.
Cost engineering
Context caching, right-sized prompting and routing non-reasoning work to lightweight models.
Bounded agents
Explicit autonomy limits, approval gates on writes, and a defined termination condition.
Where prompts and completions live
Sophisticated buyers no longer ask only where data is stored. They ask where it is processed, who can access it and which legal regime applies. We design for that up front: PII redaction before inference where the use case allows it, region-pinned processing where residency is contractual, retention limits on prompt and completion logs, and an audit trail that can show what a system was asked and what it answered. Where a signed processor agreement is required before any personal data reaches a model provider, that gates the build.
From kickoff to live in four phases.
Each phase has named deliverables, named owners and a named gate, and every one respects the systems you already have in place.
Discover
Corpus review, question taxonomy from real user language, and a baseline measurement of what today's system gets wrong and how.
Design
Retrieval architecture, model selection and routing plan, evaluation criteria per question category, and the data-handling boundary.
Build
Ingestion and retrieval pipeline, prompt architecture, eval harness wired into CI, and integration with your application and identity layer.
Run & hand off
Production monitoring on quality and cost per request, drift in question mix, and a runbook so your team can tune retrieval without us.
What a good fit looks like.
Stated up front, so neither of us spends a call finding out this was the wrong conversation.
Engagement shape
A scoped sprint of roughly eight weeks to a working proof of concept against your real corpus, with an evaluation harness you keep.
From your side
Access to the actual document corpus or knowledge source, sample questions from real users, and a decision on where inference is allowed to run.
When not to hire us
If you want a chatbot on top of a public website with no proprietary knowledge, an off-the-shelf product will be cheaper and faster than anything we would build for you.
Generative AI & LLM questions, answered.
Usually not at first. Retrieval quality and prompt architecture account for most of the gap between a weak and a strong system. Fine-tuning is worth it when output structure or domain register will not hold through prompting, and when you have stable labelled data — we treat it as an optimisation after retrieval works.
Three layers: retrieval tuned so the right passage is actually returned, prompting that requires the answer to be grounded in retrieved context and to decline when it is not, and an eval set that specifically includes questions whose answer is absent from the corpus. The third is the one most teams skip and it is where unfounded answers get caught.
Yes, with trade-offs. Self-hosted open-weight models remove the external processing question entirely at the cost of more infrastructure work and typically some capability. Where a frontier model is genuinely needed, region-pinned processing plus PII redaction before inference is the usual middle ground. The right answer depends on your regulatory position, and we scope it before building.
By treating cost as an engineering problem rather than a pricing negotiation. Retrieving only the passages a request needs shrinks prompts and often improves answers. Context shared across many requests is cached rather than rebuilt. Work that does not need a large model — intent detection, entity extraction, classification — routes to lightweight models instead. We measured exactly this pattern on the Fanziz inference pipeline.