AI Platform Engineering
The first AI use case is a project. The fourth is a platform problem. By then three teams have each built their own retrieval stack, nobody can say what GPU spend belongs to whom, and there is no single place to enforce a rate limit or capture an audit log.
The cost of no platform
Without shared substrate, every team re-implements ingestion, chunking, embedding, retrieval, prompt management and evaluation — badly, and differently. The visible cost is duplicated effort. The expensive costs are invisible: no consistent audit trail of what was asked and answered, no way to enforce tenant isolation or rate limits centrally, no chargeback so GPU spend has no owner and therefore no downward pressure, and no ability to swap a model provider without touching nine applications. Platform work is unglamorous and it is what makes the fourth use case cost a fraction of the first.
The three calls that decide the outcome.
Made explicitly, with the trade-off written down, before anything gets built.
Which vector store, and whether you need a dedicated one
If your data already lives in Postgres, pgvector avoids an entire system and is sufficient at surprisingly large scale. If it lives in MongoDB, Atlas Vector Search keeps retrieval next to the documents — the pattern we used for Fanziz's article-to-video matching. Dedicated vector databases earn their operational cost at high write throughput, aggressive filtering requirements or very large indexes. Choosing one before you know your filter patterns is a common and expensive mistake.
Model gateway as a control point
A gateway in front of every model call is the cheapest governance you will ever buy. It gives you one place for authentication, rate limiting, per-team quota and chargeback, prompt and completion logging with retention rules, provider failover, and the ability to change models without redeploying applications. Teams that skip it end up with API keys distributed across repositories and no answer when audit asks what the system was asked last quarter.
How GPUs get shared
Dedicating a GPU per workload is simple and wasteful, because most inference workloads are bursty. Time-slicing raises utilisation with no memory isolation, which is fine for trusted internal workloads and unacceptable across tenants. MIG partitioning gives hard isolation on supported hardware at the cost of fixed partition sizes. For CPU-servable models the correct answer is often no GPU at all — the moderation service we built runs CPU-optimised inference as a stateless container precisely so it scales horizontally without one.
The capabilities we bring on day one.
Each engagement assembles from this menu, sized to your scope, paced to your calendar.
Retrieval substrate
Shared ingestion, embedding and retrieval services so teams stop rebuilding the same pipeline.
Model gateway
One control point for auth, quota, failover, logging and provider changes without application redeploys.
GPU orchestration
Scheduling and partitioning matched to workload shape, including the case where you do not need a GPU.
Cost attribution
Per-team, per-use-case spend visibility, so optimisation has an owner.
Auditability and isolation by construction
Retrofitting audit trails onto a platform that was not designed for them is painful and usually incomplete. We build the record of what was asked, what was retrieved and what was answered as a property of the gateway rather than a per-application responsibility, with retention rules set by policy rather than by whoever wrote that service. Tenant isolation is enforced at the retrieval layer, so a filter bug in one application cannot surface another tenant's documents — the failure mode that turns an AI feature into a disclosure incident.
From kickoff to live in four phases.
Each phase has named deliverables, named owners and a named gate, and every one respects the systems you already have in place.
Discover
Inventory of existing AI workloads, retrieval implementations and GPU spend. Where duplication and unowned cost actually sit.
Design
Platform architecture, vector store selection against your real filter and throughput patterns, gateway design and isolation model.
Build
Shared services stood up incrementally, with one real workload migrated onto each component to prove it before the rest follow.
Run & hand off
Platform runbooks, capacity and cost dashboards, and enablement so your engineers extend it without depending on us.
What a good fit looks like.
Stated up front, so neither of us spends a call finding out this was the wrong conversation.
Engagement shape
An embedded pod, usually monthly, since platform work lands incrementally and has to be proven against real workloads as it goes.
From your side
Access to existing AI workloads and their owners, cloud and infrastructure access, and a decision-maker for platform standards.
When not to hire us
If you have one AI use case in production and no immediate plans for a second, build the use case properly and revisit platform work when duplication actually appears.
AI Platform Engineering questions, answered.
Often no. If your data already lives in Postgres or MongoDB, pgvector or Atlas Vector Search keeps retrieval next to the documents and removes an entire system from your operational surface. Dedicated vector databases earn their cost at high write throughput, heavy metadata filtering or very large indexes. We size this against your actual filter and throughput patterns rather than a benchmark.
Yes, and it is far cheaper to add at two than at nine. The gateway is where auth, quota, logging, failover and provider swaps live. Adding it later means touching every application that already hardcoded a provider SDK, and reconstructing an audit trail that was never captured.
That is the default. Platform work that requires re-platforming defeats its own purpose. We build on the cloud, identity provider, orchestration and observability stack you already run, and treat anything that forces a migration as a cost to be justified rather than assumed.
First by checking whether the workload needs a GPU at all — a meaningful share of production inference is CPU-servable, and CPU-optimised stateless containers scale horizontally without GPU scheduling complexity. Where GPUs are genuinely required, the choice between dedicated, time-sliced and MIG-partitioned depends on isolation requirements and burst profile. Chargeback comes first regardless, because unattributed spend never falls.