Machine Learning
A model that scores well offline and does nothing online is the most common outcome in enterprise machine learning. The gap is rarely the algorithm. It is leakage in validation, skew between training and serving, and the absence of an experiment design that could have detected the problem before launch.
Why offline metrics mislead
Three failure modes account for most of it. Leakage: a feature that is available at training time but not at prediction time, or a random split on data with temporal structure, both of which inflate offline scores in ways that vanish in production. Training-serving skew: the feature computed in the training pipeline is subtly not the feature computed at inference — a different null handling, a different aggregation window — and the model receives inputs it never saw. Feedback loops: a recommender trained on clicks it caused learns to reinforce its own past behaviour, and offline accuracy against logged data measures agreement with the old system rather than value to the user.
The three calls that decide the outcome.
Made explicitly, with the trade-off written down, before anything gets built.
Gradient-boosted trees or deep learning
For tabular enterprise data, gradient-boosted trees remain the correct default: stronger on typical dataset sizes, far cheaper to train and serve, and considerably easier to explain to a risk function. Deep models earn their place on text, images, audio and genuinely large interaction data. Choosing a neural architecture for a tabular problem usually buys longer training cycles and a harder conversation with compliance in exchange for no measurable lift.
Batch scoring or real-time inference
Real-time inference is assumed far more often than it is needed. If the decision is consumed by a daily process, batch scoring removes latency budgets, online feature infrastructure and a large class of failure modes at once. Real-time is justified when the features themselves are only knowable at request time. The honest question is not how fast the model can respond but how fresh the decision genuinely has to be.
How the model gets evaluated online
Offline metrics select candidates; they do not establish value. That requires an online design — a holdout, an A/B test, or an interleaving experiment for ranking — specified before launch, with the metric and the minimum detectable effect agreed by whoever owns the business outcome. Deciding how to measure success after a model is live reliably produces a debate about attribution instead of an answer.
The capabilities we bring on day one.
Each engagement assembles from this menu, sized to your scope, paced to your calendar.
Honest validation
Temporal splits, leakage checks and baselines that make the real lift visible rather than flattering.
Training-serving parity
Features computed once and reused, so the model sees at inference exactly what it saw in training.
Online experiment design
The measurement agreed with the business owner before launch, not reconstructed afterwards.
Documented by default
Model cards, data lineage and impact testing produced during the build, not retrofitted.
Models that make decisions about people
Where a model influences hiring, credit, insurance, education or access to essential services, it is likely to fall inside the EU AI Act's high-risk tier, whose obligations have applied since 2 August 2026 — and classification follows the decision being made, not the sophistication of the model. That brings documentation, human oversight, logging and post-market monitoring requirements into scope at design time. We produce model documentation as a build artefact rather than a retrospective exercise, and test for disparate impact across affected groups before deployment rather than after a complaint.
From kickoff to live in four phases.
Each phase has named deliverables, named owners and a named gate, and every one respects the systems you already have in place.
Discover
Problem framing against the decision the model will actually inform, data audit, and a baseline that any model has to beat to be worth deploying.
Design
Feature strategy, validation design that respects temporal structure, model selection, and the online evaluation plan agreed with the outcome owner.
Build
Feature and training pipelines built for reuse at inference, model iteration against honest validation, and integration into the consuming system.
Run & hand off
Monitoring on inputs and outputs, retraining triggers, documentation, and enablement so your team owns the retraining cycle.
What a good fit looks like.
Stated up front, so neither of us spends a call finding out this was the wrong conversation.
Engagement shape
A scoped sprint of roughly eight weeks to a validated model integrated into one real decision path, with the online experiment running.
From your side
Historical data with enough time depth to validate temporally, a named owner for the business outcome, and access to the system that will consume the prediction.
When not to hire us
If the decision you want to automate has no historical record of past decisions and outcomes, there is nothing to learn from yet. Instrument the process first; the model comes later.
Machine Learning questions, answered.
It depends far more on the number of positive examples and the stability of the process than on total row count. A few thousand well-labelled examples of a stable phenomenon often beat millions of rows spanning a process that changed twice. We assess this in discovery and will tell you if the honest answer is that there is not enough signal yet.
Most often leakage or training-serving skew. Leakage means the validation used information that would not exist at prediction time — including a random split on data with temporal structure. Skew means the feature computed at inference differs subtly from the one computed in training. Both inflate offline scores and both are detectable before launch if you look for them.
No. For tabular enterprise data, gradient-boosted trees are usually stronger, cheaper to serve and much easier to explain to a risk function. Deep models are the right choice for text, images, audio and large interaction data. We pick on evidence rather than on what is fashionable.
Through an online experiment designed before launch — a holdout, A/B test or interleaving design, with the success metric and minimum detectable effect agreed by whoever owns the business outcome. Offline metrics select candidates; only the online design establishes value, and deciding how to measure it after launch produces an attribution argument rather than an answer.