Machine Learning Models: The Production Gap

You know how it goes. A data science team spends six months building a model. It works beautifully in the demo. Stakeholders are impressed. Someone uses the word “revolutionary” in a slide deck. And then… nothing happens. The model sits in a notebook. Months pass. The team moves on to the next experiment. The model never reaches a single real user. This is how most machine learning models never reach production — quietly, without anyone deciding to kill them.

This is not a rare story. According to a widely cited VentureBeat analysis, 87% of machine learning models never reach production. And while the exact number varies depending on how you define “production” and who you ask, the pattern it describes is real and widely recognized across the industry. McKinsey’s State of AI research found that only about 5.5% of companies see more than 5% of their EBIT attributable to AI. A 2025 S&P Global survey found that 42% of companies had abandoned most of their AI initiatives before going live — up from 17% the year before.

So the question worth asking is not “is the 87% accurate?” It’s “why does this keep happening, and what does it actually tell us?”

The answer is almost never “the model was bad.” Most machine learning models never reach production because nobody built the production system around the model. The data science team did their job. The infrastructure, ownership, and operational layer around them simply didn’t exist.

A model in a notebook is like an engine sitting on a workbench. It runs. It performs. You can measure it. But it’s not a car. Production ML is the whole car: the fuel system, the dashboard, the brakes, the maintenance schedule, the insurance, the person who drives it, and the road it needs to travel. Most teams celebrate the engine and forget to build the rest.

Have machine learning models stuck in notebooks? Iterators helps teams turn AI experiments into production systems with MLOps, ETL pipelines, monitoring, and scalable infrastructure. If your models work in a demo but never reach users, the missing piece is usually the production layer — and that’s exactly what we build.

Why Machine Learning Models Never Reach Production: What 87% Really Means

Before getting into the why, it’s worth being precise about what production means for a machine learning model — because “production” gets used loosely, and that vagueness is part of the problem.

A machine learning model is in production when it is deployed into a live workflow, receives validated real-world data, generates predictions for real users or business processes, is monitored for quality and drift, and can be retrained, rolled back, or improved safely.

That’s a very different thing from a working demo.

A demo uses curated historical data, runs manually, and lives on one data scientist’s laptop. It proves the concept. It does not prove the system. Production ML means a fraud model is actually blocking transactions. A churn model is actually triggering retention workflows. An invoice classification model is actually routing tasks in a finance team’s daily process. A recommendation system is actually personalizing what real users see.

The gap between those two things — working demo and working production system — is where most ML projects die. And it’s not a small gap. According to Google’s research on Hidden Technical Debt in Machine Learning Systems, the actual ML code in a production system represents roughly 5% of the total codebase. The other 95% is everything around it: data pipelines, configuration, feature extraction, monitoring, logging, serving infrastructure, and integration glue.

“It is remarkably easy to incur massive ongoing maintenance costs at the system level when applying machine learning.” — D. Sculley et al., Google, Hidden Technical Debt in Machine Learning Systems

That’s the thing nobody tells you when the demo goes well. And it’s why the debate about why machine learning models never reach production so often blames the wrong team.

The ML Project Failure Rate Is Not Just a Data Science Problem

Here’s the most common failure pattern in ML implementation, and it plays out in organizations of every size:

A data scientist builds a model. It performs well on historical data. The demo impresses stakeholders. Someone asks when it can go live. Engineering gets involved and discovers there are no automated pipelines, the data sources are undocumented, there are security gaps, compliance hasn’t been consulted, and nobody allocated budget for the production layer. The model sits in “pilot purgatory” for months. Eventually, the team moves on.

The data scientist gets blamed. But the data scientist did exactly what they were hired to do — build a model. They were never given the infrastructure, the team, or the mandate to turn it into a working software system.

This is the handoff problem. And it’s fundamentally an organizational problem, not a technical one.

Production ML is less about data science alone and more about cross-functional software engineering.

It requires:

• Data scientists to optimize the model
• Data engineers to build and maintain the pipelines that feed the model
• DevOps or platform engineers to deploy, containerize, and serve the model
• Product managers to define the workflow the model fits into and the metric that proves it worked
• Security and compliance teams to define what’s allowed and what needs to be audited
• Business owners to connect the model’s output to an actual decision or action

When any one of those groups is missing, the project stalls. And in most organizations, the data scientist is the only one in the room.

The ML project failure rate conversation tends to focus on data quality or model accuracy. Those matter. But the deeper issue is that data science creates the model, and someone else entirely has to turn it into a system. If that “someone else” was never defined, funded, or included from the beginning, the model was always going to end up in a notebook.

Why Machine Learning Models Never Reach Production After a Good Demo

The demo is the trap. It’s where more machine learning models stall than anywhere else — the moment everyone celebrates the engine and assumes the car is basically done. It isn’t.

A Notebook Is Not a Production ML System

It helps to be concrete about the difference, because the gap is wider than it looks.

The notebook answers the question “can this work?” Production ML answers the question “does this work, reliably, for real users, day after day?”

One of the most illustrative failure modes here is what’s called training-serving skew — when the model performs differently in production than it did in testing. This happens because notebooks often use historical data that was manually cleaned and carefully prepared. In production, the data arrives messy, incomplete, and in real time. The model was never tested on the actual conditions it would face.

A related problem: time travel in feature engineering. If a dataset joins historical features in a way that accidentally exposes the model to future information during training — say, a chargeback count that was calculated at midnight gets attached to a transaction that happened at 9 AM — the model learns from a signal that won’t exist in real time. It looks brilliant in testing. It falls apart in production. And because there’s no monitoring, nobody notices for months.

These are not exotic edge cases. They’re extremely common. And they’re almost impossible to catch without the kind of automated validation and testing infrastructure that most teams skip because they’re focused on getting the model right, not the system around it.

![Comparison of a data science notebook and a production machine learning system, notebook on the left with historical data, manual execution, one data scientist and no monitoring, production system on the right with live data, automated pipeline, cross-functional ownership and monitoring and retraining][notebook_vs_production] Alt text: Comparison of a data science notebook and a production machine learning system.

Stakeholders Confuse Model Accuracy With Business Readiness

Here’s the uncomfortable part: a model can be accurate and still completely useless. Accuracy is a lab result. Business readiness is a different test entirely.

A model that scores 94% in a notebook can still fail in production if:

• It is too slow to fit inside the workflow it’s supposed to support.
• It is too expensive to run at the volume the business needs.
• It cannot explain its predictions to the people who have to act on them.
• It cannot handle edge cases that never appeared in the training data.
• It requires manual data prep before every run.
• Users don’t trust it, so they quietly ignore it.
• It doesn’t connect to a real workflow, so nobody acts on the output.

Every one of those is a production problem, not a modeling problem. And every one of them can kill machine learning models that looked perfect in the demo.

MLOps: Why ML Fails Without Infrastructure and Operations

MLOps is not a tool category or a trend. It’s the operating model that makes machine learning work as software rather than as research. Think of it as the operating system for production ML — and solid MLOps implementation is what separates a model that ships from a model that stalls.

Why Data Pipelines Matter More Than Algorithms

The first month of an AI implementation project is often not “building AI.” It’s cleaning up the mess that prevents AI from working.

Bad data doesn’t just produce bad predictions. It produces confident wrong predictions — which is worse. A model trained on dirty data learns the wrong patterns, passes accuracy tests because the test data is dirty too, and then quietly fails in production where the real-world data looks different.

Common data problems that block ML deployment (getting a model into a live workflow) include:

• Missing values that aren’t random — they signal something (a new user, a failed API call, a data gap) that the model misinterprets
• Inconsistent schemas across data sources — the same field means different things in different systems
• Undocumented data sources — nobody knows where a column came from or whether it’s still being updated
• Features that default to zero when a data service is unavailable — the model learns “zero means safe” and then breaks when a database timeout makes active users look like new ones

The fix is not a better algorithm. It’s data cleaning before ML deployment, basically checking the data before the model trusts it: automated validation that catches bad inputs, schema checks that make sure fields look the way they should, null-rate monitoring that watches how often values go missing, data ownership documentation, and access controls that ensure the training data and the serving data are actually the same thing.

The choice of data architecture for machine learning matters here too. Where features are stored, how they’re versioned, and whether the same feature definitions are used in training and inference — these are infrastructure decisions that have a direct impact on whether the model works in production.

A feature store — a centralized repository that serves consistent feature values for both training and real-time inference — is one of the most effective ways to close the training-serving gap. It’s also one of the most commonly skipped components in early ML implementations, because it feels like overhead until the model starts failing and you trace it back to inconsistent feature definitions.

CI/CD for ML Is Different From CI/CD for Software

The Google Cloud MLOps architecture describes three levels of pipeline automation, from fully manual (someone runs the training script by hand) to continuous training and automated release pipelines (the system detects drift, retrains, evaluates, and deploys without human intervention). Most organizations are at level zero or one. Production-grade ML requires level two or three.

CI/CD, the automated path that tests and ships changes, is different for machine learning than it is for regular software. Traditional software behavior depends mostly on code. ML system behavior depends on code, model weights, training data, feature definitions, the training process itself, and real-world feedback. A deployment pipeline for ML has to test all of those things — not just whether the code compiles, but whether the model still performs after retraining, whether the data schema matches what the model expects, and whether the serving infrastructure can handle the load.

Model registries (version history for models) and dataset versioning (version history for training data) are not optional. If you can’t reproduce a training run, you can’t debug a production failure. If you can’t roll back to a previous model version, a bad deployment can take down a critical workflow with no recovery path. These aren’t nice-to-haves. They’re the equivalent of version control for code — basic infrastructure that production systems require.

Monitoring Machine Learning Models Is Where Data Science ROI Survives or Dies

Deployment is not the finish line. It’s the starting line for a different set of problems.

Here’s something that doesn’t get talked about enough: machine learning models don’t fail loudly. They degrade quietly.

Unlike a software bug that throws an error and breaks a workflow, a model experiencing drift just becomes gradually less useful. Predictions get worse. Business outcomes slip. Nobody connects it to the model because the model is still running, still returning results, still technically “working.”

This is what happened to mobility models during COVID. Before 2020, ride-sharing demand models were trained on years of stable commuting patterns. When lockdowns hit, the data changed dramatically — overnight. Machine learning models trained on the past became less useful for predicting the present. Without monitoring, teams wouldn’t have known the model was quietly failing until the business outcomes made it obvious. By then, months of bad predictions had already compounded.

As the Iterators team puts it: if you’re not using new data, you’re learning to survive the past.

There are two types of drift to watch for:

Data drift (covariate shift) happens when the distribution of input features changes, even if the relationship between inputs and outputs stays the same. User behavior shifts. New customer segments appear. Seasonal patterns change. The model was trained on a world that no longer exists.

Concept drift happens when the relationship between inputs and outputs changes. A credit scoring model trained before an interest rate spike may have learned patterns that no longer predict default risk accurately. The inputs look the same. The world they represent has changed.

Production ML monitoring needs to track both — using statistical tests on feature distributions, prediction output shifts, and business outcome metrics. Not just model accuracy on a held-out test set, but real-world performance on real decisions. Google’s SRE guidance on monitoring distributed systems applies here directly: you want a small number of meaningful signals — latency, errors, saturation, and in ML’s case drift — that page a human before users feel the damage.

The AWS MLOps foundation roadmap recommends treating monitoring as a first-class engineering concern, not an afterthought. That means dashboards, alerting thresholds, automated retraining triggers, and a rollback plan before the model goes live — not after the first production incident.

Data Science ROI: Why Machine Learning Models Never Reach Production Without Business Metrics

This is the conversation that tends to go badly in executive reviews.

Accuracy Is Not ROI for Machine Learning Models

A data science team presents a model with 94% accuracy. The business asks: “What does that mean for us?” The data science team says: “It’s very accurate.” The business says: “But what does it do?”

Model metrics and business metrics are not the same thing:

The business metrics that actually matter: hours saved, fraud losses reduced, revenue recovered, churn reduced, support tickets avoided, analyst throughput increased, cost per decision lowered. These are the numbers that connect ML implementation to budget decisions and continued investment.

Tie ML Implementation to a Workflow Before Training Begins

Before a model goes into training, the team should be able to answer: who uses this prediction? What decision changes because of it? What action happens next? How fast does the prediction need to be? What happens when the model is wrong? Who can override it? What metric proves it worked?

If those questions don’t have answers, the model has no production path — regardless of how accurate it is.

Data Science ROI Needs a Feedback Loop

The feedback loop matters here too. A recommendation system needs click, purchase, and ignore data to improve. A fraud system needs confirmed fraud labels. A support classification system needs human correction data. A forecasting system needs actual-versus-predicted tracking. ML ROI doesn’t just require deployment — it requires a mechanism for the system to learn from outcomes. Without that, the model is frozen at the accuracy it had on launch day, degrading quietly while the world changes around it.

For more on connecting ML outputs to business metrics tracking, the framework matters as much as the model.

Machine Learning Production Checklist: What Has to Exist Before Launch

This is the checklist that most teams skip because they’re focused on the model. It’s also the checklist that determines whether the model ever does anything useful.

Before a machine learning model goes into production, the team needs:

• A clear business owner who can define success
• A clear technical owner who is responsible for the system after deployment
• A defined production workflow — where does the prediction go, and what happens next
• Validated data sources with documented ownership and access controls
• An automated data pipeline with validation checks
• A model registry for versioning and reproducibility
• Versioned datasets so training runs can be reproduced
• A deployment environment that matches production conditions
• A monitoring dashboard tracking predictions, latency, drift, and business outcomes
• Drift detection with alerting thresholds
• A rollback plan that has been tested
• A security review and compliance sign-off
• A human escalation path for when the model is wrong or uncertain
• An ROI measurement plan tied to business metrics
• A retraining schedule or automated retraining trigger

None of this is exotic. It’s the equivalent of what any serious software product requires before going live. The reason it gets skipped in ML projects is that the model feels like the product. It’s not. The model is a component. The system is the product.

Batch vs Real-Time Machine Learning Production

Not every model needs to answer in milliseconds, and pretending otherwise is a fast way to overbuild infrastructure you don’t need.

Batch inference — scoring data on a schedule — is good for:

• Nightly scoring jobs
• Churn predictions
• Demand forecasting
• Periodic recommendations
• Back-office prioritization and queue ranking

Real-time inference — scoring a request the moment it arrives — is needed for:

• Fraud detection at the point of transaction
• Personalization as a user browses
• Dynamic pricing
• Matching algorithms
• AI assistants
• Risk scoring inside a live decision
• Operational decisioning

The distinction matters because it drives everything downstream: serving architecture, cost, latency budgets, and monitoring. A batch pipeline that runs at 2 AM has very different requirements from a real-time endpoint that has to respond in 50 milliseconds under load. Choosing the wrong one either wastes money or breaks the workflow.

Security, Governance, and Audit Logs Are Not Optional

The NIST AI Risk Management Framework provides a useful structure here — not as a compliance checkbox, but as a way to think about the full lifecycle of an AI system: Govern, Map, Measure, Manage. The governance and mapping work — who owns this, what data does it touch, what are the failure modes, who can override it — should happen before training begins, not after the model is ready to deploy.

SOC 2 compliance for SaaS and AI systems is increasingly a prerequisite for enterprise ML deployment, not an optional certification. SOC 2, GDPR, the EU AI Act, and HIPAA where healthcare data is involved all shape what a production ML system is allowed to do. Audit logs, access controls, environment separation, encrypted data pipelines, anonymization, and model explainability where required — these are not bureaucratic overhead. They’re what enterprise customers ask for before they’ll let a model touch their workflows.

Why Machine Learning Models Never Reach Production in Enterprise Environments

In enterprise environments, the production gap gets wider for reasons that have nothing to do with data science quality.

Pilots get funded as research projects. Production requires platform engineering. Those are different budget lines, different teams, and often different approval processes. Machine learning models can survive six months of experimentation funding and then die waiting for a production infrastructure budget that was never allocated.

Add to that: procurement timelines, security reviews, data residency requirements, legacy system integrations, fragmented data ownership across business units, compliance reviews that weren’t included in the original project scope, siloed data, slow access approvals, and internal sponsors who moved on before the model was ready to deploy.

The point is blunt: enterprise ML failure often happens because the pilot is funded as research, but production requires platform engineering. Until someone budgets for the second thing, the first thing just produces impressive demos.

How to Stop Machine Learning Models From Never Reaching Production

Enough diagnosis. Here’s the fix.

Build the Production Path Before Choosing Machine Learning Models

Before anyone argues about model architecture, the team should define the data sources, the deployment environment, the user workflow, the monitoring approach, the retraining trigger, the security requirements, and the business KPIs. The model is the last decision, not the first. Teams that pick the algorithm before the production path have the whole thing backwards.

Treat MLOps as Spike-Shaped Work

Here’s a practical observation from building these systems: the MLOps workload is usually not constant. It’s spike-shaped.

At the start, you need a strong engineering push to build the ETL flows, the deployment architecture, the model registry, the monitoring, the inference endpoints, and the retraining pipelines. That’s heavy, concentrated work. Then, once the plumbing exists, the workload drops. The team spends its time collecting data, watching the monitors, and improving its machine learning models incrementally.

This is why staffing production ML like a permanent, fixed headcount problem often gets it wrong. You need serious capacity up front and lighter, steady capacity afterward. A flexible partner who can bring the heavy engineering push when you need it — and step back when you don’t — usually fits the shape of the work better than a permanent team sized for the peak.

Know When Custom MLOps Beats Off-the-Shelf Tools

There are good off-the-shelf MLOps tools. MLflow for model registries. Feast for feature stores. Various managed inference platforms. For standard use cases, buying components makes sense.

But tools don’t solve:

• Messy business logic that doesn’t fit standard abstractions
• Legacy system integrations that require custom connectors
• Compliance requirements that demand specific audit trails or data residency controls
• Real-time inference constraints that off-the-shelf serving layers can’t meet
• Domain-specific feedback loops that require custom labeling workflows

The question isn’t “buy or build?” It’s “which parts of this system are standard, and which parts are specific to how our business actually works?” Iterators doesn’t just advise teams to “use MLOps.” We help build the systems that make ML operational: ETL, infrastructure, deployment, observability, and feedback loops.

Iterators Case Lens: From ML Experiment to Production System

Theory is cheap. Here’s what moving from experiment to production looks like in practice, drawn from Iterators’ work.

Citrine Informatics — MLOps for Materials Informatics

Citrine runs a materials informatics platform where data scientists build machine learning models to accelerate materials discovery. The challenge was operational, not scientific: notebooks needed to move into a scalable, versioned, deployable environment. Iterators supported an MLOps platform oriented toward data scientists — scalable notebook deployment and versioning, plus Scala developer support — so the science team could ship and iterate without fighting infrastructure. The result was materially improved efficiency for the data science team.

SCHWARTZ.AI — AI Invoice Analysis Prototype

SCHWARTZ.AI was a corporate innovation pilot: an AI model paired with a web platform to analyze cost and purchase invoices. The work combined the model with UX workshops and usability testing, because a prediction nobody acts on is worth nothing. The outcome was reduced workload in cost and purchase invoice analysis and a measurably more efficient workflow — the model tied to an actual business process rather than a demo.

Bondspoke — LLM-Based Bond Scoring

Bondspoke works in sustainable fixed-income investing, where analysts drown in unstructured financial documents. Iterators helped build an LLM-based system that automatically extracts and scores information from those documents. It saved analysts hundreds of hours and, more importantly, turned scarce domain expertise into a production-grade AI workflow — the kind of system that keeps delivering value after the pilot ends.

30/60/90-Day Plan to Get Machine Learning Models Into Production

For teams that have machine learning models sitting in notebooks and want to move them toward production, the work is sequenced — not all at once.

Days 1–30: Audit Where Machine Learning Models Get Stuck

Inventory every model, notebook, and prototype in the organization. For each one, identify who the business owner is, where the data comes from, whether the data sources are documented and validated, what compliance constraints apply, and what metric would prove the model is working. Identify the deployment blockers honestly. Then prioritize one model for production — the one with the clearest business owner, the cleanest data, and the most direct connection to a workflow that already exists.

Days 31–60: Build the Minimum Production Path

Create an automated ETL pipeline, basically the route that pulls data from source systems, cleans it, and sends it where the model needs it. Add validation checks. Set up a model registry. Containerize the model serving layer. Create a staging environment that mirrors production conditions. Add CI/CD so that retraining and redeployment are automated, not manual. Define the rollback process before it’s needed. Build basic monitoring — prediction volume, latency, and at least one business outcome metric.

Days 61–90: Launch, Monitor, and Improve

Deploy to limited production. Run a shadow deployment or A/B test so the model’s predictions can be compared to existing processes before full rollout. Monitor predictions against business outcomes. Collect human feedback and correction data. Tune retraining triggers based on observed drift patterns. Improve observability metrics and adoption tracking. Then decide — based on actual production evidence — whether to scale, pause, or redesign.

The Question Worth Asking Before the Next AI Pilot

If your organization is about to fund another AI initiative, there’s one question worth asking before the kickoff meeting: what is the production path for this model, and who owns it?

Not “what’s the accuracy target?” Not “what data do we have?” Those matter, but they’re downstream of the more fundamental question. If there’s no answer to “who builds the system around the model, and what does that system look like?” — the model will probably join the 87%.

The production gap is not a data science problem. It’s a software engineering problem, an organizational problem, and a product problem. Data scientists who are blamed for ML project failure rates are usually being blamed for a system they were never equipped to build alone.

The teams that get machine learning models into production treat ML implementation the same way they treat any serious software product: with infrastructure, ownership, monitoring, and a direct connection to a business outcome that someone is accountable for. That’s not a higher bar. It’s just the actual bar for building something that works.

If you want to build an AI software solution that reaches production and stays there, the conversation starts with the system, not the model. And if you want to understand what machine learning applications across industries actually look like when they work, the pattern is consistent: production infrastructure first, model optimization second.

The engine on the workbench is impressive. But nobody drives a workbench.

FAQ: Why Machine Learning Models Never Reach Production

Why do 87% of machine learning models never reach production? Because many teams build machine learning models without the infrastructure, data pipelines, monitoring, ownership, and business workflows needed to deploy and maintain them. The model is usually fine. The production system around it was never built.

What does machine learning production mean? It means the model is running in a real workflow, using live data, producing decisions or recommendations for real users or processes, and being monitored for quality, drift, cost, and business impact — with the ability to retrain or roll back safely.

Is MLOps the same as DevOps? No. MLOps borrows heavily from DevOps but adds model versioning, dataset versioning, feature management, drift monitoring, retraining, and ML-specific governance. Software behavior depends mostly on code; ML behavior also depends on data, features, and the training process.

Why do ML demos fail after the pilot stage? Demos often use curated data and manual processes. Production systems need automation, security, observability, rollback, and integration with real business workflows — none of which the demo had to prove.

How do you measure data science ROI? Measure business outcomes such as revenue gained, hours saved, churn reduced, fraud prevented, analyst productivity, or cost per decision — not just model accuracy. Accuracy is a lab result; ROI is a business result.

What infrastructure is required for production ML? Typical production ML infrastructure includes ETL pipelines, a model registry, CI/CD, a feature store, a serving layer, monitoring dashboards, logging, access control, and retraining workflows.

What is model drift? Model drift happens when real-world data changes and the model’s predictions become less accurate or less useful over time. Data drift is a shift in the inputs; concept drift is a shift in the relationship between inputs and outputs.

Should companies build or buy MLOps tools? Buy standard components where possible, but build custom workflows when you have unique data, legacy systems, compliance requirements, or domain-specific feedback loops. The real question is which parts of the system are standard and which are specific to your business.

If your machine learning models are stuck in notebooks, the problem may not be your data science team. It may be the missing production layer around them. Iterators helps teams build the MLOps pipelines, ETL systems, monitoring, and scalable infrastructure needed to turn AI experiments into working software. Let’s get your models out of pilot purgatory.