Data science teams are increasingly asked to help estimate software projects: a churn prediction platform, a document intelligence tool, a recommendation engine, an internal analytics product. And those estimates are often wrong in the same direction. The model turns out to be the cheap part. Everything around it is where the budget goes.

This article looks at why data-heavy software is so hard to estimate, which cost drivers are most often missed, and how data teams can contribute to more realistic budgets.

The model is the smallest part of the system

In 2015, a group of Google engineers published “Hidden Technical Debt in Machine Learning Systems,” a paper that has become required reading for anyone building production ML. Its central illustration shows a small box labeled “ML Code” surrounded by much larger boxes: data collection, feature extraction, data verification, configuration, serving infrastructure, monitoring, and more.

The point still holds more than a decade later. A working notebook proves that a signal exists in the data. It does not prove that the organization can collect that data reliably, serve predictions at the required latency, detect when the model degrades, or pass a security review. Each of those is an engineering project in its own right, and each carries its own cost.

When a stakeholder asks “how long will the model take,” the more useful question is “how long will the system take.”

Seven cost drivers data teams should flag early

1. Data readiness

Most projects begin with an assumption that the data exists and is usable. That assumption is rarely tested before the budget is approved.

Common surprises include missing history, inconsistent identifiers across systems, undocumented business logic buried in legacy databases, and fields whose meaning changed over time. Migrating and cleaning years of records from an old system can be a substantial project in itself, separate from building anything new.

A short data audit before estimation, covering volume, completeness, quality, access, and ownership, is one of the highest-return activities in the entire project.

2. Data pipelines and integrations

Production systems need data to arrive continuously, not as a one-off export. Every source system adds work: authentication, schema mapping, error handling, retries, backfills, and monitoring. Sources owned by third parties add a dependency you do not control, including API rate limits, sandbox access delays, and unannounced schema changes.

Integration work is consistently underestimated because it looks simple on an architecture diagram. In practice, each connection is a small product with its own failure modes.

3. Labeling and ground truth

Supervised models need labels. If labels do not exist, someone has to create them, and that someone is often a domain expert whose time is expensive and limited. Labeling also needs guidelines, quality checks, and inter-annotator agreement measurement, all of which take time to design.

For some problems, the cost of getting reliable ground truth exceeds the cost of building the model.

4. MLOps and model lifecycle

A model in production needs versioning, reproducible training, automated evaluation, deployment pipelines, rollback, and drift monitoring. Without these, every retraining becomes a manual, risky event, and the team slowly loses the ability to explain why the model behaves the way it does.

MLOps is not an optional extra to add later. Retrofitting it onto a live system is usually more expensive than building it in from the start.

5. Inference and infrastructure costs

Training costs get attention. Inference costs often do not, even though they recur for as long as the system runs. Real-time predictions, GPU-backed services, large language model API calls, and vector databases all generate ongoing spend that scales with usage.

For LLM-based features in particular, the cost per request multiplied by projected volume should be part of the estimate, not a discovery after launch.

6. Compliance and governance

Data-intensive systems in healthcare, finance, and other regulated sectors carry extra requirements: access controls, audit trails, data residency, retention policies, and, increasingly, model documentation and explainability. Frameworks such as HIPAA, GDPR, and the EU AI Act can shape architecture decisions from day one.

Compliance tends to act as a multiplier on the whole project rather than a single line item, because it affects how every component is designed, tested, and documented.

7. Evaluation and iteration

Model performance is not known in advance. A realistic plan includes time for baseline experiments, error analysis, and several iteration cycles, plus the possibility that the first approach will not meet the target. Fixed-scope estimates that assume the first model will work are optimistic by design.

Summary: where the budget actually goes

How data teams can improve estimates

Data scientists are in a strong position to make software estimates more realistic, because they understand where uncertainty lives. A few practical habits help:

Separate discovery from delivery. Propose a short, fixed discovery phase that covers the data audit, a baseline model, and an architecture outline. Estimate the full build only after that phase, when the biggest unknowns have been reduced.

Estimate in ranges, not points. A range with stated assumptions (“if historical data covers at least three years with consistent IDs”) is more honest and more useful than a single number.

Make recurring costs visible. Present one-time build costs and monthly running costs separately, including inference, storage, monitoring, and retraining.

Name the dependencies. List every external system, data owner, and approval the project relies on. Each one is a schedule risk.

Benchmark against real projects. Published breakdowns of custom software development cost are useful reference points for sanity-checking the non-ML parts of the build, such as integrations, compliance, and team rates, which data teams may have less experience pricing.

A quick pre-estimate checklist

Before committing to a budget for any data-intensive project, confirm that you can answer these questions:

• Where does each required data source live, and who owns it?
• How much history exists, and has anyone checked its quality?
• Do labels exist, and if not, who will create them?
• What latency and throughput does the system need in production?
• What will inference cost per month at expected volume?
• Which regulations apply, and have compliance stakeholders reviewed the design?
• What happens if the first modeling approach does not meet the target?

If several of these remain open, the project needs a discovery phase before it needs a budget.

Final thoughts

Data-intensive software rarely fails because the model was too hard to build. It struggles because the surrounding system, data, pipelines, infrastructure, and governance, was treated as an afterthought in the estimate.

Data teams that bring these cost drivers into the conversation early do more than improve accuracy. They build credibility with business stakeholders and give their projects a much better chance of reaching production on a budget everyone agreed to at the start.