Reproducibility

Why your forecast changes every time you run it.

You rerun last month's forecast on the exact same data and get a different number. Nothing changed, yet the answer moved. That is not a quirk. It is the single reason finance stops trusting the plan, and it has a fix.

Why does my demand forecast change when I rerun it on the same data?

Because many planning engines are not deterministic at runtime. Machine-learning and probabilistic models retrain, reseed or reorder data on each run, so identical inputs can return a slightly different number. The data did not change; the model's internal state did. That gap, same inputs and a changed output, is a reproducibility failure, and it is more common than vendors admit.

Is it normal for a forecast to change on the same data?

A forecast should change when the inputs or the model version change, and that change should be traceable. It should not change when the inputs and the model version are identical. Legitimate, recorded change is healthy; silent drift on identical inputs is the problem, because it means nobody can reproduce or defend the number a month later. We are not saying every tool whose output ever moves is untrustworthy; we are saying identical inputs should give an identical, reproducible result, with every change on the record.

Why does forecast reproducibility matter?

Because a number you cannot reproduce is a number you cannot defend, to a CFO, to an auditor, or to yourself when you are trying to tell whether a miss came from the plan or from the market. Research on demand forecasting makes the same point: significant run-to-run variability on identical inputs erodes planner trust and forces excessive manual overrides. That is why we argue reproducibility comes before accuracy. Get the same number first; then make it sharper. A precise number you cannot repeat is worth less than a solid one you can.

How do you make a demand forecast reproducible?

Treat the forecasting configuration like software. Version the methods, parameters and hierarchy rules, fix the model version, and keep the decision layer, safety stock, reorder points and cash, strictly deterministic. Then the same data and settings always return the same number, with every change recorded. That is exactly how DemandIntel is built: machine learning sharpens the forecast, then every decision runs deterministically, so you get the intelligence without the drift. Run it today and next quarter on the same data and the number is identical, and planning and execution can finally share it.

The test

Rerun it. Does the number hold?

Not reproducible

The slot machine

  • Same data, different number
  • No recorded reason for the change
  • Planners override on instinct
  • Finance quietly distrusts the plan
  • A miss cannot be traced
Reproducible

The glass box

  • Same data, same number, every time
  • Every change versioned and dated
  • Planners trust what they can repeat
  • Finance can defend the figure
  • Audit-ready months later
The payoff

One number that holds

  • Fewer overrides, less firefighting
  • Cash decisions you can defend
  • Faster, calmer S&OP
  • Adoption, because people believe it
  • Accuracy built on a stable base
Questions

Common questions

How do I test whether my forecast is reproducible? +

Rerun last month's forecast on the exact same data and settings. If the number is not identical, you have a reproducibility gap. Two more tests: can you explain the figure to a CFO in two minutes, and is every override logged with who, when and why.

Does deterministic mean no AI? +

No. Machine learning still sharpens the forecast. What stays deterministic is the decision, so the part that moves your money is the part you can reproduce and explain.

Is reproducibility the same as accuracy? +

No. Reproducibility is getting the same number from the same inputs; accuracy is being close to reality. You need both, but reproducibility comes first: you cannot reliably improve a number you cannot repeat.

Value in an hour, not a year

Rerun it on your own data in 45 minutes.

Bring last month's numbers. We will run them, run them again, and show you the same figure both times, traced from the headline to its parts. If it does not hold up, we part as friends.