Once a forecast enters procurement or scheduling, the deliverable changes from an estimate of future demand to a commitment of resources today. That change in use alters acceptance criteria. This is an enduring methodological discussion, not a claim that probabilistic forecasting is new. All numerical examples are constructed; no enterprise deployment experiment was performed.
Define the loss carried by a decision before comparing forecast errors
Underpredicting by ten units might require an extra delivery next shift, or it might leave an entire process waiting. Ten excess units might sell next week, or expire tonight. A metric that treats both deviations as ten units describes predictive distance, but may not describe allocation quality. Technical leaders should have procurement, planning and data teams approve the target, permitted actions and loss definitions on the same acceptance record.
Probabilistic forecasting extends a single output into quantiles or a distribution. Amazon Forecast documents quantile loss with different penalties for underprediction and overprediction.[2] We use that methodological definition, not a product recommendation. The engineering opportunity is to let a decision layer choose an appropriate risk position rather than giving every department one supposedly optimal point estimate.
A suitable mean forecast may suffice for a budget concerned with average demand. Continuously adjustable replenishment may also make a one-off inventory model too simple. Distributional outputs and calibration maintenance are most defensible when resources must be committed in advance, deviation costs are describable, and adequate historical evidence exists.
P90 is a demand quantile, not a 90% unit fill rate
P90 locates roughly 90% of the modeled probability at or below a demand value; for discrete demand, cumulative probability at that value can exceed 90%. It does not mean this particular prediction is 90% correct, or guarantee exactly nine shortage-free cycles out of ten. Outcomes must still establish whether the modeled distribution resembles reality. P50 is the median and need not equal the mean in a skewed distribution.
Forecasting: Principles and Practice distinguishes point forecasts from prediction intervals and describes quantile scores and scores penalizing both interval width and observations outside the interval.[1] A very wide interval may cover more outcomes without helping a planner. Evaluate coverage together with usefulness, rather than accepting one coverage percentage.
Separate the proportion of cycles without a shortage from the proportion of demanded units supplied. A cycle short by one unit and a cycle short by one hundred both count as one shortage cycle. Procurement must specify the relevant objective; relabeling P90 as a 90% unit fill rate does not establish a service commitment.
A constructed example: changing costs changes the appropriate quantile
Consider one advance stocking decision serving a single period, with no replenishment within that period. Let Co be the net loss per excess unit and Cu the loss per missing unit, both linear in quantity. MIT's newsvendor teaching material describes the corresponding trade-off: choose the smallest stocking quantity whose cumulative demand probability reaches Cu/(Cu+Co).[3] This rule depends on those assumptions; it is not universal inventory policy.
Construct demand of 80 units with probability 0.2, 100 with probability 0.5, and 140 with probability 0.3. With Cu=9 and Co=3 currency units per item, the target quantile is 0.75. Cumulative probability is 0.7 at 100 and 1 at 140, so the unconstrained choice is 140. Stocking 100 produces expected deviation loss of 0.2×20×3 + 0.3×40×9 = 120; stocking 140 produces 0.2×60×3 + 0.5×40×3 = 96.
Mean demand here is 108. Stocking that amount produces expected loss of 0.2×28×3 + 0.5×8×3 + 0.3×32×9 = 115.2. The mean remains meaningful, but does not minimize the specified asymmetric loss. If Cu becomes 3 and Co becomes 9, the target quantile becomes 0.25 and the choice becomes 100. An unchanged demand distribution does not justify retaining a policy after its losses change.
This is arithmetic under a stipulated distribution, not evidence that a model can learn it. Avoid double counting costs: do not add revenue to an opportunity loss that already includes the relevant profit; account for recovery or later use of surplus. When production interruption or broken commitments cannot be priced credibly, use explicit constraints and human decisions rather than forcing every consequence into one monetary value.
Evaluate probability quality separately from action value
Maintain two linked records. A forecast record contains issue time, item, target period, input version and quantiles. A decision record contains loss parameters, available inventory, incoming stock, capacity restrictions, final order quantity and reasons for human changes. This separates an incorrect distribution from a reasonable forecast used by an unsuitable policy, and makes operational overrides traceable.
First check data and timing. Demand is not necessarily sales censored by limited stock. Unfulfilled demand during a stockout may be unobserved, so learning sales directly can understate demand. Identify stockouts, cancellations and missing channel data, and disclose unrecoverable gaps. Apply the data-availability approach to time-series backtesting to historical inputs and retraining schedules, so later corrections cannot improve past predictions.
Second check probability outputs. For each lead time, product group and important operating condition, compare outcomes with quantile thresholds and report sample counts, quantile loss and interval width. Overall coverage near its nominal level does not prove calibration in every group; fluctuations in sparse groups do not automatically establish model failure. Fix evaluation periods and grouping rules beforehand, and do not treat serially correlated cycles as independent evidence.
Third replay policies. Compare the current rule, a simple baseline and the proposed policy at the same historical origins with identical data and execution constraints. Measure missing units, leftover units, delays and total loss. Tune on development periods and retain a final period untouched by selection. Missing or late predictions must use the actual fallback policy in the evaluation rather than disappear from the reported sample.
Specify five boundaries before turning quantiles into orders
First, define the target horizon. If procurement covers total demand over a delivery lead time, predict that total. Adding daily P90 values does not generally produce the period's P90. Dependence between days affects the total distribution; model it directly or construct joint scenarios retaining temporal dependence, then validate the aggregate separately.
Second, define feasible actions. Pack sizes, minimum orders, storage capacity, shared budgets and supplier limits can prevent direct execution of a quantile rule. Generate candidate actions and compare feasibility and loss. Shared constraints across products require joint allocation. Random lead times, repeated replenishment opportunities or shortages that alter future demand call for an appropriate dynamic policy model.
Third, establish cost credibility. Vary Cu and Co within business-approved ranges and check whether recommendations change sharply. If a small cost change moves an order from one case to several, show the sensitivity range. When actions have similar estimated losses, an easier-to-operate choice may be preferable to overstating a tiny numerical advantage.
Fourth, assign maintenance responsibility: who updates costs, interprets promotion and new-product conditions, and investigates coverage changes? Fifth, define stopping conditions. Missing stockout information, crossing quantiles, persistent miscalibration in critical groups or changed execution constraints should trigger the agreed baseline and further evidence gathering. Start with a few traceable decisions, establish benefit across the complete process, and only then expand automated replenishment.