Forecast Accuracy and Bias for Imported SKUs: A Practical Review Method

Measure an imported-SKU forecast against the version that actually informed the decision—not a revised forecast saved after the result became visible. Compare like-for-like periods and units, calculate both absolute error and signed bias, then investigate the SKUs and events that created operational risk.

One percentage cannot tell the full story. Absolute error shows the size of the miss. Bias shows its direction. You need both because repeated over-forecasting can build excess inventory while repeated under-forecasting can leave purchase plans short, even when errors cancel at a portfolio total.

Freeze the forecast that the business actually used

Store the forecast version, creation date, horizon, unit, SKU, location and owner at the decision cut-off. If purchasing used a forecast issued on 1 May for August demand, that is the version to evaluate when August closes. A 15 July update may be useful for execution, but it must not replace the earlier commitment forecast in the accuracy record.

Keep separate fields for:

• forecast issue date;
• demand period;
• SKU and location or channel;
• forecast units;
• forecast method or version;
• approved override;
• decision that consumed the forecast; and
• actuals-close date.

The demand forecasting inputs guide explains how to prepare a baseline and keep overrides visible. This article starts after that forecast has been frozen.

Make actuals comparable before calculating error

Define what “actual” means. Units ordered, units shipped, units invoiced and units sold can produce different answers. Use the measure that matches the forecast's stated target, at the same time bucket and level of aggregation.

Before calculating, flag:

• stockouts or partial availability;
• listing outages or closed channels;
• cancelled or returned orders;
• promotions not represented in the frozen forecast;
• one-off bulk orders;
• SKU substitutions or code changes; and
• incomplete actuals.

If the forecast target was unconstrained demand but the actual field contains sales, a stockout may leave the review observing sales rather than all demand. Preserve the observed sales, flag the period and do not present that metric as unconstrained-demand error. Do not manufacture a replacement actual merely to improve the metric.

Use absolute error and signed bias together

This article uses one sign convention throughout:

forecast error = actual − forecast

Positive error means actual demand exceeded forecast: under-forecasting. Negative error means forecast exceeded actual demand: over-forecasting. Some systems use the opposite sign, so record the convention in the dashboard and data dictionary.

Amazon Forecast documents WAPE, MAPE, MASE and RMSE as different ways to evaluate forecast error. It notes that WAPE is undefined when the sum of observed values is approximately zero, and that RMSE gives larger misses more influence because errors are squared. Metric choice depends on the business question.

WAPE for portfolio-scale magnitude

WAPE adds the absolute SKU-period errors before dividing by total actual volume. This prevents positive and negative errors cancelling in the numerator.

Use the ratio as an error measure. Avoid automatically labelling 100% − WAPE as accuracy: WAPE can exceed 100%, and the subtraction can create a negative “accuracy” label that is harder to interpret than the underlying error.

Bias for direction

Bias retains the signs. With this article's convention, a positive ratio means the group was under-forecast; a negative ratio means it was over-forecast. A result near zero does not prove good forecasting. Large positive and negative SKU errors may have cancelled.

MAE for unit-scale interpretation

MAE expresses the average absolute miss in the same units as the demand record. It is useful within comparable SKUs or time buckets. It becomes less informative when one portfolio mixes products with radically different volumes.

Worked example: three SKU-periods

The figures below are fictional. All three rows use the same closed period and frozen forecast version.

For this fictional set:

• bias = (300 − 320) ÷ 300 = −6.7%;
• WAPE = (10 + 10 + 20) ÷ 300 = 13.3%; and
• MAE = (10 + 10 + 20) ÷ 3 = 13.3 units.

The bias says the group was over-forecast overall. WAPE shows that absolute misses were larger than the net error because A and B partly cancelled. The metrics diagnose the record; they do not reveal why the misses occurred.

Do not let aggregate totals cancel SKU errors

If total forecast and total actual are both 1,000 units, aggregate bias is zero. Yet one SKU might be 200 units over and another 200 under. The cash, stock and service consequences can be materially different.

Review at three levels:

1. portfolio or category, to see total magnitude and direction;
2. SKU or SKU-location, to identify offsetting misses; and
3. decision segment, such as high-value, long-lead-time or constrained items.

Use the same aggregation rule period to period. Do not change from SKU-location to product-family totals because the detailed result is uncomfortable.

Treat zero demand and stockouts by metric

MAPE divides each period's error by that period's actual. An actual of zero makes that row's percentage undefined; very small actuals can make a small unit miss look extreme. WAPE avoids row-by-row division. A zero-actual row can remain in its absolute-error numerator when total actual across the evaluated group is non-zero. AWS notes that WAPE itself is undefined when the total observed value is approximately zero. The ratio bias used here also needs a non-zero total actual denominator.

Apply a metric-specific rule rather than forcing or hiding a number:

• zero actual row — no row-level MAPE; retain absolute error in aggregate WAPE;
• zero aggregate actual — hold WAPE and ratio bias; report absolute error units;
• launch — no comparable history;
• stockout — flag sales versus unconstrained-demand target;
• discontinued — forecast governance review; or
• data incomplete — hold calculation.

Report how many rows were excluded from each metric and why. A clean-looking percentage that silently drops a zero-actual miss is not a governed measure.

Segment the review by decision risk

Not every miss deserves the same attention. Segment the portfolio using a declared business rule, then show the metric and major exceptions within each segment.

Useful segmentation dimensions include:

• landed inventory value;
• replenishment lead time;
• supplier MOQ or pack constraint;
• shelf-life or obsolescence exposure;
• service criticality;
• gross error contribution; and
• lifecycle status.

The ABC inventory and cycle-count plan shows how to declare a ranking criterion rather than assuming “A” always means the same thing. Do not weight by cost or priority without naming the weight and explaining the decision it supports.

Turn exceptions into process changes

For the largest errors, add a cause code only after reviewing evidence. Separate demand, supply and data causes.

Do not change safety stock automatically because WAPE worsened. The reorder-point and safety-stock process needs demand and lead-time evidence plus a declared service decision. Forecast measurement is one input, not the whole policy.

Run one governed monthly review

A compact monthly record should include:

• frozen forecast version and actuals cut-off;
• coverage: SKUs included, excluded and missing;
• WAPE, signed bias and MAE at the declared levels;
• top absolute-error contributors;
• repeated same-sign bias;
• stockout, launch and data-quality exceptions;
• owner and due date for each corrective action; and
• the date the change will be tested.

Preserve the prior result after a method change. If the sign convention, hierarchy, horizon or actuals definition changes, version the metric and explain the break. Use improved measurement to sharpen future economic order quantity and purchase decisions—not to rewrite the forecast history.

Sources

• AWS — Evaluating Predictor Accuracy
• AWS Prescriptive Guidance — Forecast error