Forecast Bias over Tracking Signal

Tracking Signal is used to measure forecasting bias over time for the same SKU/Product especially when bias approaches extreme proportions and crosses a threshold. This comes from the Quality Control literature that uses control limits to valuate if a process is in control. I agree bias may be easier to understand but bias has been measured differently by different Gurus!

Again people confuse cross-sectional and calendar bias –

1. Sales people often induce cross-sectional bias. They would like to generally over-forecast all skus so there is supply available when they don’t know which skus the customer is going to order. This is a measure across all SKUs in one Period.

2. Calendar Bias is forecast bias on one SKU across many periods over time. I generally don’t believe sales people have the incentive to bias just any SKU unless it is a hot new product that is in heavy demand.

Other than clarity issues, at times the bias measure also suffers from a lack of testing for Statistical significance.

Think about a sku having forecast errors as below:

Mon1 +20%, Mon2 -20%, Mon3 14%, Mon4 -14%, Mon5 + 20%. Measuring at month 5 would show a positive bias, although statistically this is no different from zero.

Generally we advise using a T test to complement the bias measure.

Tracking signal is itself is a test of statistically significant bias. But the problem is you can do nothing with it other than to conclude that it is out-of-control bias.

But the bias measure when correctly computed and established, has more utility in planning. If you establish there is a consistent 20% upward bias, you can improve results dramatically by cutting the forecast by 20%.

So we actually prefer and recommend the forecast bias measure, although Tracking Signal is an indicator of extreme bias.

Dr. Chockalingam

Dr. Mark Chockalingam is the founder and President of Demand Planning LLC, a Business Process and Strategy Consultancy helping clients across industries: Pharmaceuticals, Consumer Products, Chemicals and Fashion Apparel. His specialty consulting areas include Sales forecasting, Supply Chain Analytics, and Sales and Operations Planning. He has conducted numerous training and strategy facilitation workshops for a variety of clients in the US and abroad. Mark has worked with a variety of companies from the Fortune 500 such as Wyeth, Miller SAB, FMC, Teva to the small and medium size companies such as Au Bon pain, Multy Industries, Ticona, a divison of Celanese AG. With significant expertise in business forecasting and modeling, he is a frequent speaker at major supply chain events on topics ranging from demand management to sales and operations planning. Prior to establishing his consulting practice, Mark has worked with manufacturing companies in important supply chain positions. Mark was Director of Market Analysis and Demand Planning for the Gillette Company, now part of Proctor and Gamble. Before Gillette, Mark led the Suncare, Footcare and OTC forecasting processes for Schering-Plough Consumer HealthCare in Memphis. Mark has a Ph. D. in Finance from Arizona State University, an MBA from the University of Toledo and is a member of the Institute of Chartered Accountants of India.

You may also like...

1 Response

  1. Ryan Bechtel says:

    Mark,

    I found this post to be very informative. I have not thought to test the statistical significance of the bias. I am familiar with t-test, but can you speak more about how you would test the statistical significance? Are you testing that BIAS 0? Is it better to test this on a monthly or weekly scale? Also should you limit the range at which you are testing the BIAS, for example a rolling a 5-6 months? I appreciate any feedback you can provide.