Archive for the ‘Forecasting Process’ Category

Demand Volatility vs Forecast Accuracy

Saturday, December 19th, 2009
Forecast accuracy and Demand volatility are two different things, though there is a strong relationship between accuracy and volatility in practice.  For the purposes of this discussion, I want to use just common man terminology, since quite a few “statisticians” here are so wedded to terminology and easily get excited.
Demand volatility does NOT necessarily mean the demand is unpredictable.  Demand volatility also does not mean the demand is an ugly scatter of points across the map.
For example, the calculated demand volatility for a product could be the result of a highly seasonal demand profile.  Such highly seasonal demand can look very pretty on a graph and could be easily predictable by what I call as an extrapolation by hand-drawing.
In summary, volatility is not necessarily bad – growth causes volatility, so do seasonal and cyclical swings.  There could be other reasons – promotions and phase in, phase out, bonus packs, price actions, just plain old management actions to make the quarter.
Corporate management can try to stay away from some sub-optimal business practices and hence cause a reduction in demand volatility.  They can promote so customers may buy through out the year.  But can we temper seasonality and growth?  Do we want to?
Life is essentially seasonal and we have to live with it.  Christmas does not occur every month of the year!  People like to go to the beach on sunny days, and more sunny days occur during the summer.
Trevor hits it right on when he says Forecast accuracy is an expression of how well one can predict the actual demand – volatile or not.  He can create an accurate forecast from a volatile demand distribution.
How ever there is another distribution that the forecaster needs to consider.  The distribution of the actual outcomes around the forecast — if the forecaster can say his forecast has at most 5% upside and a 5% downside, then hats off!  However if the plausible outcomes are widely varying, then more needs to be done to lower the expected error:
-  Model and forecast at the right level and use proportioning techniques
-  introduce more magic in modeling (however you do it, software or otherwise)
-  collect that market intelligence only foretellers can come up with
-  discipline your corporate management to reduce demand volatility (hopefully without losing your job)
-  S&OP process so one hand knows what the other hand is doing and reduce the bias in the forecast coming from functional groups.
The last point is more important.  More seasonal products suffer from more game playing before the season, so do scarce promotional packs.  Similarly, long lead-time products also cause more game playing.  You can expect more  upward forecast bias in these products as well.

There was a post on one of the Linked-in Groups that got my attention today.  How do you reduce demand volatility using better forecasting?

Forecast accuracy and Demand volatility are two different things, though there is a strong relationship between accuracy and volatility in practice.  For the purposes of this discussion, I want to use just common man terminology.

Demand volatility does NOT necessarily mean the demand is unpredictable.  Demand volatility also does not mean the demand is an ugly scatter of points across the map.

For example, the calculated demand volatility for a product could be the result of a highly seasonal demand profile.  Such highly seasonal demand can look very pretty on a graph and could be easily predictable by what I call as an extrapolation by hand-drawing.

In summary, volatility is not necessarily bad – growth causes volatility, so do seasonal and cyclical swings.  There could be other reasons – promotions and phase in, phase out, bonus packs, price actions, just plain old management actions to make the quarter.

Corporate management can try to stay away from some sub-optimal business practices and hence cause a reduction in demand volatility.  They can promote so customers may buy through out the year.  But can we temper seasonality and growth?  Do we want to?

Life is essentially seasonal and we have to live with it.  Christmas does not occur every month of the year!  People like to go to the beach on sunny days, and more sunny days occur during the summer.

Forecast accuracy is an expression of how well one can predict the actual demand – volatile or not.  You may be able to create a very accurate forecast from a volatile demand distribution.   This can be done by good modeling and diligent forecasting.

How ever there is another distribution that the forecaster needs to consider.  The distribution of the actual outcomes around the forecast — if the forecaster can say his forecast has at most 5% upside and a 5% downside, then hats off!  However if the plausible outcomes are widely varying, then more needs to be done to lower the expected error:

-  Model and forecast at the right level and use proportioning techniques

-  introduce more magic in modeling (however you do it, software or otherwise)

-  collect that market intelligence only foretellers can come up with

-  discipline your corporate management to reduce demand volatility (hopefully without losing your job)

-  S&OP process so one hand knows what the other hand is doing and reduce the bias in the forecast coming from functional groups.

The last point is more important.  More seasonal products suffer from more game playing before the season, so do scarce promotional packs.  Similarly, long lead-time products also cause more game playing.  You can expect more  upward forecast bias in these products as well.

In conclusion, improving forecast accuracy involves doing a variety of things and reducing or mitigating volatility is just one of those things.

Forecast Alerts

Thursday, August 20th, 2009

Popular software packages have forecast alert monitors. Each software may term it differently:

  1. Manugistics calls them lists
  2. APO and Dematra call them as alerts

An overworked demand planner typically plans in the order of 1000+ skus each month.  That is where these alerts come in handy.  You set a pre-determined alert based on the condition meeting a threshold……. Voila!  As soon as the month-end job runs and re-creates the models, the alerts kick in and do their job.

You get a report, perhaps emailed to you at night comprised of all those items that exceeded your threshold.  Then your job becomes a tad easier.  You walk in to the office with your day planned out thinking to yourself:

“I am going to address those 45 SKUs that show a MAPE above 30% first, then address the next 100 that are above 20%”.

Typically the alerts are set using a comparison of new model values to the in-sample historical values.  In essence, they address the model fit issue and alert you to specific situations where the models are insufficient or just plain wrong.  Quite possibly, the history may have an anamoly as well.

Now ironically I have seen many planners complain that they are getting too many alerts from the software making them practically not very useful.  One planner mentioned that his system was spitting out 9000 alerts for a total of 1000 skus?!

This only underscores the importance of thresholds and correct definitions.  If you note this cardinal rule, then you will be well on your way to leveraging the exception capabilities of your software:

  1. Automated alert measures can only be set for relative performance meausres – MPE, MAPE, or RMSE relative to Average demand, not for absolute measures such as MAD.
  2. The settings are just as good as your understanding of the business – the thresholds have to be reasonable in comparison to historical standards.

Also remember that Alerts can be used for another important monthly activity:  comparing the difference between two successive forecast runs.  DemandPlanning.Net calls it the Delta ratio.  Perhaps you are interested in all items where the forecast delta is 25% or above!

Until next time,

Mark C