Forecasting Blog

In the demand forecasting tutorial in New Jersey, there was a key question on forecasting during a recession.  Forecasting is relatively easier in a boom.  As they say the Trend is your friend!  But not so during a recession, since we don’t want the trend to be persistent ==>  this would result in a worsening economic situation.

We want an inflection point for the drop to stabilize and then we expect a bottoming up and a pick up so we are back on positive trend.  However, turning points are difficult to forecast.  There was a lot of talk about the V shaped recovery and its challenges for demand planning.  The drop was sudden and the demand fell off a cliff.  Then the pick up was sudden as well.

Companies had substantial drop in their demand earlier this year but the consequent inventory depletion and the sudden pick up in demand had resulted in businesses scrambling to ramp up production.  The decision to deplete inventories earlier this year had compounded the woes.  Companies through most of this year have been suffering from customer service issues and higher costs on expediting and execution.

Forecasting is indeed difficult when things are not business as usual.  You need extra information or additional insights to call these turns correctly.  However, when that additional driver is outside of your control, then it is scenario planning that you need to count on.

Exponential smoothing will catch up with the new trend with a time lag.  The first few forecasts will be way off.  This depends on the smoothing parameter in the model.

We have now settled that MAPE or Mean Absolute Percent Error is the measure for forecast performance for a planner/Division at the end of the month.  It is more popularly used as a cross-sectional measure across multiple items and products to come up with one metric to denote forecast performance.

However, what should we use to assess the quality of the statistical models?  Here I am trying to deal with the measure for diagnostics.  When you have a forecast model for a product or customer, what measure would you use to determine if the model is a good fit. 

We can use the Mean Absolute Deviation or MAD which is closely related MAPE. However, most software tools and applications propose R-squared or its derivate measure root mean squared error as the measure of choice.

The big “subtle” confusion is the definition and interpretation of MAPE.  In academics, we commonly understand MAPE is the average of the percent errors. But business planners are puzzled by this behavior of averaging percent numbers without regard to scale.  So most discussions and use of MAPE in the industry is always volume-weighted. MAPE = Weighted Mean Absolute Error percent.

So more specifically with reference to the model fit in a time series context, what is your recommendation? To rephrase the question, let us say we have come up with a Holt-Winters model that seemingly represents best the history. To assess this model and compare it with others, what should we use?

1. MAPE = An average of the percent errors of Abs(A-F)/A?
2. Root Mean Squared Error?

It appears that RMSE would be a better metric given it punishes bigger deviations more so due to the squaring of the error.  The only downside is it is not relative since it is an absolute number.  The best bet is to compare the RMSEs of different models and pick the one that is reasonably smaller without overfitting……

Welcome to the forecasting blog! 

These are collection of thoughts on forecasting, demand planning, Sales and Operations planning and corporate strategy!  Please contribute your thoughts and comments to enhance the usefulness of business forecasting around the world!