Posts Tagged ‘Modeling’

Forecast Modeling capabilities in SAP APO vs. other statistical tools

Saturday, May 29th, 2010

I will take SAS software and Forecastpro as benchmark tools here to compare the capabilities of APO DP.

Between SAS and APO DP, the difference is huge. You have an array of statistical models to choose from in SAS while APO DP, has just a few limited set of models. Even among the models available in APO, the optimization and convergence of model selection to result in the parameters are some what inefficient and does not compare to even inexpensive off-the-shelf products like Forecast Pro or Autobox.

For univariate forecasting, you can use Exponential smoothing models, intervention models, Box-Jenkins models, distributed lag models, vector autoregression models etc. SAS allows you to model using a variety of techniques and allows you to customize them, although you require deep knowledge in statistics and programming. It is not a plug and play tool.

APO DP offers the basic exponential smoothing models and linear regression model that use a deterministic time index as the independent variable. The exponential smoothing models work for most purposes but they are not the most efficient – for example the iterations of the smoothing parameters namely alpha beta and gamma are in increments of .5 not anything in between.

And some configurations I have seen and versions I have seen can be quite sticky and may just resort to one model with all parameters set to .3 or some user-defined selection. This is also quite surprising because it assumes the user already knows what the parameter is – from where? By doing the modeling in an external statistical tool and sticking them back in?! But this is not APO’s fault – mostly the configuration team that did not know how to leverage the power of the tool.

SAS also can be used for generating transfer function models when you need to do true causal models. The only option available in APO is to use the MLR models using a multi-variate profile and then inserting your events in a linear model. This may actually produce incorrect results if there is auto-correlation in your data, which is the most likely probability since most sales data will be serially auto-correlated.

So in summary, SAP has some usable statistical models but one has to be careful in choosing what to use where. You need a demand planning expert to set the profile settings and the model selection process and train your users on what to do and what not to do.

For additional information on APO DP modeling, please see

http://demandplanning.net/sapAPO_workshop.htm

Demand Volatility vs Forecast Accuracy

Saturday, December 19th, 2009
Forecast accuracy and Demand volatility are two different things, though there is a strong relationship between accuracy and volatility in practice.  For the purposes of this discussion, I want to use just common man terminology, since quite a few “statisticians” here are so wedded to terminology and easily get excited.
Demand volatility does NOT necessarily mean the demand is unpredictable.  Demand volatility also does not mean the demand is an ugly scatter of points across the map.
For example, the calculated demand volatility for a product could be the result of a highly seasonal demand profile.  Such highly seasonal demand can look very pretty on a graph and could be easily predictable by what I call as an extrapolation by hand-drawing.
In summary, volatility is not necessarily bad – growth causes volatility, so do seasonal and cyclical swings.  There could be other reasons – promotions and phase in, phase out, bonus packs, price actions, just plain old management actions to make the quarter.
Corporate management can try to stay away from some sub-optimal business practices and hence cause a reduction in demand volatility.  They can promote so customers may buy through out the year.  But can we temper seasonality and growth?  Do we want to?
Life is essentially seasonal and we have to live with it.  Christmas does not occur every month of the year!  People like to go to the beach on sunny days, and more sunny days occur during the summer.
Trevor hits it right on when he says Forecast accuracy is an expression of how well one can predict the actual demand – volatile or not.  He can create an accurate forecast from a volatile demand distribution.
How ever there is another distribution that the forecaster needs to consider.  The distribution of the actual outcomes around the forecast — if the forecaster can say his forecast has at most 5% upside and a 5% downside, then hats off!  However if the plausible outcomes are widely varying, then more needs to be done to lower the expected error:
-  Model and forecast at the right level and use proportioning techniques
-  introduce more magic in modeling (however you do it, software or otherwise)
-  collect that market intelligence only foretellers can come up with
-  discipline your corporate management to reduce demand volatility (hopefully without losing your job)
-  S&OP process so one hand knows what the other hand is doing and reduce the bias in the forecast coming from functional groups.
The last point is more important.  More seasonal products suffer from more game playing before the season, so do scarce promotional packs.  Similarly, long lead-time products also cause more game playing.  You can expect more  upward forecast bias in these products as well.

There was a post on one of the Linked-in Groups that got my attention today.  How do you reduce demand volatility using better forecasting?

Forecast accuracy and Demand volatility are two different things, though there is a strong relationship between accuracy and volatility in practice.  For the purposes of this discussion, I want to use just common man terminology.

Demand volatility does NOT necessarily mean the demand is unpredictable.  Demand volatility also does not mean the demand is an ugly scatter of points across the map.

For example, the calculated demand volatility for a product could be the result of a highly seasonal demand profile.  Such highly seasonal demand can look very pretty on a graph and could be easily predictable by what I call as an extrapolation by hand-drawing.

In summary, volatility is not necessarily bad – growth causes volatility, so do seasonal and cyclical swings.  There could be other reasons – promotions and phase in, phase out, bonus packs, price actions, just plain old management actions to make the quarter.

Corporate management can try to stay away from some sub-optimal business practices and hence cause a reduction in demand volatility.  They can promote so customers may buy through out the year.  But can we temper seasonality and growth?  Do we want to?

Life is essentially seasonal and we have to live with it.  Christmas does not occur every month of the year!  People like to go to the beach on sunny days, and more sunny days occur during the summer.

Forecast accuracy is an expression of how well one can predict the actual demand – volatile or not.  You may be able to create a very accurate forecast from a volatile demand distribution.   This can be done by good modeling and diligent forecasting.

How ever there is another distribution that the forecaster needs to consider.  The distribution of the actual outcomes around the forecast — if the forecaster can say his forecast has at most 5% upside and a 5% downside, then hats off!  However if the plausible outcomes are widely varying, then more needs to be done to lower the expected error:

-  Model and forecast at the right level and use proportioning techniques

-  introduce more magic in modeling (however you do it, software or otherwise)

-  collect that market intelligence only foretellers can come up with

-  discipline your corporate management to reduce demand volatility (hopefully without losing your job)

-  S&OP process so one hand knows what the other hand is doing and reduce the bias in the forecast coming from functional groups.

The last point is more important.  More seasonal products suffer from more game playing before the season, so do scarce promotional packs.  Similarly, long lead-time products also cause more game playing.  You can expect more  upward forecast bias in these products as well.

In conclusion, improving forecast accuracy involves doing a variety of things and reducing or mitigating volatility is just one of those things.

Too Big to Fail? Too small to save? Too inefficient to work with? Too old school to educate?

Tuesday, November 17th, 2009

Policy makers hang on to the idea, or more like fear, that it is too risky to let big banks fail. There is a lot of talk and surmise that Lehman Brothers failure may have caused some of the precipitation last year. Not for sure.

This fear on the part of the policy makers has created a put option or a gamble of sorts for Wall Street. This TBTF policy has created loads of easy money. The printing press is running non-stop and the Fed is flying helicopters all over the country and sprinkling greenbacks (or may be just on the backyards of the big banks).

V-Shaped Recovery

V-Shaped Recovery

With essentially zero-cost money, the likes of Goldman Sachs, are now taking bigger risks in the markets and other asset investments. Savers faced with zero yield on their deposits are also pushing to take on more speculative risks and risky investments.

If this TBTF policy is going to cripple us, then why let anything be too big? The Big ones think of it as a right to come hat-in-hand, at the slightest problem that they suspect is systemic. If the Government and the Fed have the obligation to save the Big ones when in trouble, do they also earn the right to NOT let them become too big? Sounds fair enough.

W-Shaped Recovery

W-Shaped Recovery

Perhaps policy makers of the 21st century need to think about a TBTF premium and collect monies from financial companies that are beyond an acceptable size. Perhaps policy makers can collect this insurance premium or levy on a tiered basis that increases with the size. Maybe they also have a say on how big they can become.

The reluctance of the Fed to mop up the excess liquidity and the lack of any developing financial regulation are puzzling facts.

* Will this lead to Financial crisis 2?
* Is America disadvantaged by the sliding dollar?
* How will it affect your plans and forecasts?

Is the recovery V-shaped or W-Shaped?

If you are c-Level executive, you need to wonder about these issues in your intermediate and long-term planning process.

Excellent participation in the Forecasting Tutorial

Tuesday, November 3rd, 2009

The forecasting workshop on Oct 22 and 23 held at Whippany, NJ was well attended by companies from different industries from Consumer goods, to medical devices to Technology companies. We had participants from

  • SMSC
  • J&J
  • Tools Group
  • Bush Brothers
  • Merck
  • Crabtree and Evelyn
  • Avon Products
  • Niles Audio

Thanks to the great audience, the workshop was very interactive with people talking about their real world planning experiences and comparing notes on how they dealt with specific situations.  A number of questions were addressed during this interactive forum.  I truly enjoyed the participation and nothing better than conducting a workshop with an engaging audience.  Thank you.

The major focus on the first day was forecasting in the current economy.  There was a lot of discussion on the V-shaped recovery and how to forecast for it.  Although exponential smoothing is an adaptive technique that normally catches up with a lag, it is difficult when the demand suddenly drops and then sharply recovers a few months later.  We all discussed the importance of scenario planning and other techniques that are important like leading indicators.

The guest speakers Mark Temkin and Jay Nearnberg also provided valuable insights to the group.  There was discussion on inventory optimization, S&OP and demand metrics.  There were number of questions on modeling and metrics, particularly the usage of MAPE and the methodology to compute the MAPE.  The weighted Mean Absolute Percent Error seemed to be the most common performance metric used by most organizations, although some had used a variation of it.

Please feel free to post any follow-up questions on this workshop either here or in our Linked-In Group for DemandPlanning Net Training.

Mark Chockalingam

Woburn, MA

November 3, 2009.

SAP APO Demand Planning – usability versus functionality

Wednesday, September 2nd, 2009
The complexity and scale of the data and functionality actually drowns out the requirement for business process mapping and leveraging the functionality to meet the business needs. So the key ingredient of building the bridge between what the users want versus what the tool can do is perhaps left out in the cold – either due to the hectic project management schedules or due to the lack of understanding by IT staff/Integrators etc.
I think this is more than just “selling to management” – I am sure management will love to buy the concept of increasing the usability of any gadget/tool. Nothing worse than creating shelfware that is trashed after a few years and few million dollars down.
In many implementations, the business process blue printing is either poorly done or not done at all. May be it is in the project plan, but this task is marked complete sheerly by a group of tech consultants mapping the old database to the new system or just the same inefficient process into a system.
I agree with the fact that the statistical knowledge is lacking from both the user representation as well as from the consulting resources. For example, I have found many consultants still struggle with questions in implementations:
1. How does one manipulate the trend profiles to adjust model trend?
2. What is the difference between using the ex-post forecast vs. Median?
3. Why does a moving average forecast model produces a forecast that rarely moves?
4. How much history should one have to create usable models?
5. When does one use Croston’s models?
and a host of other simple questions that can be resolved with an expert user/consultant that understands the business of demandplanning and the math behind the models.
Finally, there are some short comings and challenges with the tool itself, just like any other tool. Consultants and Integrators either do NOT know what these are or just afraid to acknowledge these. It is important to recognize this, so they can help the users navigate around these pitfalls to improve usability of the tool.
regards,
Mark
http://www.demandplanning.net

I have heard from Consultants and business managers across many companies: Very rarely there is user adoption of standard APO forecasting techniques.  Users typically use an off-line forecasting process and enter the forecast into the planning book or even upload it with IT help.  Why is it?

Are the standard models and algorithms not sufficient for the business models?  Or is it becuase the users and IT staff do not understand the statistics and functionality behind APO?

Let me address some possible reasons for this unfortunate situation.

The complexity and scale of the data and functionality actually drowns out the requirement for business process mapping and leveraging the functionality to meet the business needs.  So the key ingredient of building the bridge between what the users want versus what the tool can do is perhaps left out in the cold – either due to the hectic project management schedules or due to the lack of understanding by IT staff/Integrators etc.

I think this is more than just “selling to management” – I am sure management will love to buy the concept of increasing the usability of any gadget/tool. Nothing worse than creating shelfware that is trashed after a few years and few million dollars down.

In many implementations, the business process blue printing is either poorly done or not done at all. May be it is in the project plan, but this task is marked complete sheerly by a group of tech consultants mapping the old database to the new system or just the same inefficient process into a system.  Although we all acknowledge that the business process need to be mapped, re-designed and documented before the system implementation begins, very rarely this is funded by corporate managers as a worthy effort.  They like the tool and they want the tool……..

There is also this other possibility that the statistical knowledge is lacking from both the user representation as well as from the consulting resources. For example, I have found many implementation consultants still struggle with questions in implementations:

1. How does one manipulate the trend profiles to adjust trend?

2. What is the difference between using the ex-post forecast vs. Median?

3. Why does a moving average forecast model produces a forecast that rarely moves?

4. How much history should one need to create usable models?

5. When does one use Croston’s models?

and a host of other simple questions that can be resolved with an expert user/consultant that understands the business of demandplanning and the math behind the models.

Finally, there are some short comings and challenges with the tool itself, just like any other tool. Consultants and Integrators either do NOT know what these are or just afraid to acknowledge these. It is important to recognize this, so they can help the users navigate around these pitfalls to improve usability of the tool.

Please read http://demandplanning.net/solutions.htm to understand what can be done to improve the tool.  Some of the stuff is simple and perhaps will mean consulting costs in the order of 5% of your total budget to implement the tool in the first place.

Can we use Multiple Linear Regressions to model sales promotions?

Wednesday, July 29th, 2009

The simplest approach to model promotions is through interventions. This is also called event modeling in the literature. We collect a list of events and classify into types of events by the intended effect or by the sensitivity on the volume. For example, retailer tabs may have a 10% lift on volume while an FSI may have a 20% lift.

Then we create different indicator variables and use them as independent variables to do the volume forecast. In some of the tools, you can specifically calculate the volume lift by event type.

In our training workshops we have used Forecastpro and Excel to illustrate event modeling. Excel takes a little bit more effort to set up these models. Ideally you want to use event models with either ARIMA models or using Exponential smoothing models with events.

If you want to learn more on event modeling in demand planning, you may want to join us for the DP Web workshop to be held on Friday July 31 at 10AM EST.

For more information, see

http://demandplanning.net/workshops

The mystery of Moving Averages

Friday, July 10th, 2009

A user in an APO forum lamented:

“I am trying to use the Moving Average option in APO DP but I get a static forecast that remains constant for all the future months based on the history.  I would like the statistical forecast to be a moving target not a constant.  Not sure how to achieve this?!”
“All I get is the same statistical forecast volume for all 24 months. This is not want we are looking for.  The system is giving me a forecast like this:  month1 – 915, month2 – 915,… month 24 – 915″
“But I want the system to give me forecast that is different each month: 
Future: month1 – 915, month2 – 902,… month 24 – 908.”

Before we go too far with the micro details of these numbers, let us first realize some qualities of a good statistical forecast. 

One of the important qualities of a forecast is robustness. Robustness means the forecast does not change like a yo-yo just based on new historical data point.

So the objective is not to mimic the history but to produce a forecast that minimizes the error. Theoretically, Moving averages should produce a constant forecast into the future at least after the first two periods.  The idea of the moving average is that it will change by at least one third of the impact of the new observation that is different from the
forecast.
Other than this, let us understand the difference in error in what is being proposed here.  A forecast that is 915 each month is off by 1% from another forecast that varies between 902 and 915 over the entire forecast horizon. 

A contrived model that looks fancier with oscillations in results between 902 and 915 perhaps can be 1% better on average than another model that proposes to use a constant 915 every month. I don’t think the trade-off to improve forecast quality by 1% is worth the model search to come up with a more complex model that mimics
the history better.

The fact that you are choosing moving average means that the data series is relatively more stable. We as planners should let moving average do its job and move on to more complex items that need your attention – items that
have a persistent trend or seasonality or both.
Trying to fit a holt-winters model when the series begs you for a constant model is NOT a good use of time. Note that SAP APO classifies moving average under constant models.  In fact, specifying that you want Holt-Winters models in such a scenario will give you a First Order smoothing model which again gives you constant forecasts into the horizon. 
If you want to learn more about error metrics, you may be interested in http://www.demandplanning.net/apoMetrics_webworkshops.htm.