The Forecaster’s Luck by Ger Koole

 I am lucky not to be a forecaster… because they really have a difficult job. They have to differentiate between the effects of many factors, such as day-of-the-week, week-of-the-year, long-term trend, marketing events – without being able to tell exactly what the effect of each factor is. The reason for this is twofold: factors change over time, depending on the time of the year, changes in the call center or the company’s business, etc. But also because everything is obscured by a layer of noise. Think about it: if you forecast 30 calls in half an hour, then 30 customers won’t agree together to call one after another with exactly 1-minute intervals. Instead, there will be longer and shorter inter-arrival times and periods looking like clusters, like in the graph below. Mathematicians call this a Poisson process.

This noise has three important consequences. A forecaster can never do it completely right: a perfect forecast is purely luck, the noise will most of the time prevent the actuals from being exactly equal to the forecasts. To measure forecasting errors, we use the WAPE, the weighted absolute percentage error. It is the sum of the absolute interval errors divided by the total volume. For example: you forecast a volume of 100 for 4 consecutive quarters. The actuals are 90, 105, 100, 110. Then the WAPE is:

Some advanced mathematics gives a beautiful simple formula for the minimal WAPE:

In the figure below we plotted the minimal WAPE for different volumes. To give an example: for a forecast of 250 the minimal WAPE is √ 2 / (π 250) = 5%. Thus, as the minimal WAPE decreases with the volume, for a volume below 250 the accuracy of the forecast can never be below 5%!

Noise also makes forecasting more difficult. It obscures the signal hidden behind the noise. To achieve an accuracy as high as possible given the noise requires lots of data and a deep understanding of on the one hand the nature of call centers and on the other hand  advanced statistics and machine learning.

But the noise can also work in the advantage of the forecaster: the forecasting errors are, to some extent at least, obscured by the noise. When the volume is low you won’t see a small forecasting error because the noise is much bigger. Of course, you cannot hide behind the noise when it is smaller than the error you made in the other components.

This raises the question: when do you see the error through the noise? And can we find the error simply by deducting the minimal WAPE from the observed WAPE? If the FC was 250 and the WAPE 12% was the forecasting error then 12 – 5 = 7%? Unfortunately reality is not that simple. To obtain some insight look at the figure below. It shows volume on the horizontal axis and WAPE on the vertical one, for different forecasting errors plotted at the right axis. The bottom red line is the minimal WAPE: the WAPE with a 0% forecasting error. For example, when the volume is 20, then the error by the Poisson noise is also around 20%.

Now let’s look at the other lines. They represent the WAPE for different levels of FC errors – the percentages are given on the right axis. When, again, the volume is 20 but you made a 20% FC error, i.e., you predicted 16 or 24, then the WAPE is around 25%. Thus you cannot simply add error and minimal WAPE: in this case 15% of the error is absorbed by the noise. On the other hand, when volumes and/or errors are high the WAPE is equal to the error: you do not see the noise, it disappears in the error.

You can also reverse this reasoning: when the volume was 100 and the WAPE around 10%, then the error can be anything up to say the minimal WAPE thus 10%, the lines close together. But when the error is significantly higher than the minimal WAPE, then the error is close to the WAPE, you cannot blame the noise.

To conclude: despite the noise, enough data plus advanced methods and understanding can give reliable forecasting – but up to a certain level, given by the minimal WAPE. For low volumes, forecasting errors are obscured by the noise, but for higher volumes the WAPE has to be completely contributed to the forecasting errors, the noise is too small to hide behind.

Ger is full professor at Vrije Universiteit Amsterdam and founder and chief scientist at CCmath.
To find out more about CCmath and its products, see www.CCmath.com.
To learn more about the background and practice of WFM, see WFMacademy.CCmath.com
The author thanks Giuseppe Catanese for his feedback.