International Journal of Forecasting 25 (2009) 526–530 www.elsevier.com/locate/ijforecast
Incorporating a tracking signal into a state space model Ralph D. Snyder a , Anne B. Koehler b,∗ a Monash University, Australia b Miami University, United States
Abstract It is a common practice to complement a forecasting method such as simple exponential smoothing with a monitoring scheme to detect those situations where forecasts have failed to adapt to structural change. It will be suggested in this paper that the equations for simple exponential smoothing can be augmented by a common monitoring statistic to provide a method that automatically adapts to structural change without human intervention. The resulting method, which turns out to be a restricted form of damped trend corrected exponential smoothing, is compared with related methods on the annual data from the M3 competition. It is shown to be better than simple exponential smoothing and more consistent than traditional damped trend exponential smoothing. c 2008 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.
Keywords: Exponential smoothing; Monitoring forecasts; Structural change; Adjusting forecasts; State space models; Damped trend
1. Introduction A tracking signal is often used in computerized inventory systems for monitoring forecast errors to detect structural changes in demand time series. The goal of this oversight is to maintain control of a forecasting system by responding to out-ofcontrol signals that are based on the forecast errors. One widely used tracking signal was introduced by Trigg (1964) and was based in part on a smoothed forecasting error statistic (cf. Farnum & Stanton, ∗ Corresponding address: Department of Decision Sciences and MIS, Miami University, Oxford, 45056 OH, United States. Tel.: +1 513 529 4826. E-mail address:
[email protected] (A.B. Koehler).
1989). We will show that the effect of incorporating this smoothed forecasting error statistic directly into the recurrence equation of simple exponential smoothing is equivalent to using a restricted form of traditional damped trend exponential smoothing (Gardner & McKenzie, 1985). Thus, we provide an explanation for the widespread success of damped trend exponential smoothing (Fildes, Nikolopous, Crone, & Syntetos, 2008; Gardner, 2006; Hyndman, Koehler, Ord, & Snyder, 2008) in that it possesses a special capacity to adapt to structural change without direct intervention. The structure of the paper is as follows. In Section 2, we introduce the local level model, the basic state space model underlying simple
c 2008 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved. 0169-2070/$ - see front matter doi:10.1016/j.ijforecast.2008.12.003
R.D. Snyder, A.B. Koehler / International Journal of Forecasting 25 (2009) 526–530
exponential smoothing. We also define the smoothederror tracking signal and argue that it represents the average deviation in the underlying level of the time series from an ‘ideal’ level. We suggest that this deviation can be incorporated as an adjustment to the underlying level in a local level model to yield a new model that underpins the restricted version of the damped trend exponential smoothing method. In Section 3, the new restricted damped trend model is compared with simple exponential smoothing and conventional damped trend exponential smoothing in a study based on the annual time series from the M3 competition database (Makridakis & Hibon, 2000). 2. A model combining simple exponential smoothing with a tracking signal The white noise model forms a convenient starting point for our study. It takes the form yt = µ + εt , where yt is the series value in typical period t, and µ is a time invariant mean representing the global underlying level of the series. The error term εt is a normally distributed shock with a mean 0 and a constant variance σ 2 and is uncorrelated with the error terms in other time periods. The effect of a shock on the series is restricted to only one period, the one in which it occurs; it has no spillover effect on future series values. The white noise model invariably fails in business and economic applications, one of a number of reasons for this being that it ignores the impact of the structural change so often found in business and economic processes. Structural change is said to occur when a shock leads to changes to the underlying states of a process that in turn impact future values of a time series. The white noise model can be adapted to allow for structural change; the mean µ is replaced by a time dependent random variable `t called the local level. The resulting innovations state space model, called a local level model, is yt = `t−1 + εt `t = `t−1 + αεt .
(2.1) (2.2)
The one period lag on the local level in the measurement equation (2.1) is used to indicate that this random variable references the beginning of period t. The term αεt is the change in the underlying level from one period to the next. It is an underlying
527
growth rate, but one which is uncertain because it is affected by the shock. Such change has a permanent effect on future values of the time series, a feature that defines a stochastic trend (Beveridge & Nelson, 1981). In this context the underlying growth rates are normally distributed, have a common mean of zero and a variance α 2 σ 2 , and are temporally uncorrelated. The parameter α determines the magnitude of the underlying growth. When α = 0 there is no underlying growth, a situation corresponding to the white noise process. In most applications α is positive, its magnitude being a reflection of the amount of structural change in a time series. When α = 1, the model corresponds to a random walk. The local level model is estimated using simple exponential smoothing (Brown, 1959). When a time series value yt has been observed, it may be compared with the one-step-ahead prediction `t−1 , to give the error εt = yt − `t−1 . The underlying level is then updated with the recurrence equation `t = `t−1 + αεt . For a time series of length T , the seed level `0 and the parameter α, both of which determine the trajectory of the underlying levels, may be chosen minimize PT to the sum of squared errors SS E = t=1 εt2 . The point forecasts for periods beyond the prediction origin at the end of period T all equal the final underlying level `T . The structural change embedded in the local level model occurs from one period to the next. Being entirely reflected by the underlying growth which is subject to the properties mentioned above, this structural change is relatively stable. Yet a series can also be affected by episodic events that cause structural breaks disproportionate in size to the stable structural change embedded in the local level model. Although an extensive body of literature exploring the possibility of incorporating structural breaks into a model exists, the occurrence of these breaks is typically too infrequent to permit the reliable prediction of their timing and size. It makes more sense, from a practical perspective, to ignore the existence of structural breaks in the model itself, but instead to adopt measures to detect structural breaks when they occur so that commensurate adjustments can then be made to the forecasts. The smoothed-error statistic is defined recursively by the equation bt = φbt−1 + βεt ,
(2.3)
528
R.D. Snyder, A.B. Koehler / International Journal of Forecasting 25 (2009) 526–530
where β is a smoothing parameter such that φ +β = 1. This statistic is an exponentially weighted average of the errors, the weights declining back through time. It may be used in conjunction with control limits to monitor forecasts and is known as the smoothed-error tracking signal (cf. Farnum & Stanton, 1989). Under the null hypothesis that the error terms are independent and normally distributed, bt has a normal distribution p with mean 0 and standard deviation σβ/ 1 − φ 2 . The smoothed-error statistic bt , when employed as a tracking signal, has an out-of-control region defined by q |bt | > zσβ/ 1 − φ 2 (2.4) where z is the value of the standard normal variable corresponding to a specified level of significance. This tracking signal is equivalent to applying the well-known exponentially weighted moving average (EWMA) control chart (Roberts, 1959) to the error term εt . The choice of β is a challenge. If the simple stochastic trend model (i.e. simple exponential smoothing) is used for forecasting, β could be chosen to be equal to α. However, McKenzie (1978) and Gardner (1985) have provided evidence that the same smoothing parameter should not be used for both smoothing and monitoring. When distinct smoothing parameters are used, it is incumbent on us to suggest a mechanism for determining a separate value for β. It should be stressed that systematic patterns also emerge in the smoothed-error statistic when the process generating the time series changes or the model has been incorrectly specified. For example, suppose the change in the underlying level is given by b+αεt instead of just αεt , where b is a constant termed the drift. A time series under this assumption typically displays an upward drift when b > 0. So, when this new regime applies, the smoothed-error statistic has a constant upward bias which can be shown to be well approximated by the quantity b/α. Other plausible deviations from the assumptions of the local level model lead to further possible systematic patterns in the smoothed-error statistic. It may be conjectured that a more robust approach to forecasting emerges if the predictions are corrected each period by the average error, as depicted by the smoothed-error statistic bt . At the beginning of period
t, the expected value of bt is φbt−1 . The Eqs. (2.1) and (2.2) of the simple stochastic trend can be augmented by this expected change to give the new equations yt = `t−1 + φbt−1 + εt
(2.5)
`t = `t−1 + φbt−1 + αεt
(2.6)
bt = φbt−1 + βεt .
(2.7)
This is a damped local trend model, the statistical framework underpinning damped trend exponential smoothing. It differs, however, in one respect. Because the smoothed-error statistic is an exponentially weighted average, the parameters φ and β satisfy the additional restriction φ + β = 1. The new model is a restricted version of the classical damped stochastic trend model. The use of this new model enables us to resolve the problem of determining the appropriate value of β. Under the null hypothesis that there are no structural breaks, the value of β can quite simply be estimated in conjunction with the parameter α while fitting the restricted stochastic damped trend model to the time series data. It is possible to still resort to the use of simple exponential smoothing in conjunction with the smoothed error statistic to detect structural breaks, using the value of β that emerges from this fitting exercise. Such an approach would successfully detect structural breaks provided the assumptions underpinning simple exponential smoothing apply. Because it effectively adjusts the underlying level by the average error, restricted damped trend exponential smoothing should automatically adapt to many of the possible deviations from the underlying conditions that make simple exponential smoothing the appropriate approach to forecasting. On this basis, one might expect it to yield more robust forecasts than simple exponential smoothing. A number of interesting questions now emerge. 1. Is it more effective to use the restricted damped trend model than the local level model with monitoring? In general, the answer is ‘maybe’. The parameters of the restricted damped trend model are tuned to the stable structural change situation. When the prospect of a structural break is remote, it must then be better. In an unstable context, if appropriate responses are made when the out-ofcontrol situations are detected, a local level model
529
R.D. Snyder, A.B. Koehler / International Journal of Forecasting 25 (2009) 526–530
coupled with the tracking signal might be expected to perform better. 2. Second, the restricted damped trend model effectively has one less parameter than the conventional damped trend model. This makes it inherently simpler. However, does the more parsimonious model yield competitive forecasts? 3. Empirical study The second question was addressed in a study undertaken on the 645 annual time series from the M3 competition database (Makridakis & Hibon, 2000). Each series was divided into two parts: a fitting sample and an evaluation sample. The series vary in length, but the evaluation part always corresponded to the final six observed values. Three forecasting approaches were compared: simple exponential smoothing; restricted damped exponential smoothing; and conventional damped exponential smoothing. Maximum likelihood estimates of the associated parameters and seed states were obtained, the smoothing parameters and damping factors being restricted to the unit interval. Prediction performances on the forecasting horizons 1 to 6 were gauged using the mean absolute scaled error (MASE) statistic (Hyndman & Koehler, 2006) applied to the evaluation samples. Ideally, exponential smoothing with the tracking signal should have been included in the study. However, its performance depends very much on the type of interventions undertaken in response to detected out-of-bounds situations. In practice, such interventions can depend on market intelligence not already embedded in the sample, so it is not possible to reproduce such responses in a study like this. Our strategy, given this constraint, was to apply the optimized simple exponential smoothing to the fitting sample to obtain one-step ahead prediction errors, their standard deviation, and the out-of-control regions (2.4). Then a second pass was made of the fitting sample using the smoothed-error statistic (2.3), with b0 = 0 and β = 0.1, to count the frequency of occurrence of out-of-bounds events, but without interventions. About 70% of the series were classified as stable on the basis that they had five percent or less out-of-bounds events; and, because of this, the remaining 30% were classified as unstable. It might
Table 1 Median MASEs on annual M3 time series. Models
Stable
Unstable
All
Local level Damped trend (restricted) Damped trend (classical)
1.99 1.67 1.69
3.02 2.23 2.11
2.26 1.85 1.76
be anticipated that simple exponential smoothing will perform best on the stable series and that the damped trend approaches will work best on the unstable series. The results shown in Table 1 were obtained using Matlab. First, the conjecture about the local level model being best for the stable series proved to be wrong. The local level model accommodates smooth structural change in its levels, but its forecasts lagged behind on series containing trends. Second, the restricted version of damped trend exponential smoothing proved to be more reliable than its classical counterpart. The box plot in Panel A of Fig. 1 indicates that, although the classical approach has a marginally better median MASE, its distribution is more skewed, thus suggesting a lower level of reliability. This finding is reinforced by the box plot in Panel B of the percentage MASE differences 100 log (MASErestricted /MASEunrestricted ), which is skewed more towards the bottom. The single source of error state space framework of Hyndman, Koehler, Snyder, and Grose (2002) already includes the traditional damped local trend model. The results from our study indicate that this framework can be made more reliable by extending it to include the restricted form of the damped local trend model. 4. Conclusions The most significant contribution of this paper was the demonstration that a restricted form of damped trend exponential smoothing can be thought of as simple exponential smoothing with an embedded tracking signal that automatically adapts forecasts to unanticipated structural change. This helps to explain why damped trend exponential smoothing has been such a successful method of forecasting; it adapts automatically to situations that may differ in a number of potential ways from the conditions needed for simple exponential smoothing to be the optimal form of algorithm. A further interesting finding was that a restriction on the parameters of the damped local
530
R.D. Snyder, A.B. Koehler / International Journal of Forecasting 25 (2009) 526–530
Fig. 1. Box plots of results for damped local trends.
trend model can lead to a more consistent forecasting performance without seriously impeding forecasting accuracy. References Beveridge, S., & Nelson, C. R. (1981). A new approach to decomposition of economic time series into permanent and transitory components with particular attention to measurement of the business cycle. Journal of Monetary Economics, 7, 151–174. Brown, R. G. (1959). Statistical forecasting for inventory control. New York: McGraw-Hill. Farnum, N. R., & Stanton, L. W. (1989). Quantitative forecasting methods. Boston: PWS-Kent. Fildes, R., Nikolopous, K., Crone, S., & Syntetos, A. (2008). Forecasting and operational research: A review. Journal of the Operational Research Society, 59, 1150–1172. Gardner, E. S., Jr. (1985). CUSUM vs. smoothed-error forecast monitoring schemes: Some simulation results. Journal of the Operational Research Society, 36, 43–47. Gardner, E. S., Jr. (2006). Exponential smoothing: The state of the
art – part II. International Journal of Forecasting, 22, 637–666. Gardner, E. S., Jr., & McKenzie, E. (1985). Forecasting trends in time. Management Science, 31, 1237–1246. Hyndman, R. J., Koehler, A. B., Ord, J. K., & Snyder, R. D. (2008). Forecasting with exponential smoothing: The state space approach. Berlin: Springer. Hyndman, R. J., Koehler, A. B., Snyder, R. D., & Grose, S. (2002). A state space framework for automatic forecasting using exponential smoothing methods. International Journal of Forecasting, 18, 439–454. Hyndman, R. J., & Koehler, A. B. (2006). Another look at measures of forecast accuracy. International Journal of Forecasting, 22, 679–688. Makridakis, S., & Hibon, M. (2000). The M3-competition: Results, conclusions and implications. International Journal of Forecasting, 16, 451–476. McKenzie, E. (1978). The monitoring of exponentially weighted forecasts. Journal of the Operational Research Society, 29, 449–458. Roberts, S. W. (1959). Control chart tests based on geometric moving averages. Technometrics, 1, 239–250. Trigg, D. W. (1964). Monitoring a forecasting system. Operational Research Quarterly, 15, 271–274.
