Symmetric mean absolute percentage error

The symmetric mean absolute percentage error (SMAPE or sMAPE) is an accuracy measure based on percentage (or relative) errors. It is usually defined^{[ citation needed ]} as follows:

${\displaystyle {\text{SMAPE}}={\frac {2}{n}}\sum _{t=1}^{n}{\frac {\left|F_{t}-A_{t}\right|}{|A_{t}|+|F_{t}|}}}$

where ${\displaystyle A_{t}}$ are the actual values and ${\displaystyle F_{t}}$ are the forecasted values. Note that if ${\displaystyle A_{t}=F_{t}=0}$, then term ${\displaystyle t}$ is undefined (${\displaystyle 0/0}$), and is usually ignored in the summation.

Explaining this equation in words, the absolute difference between A_t and F_t is divided by half the sum of absolute values of the actual value A_t and the forecast value F_t. The value of this calculation is summed for every fitted point t and divided again by the number of fitted points n.

History

The earliest reference to a similar formula appears to be Armstrong (1985, p. 348), where it is called "adjusted MAPE" and is defined without the absolute values in the denominator. It was later discussed, modified, and re-proposed by Flores (1986).

Armstrong's original definition is as follows:

${\displaystyle {\text{SMAPE}}={\frac {1}{n}}\sum _{t=1}^{n}{\frac {\left|F_{t}-A_{t}\right|}{(A_{t}+F_{t})/2}}}$

The problem is that it can be negative if ${\displaystyle A_{t}+F_{t}<0}$. Therefore, the currently accepted version of SMAPE assumes the absolute values in the denominator.

Discussion

Comparison with MAPE

The idea behind SMAPE is that over and under-forecasts are treated in a relative way, rather than an absolute way, as with the mean absolute percentage error (MAPE). For example, applying the formula above to some actual ${\displaystyle A}$ and forecasted ${\displaystyle F}$ values:

${\displaystyle A}$	${\displaystyle F}$	MAPE	SMAPE
100	110	10%	9.52%
100	90	10%	10.53%

we see that MAPE considers an over and underestimation of 10% as equivalent, whereas SMAPE considers the underestimation to be slightly "worse" than the overestimation.

Extending this to larger forecast errors:

${\displaystyle A}$	${\displaystyle F}$	MAPE	SMAPE
100	200	100%	66.67%
100	50	50%	66.67%

Here, double overestimation and half underestimation are treated equivalently by SMAPE, whereas MAPE considers the overestimation to be "twice as bad" as the underestimation.

Extending to an even more extreme case:

${\displaystyle A}$	${\displaystyle F}$	MAPE	SMAPE
100	1,000	900%	163.63%
100	10	90%	163.63%

Here it becomes clear that MAPE is unbounded from above, and can provide extremely large penalties for overestimations – but cannot do the same for extreme underestimations. SMAPE, on the other hand, is bounded between 0% and 200%, and penalises these larger over and underestimations in a more "symmetric" manner.

Therefore, the choice between MAPE and SMAPE depends entirely on the problem at hand, and whether or not a relative metric is more appropriate. This may be the case if the expected forecasting errors exceed ${\displaystyle \gg 10\%}$; for smaller errors, the MAPE is more frequently chosen, due to its simplicity and ease of interpretation.

Alternative Versions

As a "percentage error", SMAPE values between 0% and 100% can be considered easier to interpret, and an alternative formula is sometimes used in practice:

${\displaystyle {\text{SMAPE}}={\frac {1}{n}}\sum _{t=1}^{n}{\frac {|F_{t}-A_{t}|}{|A_{t}|+|F_{t}|}}}$

There is also a third version of SMAPE, which allows measuring the direction of the bias in the data by generating a positive and a negative error on line item level. Furthermore, it is better protected against outliers and the bias effect^{[ clarification needed ]}. The formula is:

${\displaystyle {\text{SMAPE}}={\frac {\sum _{t=1}^{n}\left|F_{t}-A_{t}\right|}{\sum _{t=1}^{n}(A_{t}+F_{t})}}}$

Alternatives

Provided the data are strictly positive, an alternative measure of relative accuracy can be obtained based on the log of the accuracy ratio: log(F_t / A_t). This measure is easier to analyze statistically and has valuable symmetry and unbiasedness properties. When used in constructing forecasting models, the resulting prediction corresponds to the geometric mean (Tofallis, 2015) ^{[ clarification needed ]}.

See also

• Relative change and difference
• Mean absolute error
• Mean absolute percentage error
• Mean squared error
• Root mean squared error

References

• Armstrong, J. S. (1985) Long-range Forecasting: From Crystal Ball to Computer, 2nd. ed. Wiley. ISBN   978-0-471-82260-8
• Flores, B. E. (1986) "A pragmatic view of accuracy measurement in forecasting", Omega (Oxford), 14(2), 93–98. doi : 10.1016/0305-0483(86)90013-7
• Tofallis, C (2015) "A Better Measure of Relative Prediction Accuracy for Model Selection and Model Estimation", Journal of the Operational Research Society, 66(8),1352-1362. archived preprint

External links

• Rob J. Hyndman: Errors on Percentage Errors