Forecast error

Last updated

In statistics, a forecast error is the difference between the actual or real and the predicted or forecast value of a time series or any other phenomenon of interest. Since the forecast error is derived from the same scale of data, comparisons between the forecast errors of different series can only be made when the series are on the same scale. [1]

Contents

In simple cases, a forecast is compared with an outcome at a single time-point and a summary of forecast errors is constructed over a collection of such time-points. Here the forecast may be assessed using the difference or using a proportional error. By convention, the error is defined using the value of the outcome minus the value of the forecast.

In other cases, a forecast may consist of predicted values over a number of lead-times; in this case an assessment of forecast error may need to consider more general ways of assessing the match between the time-profiles of the forecast and the outcome. If a main application of the forecast is to predict when certain thresholds will be crossed, one possible way of assessing the forecast is to use the timing-errorthe difference in time between when the outcome crosses the threshold and when the forecast does so. When there is interest in the maximum value being reached, assessment of forecasts can be done using any of:

Forecast error can be a calendar forecast error or a cross-sectional forecast error, when we want to summarize the forecast error over a group of units. If we observe the average forecast error for a time-series of forecasts for the same product or phenomenon, then we call this a calendar forecast error or time-series forecast error. If we observe this for multiple products for the same period, then this is a cross-sectional performance error. Reference class forecasting has been developed to reduce forecast error. Combining forecasts has also been shown to reduce forecast error. [2] [3]

Calculating forecast error

The forecast error is the difference between the observed value and its forecast based on all previous observations. If the error is denoted as then the forecast error can be written as:

where,

= observation

= denote the forecast of based on all previous observations

Forecast errors can be evaluated using a variety of methods namely mean percentage error, root mean squared error, mean absolute percentage error, mean squared error. Other methods include tracking signal and forecast bias.

For forecast errors on training data

denotes the observation and is the forecast

For forecast errors on test data

denotes the actual value of the h-step observation and the forecast is denoted as

Academic literature

Dreman and Berry in 1995 "Financial Analysts Journal", argued that securities analysts' forecasts are too optimistic, and that the investment community relies too heavily on their forecasts. However, this was countered by Lawrence D. Brown in 1996 and then again in 1997 who argued that the analysts are generally more accurate than those of "naive or sophisticated time-series models" nor have the errors been increasing over time. [4] [5]

Hiromichi Tamura in 2002 argued that herd-to-consensus analysts not only submit their earnings estimates that end up being close to the consensus but that their personalities strongly affect these estimates. [6]

Examples of forecasting errors

Michael Fish - A few hours before the Great Storm of 1987 broke, on 15 October 1987, he said during a forecast: "Earlier on today, apparently, a woman rang the BBC and said she heard there was a hurricane on the way. Well, if you're watching, don't worry, there isn't!". The storm was the worst to hit South East England for three centuries, causing record damage and killing 19 people. [7]

Great Recession - The financial and economic "Great Recession" that erupted in 2007—arguably the worst since the Great Depression of the 1930s—was not foreseen by most forecasters, though a number of analysts had been predicting it for some time (for example, Brooksley Born, Dean Baker, Marc Faber, Fred Harrison, Raghuram Rajan, Stephen Roach, Nouriel Roubini, Peter Schiff, Gary Shilling, Robert Shiller, William White, and Meredith Whitney). [8] [9] [10] [11] The UK's Queen Elizabeth herself asked why had “nobody” noticed that the credit crunch was on its way, and a group of economists—experts from business, the City, its regulators, academia, and government—tried to explain in a letter. [12]

It was not just forecasting the Great Recession, but also its impact where it was clear that economists struggled. For example, in Singapore, Citi argued the country would experience "the most severe recession in Singapore’s history". The economy grew in 2009 by 3.1%, and in 2010 the nation saw a 15.2% growth rate. [13] [14]

Similarly, Nouriel Roubini predicted in January 2009 that oil prices would stay below $40 for all of 2009. By the end of 2009, however, oil prices were at $80. [15] [16] In March 2009, he predicted the S&P 500 would fall below 600 that year, and possibly plummet to 200. [17] It closed at over 1,115 however, up 24%, the largest single-year gain since 2003. CNBC's Jim Cramer wrote that Roubini was "intoxicated" with his own "prescience and vision," and should realize that things are better than he predicted; Roubini called Cramer a "buffoon," and told him to "just shut up". [15] [18] Although in April 2009, Roubini prophesied that the United States economy would decline in the final two quarters of 2009, and that the US economy would increase just 0.5% to 1% in 2010, in fact the U.S. economy in each of those six quarters increased at a 2.5% average annual rate. [19] Then in June 2009 he predicted that what he called a "perfect storm" was just around the corner, but no such perfect storm ever appeared. [20] [19] In 2009 he also predicted that the US government would take over and nationalize a number of large banks; it did not happen. [21] In October 2009 he predicted that the price of gold "can go above $1,000, but it can’t move up 20-30%”; he was wrong, as the price of gold rose over the next 18 months, breaking through the $1,000 barrier to over $1,400. [22]

2020 Global Growth - At the end of 2019 the International Monetary Fund estimated global growth in 2020 to reach 3.4%, but as a result of the coronavirus pandemic, the IMF have revised its estimate in November 2020 to expect the global economy to shrink by 4.4%. [23] [24]

See also

Related Research Articles

<span class="mw-page-title-main">Pessimism</span> Negative mental attitude

Pessimism is a mental attitude in which an undesirable outcome is anticipated from a given situation. Pessimists tend to focus on the negatives of life in general. A common question asked to test for pessimism is "Is the glass half empty or half full?"; in this situation, a pessimist is said to see the glass as half empty, or in extreme cases completely empty, while an optimist is said to see the glass as half full. Throughout history, the pessimistic disposition has had effects on all major areas of thinking.

Forecasting is the process of making predictions based on past and present data. Later these can be compared (resolved) against what happens. For example, a company might estimate their revenue in the next year, then compare it against the actual results creating a variance actual analysis. Prediction is a similar but more general term. Forecasting might refer to specific formal statistical methods employing time series, cross-sectional or longitudinal data, or alternatively to less formal judgmental methods or the process of prediction and resolution itself. Usage can vary between areas of application: for example, in hydrology the terms "forecast" and "forecasting" are sometimes reserved for estimates of values at certain specific future times, while the term "prediction" is used for more general estimates, such as the number of times floods will occur over a long period.

In statistical inference, specifically predictive inference, a prediction interval is an estimate of an interval in which a future observation will fall, with a certain probability, given what has already been observed. Prediction intervals are often used in regression analysis.

<span class="mw-page-title-main">Regression analysis</span> Set of statistical processes for estimating the relationships among variables

In statistical modeling, regression analysis is a set of statistical processes for estimating the relationships between a dependent variable and one or more independent variables. The most common form of regression analysis is linear regression, in which one finds the line that most closely fits the data according to a specific mathematical criterion. For example, the method of ordinary least squares computes the unique line that minimizes the sum of squared differences between the true data and that line. For specific mathematical reasons, this allows the researcher to estimate the conditional expectation of the dependent variable when the independent variables take on a given set of values. Less common forms of regression use slightly different procedures to estimate alternative location parameters or estimate the conditional expectation across a broader collection of non-linear models.

In finance, bond convexity is a measure of the non-linear relationship of bond prices to changes in interest rates, and is defined as the second derivative of the price of the bond with respect to interest rates. In general, the higher the duration, the more sensitive the bond price is to the change in interest rates. Bond convexity is one of the most basic and widely used forms of convexity in finance. Convexity was based on the work of Hon-Fei Lai and popularized by Stanley Diller.

<span class="mw-page-title-main">Nouriel Roubini</span> Iranian-American economist

Nouriel Roubini is a Turkish-born Iranian-American economic consultant, economist, speaker and writer. He is a Professor Emeritus since 2021 at the Stern School of Business of New York University.

Economic forecasting is the process of making predictions about the economy. Forecasts can be carried out at a high level of aggregation—for example for GDP, inflation, unemployment or the fiscal deficit—or at a more disaggregated level, for specific sectors of the economy or even specific firms. Economic forecasting is a measure to find out the future prosperity of a pattern of investment and is the key activity in economic analysis. Many institutions engage in economic forecasting: national governments, banks and central banks, consultants and private sector entities such as think-tanks, companies and international organizations such as the International Monetary Fund, World Bank and the OECD. A broad range of forecasts are collected and compiled by "Consensus Economics". Some forecasts are produced annually, but many are updated more frequently.

A corporate bond is a bond issued by a corporation in order to raise financing for a variety of reasons such as to ongoing operations, mergers & acquisitions, or to expand business. It is a longer-term debt instrument indicating that a corporation has borrowed a certain amount of money and promises to repay it in the future under specific terms. Corporate debt instruments with maturity shorter than one year are referred to as commercial paper.

In statistics the mean squared prediction error (MSPE), also known as mean squared error of the predictions, of a smoothing, curve fitting, or regression procedure is the expected value of the squared prediction errors (PE), the square difference between the fitted values implied by the predictive function and the values of the (unobservable) true value g. It is an inverse measure of the explanatory power of and can be used in the process of cross-validation of an estimated model. Knowledge of g would be required in order to calculate the MSPE exactly; in practice, MSPE is estimated.

The mean absolute percentage error (MAPE), also known as mean absolute percentage deviation (MAPD), is a measure of prediction accuracy of a forecasting method in statistics. It usually expresses the accuracy as a ratio defined by the formula:

<span class="mw-page-title-main">VIX</span> Volatility index

VIX is the ticker symbol and the popular name for the Chicago Board Options Exchange's CBOE Volatility Index, a popular measure of the stock market's expectation of volatility based on S&P 500 index options. It is calculated and disseminated on a real-time basis by the CBOE, and is often referred to as the fear index or fear gauge.

Zvi Bodie is an American economist, author and professor. He was the Norman and Adele Barron Professor of Management at Boston University, teaching finance at Questrom for 43 years before retiring in 2015. His textbook, Investments, (with Kane and Marcus) is the market leader and is used in the certification programs of the CFA Institute and the Society of Actuaries. Bodie's work has centered on pension finance and investment strategy. He continues to do consulting work and media interviews.

In statistics, mean absolute error (MAE) is a measure of errors between paired observations expressing the same phenomenon. Examples of Y versus X include comparisons of predicted versus observed, subsequent time versus initial time, and one technique of measurement versus an alternative technique of measurement. MAE is calculated as the sum of absolute errors divided by the sample size:

The root mean square deviation (RMSD) or root mean square error (RMSE) is either one of two closely related and frequently used measures of the differences between true or predicted values on the one hand and observed values or an estimator on the other.

United States housing prices experienced a major market correction after the housing bubble that peaked in early 2006. Prices of real estate then adjusted downwards in late 2006, causing a loss of market liquidity and subprime defaults.

While beginning in the United States, the Great Recession spread to Asia rapidly and has affected much of the region.

In statistics, the mean absolute scaled error (MASE) is a measure of the accuracy of forecasts. It is the mean absolute error of the forecast values, divided by the mean absolute error of the in-sample one-step naive forecast. It was proposed in 2005 by statistician Rob J. Hyndman and Professor of Decision Sciences Anne B. Koehler, who described it as a "generally applicable measurement of forecast accuracy without the problems seen in the other measurements." The mean absolute scaled error has favorable properties when compared to other methods for calculating forecast errors, such as root-mean-square-deviation, and is therefore recommended for determining comparative accuracy of forecasts.

Comparisons between the Great Recession and the Great Depression explores the experiences in the United States and the United Kingdom.

Pessimism porn is a neologism coined in 2009 during the Great Recession to describe the alleged eschatological and survivalist thrill some people derive from predicting, reading and fantasizing about the collapse of civil society through the destruction of the world's economic system.

Lisa Goldberg is a financial economist and statistician who serves at the University of California, Berkeley as director of research at the Center for Risk Management Research and as Adjunct Professor of Statistics. She is also the Co-Director for the Consortium for Data Analytics in Risk at UC Berkeley.

References

  1. 2.5 Evaluating forecast accuracy | OTexts . Retrieved 2016-05-12.{{cite book}}: |website= ignored (help)
  2. J. Scott Armstrong (2001). "Combining Forecasts". Principles of Forecasting: A Handbook for Researchers and Practitioners (PDF). Kluwer Academic Publishers.
  3. J. Andreas Graefe; Scott Armstrong; Randall J. Jones, Jr.; Alfred G. Cuzán (2010). "Combining forecasts for predicting U.S. Presidential Election outcomes" (PDF).
  4. Brown, Lawrence D. (1996). "Analyst Forecasting Errors and Their Implications for Security Analysis: An Alternative Perspective". Financial Analysts Journal. 52 (1): 40–47. doi:10.2469/faj.v52.n1.1965. ISSN   0015-198X. JSTOR   4479895. S2CID   153329250.
  5. Brown, Lawrence D. (1997). "Analyst Forecasting Errors: Additional Evidence". Financial Analysts Journal. 53 (6): 81–88. doi:10.2469/faj.v53.n6.2133. ISSN   0015-198X. JSTOR   4480043. S2CID   153810721.
  6. Tamura, Hiromichi (2002). "Individual-Analyst Characteristics and Forecast Error". Financial Analysts Journal. 58 (4): 28–35. doi:10.2469/faj.v58.n4.2452. ISSN   0015-198X. JSTOR   4480404. S2CID   154943363.
  7. "Michael Fish revisits 1987's Great Storm". BBC. 16 October 2017. Retrieved 16 October 2017.
  8. Helaine Olen (March 30, 2009). "The Prime of Mr. Nouriel Roubini", Entrepreneur .
  9. Jerry H. Tempelman (July 30, 2014). "Austrian Business Cycle Theory and the Global Financial Crisis: Confessions of a Mainstream Economist," The Quarterly Journal of Austrian Economics]
  10. "The Economic Crisis and the Crisis in Economics". www.eatonak.org. Archived from the original on 2023-03-02. Retrieved 2023-03-02.
  11. Bezemer, Dirk J, 16 June 2009. "“No One Saw This Coming”: Understanding Financial Crisis Through Accounting Models"
  12. British Academy-The Global Financial Crisis Why Didn't Anybody Notice?-Retrieved July 27, 2015 Archived July 7, 2015, at the Wayback Machine
  13. Chen, Xiaoping; Shao, Yuchen (2017-09-11). "Trade policies for a small open economy: The case of Singapore". The World Economy. 40 (11): 2500–2511. doi:10.1111/twec.12555. ISSN   0378-5920. S2CID   158182047.
  14. Subler, Jason (2009-01-02). "Factories slash output, jobs around world". Reuters. Retrieved 2020-09-20.
  15. 1 2 Joe Keohane (January 9, 2011). "That guy who called the big one? Don’t listen to him." The Boston Globe .
  16. Eric Tyson (2018). Personal Finance For Dummies
  17. Maneet Ahuja (2014). The Alpha Masters; Unlocking the Genius of the World's Top Hedge Funds
  18. "Roubini to Cramer: ‘Just shut up’", The Los Angeles Times , April 8, 2009.
  19. 1 2 Daniel Altman (October 8, 2012). "Nouriel Roubini; He may not be perfect, but there's never been a better time to be in the prophet of doom business," Foreign Policy Magazine .
  20. Nouriel Roubini (June 16, 2009). "Financial Gain, Economic Pain," Project Syndicate.
  21. Joseph Lazzaro (March 26, 2009). "'Dr. Doom' predicts some big banks will be nationalized," AOL.com.
  22. Alice Guy (January 16, 2023). "Seven times the experts got it very wrong on the economy," Interactive Investor .
  23. "IMF warns world growth slowest since financial crisis". BBC News. 2019-10-15. Retrieved 2020-11-22.
  24. "IMF: Economy 'losing momentum' amid virus second wave". BBC News. 2020-11-19. Retrieved 2020-11-22.