What About the Accuracy of Other USDA Forecasts?

What About the Accuracy of Other USDA Forecasts?

Among other USDA forecasts, WASDE ending stocks forecasts, NASS Grain Stocks estimates, and USDA’s baseline production and price forecasts received most attention.  “Ending stocks measure the carryover of a commodity that enters the supply side of the market in the following marketing year.  These stocks are a measure of the scarcity of the crop just before the next crop harvest, and they play an important role in the decision-making process for agricultural producers, processors, and policymakers.” (Xiao, Hart, and Lence, 2017, p. 221)  At the same time, ending stocks is a residual category in the WASDE balance sheet, calculated as a difference between total supply and total use for the marketing year. Even though most WASDE balance sheets contain a “residual” category on the use side, Botto et al. (2006) demonstrated that errors in supply and use forecasts tend to contribute to ending stocks forecast errors, as shown in figure 2.  This graph shows that a 1% overestimation in production in May reports resulted in approximately a 4% overestimation in corn ending stocks.  Xiao, Hart, and Lence (2017) demonstrated that USDA ending stocks forecasts are inefficient with predictability in revisions consistent with smoothing. The authors point out that “Concerns, such as those voiced by the soybean industry, that the USDA ending stock estimates were not adequately capturing the export demand growth resulting in higher ending stock estimates and lower crop prices likely have some merit. (p.239)”  Isengildina-Massa, Karali, and Irwin (2013) examined the sources of errors in WASDE balance sheet forecasts and contributed them largely to structural changes in commodity markets that took place in the mid-2000s, challenges with predicting periods of economic growth and changes in exchange rates, while inflation and changes in oil price had a much smaller impact. 

NASS Grain Stocks reports are widely used by the industry to gauge a pace of domestic use based on how much crop was still left in storage. The accuracy of these reports in the post 2006 sub-period has been questioned in recent years. The challenge with evaluating the accuracy of these reports is the lack of the “final value” as these reports deal with a “flow” variable.  In a number of studies by Irwin, Sanders and Good (2014a, 2014 b, 2014c, 2014d, 2014e) published grain stock values have been compared to industry expectations to point out that there was a notable decline in the ability of market participants to anticipate USDA stock estimates for corn through 2013. The authors argued that the potential reasons for larger surprises in corn grain stocks estimates are not due to commonly proposed reasons, but rather to unresolved sampling errors in production estimates.  They demonstrated that USDA stocks estimates undoubtedly encompassed sampling errors for both production and stocks estimates and it is highly likely that unresolved sampling errors for corn production estimates are large enough to explain even the largest surprises.  Their analysis highlighted the potential value of adding a survey of corn feed use that would allow a fuller accounting of corn usage as well as a revision of January corn production estimates similar to what has historically been done for soybeans. (Irwin, Sanders, and Good, 2014e)

USDA 10-year baseline forecasts are also important because they are widely used for policy analysis (Irwin and Good, 2015).  The studies of these forecasts typically show that it is extremely difficult to forecast prices that far into the future.   For example, percentage errors for USDA 5-year ahead baseline corn price forecasts from Irwin and Good (2015), shown in figure 3 reveal that forecast errors were negative (USDA forecasts greater than actual prices) in the early years and positive (USDA forecasts less than actual prices) in later years. Errors were also larger in the later years, with errors exceeding 40 percent in five of the last eight years.  When compared to futures-based forecasts, USDA forecast errors tended to be smaller for the one-, two-, and three-year time horizons, but larger for all longer-term forecasts except at ten years.  These findings suggest that the low accuracy of the long-term forecasts should be recognized and considered for decision making.

Boussios, Skorbianky, and MacLachlan (2021) examined USDA’s baseline forecasts for U.S. harvested area over 1997-2017 and found a tendency to consistently overestimate the harvested area for wheat and underestimate soybean and corn harvested area and proposed to use alternative econometric models to improve the accuracy of these forecasts. Their results suggest that the forecasts generated using the proposed econometric models produced predictions with an average absolute forecasting error 10 years out that was between 26 percent to 60 percent smaller than those provided by baseline projections.  These models have been adopted by USDA to complement their existing forecasting tools.