Often, when discussing weather service needs with a potential client, I am asked about the possibility of providing long-range weather forecasts. By “long-range,” I’m referring to weather forecasts beyond seven to 10 days — ranging from three-month seasonal outlooks to as much as a year into the future. As my high school physics teacher liked to say, “It seems obvious to the most casual observer” that if one could provide really accurate weather forecasts for such long time scales, there would be tremendous value and the provider would be so wealthy they might just do something else with all their money and new-found free time. At least, that is my typical response! For this month’s column, I would like to address three questions about long-range forecasts:
- What is the current state of the science of long-range forecasting?
- How accurate are the existing services?
- How useful are they specifically for precision agriculture?
In its most recent report on Weather Analysis and Forecasting, the American Meteorological Society (AMS) said the following about extended-range forecasts:
“Presently, forecasts of daily or specific weather conditions do not exhibit useful skill beyond eight days, meaning that their accuracy is low. However, probabilistic forecasts issued to highlight significant trends (e.g., warmer than normal, wetter than normal) can be skillful when compared to a baseline forecast.”
This confirms what most of us know experientially, especially if you live in a place where the weather routinely changes. Forecasts of specific conditions (i.e., actual temperatures, rainfall amounts, etc.) are impossible to make with useful accuracy beyond a week or two.
MORE BY BRENT SHAW
However, the AMS statement does address a type of long-range forecast, and how the NOAA Climate Prediction Center’s (CPC) skill in producing these forecasts has been steadily improving:
“Finally, monthly and seasonal forecasts are typically issued for meteorological phenomena that cover areas ranging from the size of a continent to the planet as a whole. Skill in monthly and seasonal forecasts is extremely variable from period to period, but the skill of NOAA Climate Prediction Center one- and three-month forecasts of temperature and precipitation increased by more than 25% between 2006 and 2013. Increases in forecast skill at these lead times can largely be attributed to improved understanding of and ability to forecast major modes of large-scale climate variability … “
A key point from these quotes is that specificity decreases with increasing forecast length. Monthly and seasonal forecasts are large-scale in nature and don’t provide specific numbers for the various weather parameters. In other words, the science currently cannot provide accurate long-range forecasts of actual weather conditions for a given day specific to your location. But, it does support the ability to provide regional assessments on whether the monthly to seasonal average conditions will be above or below longer-term averages (typically based on a 30-year climatology). In meteorology, we call these “anomaly forecasts.” These types of forecasts are widely used in the energy industry, as it helps them plan for seasonal energy demand at a regional level and can affect the value of energy futures in the trading market.
Accuracy of Forecasts
Even though accuracy is improving, how accurate are these forecasts, really? The CPC is a world leader in government weather services and represents the state of the science in seasonal forecasting, and they provide online access to their statistical assessment of forecast accuracy, along with archives of previous forecasts and the actual observed conditions. Their forecasts are produced by expert forecasters with specific expertise in the evolution of large-scale atmospheric patterns, combined with advanced, long-range ensemble numerical weather prediction systems. The forecasts are produced on a global grid, where one grid square is approximately 130 miles on each side. Each grid cell contains probabilities of temperature and precipitation being above normal, below normal, near normal, or equal chance that it can be above or below. From these grids, they create anomaly maps that communicate the final forecast. For their seasonal product, they produce a three-month forecast that is issued about a half month in advance. So, the forecast for expected average conditions for the period covering July through September of 2017 was issued in mid-June, 2017.
The figure below shows the July through September seasonal CPC forecast maps for temperature and precipitation on the top row. What actually occurred, based on observed conditions translated to the same grid, is shown for comparison on the bottom row.
In the case of temperature, almost the entire U.S. was forecast to be above normal (red area). Based on this visual comparison, this forecast did not pan out very well, as almost the entire eastern two-thirds of the country experience near or below normal conditions (gray and blue areas). For precipitation, a large part of the county was forecast to be in the “equal chance” category (white areas). Ideally, the amount of area contained in the equal chance forecast should be minimized for maximum usefulness, since it is essentially saying that it is a toss-up.There were only two areas that had a non-equal chance forecast: a portion of the Northern Plains and Southeast Texas, both of which were forecast to be above normal. The Southeast Texas area represented an accurate forecast, but the forecast in the Northern Plains was a mixed bag, with a significant percentage of the area actually receiving near or below normal rainfall, especially the Montana area, which has been ravaged by wildfires this year as a result of the dry conditions.
The next two figures show a statistical measure of the forecast skill over the entire grid for all of the three-month temperature and precipitation forecasts issued by CPC for the past five years. The first figure is for the temperature forecasts, and the second is for precipitation forecasts. Each dot on the graph represents the skill of the forecast product issued for middle month of the three-month period (e.g., the July through September period is represented as August). For a forecast to be considered as having any useful skill, the dot has to be above the solid line (“0” on the y-axis), which represents no skill. Dots below the line represent negative skill, that is, that they are less accurate than a random forecast or even just assuming conditions will be “normal” compared to the longer-term climatology. A perfect forecast yields a value of 1.0, and a completely wrong forecast yields a score of -0.5. The dashed line on the graph shows the average skill of all the forecasts over the entire five-year record.
You can see that over the last five years, the skill of forecasting temperature anomalies has been trending upward, and that the overall average is on the positive side, even though the past three forecasts were not very good, corroborating the example forecast we showed above. The dashed line represents the average over all of the forecasts, and it is positive but still a very low value, approximately 0.08 on a scale of 0.0 to 1.0 for positive skill.
For precipitation, the story is much bleaker. On average, the forecasts demonstrate no skill, with almost as many years having a negative score on the graph as having a positive score. This means that using the climatological normals to predict what will happen regarding precipitation over the next three months is as good as using these forecasts.
Given all of this, how useful are these products? It is easy to see the benefit to energy commodity futures trading, and perhaps even agricultural futures commodity trading or agribusiness supply chain management decisions, where large scale trends may be more important than day-to-day variability. But accuracy remains a limitation, so this translates to a “high-risk, high-reward” scenario, much like any other gambling activity.
But assuming they were much more accurate, how can seasonal forecasts benefit actual decisions made by a grower, retailer, or consultant, especially in the context of precision agriculture? If one can project that a warmer than normal spring might create earlier onset of planting, a producer might select different varieties of crops based on a potentially longer growing season. Or perhaps a forecast of drier than normal conditions might cause one to select a more drought-tolerant crop. But, the reliability to make such a call just doesn’t seem to be there yet.
Because these are forecasts of average conditions over a three-month period, it is quite possible for a forecast to end up being correct, even though high-impact individual events might have occurred within the period. That is, you could have copious rainfall for some small portion of the period, leading to crop-destroying floods, and then have a drought during critical periods of crop development. This type of forecast does not (and cannot) address such things. It is the noise (weather) around the signal (averages) that really impact our daily lives and economy. Making decisions based on seasonal forecasts, even if they continue to improve in skill, for precision agricultural operations, remain very high risk with little or no reward or even applicability. I would love to hear your thoughts!