Weather Forecasting: How Does It Work, and How Reliable Is It?

Do you ever wonder exactly how the weather forecast is made? And how good are they, really? In last month’s article, I discussed the accuracy and usefulness of seasonal weather outlooks. This month, I want to hone in on near-term weather forecasts, defined for our purposes as forecasts of specific day-by-day or even hourly conditions for periods less than two weeks into the future. These are the forecasts by which we plan our weather-sensitive operations and personal activities. This is my career passion because it is in these time frames that confidence is high enough to make significant decisions that directly affect agricultural operations and output. Yet, my observation is that weather forecasts are still under-utilized in agricultural decision-making processes, with an over-emphasis on using recent and past data for determining future actions, perhaps due to the historical perception that forecasts are no better than the flip of a coin. Hopefully, this article can help build confidence in modern weather forecasting capabilities.

The weather forecasting process starts with the collection, quality control, and fusion of large amounts of diverse raw observational data from space, airborne, and ground-based assets to produce a coherent, three-dimensional view of the atmosphere at any point in time, from which we project the future conditions, adjust, and repeat. Those are all large topics for another time. For now, let’s just focus on the actual methods used in projecting the weather forward in time from the current conditions.

It turns out there are multiple techniques that are used, each of which has specific time frames for which they are most relevant. These methods are shown in the figure below, which depicts their comparative usefulness as a function of forecast lead time (meaning, how far into the future are you trying to predict, e.g., a forecast for tomorrow would have a lead time of one day).

Automated-Forecast-Method-Comparison-Graph

The two simplest methods are persistence and climatology. A persistence forecast means that conditions are assumed to remain unchanged. We all do this inherently when we look out the window to see if an umbrella or sunglasses are needed for an immediate, short-lived outside activity because in most cases we can assume things remain constant for a few minutes or longer.  A climatology-based forecast assumes conditions for a particular day or time period will fall within statistical normals derived over many years for that same day or period. Interestingly, these two “easy” methods are applicable at opposite ends of the time spectrum, with persistence often working very well for forecasting very near term conditions, and climatology being the best predictor for long-range forecasts. For a forecast to be considered to have any skill, it must be able to at least beat both of these two methods.

Extrapolation is slightly more complicated, and assumes an existing weather feature, such as a cold front or a line of storms, will continue generally moving in the same speed and direction without significant change. But it cannot account for new development or dissipation. We will save teleconnections for another time, as it is primarily for long-range forecasting, using large scale indicators such as the El Niño Southern Oscillation to infer what might be expected for the longer-range patterns.

That leaves numerical weather prediction, which is by far the most important method used in modern meteorology for forecasting lead times beyond a couple of hours. The advent of affordable, high-performance computing has revolutionized our ability to create computer models that simulate the motion and physics of the three-dimensional atmosphere, accounting for its interaction with topography, land surface, and the oceans with amazing realism, without any need for historical data or statistical relationships.

You often hear the TV meteorologist refer to “the models” when talking about the forecast, often accompanied with realistic animations of future clouds and precipitation.  In the last few decades, the advancement of these models have dramatically improved forecast accuracy and changed the role of the human weather forecaster. Yet, they still are subject to errors due to our incomplete understanding of all process interactions, our inability to explicitly simulate down to chaotic, molecular levels, and insufficient sampling of the initial state of the atmosphere, particularly in the mid and upper levels of the atmosphere over the oceans and other unpopulated areas. Thus, in most cases, the best weather forecast is an optimum blend of some or all of these methods, most often performed by a combination of automation and human oversight.

So, how good are the near-term forecasts now? ForecastWatch.com is an independent weather forecast verification service. Each day they collect and archive daily forecasts from a large number of public and private weather forecast providers. They compare these forecasts to weather observations from high-quality, government-operated weather stations and generate statistics on a variety of weather parameters of interest to the general public, including daily high and low temperature, wind speed, and occurrence of precipitation. This provides an excellent “apples-to-apples” comparison across providers because the same methods and truth data are applied to all sources, and the consistency allows for the assessment of long-term changes in accuracy. They recently published their study of daily high-temperature forecast accuracy, based on a 12-year period spanning 2005 through 2016, aggregating almost 200 million individual data sample for over 750 U.S. locations. If you like numbers and graphs, it’s worth a read. But, some key takeaways are summarized here:

  • One-day forecasts have an error of less than 3° Fahrenheit.
  • The five-day forecast is now almost as accurate as a one day forecast was in 2005, and overall error of the daily high-temperature forecast decreased by 33% over that period.
  • We have now reached a point where the nine-day temperature forecast is slightly better than climatology.

The graph below shows statistics from ForecastWatch for this year, spanning January through September. It shows the forecast error in degrees by forecast lead time for a climatology-based forecast (black), persistence forecast (red), and the average error of all providers analyzed by their service (blue). As mentioned, a forecast is really only useful if it performs better than persistence and climatology, and you can see that is the case all the way through day nine. This is consistent with the conclusions drawn from the 12-year study. The error tends to increase roughly 0.5° per day, so you can also infer that by around 10 days out, the forecasts are no longer more accurate than climatology, corroborating my assertion in last month’s article that forecasts of specific conditions at a particular location beyond a couple of weeks are beyond the state of the science.

As models continue to improve, we can expect to see a continuation of improving forecast accuracy, at least to a point. There is much debate about what the realistic limits are for weather forecasting, and it may not ever reach a point where we can project exact weather for a location beyond a few weeks in our lifetimes. But, in terms of things we attempt to predict in this world, the forecasting of near term weather is a major success story. Hopefully, as confidence grows in our ability to forecast weather and soil conditions, much value will be extracted for more efficient operations, higher yields, and better environmental stewardship.

Leave a Reply

One comment on “Weather Forecasting: How Does It Work, and How Reliable Is It?

  1. Very interesting and important. We, in Netafim, use short term weather forecast as part of our crop irrigation models and this is an essential part of it.
    Temperature, I guess, is a relatively an easy parameter for forecasting. We mainly straggle with precipitation which is mostly a local event and thus very hard to forecast

Data Management Stories
Field
Data ManagementThe Climate Corp., Deveron Form Drone Analytics Partnership
November 16, 2017
The Climate Corp., a subsidiary of Monsanto Co., has announced a partnership with Deveron UAS Corp., that will deliver farmers Read More
Data ManagementAgGateway Grain Transfer Proof of Concept Shows Promise
November 10, 2017
We talk about closed-loop systems quite a bit in modern production agriculture. Having a closed-loop, or completely self-contained, system is Read More
Spray Drift
Business ManagementProtecting Your Business with Ag Data: What Dicamba Can Teach Us (Guest Column)
November 9, 2017
All over the country, in ag communities large and small, farmers are reporting crop damage, neighbors are fighting, lawsuits and Read More
Data ManagementDJI Announces FlightHub Drone Operations Management Platform, Other New Developments
November 7, 2017
DJI showcased new hardware and software advancements that make it easier for businesses across multiple industries to incorporate drones into Read More
Trending Articles
Blue River Tech See and Spray prototype in CA
Business ManagementIs the Agtech Startup Boom Over?
November 13, 2017
In the past few months there has been a flurry of acquisitions of agtech startups by major players in the Read More
Spray Drift
Business ManagementProtecting Your Business with Ag Data: What Dicamba Can Teach Us (Guest Column)
November 9, 2017
All over the country, in ag communities large and small, farmers are reporting crop damage, neighbors are fighting, lawsuits and Read More
Kokoda-Trek2
Australia/New ZealandPrecision Agriculture in Papua New Guinea
November 2, 2017
I recently spent two weeks in the Central Highlands area of Papua New Guinea (PNG) on the Kokoda Track. The Read More
Robotic-Field-Scanner
Specialty CropsOpinion: Agriculture’s Big Data is Coming to You
October 20, 2017
If you think your world is changing at a dizzying pace, you ain’t seen nothin’ yet. That’s my takeaway from Read More
Brazil-Soybean-field
AmericasIPNI Brazil: Precision Agriculture to Improve Nutrient Use Efficiency
October 16, 2017
As I mentioned in my last article, the IPNI Brazil Symposium on Precision Agriculture was held in early October in Read More
Decision Support SoftwareFarmers Edge, Service Provider Talk High-Frequency Imagery Deal
October 11, 2017
As the domestic ag tech market continues to mature and develop as we barrel into 2018, another intriguing partnership has Read More
Latest News
DronesDrones Play Critical Role in Ohio Agricultural Research
November 17, 2017
The Springfield Beckley-Municipal Airport now has a new hangar for unmanned aircraft and precision agriculture, writes Allyson Brown on GovTech.com. Read More
Iowa-Farmer China
AsiaIowa Farm a Model for Modernizing China Agriculture
November 16, 2017
The buddy seat on Rick Kimberley’s combine-harvester is a fine vantage point from which to observe precision farming. The combine’s Read More
Field
Data ManagementThe Climate Corp., Deveron Form Drone Analytics Partner…
November 16, 2017
The Climate Corp., a subsidiary of Monsanto Co., has announced a partnership with Deveron UAS Corp., that will deliver farmers Read More
aggateway
Service ProvidersAgGateway Recognizes Leaders at 2017 Annual Conference
November 16, 2017
At its annual conference last week, AgGateway honored individuals and companies that are advancing electronic connectivity efforts in agriculture and Read More
corn field
UncategorizedTopcon Agriculture Prepares to Release Horizon 4.02
November 15, 2017
Topcon Agriculture has announced the addition of new functions including Auto Turn and Steer to Boundary applications in its upcoming Read More
Used Combine
Service ProvidersFinding New Precision Ag Opportunities in Older Equipme…
November 15, 2017
Whether to buy new or used equipment has been and continues to be an important topic for growers. When it Read More
John-Deere-Air-Carts
Systems ManagementJohn Deere 1910 Air Carts Can Now Be Retrofitted with S…
November 14, 2017
Intelligent Agricultural Solutions, LLC (Intelligent Ag), a leader in electronics innovations for modern farming, has released Engage Zone Control, the Read More
Climate FieldView Script Creator
EuropeThe Climate Corp. Expands Global Digital Agriculture Pl…
November 14, 2017
The Climate Corp., a subsidiary of Monsanto Co., has announced at Agritechnica, the world’s leading trade fair for agricultural machinery, Read More
Soybean-Field-Photo-Barbara-Eckstein
Data ManagementWeather Forecasting: How Does It Work, and How Reliable…
November 14, 2017
Do you ever wonder exactly how the weather forecast is made? And how good are they, really? In last month’s Read More
Topcon_TAP
Decision Support SoftwareTopcon Agriculture Introduces New Connected Agricultura…
November 13, 2017
Topcon Agriculture has introduced TAP, the Topcon Agriculture Platform, a cloud-based agricultural ecosystem designed to provide real-time monitoring and data Read More
Topcon A6
Systems ManagementTopcon Agriculture Adds ISOBUS Communication to Opus Co…
November 13, 2017
Topcon Agriculture has added industry-leading ISOBUS universal terminal and task controller functionality to the Opus A6 and A8 consoles. The Read More
Blue River Tech See and Spray prototype in CA
Business ManagementIs the Agtech Startup Boom Over?
November 13, 2017
In the past few months there has been a flurry of acquisitions of agtech startups by major players in the Read More
Data ManagementAgGateway Grain Transfer Proof of Concept Shows Promise
November 10, 2017
We talk about closed-loop systems quite a bit in modern production agriculture. Having a closed-loop, or completely self-contained, system is Read More
Spray Drift
Business ManagementProtecting Your Business with Ag Data: What Dicamba Can…
November 9, 2017
All over the country, in ag communities large and small, farmers are reporting crop damage, neighbors are fighting, lawsuits and Read More
Farmers Business Network, FBN Procurement
Industry NewsFarmers Business Network Details Updated FBN Direct Sho…
November 8, 2017
Farmer’s Business Network, Inc. is modernizing the way farmers shop with a new e-commerce platform designed and engineered exclusively for Read More
Excellence AwardsNominations Open For 2018 PrecisionAg Awards Of Excelle…
November 8, 2017
The PrecisionAg Award of Excellence program sheds light on some of the most important people and programs that have helped agriculture realize the full benefits of technology to improve agronomy, efficiency, and stewardship in crop production. Read More
Data ManagementDJI Announces FlightHub Drone Operations Management Pla…
November 7, 2017
DJI showcased new hardware and software advancements that make it easier for businesses across multiple industries to incorporate drones into Read More
Deere-Combine
Data ManagementTech’s Perspective: Closeout to #Harvest2017
November 7, 2017
As the harvest season comes to a close, a quick look at Twitter gives us some insights into how things Read More