Skill or Luck?
There’s one thing that many of us are missing right now while we’re occupying ourselves at home: sports. We should have been all set for the playoffs in major league hockey and basketball, and we would be excited about the beginning of the major league baseball and soccer seasons. We also would have been eagerly anticipating some of this spring and summer’s major sporting events, including the Olympics. So let’s dream a little…
When we set out to write this blog post for Inside the Eye, we wanted to show how National Hurricane Center (NHC) forecasters use their skill and expertise to predict the future track of a hurricane. And then it got us thinking, how does luck factor into the equation? In other words, when meteorologists get a weather forecast right, how much of it is luck, and how much of it is forecasters’ skill in correctly interpreting, or even beating, the weather models available to them?
Investment strategist Michael Mauboussin created a “Skill-Luck Continuum” where individual sports, among other activities in life, are placed on a spectrum somewhere between pure skill and pure luck (Figure 1). Based on factors such as the number of games in a season, number of players in action, and number of scoring opportunities in a game or match, athletes and their teams in some sports might have to rely on a little more luck than other sports to be successful. On this spectrum, a sport like basketball would be closest to the skill side (there are a lot of scoring opportunities in a basketball game) whereas a sport like hockey would require a little more luck (there are fewer scoring opportunities in a hockey match, and sometimes you just need the puck to bounce your way). Fortunately for hockey fans, there are enough games in a season for their favorite team’s “unlucky” games to not matter so much.
Figure 1. The Skill-Luck Continuum in Sports, developed by investment strategist Michael Mauboussin.
Where would hurricane forecasting lie on such a continuum? There’s no doubt that luck plays at least some part in weather forecasting too, particularly in individual forecasts when random or unforeseen circumstances could either play in your favor (and make you look like the best forecaster around) or turn against you (and make you look like you don’t know what you’re doing!). But luck is much less of a factor when you consider a lot of forecasts over longer periods of time, where the good and bad circumstances should cancel each other out and true skill shines through (just as in sports). At NHC, we routinely compare our forecasts with weather models over these long periods of time to assess our skill at predicting, for example, the future tracks of hurricanes.
An International Friendly?
From our experience of talking to people about hurricanes and weather models, it seems to be almost common “knowledge” that only two models exist – the U.S. Global Forecast System (GFS) and the European Centre for Medium Range Weather Forecasts (ECMWF) model. It’s true that those two models are used heavily at NHC and the National Weather Service in general, but there are many more weather models that can simulate a hurricane’s track and general weather across the globe. (Here’s a comprehensive list showing all of the available weather models that are used at NHC today, if you’re interested: https://www.nhc.noaa.gov/modelsummary.shtml.) We’ve also heard and seen people compare the GFS and ECMWF models and talk about which model scenario might be more correct for a given storm. This blog entry summarizes the performances of those models and discusses how, on the whole, NHC systematically outperforms them on predicting the track of a storm.
Below are the most recent three years of data (2017, 2018, and 2019) of Atlantic basin track forecast skill from NHC and the three best individual track models: the GFS, ECMWF, and the United Kingdom Meteorological Office model (UKMET) (Figure 2). Track forecast skill is assessed by comparing NHC’s and each model’s performance to that of a baseline, which in this case is a climatology and persistence model. This model makes forecasts based on a combination of what past storms with similar characteristics–like location, intensity, forward speed, and the time of year–have done (the climatology part) and a continuation of what the current storm has been doing (the persistence part). This model contains no information about the current state of the atmosphere and represents a “no-skill” level of accuracy.
Figure 2. NHC and selected model track forecast skill for the Atlantic basin in 2017, 2018, and 2019.
On the skill diagrams above, lines for models or forecasts that are above other lines are considered to be the most skillful. It can be seen that in each year shown, NHC (black line) outperforms the models and has the greatest skill at most, if not all, forecast times (the black line is above the other colored lines most of the time). Among the models, the ECMWF (red line) has been the best performer, with the GFS (blue line) and UKMET (green line) trading spots for second place.
Yet another metric to estimate how often NHC outperforms the models is called “frequency of superior performance.” Based on this metric, over the last 3 years (2017-19), NHC outperformed the GFS 65% of the time, the UKMET 59% of the time, and the ECMWF 56% of the time. This means that more often than not, NHC is beating these individual models. So the question is, how do the NHC forecasters beat the models?
Keep Your Eyes on the Ball
Forecasters at NHC are quite skilled at assessing weather models and their associated strengthens and weaknesses. It is that experience and a methodology of using averages of model solutions (consensus) that typically help NHC perform best. If you ever read a NHC forecast discussion and see statements like “the track forecast is near the consensus aids,” or “the track forecast is near the middle of the guidance envelope,” the forecaster believed that the best solution was to be near the average of the models. Although this strategy often works, NHC occasionally abandons this method when something does not seem right in the model solutions. One recent example of this was Tropical Storm Isaac in 2018. The figure below (Figure 3) shows the available model guidance, denoted by different colors, at 2 PM EDT (1800 UTC) on September 9 for Isaac, with the red-brown line representing the model consensus (TVCA).
Figure 3. NHC forecast (dashed black line) and selected model tracks at 2 PM EDT (1800 UTC) September 9, 2018 for then-Tropical Storm Isaac. The solid black line represents the actual track of Isaac and the red-brown line represents the model consensus.
Although the models were in fair agreement that the storm would head westward for some time, a few models diverged by the time Isaac was expected to be near the eastern Caribbean Islands, mostly because they disagreed on how fast Isaac would be moving at that time. Instead of being near the middle of the guidance envelope, NHC placed the forecast on the southern side of the model suite (dashed black line) at the latter forecast times since the forecaster believed that the steering flow would continue to force Isaac westward into the central Caribbean. Indeed, NHC was correct in this case, and in fact, for the entire storm, NHC had very low track errors.
In some cases all of the models turn out to be wrong, which usually causes the official forecast to suffer as well. That was the case for a period during Dorian in 2019. Figure 4 shows many of the available operational models at 8 PM EDT on August 26 (0000 UTC August 27) for then-Tropical Storm Dorian. As you can see by noting the deviation of the colored lines from the solid black line (Dorian’s actual track), none of the models or the official forecast (colored lines) anticipated that Dorian would turn as sharply as it did over the northeastern Caribbean Sea, and no model showed a direct impact to the Virgin Islands, where Dorian made landfall as a hurricane.
Figure 4. NHC forecast (dashed black line) and selected model tracks at 8 PM EDT on August 26 (0000 UTC 27 August), 2019 for then-Tropical Storm Dorian. The solid black line represents the actual track of Dorian.
Figure 5 shows many of the operational models at 2 AM EDT (0600 UTC) on August 30 when Dorian, a major hurricane at the time, was approaching the Bahamas. You can see that all of the models showed Dorian making landfall in south or central Florida in about four days from the time of the model runs, and none of them captured the catastrophic two-day stall that occurred over Great Abaco and Grand Bahama Islands. NHC’s forecast followed the consensus of the models in this case and thus did not initially anticipate Dorian’s long, drawn-out battering of the northwestern Bahamas.
Figure 5. NHC forecast (dashed black line) and selected model tracks at 2 AM EDT (0600 UTC) on August 30, 2019 for Hurricane Dorian. The sold black line represents the actual track of Dorian.
The Undervalued Player? A Consistently Good Field-Goal Kicker
In American football, probably one of the most undervalued players on the field is the kicker. They don’t see much action during the majority of the game. But at the end of close games, who has the best chance to win the game for a team? A dependably accurate field goal kicker. In that vein, it’s not just accuracy that can make NHC’s forecasts “better” than the individual models. Another important factor is how consistent NHC’s predictions are from forecast to forecast compared to those from the models. We looked at consistency by comparing the average difference in the forecast storm locations between predictions that were made 12 hours apart. For example, by how much did the 96-hour storm position in the current forecast change from the 108-hour position in the forecast that was made 12 hours ago (which was interpolated between the 96- and 120-hour forecast positions)? Figure 6 shows this 4-day “consistency,” as well as the 4-day error, plotted together for the GFS, ECMWF, UKMET, and NHC forecasts for the Atlantic basin from 2017-19. It can be seen that NHC is not only more accurate than these models (it’s farthest down on the y-axis), but it is also more consistent (it’s farthest to the left on the x-axis), meaning the official forecast holds steady more than the models do from cycle to cycle. We like to say that we’re avoiding the model run-to-run “windshield wiper” effect (large shifts in forecast track to the left or right) or “trombone” effect (tracks that speed up or slow down) that are often displayed by even the most accurate models.
Figure 6. 96-hour NHC and model forecast error and consistency for 2017-2019 in the Atlantic basin (change from cycle to cycle).
NHC’s emphasis on consistency is so great that there are times when we knowingly accept that we might be sacrificing a little track accuracy to achieve consistency and a better public response to the threat. An example would be for a hurricane that is forecast to move westward and pose a serious threat to the U.S. southeastern states. Sometimes, such storms “recurve” to the north and then the northeast and move back out to sea before reaching the coast. When the models trend toward such a recurvature, the NHC’s forecast will sometimes lag the models’ forecast of a lower threat to land. In these cases, NHC does not want to prematurely take the southeastern states “off the hook”, sending a potentially erroneous signal that the risk of impacts on land has diminished, only to have later forecasts ratchet the threat back up after the public has turned their attention and energies elsewhere if the models, well, “change their mind”. That would be the kind of windshield wiper effect NHC wants to prevent in its own forecasts. Now, there are times where the recurvature does indeed occur. Then, NHC’s track forecasts, which have hung back a little from the models, could end up having larger errors than the models. But, NHC can accept having somewhat larger track forecast errors than the models in such circumstances at longer lead times if in doing so it can provide those at risk with a more effective message–achieved in part through consistency.
The superior accuracy and higher levels of consistency of the NHC forecasts are both important characteristics since emergency managers and other decision makers have to make challenging decisions, such as evacuation orders, based on that information. It is not surprising to us that NHC’s forecasts are more consistent than the global models, since forecasters here take a conservative approach and usually make gradual changes from the forecast they inherited from the previous forecaster. Conversely, the models often bounce around more and are not constrained by their previous prediction. And, unlike human forecasters, the models also bear no responsibility or feel remorse when they are wrong!
Filling Out Your Bracket
Accuracy, consistency, and luck are important factors in one particularly favorite sport: college basketball. We just passed the time of year when we should have been crowning champions in the men’s and women’s college basketball tournaments. But before those tournaments would have kicked off, “bracketologists” (no known relation to meteorologists!) would have made predictions on which teams would make it into the tournaments and which teams would have been likely to win.
Think of it this way: a team can be accurate in that they have a spectacular winning record during the regular season, but does that mean they are guaranteed to win the tournament, or even advance far? Nope. As is often said, that’s why they play the game. An inconsistent team—one whose performance varies wildly from game to game—has a higher risk of having a bad game and losing to an underdog in the first few rounds, even if their regular season record by itself suggests they should have no problem winning. The problem is, they could have been very lucky in the regular season, winning a lot of close games that could have easily swung the other way. If that luck runs out, the inconsistent team could have an early exit from the tournament. With a consistent team, on the other hand, you pretty much know what kind of performance you’re going to get—good or bad—and that increases confidence in knowing how far in the tournament the team would advance. You’d want to hitch your wagon to a good team that is consistent and hasn’t had to rely on too much luck to get where they are.
The same can be said for hurricane forecasts from NHC and the models. NHC’s track forecasts are more accurate and more consistent than the individual models in the long run, and that fact should increase overall user confidence in the forecasts put out by NHC. Even still, there is always room to improve, and it is hoped that forecasts will continue to become more accurate and consistent in the future. It is always a good idea to read the NHC Forecast Discussion to understand the reasons behind the forecast and to gauge the forecaster’s confidence in the prediction. For more information on NHC forecast and model verification, click the following link: https://www.nhc.noaa.gov/verification/
— John Cangialosi, Robbie Berg, and Andrew Penny
Semper Paratus (Always Ready): A Shared Mission of Watching Over a Vast Blue Ocean
The National Hurricane Center (NHC) has the responsibility for issuing weather forecasts and warnings for a wide expanse of the Atlantic and eastern North Pacific Oceans. Within NHC, the Hurricane Specialist Unit (HSU) issues forecasts for tropical storms and hurricanes in these regions, issues associated U. S. watches and warnings, and provides guidance for the issuance of watches and warnings for international land areas. NHC’s Tropical Analysis and Forecast Branch (TAFB) makes forecasts of wind speeds and wave heights and issues wind warnings year-round for the eastern North Pacific Ocean north of the equator to 30°N, and for the Atlantic Ocean north of the equator to 31°N and west of 35°W (including the Gulf of Mexico and Caribbean Sea). These wind warnings include tropical storms and hurricanes as well as winter storms, tradewind gales, and severe gap-wind events (for example, the “Tehuantepecers” south of Mexico).
The United States Coast Guard (USCG) has areas of responsibility (AORs) that extend well beyond those of NHC, with potential weather hazards affecting the fleet and their missions over the ocean, inland U.S. waterways, and flood-prone U.S. land areas. Although the USCG is responsible for search and rescue missions that may occur due to weather hazards, they are also vulnerable to severe weather and must also protect their own fleet and crews from these hazards.
One of the USCG’s oldest missions and highest priorities is to render aid to save lives and property in the maritime environment. To meet these goals, the United States’ area of search-and-rescue responsibility is divided into internationally recognized inland and maritime regions. There are five Atlantic USCG Search and Rescue Regions (SRRs) (Boston, Norfolk, Miami, New Orleans, and San Juan) and two Pacific USCG SRRs (Alameda and Honolulu) that overlap with NHC’s hurricane and marine areas of responsibility. The other eastern Pacific regions north of the Alameda SRR do not typically, if ever, experience hurricane activity. The multi-million square mile area of the agencies’ overlap allows NHC to provide weather hazard Decision Support Services (DSS) for the USCG.
Building Partnerships with the Districts
The National Weather Service (NWS) signed a Memorandum of Agreement (MOA) with the USCG to provide them with weather support. Over the past couple of years, staff at NHC have had numerous discussions with several of the USCG districts in order to build stronger partnerships. These discussions, primarily involving how NHC can better serve the USCG, established criteria for requiring TAFB to provide weather briefings to key decision makers within the USCG. When criteria are met, TAFB provides the relevant USCG District with once- or twice-a-day briefing packages detailing the weather impacts on their area of responsibility. This information provides the USCG districts with the details necessary to make efficient and effective decisions about potential mobilization of their fleet.
2018 Hurricane Season Briefing Support
During the 2018 hurricane season, TAFB provided 30 briefings to USCG Districts 5 (Norfolk), 7 (Miami), 8 (New Orleans), and 11 (Alameda) for the several tropical storms and hurricanes that affected them. These interactions helped to build the relationships between NHC and the USCG districts and aided the districts in making decisions regarding fleet mobilization, conducting search and rescue missions, and preparation for USCG’s land-based assets and personnel. Some of these briefings occurred during rapidly evolving high impact scenarios, including Hurricane Michael. Michael was forecast to become a hurricane within 72 hours of developing into a tropical depression and was forecast to make landfall within 96 hours of its formation. Ultimately, Michael rapidly intensified into a category 5 hurricane only 3½ days after formation, before making landfall on the Florida Panhandle. Hurricane Michael’s track across the east-central Gulf of Mexico straddled the border of USCG Districts 7 (Miami) and 8 (New Orleans), leading to both Districts taking action in advance of the hurricane.
Support for District 5 (Norfolk)
The NWS’s Ocean Prediction Center, the NHC (through TAFB), and the NWS National Operations Center have worked together to provide weekly high-level coordination briefings to USCG District 5 on upcoming hazards focused on the Atlantic Ocean north of 31°N over the following seven days. Each Monday (except Tuesday if Monday is a holiday) by noon Eastern Time, the NWS provides a briefing that covers the mid-Atlantic region from New Jersey through North Carolina. Typically, the briefing covers the area to roughly 65°W, though the exact area covered can vary based on the week’s expected weather hazards. The USCG, in turn, has been sharing the information with mariners, port partners, and industry groups for situational awareness and critical decision-making.
NHC’s TAFB is ready to provide decision support services to the USCG Districts for the 2019 hurricane season. Plans are being developed to continue this type of support for many years to come.
— Andy Latto
The State of Hurricane Forecasting is . . .
The National Hurricane Center (NHC) has the responsibility for issuing advisories and U.S. watches/warnings for tropical cyclones (TCs), which includes tropical depressions, tropical storms, and hurricanes, for the Atlantic and east Pacific basins. NHC has a long history of issuing advisories for TCs, with the first known recorded forecast being in 1954, when 24-hour predictions of a TC’s track were made. Since then, we’ve expanded our forecasts out in time and added predictions of TC intensity, size, and associated hazards, such as wind, storm surge, and rainfall. In addition, the lead times of tropical storm and hurricane watches and warnings have increased to give the public additional time to prepare for these potentially devastating events. Since we’re at the time of year when the U.S. President and state governors have just given their “State of the Union” or “State of the State” speeches, we thought this might be a good time to give our own “State of Hurricane Forecasting” speech. This blog entry takes a look at the accuracy of NHC’s forecasts and quantifies how much more accurate they are today compared to decades ago.
Track Forecasting (a.k.a., Where the Storm Will Go)
We are usually more confident in predicting the path of TCs as compared to predicting the strength or size of a TC. The primary reason for this is because the track of a TC is governed by forces larger than the tropical system itself, since the surrounding steering currents cover a much larger area than the hurricane. Because these nearby weather patterns are big, we can usually “see” them easily, and the global weather models do a fairly good job in predicting how these steering features might evolve over the course of a few days.
The figure below shows the average NHC track forecast errors for tropical storms and hurricanes by decade beginning in the 1960s. You can see that there has been a steady reduction in the track errors over time, with the average errors in the current decade about 30-40% smaller than they were in the 2000s and about half of the size (or even smaller) than they were in the 1990s.
If that doesn’t seem impressive, let’s look at another example. The next graphic shows two circles centered on a point near Pensacola, Florida, with the blue one representing the average 48-hour track error in 1990 and the red one showing the average 48-hour error today. What it shows is that if NHC had made a forecast for a storm to be over Pensacola in 48 hours back in 1990, the TC would have ended up, on average, not exactly over Pensacola but somewhere on the blue circle. If NHC makes the same forecast today, now the storm ends up, on average, somewhere on the red circle. You can easily see that the NHC forecasts for the path of a TC today are much more accurate, on average, than they were decades ago, and these more accurate forecasts have helped narrow the warning areas, save lives, and make for more efficient and less costly evacuations.
So, you might be wondering why the track forecasts are more accurate today than in the past. Well, the primary reason is the advancements in technology, specifically the improvements in the observing platforms (satellites, for example) and the various modeling systems we use to make forecasts. The amount and quality of data available to the models so they can paint an initial picture of the atmosphere have increased dramatically in the last 20 to 30 years. Also, the resolution and physics in the models we use today are far superior to what forecasters had available in the 1990s or prior decades, in part due to the tremendous improvements in computational capabilities. In addition, NHC has found ways to even beat the individual dynamical models by using a balance of statistical approaches and experience.
We often hear a lot of questions asking which model is the best one. Although some models are usually better than others, no model is perfect, and their performance varies from season to season and from storm to storm. Two of the most well-known models for weather forecasting are the U.S. National Weather Service’s Global Forecast System (GFS) and the European Centre for Medium-Range Weather Forecasts (ECMWF). The figure below shows a comparison of the NHC forecasts (OFCL, black) and forecasts from the GFS (GFSI, blue) and ECMWF (EMXI, red) models for Hurricanes Harvey, Irma, Maria, and Nate in 2017. In all of these cases, except for Hurricane Irma, OFCL performed as well as or better than GFSI and EMXI. Among the two models, EMXI beat GFSI for Harvey, Irma, and Nate, but GFSI beat EMXI for Maria.
Over the past decade, the average track errors of GFSI and EMXI models have been quite close, so even though EMXI was the best-performing model most of the time in 2017, it does not mean that it will always be the best for every storm. The models that typically have the lowest errors are consensus aids, which blend several models together. Forecasters construct their own forecasts of how the storm will evolve, aided by model simulations and their knowledge of model strengthens and weaknesses.
Even though our track forecasts are much more accurate today – in fact preliminary estimates are that the 2017 Atlantic track forecasts set record low errors at all time periods – typical track errors currently start off at 37 n mi at 24 hours and then increase by about 35 n mi (40 mi ) per day of the forecast. This means that our 5-day track error is on average around 180 n mi (210 mi). So, keep that in mind and be sure to account for forecast uncertainty when using NHC forecasts next hurricane season.
Intensity Forecasting (a.k.a., How Strong the Storm Will Get)
Predicting the intensity of a tropical storm or hurricane is usually more challenging than forecasting its track. This is because the intensity of these weather systems is affected by factors that are both big and small. On the large scale, vertical wind shear (the change of wind speed and direction with height) and the amount of moisture in the atmosphere greatly affect the amount or organization of the thunderstorm activity that the TC can produce. Ocean temperatures also affect the system’s intensity, with temperatures below 80° F usually being too cool to sustain significant thunderstorm activity. However, smaller-scale features can also be at play. One of the more complex phenomena that affects a TC’s intensity is an eyewall replacement cycle. Initially, when two eyewalls, one inside the other, are present, the hurricane’s wind field will begin to expand, and as the inner eyewall dies, the hurricane’s peak winds start to weaken. However, if the second eyewall contracts, the hurricane can often re-intensify. The radar image below of Hurricane Irma (2017) was taken at the beginning of an eyewall replacement cycle, when the hurricane had a double eyewall structure.
Given these complex factors and the fact that errors in the track can also affect the TC’s future intensity, we have not made as much progress in this area as we have for track forecasting. The next graphic (below) shows NHC average intensity errors for Atlantic tropical storms and hurricanes by decade starting in the 1970s. Note that only small improvements were made in the intensity predictions from the 1970s through the 2000s. A much more significant reduction in error has occurred in the current decade, which could mean that the recent investment in new models and techniques is beginning to pay off. Today’s intensity errors are close to 15 kt (17 mph) from 72 to 120 h. This number is on the order of one Saffir-Simpson category, so we often encourage those who could be affected by a TC to prepare for a storm one category stronger (on the Saffir-Simpson Hurricane Wind Scale) than what we are forecasting.
Although the GFS and ECMWF models are skillful for track forecasting and help us understand the environment around the TC, did you know that these models are typically inadequate to predict how strong a TC might become? Both the GFS and ECMWF are global models, and they cannot “see” sufficient detail within the storm to represent and predict the core winds in the hurricane’s eyewall. Therefore, we use different models to predict intensity, some that are run at high resolution specifically for TCs (e.g., Hurricane Weather Research and Forecasting [HWRF] model, Hurricanes in a Multi-scale Ocean-coupled Non-hydrostatic [HMON] model) and some that are statistical in nature (e.g., Statistical Hurricane Intensity Prediction Scheme [SHIPS], Logistic Growth Equation Model [LGEM]). The statistical models tell the forecaster what typically occurs for a TC in a specific location and environment based on past storm behavior. Even though the intensity models are improving, the gains in these models are much smaller than what has occurred in the models we use for track forecasting.
If you want more information on the models, please visit the following page for details: http://www.nhc.noaa.gov/modelsummary.shtml
Will the errors keep decreasing?
The short answer is they likely won’t forever. At some point the forecasts made by NHC and other forecasting centers will likely reach the limits of predictability. No one knows for sure what those limits are or when they will be reached, but researchers are still providing great information that is helping NHC make steady advancements as discussed above.
For more information on the NHC and model verification please visit the following page: http://www.nhc.noaa.gov/verification/
— John Cangialosi
Two years ago this month, Tropical Storm Bill made landfall along the central Texas coast, just 17 hours after becoming a tropical cyclone only 145 miles offshore. The precursor disturbance, a broad and ill-defined area of low pressure, had already been producing tropical-storm-force winds, and there was little doubt that the system would soon bring those dangerous winds onshore. Although NHC’s Tropical Weather Outlooks had been talking about the possibility of those conditions two days in advance, and their likelihood one day in advance, some in the media and emergency management communities lamented the lack of earlier formal tropical storm warnings and full advisory products from NHC. A few even suggested that NHC classify the disturbance as a tropical storm when it wasn’t one. By policy and tradition, NHC advisories, track and intensity forecasts, and any associated watches and warnings begin only after a disturbance has become a tropical cyclone; in this case a tropical storm warning was issued as soon as Bill formed, about 12 hours before the hazardous winds reached the coast. For some additional discussion on why warnings couldn’t have been issued any earlier for Bill, please see our blog post written after that event.
This is hardly the only example of a tropical cyclone striking land shortly after genesis, and well within the normal 48-hour watch/warning time frame. In 2010, Tomas struck Barbados as a tropical storm 27 hours after formation, and St. Vincent and St. Lucia as a hurricane 38 hours after formation. In September of 2007, Humberto made landfall as a hurricane along the Texas coast a mere 19 hours after becoming a tropical cyclone. This recurring problem has been on our minds for a long time, and this season we’ve introduced a service enhancement to address the issue.
Starting this year, NHC has the option to issue advisories, track and intensity forecasts, watches, and warnings for disturbances that are not yet a tropical cyclone, but which pose the threat of bringing tropical storm or hurricane conditions to land areas within 48 hours. This substantial change in policy means that we won’t have to wait for a disturbance to meet the technical requirements of a tropical cyclone (such as having a well-defined center of circulation or sufficiently organized thunderstorm activity) to issue forecasts or post warnings. And boy, it didn’t take long for us to employ this new option, with both the pre-Bret and pre-Cindy disturbances requiring the initiation of potential tropical cyclone advisories on two consecutive days! But more on that in a moment.
Although we’ve been working on the technical and administrative changes to bring this about over the past two years, the effort actually began after the Deepwater Horizon disaster in 2010, when NHC was asked to provide enhanced forecast support for the response effort. Since then, NHC has been practicing making track and intensity forecasts for disturbances, and at the same time we’ve been improving our ability to forecast tropical cyclone genesis. We now believe that the science has advanced enough to allow the confident prediction of tropical cyclone impacts while these systems are still in the developmental stage.
So for these land-threatening “potential tropical cyclones” (and that’s the term we’re using in our advisories), NHC can now issue the full suite of text, graphical, and watch/warning products that previously has only been used for ongoing tropical cyclones. This includes the cone graphic, public advisory, discussion, wind speed probabilities – everything – and all the products will look exactly the same as our tropical cyclone products. The only thing that’s different is what we call the “system type”; we’ve added POTENTIAL TROPICAL CYCLONE to the roster of possible system types. And since you asked (or at least were thinking about asking), here’s the complete list:
POTENTIAL TROPICAL CYCLONE
For those who are interested in the definitions of each of these system types, you can find them in National Weather Service Instruction 10-604, Tropical Cyclone Names and Definitions.
We did consider some alternatives to the term potential tropical cyclone. “Tropical disturbance” was a fairly obvious option but we knew that some of these precursor disturbances weren’t going to be tropical in nature (such as a frontal cyclone evolving into a subtropical or tropical cyclone), so that eliminated tropical disturbance. Another option was simply “disturbance”, which aside from evoking Star Wars imagery (I felt a great disturbance in the Gulf), did not in our view adequately convey the appropriate level of threat. In the end, potential tropical cyclone seemed both accurate and appropriate to the threat, although it’ll take a bit of getting used to for some.
Potential tropical cyclones will share the naming rules currently used for depressions, with depressions and potential tropical cyclones being numbered from a single list (e.g., “One”, “Two”, “Three”, …, “Twenty-Three”, etc.). The assigned number will always match the total number of systems we’ve written advisories on within that basin during the season. For example, if three systems requiring advisories have already occurred within a basin in a given year, the next land-threatening disturbance would be designated “Potential Tropical Cyclone Four”. If a potential tropical cyclone becomes a tropical depression, its numerical designation doesn’t change (i.e., Potential Tropical Cyclone Four becomes Tropical Depression Four).
Potential tropical cyclone advisory packages will be issued at the standard advisory times of 5 AM, 11 AM, 5 PM, and 11 PM EDT, with three-hourly Intermediate Public Advisories being issued at 2 AM, 8 AM, 2 PM, and 8 PM EDT when watches or warnings are in effect. The product suite will include a five-day track and intensity forecast, just as is done for ongoing tropical cyclones. In addition, the Potential Storm Surge Flooding Map and Storm Surge Watch/Warning graphic would be issued for these systems when appropriate. We’ll continue issuing advisory packages on a potential tropical cyclone until watches or warnings are discontinued or until the threat of tropical-storm-force winds for land areas sufficiently diminishes, at which point advisories would be discontinued. However, if it seems likely that new watches or warnings would be necessary within a short period of time (say 6-12 hours), then advisories could continue during that brief gap in warnings in the interest of service continuity.
Since the primary issuance trigger is the threat of tropical storm conditions over land, there won’t be any specific threshold of formation likelihood for the initiation of advisories. For example, a fast-moving tropical wave approaching the Lesser Antilles might already have tropical-storm-force winds but no closed wind circulation. In this case, a genesis forecast of 40% – 50% would likely be enough to trigger advisories and warnings. In contrast, a genesis forecast of 70% for a system close to shore might not trigger advisories if the system were not expected to reach tropical storm strength before moving inland.
The issuance of NHC products for potential tropical cyclones is very much analogous to the change that occurred after Hurricane Sandy in 2012, when NHC advisories on post-tropical cyclones became possible. After Sandy, we realized that there was great benefit to users in NHC’s being able to continue writing advisories on systems even after they were no longer a tropical cyclone. That solved the service continuity problem on the “back end”, and now we’re completing the process by ensuring a steady flow of information on the front end of a tropical cyclone’s life cycle. In all cases, we’ll be trying to ensure that warning types (tropical vs. non-tropical) don’t have to change in the middle of an event.
There are some things to be aware of with this new capability. First, potential tropical cyclone advisories will not be issued for systems that threaten only marine areas – largely because this would pose an unmanageable workload/staffing issue for us but also because marine forecast products (the High Seas and Offshore Waters forecasts) already allow the issuance of gale and storm warnings before a tropical cyclone has formed.
Second, because potential tropical cyclones will have a standard five-day forecast track and uncertainty cone, to avoid potential confusion with the cone we’re going to stop drawing potential formation areas for these systems in the Graphical Tropical Weather Outlook.
We’re also concerned that some users may pay too much attention to the longer-range part of these new forecasts (the part beyond 72 hours). We know that forecast errors for weaker and developing systems tend to be larger than those for strong storms and hurricanes, and we even considered only going out to 72 hours with the new potential tropical cyclone advisories (since the primary purpose was to support watches and warnings). But in the end, consistency and technical issues argued for going out to five days, and that’s what we’re doing. So it’s likely that forecast-to-forecast changes in the longer-range portion of our potential tropical cyclone advisories will be larger than what folks are used to. And for those of you who like to look at forecast model intensity guidance, be aware that most of these intensity models assume the system is a tropical cyclone. Since that won’t be the case for these systems, intensity models run on potential tropical cyclones will generally have a high bias. And lastly, since many potential tropical cyclones will not have well-defined centers, there will likely be large jumps in the reported location of these systems from advisory to advisory. But even with all these caveats, we think that the ability to post warnings before a cyclone forms is an important service enhancement – one that will help save lives and protect property, while at the same time allowing NHC to analyze and report on tropical systems as accurately and as honestly as possible.
After our experiences with Bret and Cindy, we’re optimistic about the value of this new capability. Advisories on Potential Tropical Cyclone Two were started 24 hours before Bret officially became a tropical cyclone, giving residents of Trinidad and Tobago, Grenada, and northeastern Venezuela an additional day of warning for tropical storm conditions. If this were still 2016, places like Trinidad may have only had three to six hours between the time of the first advisory and the time when tropical storm force winds began on the island. And for Cindy, advisories on Potential Tropical Cyclone Three were initiated roughly 21 hours before Cindy met the criteria of a tropical cyclone. This allowed Tropical Storm Warnings to be issued for southeastern Louisiana 21 hours earlier than they would have been if the storm had occurred last year.
Just a few weeks into the new season, we’re pretty happy about the way this all worked. We think we successfully demonstrated the ability to provide more advanced warning than we could have in previous years for these developing tropical cyclones. But we’d love to hear feedback from our users, customers, and partners. Were the potential tropical cyclone advisories in advance of Bret and Cindy confusing? Helpful? Maybe both? Or bad puns aside, did the new capability fit the “Bill”?
If you’d like to provide comments on your experiences with the Potential Tropical Cyclone advisories during Bret and Cindy, please feel free to contact Jessica Schauer, the NWS Tropical Cyclone Program Leader, at Jessica.Schauer@noaa.gov.
— James Franklin
Editor’s Note: This post marks James’s last blog contribution as a member of the NHC family. After 35 years of service in the federal government (17 years at NOAA’s Hurricane Research Division and 18 years at the National Hurricane Center), James is retiring at the end of this week. We want to thank James for his contributions to not only the blog, but also for his many contributions to hurricane forecasting and NHC operations over the past several decades. Although James will no longer be “inside the eye” of the sometimes-hectic NHC scene, we know he won’t be too far away cheering on his beloved Miami Hurricanes, Miami Dolphins, and Florida Panthers. Congratulations, James, and happy retirement!
Tropical Storm Erika, coming as it did so close to the beginning of the new college and professional football seasons, is a reminder that Monday-morning quarterbacking is nearly as popular an activity as the sport itself. And we at NHC do it too. After every storm we review our operations with an eye toward improving our products and services. Erika is no different, though there’s been more questioning and criticism than usual, with few components of the weather enterprise spared. Some in the media were accused of overinflating the threat, numerical models were bashed, and some public officials were charged with overreacting. NHC’s forecasts were questioned while others lamented that NHC’s voice wasn’t strong enough amid all the chatter. So, in the spirit of searching for a tropical storm eureka, in this blog entry we present some of our own post-storm reflections.
How good were NHC’s forecasts for Erika?
The NHC official forecast errors were larger than average. A preliminary verification shows that the average 72-hr track error for Erika was 153 n mi, about 30% larger than the 5-year average of 113 n mi. And nearly all of this error was a rightward (northward) bias – that is, Erika moved consistently to the left of the NHC forecast track. As for intensity, Erika ended up being weaker than forecast; the 72-hr intensity forecasts were off by about 20 kt on average, and the official 5-day forecasts called for a hurricane over or near Florida until as late as 2 AM Friday, when it became clear that Erika was going to have to deal with the high terrain of Hispaniola.
Why was Erika so difficult to get right?
Erika was a weak and disorganized tropical cyclone, and weak systems generally present us (and the computer simulations we consider) with tougher forecast challenges. In fact, the average 72-hr track error for all tropical depressions and weak tropical storms is around 155 n mi – just about exactly what the errors for Erika were. So the track issues weren’t really a surprise to us. Of course, knowing whether such errors were going to occur and how to reduce them in real time wasn’t obvious. If it had been obvious, we would have called an audible and changed the forecast.
What made this particular situation so challenging was wind shear, mainly due to strong upper-level westerly winds in Erika’s environment. These winds tended to displace the storm’s thunderstorms away from its low-level circulation, causing the storm to lack a coherent vertical structure. When this happens, it’s very difficult to tell just how much influence those upper-level winds will have on the storm track. Sometimes storms hold together against wind shear (Andrew of 1992 is a good example), and there were times when Erika seemed to be winning its battle. If it had held together better, it would have taken a track more to the north and ended up being a much stronger system. Obviously, it didn’t play out that way, but that was an outcome far easier to see with the benefit of hindsight.
An additional complication was Puerto Rico and Hispaniola. If Erika had been able to avoid those land masses, it would have been better able to withstand the disruptive effects of wind shear. And early on, we expected Erika to mostly avoid land. In this case, not getting the track right made it much harder to get the intensity right, which made the track forecasts harder yet.
Is there too much reliance on numerical models, and did they fail for Erika?
The improvements in track forecasting over the past few decades are directly attributable to improvements in numerical models, and to the data used to initialize them, to the point where it’s become almost impossible for a human forecaster to consistently outperform the guidance. The modelling community deserves our praise for the tremendous progress they’ve made, not criticism for missing this one. While we approach each forecast with an attempt to diagnose when and how the models might fail, it is exceedingly difficult, and it’s not something we do in our public forecasts unless our confidence is very high.
Human forecasters (including those at NHC) are still able to occasionally outperform the intensity models, mainly because satellite depictions of storm structure can often be used by forecasters more effectively than by models, giving us an edge in certain circumstances. But neither human forecasters nor the models are particularly good at anticipating when thunderstorms in the cyclone core are going to develop and how they’re going to subsequently evolve, especially for weaker cyclones like Erika.
Because the atmosphere is a “chaotic” physical system, small differences in an initial state can lead to very large differences in how that state will evolve with time. This is why the model guidance for Erika was frustratingly inconsistent – sometimes showing a strong hurricane near Florida, while at other times showing a dissipating system in the northeastern Caribbean. It’s going to take a large improvement in our ability to observe in and around the tropical cyclone core (among other things), to better forecast cases like Erika for which storm structure is so important to the ultimate outcome. But our best hope for better forecasts lies in improved modeling–a major goal of the Hurricane Forecast Improvement Program (HFIP).
Given the overall model guidance we received during Erika, it’s hard even now, well after the storm, to see making a substantially different forecast with current capabilities and limitations. In fact, had our first few advisories called for a track much farther south at a much weaker intensity, or even a forecast of dissipation due to interaction with Hispaniola, our partners and the public might rightly have questioned our rationale to go firmly against many model forecasts of a stronger system farther north.
Was the message from NHC muddled?
We think that there might be some ways for NHC to make key aspects of our message easier to find. Although NHC’s Tropical Cyclone Discussions (TCDs) repeatedly talked about the uncertainty surrounding Erika’s future beyond the Caribbean, including the possibility that the cyclone could dissipate before reaching Florida, it does not appear that this was a prominent part of the media’s message to Florida residents. Making key “talking points” more distinctly visible in the TCD and the Public Advisory are options we are considering, as well as enhanced use of NHC’s Twitter and Facebook accounts. Having said that, consumers and especially re-packagers of tropical cyclone forecast information, like our media partners, should take some responsibility for making use of the information that is already available. We also invite our media partners to take more advantage of the numerous training sessions we offer, mostly during the hurricane offseason. Reaching anyone in the television industry with such training, except for on-camera meteorologists, has proven over the years to be very difficult. We would like to train more reporters, producers, news department staff, executives, etc. so they are more sensitized to forecast uncertainty and how to communicate it with the help of our products, but we realize that a more focused “talking points” approach as described above will probably be needed to assist these busy folks in conveying a consistent message.
An NHC advisory package contains a variety of products, each geared to providing a certain kind of information or to serving a particular set of users. Some of our media partners have expressed concerns over the increasing number of NHC products, but the various wind and water hazards posed by a tropical cyclone cannot be boiled down to one graphic, one scale, or one index. We are, in fact, still in the process of intentionally expanding our product and warning suite to focus more on the individual hazards and promote a more consistent message about those hazards. Even so, two of our products that have been around for many years are still crying out for greater visibility.
We’ve already mentioned the Tropical Cyclone Discussion, a two- to four-paragraph text product that is the window into the forecaster’s thinking and provides the rationale behind the NHC official forecast. In the TCD we talk about the meteorology of the situation, indicate our level of confidence in the forecast, and when appropriate, discuss alternative scenarios to the one represented by the official forecast. Anyone whose job it is to communicate the forecast needs to make the TCD mandatory reading on every forecast cycle.
Some users may not understand the amount of uncertainty that is inherent in hurricane forecasts (although we suspect Florida residents now have a greater appreciation of it than they had two weeks ago). We need to continue to emphasize, and ask our media partners to emphasize, NHC’s Wind Speed Probability Product, available in both text and graphical form, which describes the chances of hurricane- and tropical-storm-force winds occurring at individual locations over the five-day forecast period. Someone looking at that product would have seen that at no point during Erika’s lifetime did the chance of hurricane-force winds reach even 5% at any individual location in the state of Florida, and that the chances of tropical-storm-force winds remained 50-50 or less. In addition, in some of the number crunching we did after the storm, we calculated that the chance of hurricane-force winds occurring anywhere along the coast of Florida never got above 21%.
We realize that at first it seems counterintuitive that we are forecasting a hurricane near Florida while no one in that state has even a 5% chance of experiencing hurricane-force winds. That, however, is the reality of uncertainties in 5-day forecasts, especially for weaker systems like Erika, and the wind speed probabilities reliably convey the wind risk in each community. We did notice a couple of on-air meteorologists referencing the Wind Speed Probabilities, which is great – and the more exposure this product gets, the better. We would like to work with our television partners to help them take advantage of existing ways for many of them to easily bring the wind speed probabilities into their in-house graphics display systems.
A very nice training module exists for folks interested in learning about how to use the Wind Speed Probabilities: https://www.meted.ucar.edu/training_module.php?id=1190#.VenooLQ2KfQ. You will need to register for a free COMET/MetEd account in order to access the training module.
Did Floridians over-prepare for Erika?
Anyone who went shopping for water and other supplies once they were in the five-day cone did exactly the right thing (of course, it’s much better to do that at the beginning of hurricane season!). Being in or near the five-day cone is a useful wake-up call for folks to be prepared in case watches or warnings are needed later. But as it turned out, no watches or warnings were ever issued for Florida. In fact, at the time when watches would normally have gone up for South Florida, NHC decided to wait, knowing that there was a significant chance they would never be needed – and that turned out to be the right call.
Watches (which provide ~48 hours notice of the possibility of hurricane or tropical storm conditions) and warnings (~36 hours notice that those conditions are likely) seem to be getting less attention these days, with the focus on a storm ramping up several days in advance of the first watch. While the additional heads-up is helpful, we worry about focusing in on specific potential targets or impacts that far in advance. The watch/warning process begins only 48 hours in advance because that’s the time frame when confidence in the forecast finally gets high enough to justify the sometimes costly actions that an approaching tropical cyclone requires. (Although, we recognize that some especially vulnerable areas sometimes have to begin evacuations prior to the issuance of a watch or a warning, and we and the local Weather Forecast Offices of the National Weather Service directly support emergency management partners as they make such decisions.)
What can NHC do better next time?
While we’d like to make a perfect forecast every time, we know that’s not possible. Failing that, we’re thinking about ways we can improve the visibility of our key messages, particularly those that will help users better understand forecast uncertainty. As noted above, we’re considering adding a clearly labeled list of key messages or talking points to either the TCD or the Tropical Cyclone Public Advisory, or both. We’d also like to try to make increased use of our Twitter accounts (@NHC_Atlantic, @NWSNHC, and @NHCDirector), which so far have been mostly limited to automated tweets or to more administrative or procedural topics. We’re also looking at whether the size of the cone should change as our confidence in a forecast changes (right now the cone size is only adjusted once at the beginning of each season), and thinking about ways to convey intensity uncertainty on the cone graphic. But most of all, we need to keep working to educate the media and the public about the nature of hurricane forecasts generally, and how to get the information they need to make smart decisions when storms threaten.
— James Franklin
The development of Tropical Storm Bill so close to the Texas coast, with the posting of a formal tropical storm warning only about 12 hours before winds of that intensity came ashore on Tuesday June 16th, highlighted a long-standing and well-known limitation in the tropical cyclone program of the National Weather Service (NWS). An ongoing NHC initiative to improve service for such systems is the subject of today’s blog post.
Although there’s nothing new about a tropical cyclone forming on our doorstep, what is new is an increased ability to anticipate it. NHC has greatly enhanced its forecasts of tropical cyclone formation over the past several years, introducing quantitative 48-hr genesis forecasts to the Tropical Weather Outlook in 2008, and extending those forecasts to 120 hours in 2013. In 2014, we introduced a graphic showing the locations of tropical disturbances and the areas where they could develop into a depression or storm over the subsequent five days. Thirty-six hours in advance of Bill’s formation, NHC gave the precursor disturbance a 60% chance of becoming a tropical cyclone, and increased that probability to 80% about 24 hours in advance. While nothing was guaranteed, we were pretty confident a tropical storm was going to form before the disturbance reached the coast. And although we weren’t issuing specific track forecasts for the disturbance, NHC’s new graphical Tropical Weather Outlook (example below) showed where the system was generally headed.
We wouldn’t have had such confidence 20 years ago, or even 5 years ago. And so our tropical cyclone warning system, developed over several decades, doesn’t allow for a watch or warning until a depression or storm actually forms and NHC’s advisories begin; by both policy and software, warning issuances are tied to cyclone advisories. If we had wanted to issue a tropical storm watch for Bill on the morning of Sunday the 14th (48 hours prior to landfall), or a warning that evening (36 hours ahead of landfall), we would have had to pretend that the disturbance in the Gulf of Mexico was a tropical cyclone. Even during the day on Monday, data from an Air Force Reserve Hurricane Hunter aircraft showed that the disturbance had not yet become a depression or storm.
Couldn’t NHC have called the disturbance a tropical storm anyway, in the interest of enhanced preparedness? Yes, but what if the disturbance never becomes a tropical storm – remember, even an 80% chance of formation means it won’t become a tropical storm at least once every five times. So naming it early risks the credibility of the NWS and NHC, and endangers a trust we’ve worked for decades to establish. In addition, there are legal and financial consequences to an official designation of a tropical cyclone – consequences that obligate us to call it straight. And finally, as custodians of the tropical cyclone historical record, we have a responsibility to ensure the integrity of that record.
When systems that have the potential to become tropical cyclones pose hazards to life and property, NHC’s best avenue for highlighting those hazards currently is the Tropical Weather Outlook. Ahead of Bill’s formation, the possibility of tropical storm conditions along the middle and upper Texas coast was included in the Sunday evening Outlook, and by early Monday afternoon the Outlooks were saying those conditions were likely. Products issued by NWS local forecast offices (WFOs) carried similar statements.
Although most folks seemed to have gotten the message that a tropical storm was coming, it’s widely thought that the Tropical Weather Outlook and local WFO products don’t carry the visibility and weight of an NHC warning, or of an NHC advisory package with its attendant graphics. In addition, some institutions have preparedness plans that are tied to the presence of warnings. We agree that warnings during the disturbance stage could improve community response, and we’ve been working toward that goal since 2011. In that season, NHC initiated an internal experiment in which the Hurricane Specialists prepare track and intensity forecasts for disturbances with a high likelihood of development, and use these forecasts to determine where watches and warnings would have been appropriate. These internal disturbance forecasts have had some successes and failures, but may now be good enough to make public.
With our colleagues across the NWS, we’re now working through the logistics of expanding the tropical cyclone product and warning suite to accommodate disturbances. One plan under consideration calls for NHC to produce a five-day track and intensity forecast for those disturbances having a high chance of becoming a tropical cyclone, and which pose the threat of bringing tropical-storm-force winds to land areas. The forecasts would be publicly issued through the standard NHC advisory products, including the Public Advisory, Discussion, and Wind Speed Probability Product, along with the forecast cone and the other standard graphics. These advisory packages would be issued at the normal advisory times, and continue until the threat of tropical-storm-force winds over land had diminished. If and when the disturbance became a tropical cyclone, advisory packages would simply continue.
We are still evaluating these and other options for getting tropical cyclone warnings out for potential tropical cyclones. If we do begin issuing forecasts for these systems, we know from our experimental forecasts that they won’t be as accurate as our current public forecasts for tropical cyclones are – and we’ll want to make sure users know about those uncertainties. There are many details to iron out and much technical work to do, but we’re hopeful to have this service enhancement in place for the 2017 hurricane season.