Stats on accuracy of NHC TWOs (yellow/orange/red %)

[jinftl]

Great post by Dr. Jeff Masters this morning on the overall accuracy of the % and color code features on the NHC's Tropical Weather Outlooks. Last year, once an area of disturbance got the 70% probability red circle or higher - it developed 100% of the time.



How accurate is NHC's Tropical Weather Outlook?

"So the big question is, how good are these forecasts? When NHC gives a 30% chance that an "Invest" will become of tropical cyclone, does this happen 30% of the time? Well, according to the 2012 National Hurricane Center Forecast Verification Report, NHC should have drawn fewer yellow circles and more orange circles and red circles during 2012, as there was a tendency to under-predict when a threat area might develop. For example. for the 46 forecasts where a 30% chance of development was given, 50% of the threat areas actually developed. Every disturbance that was given a 70% and higher chance of development ended up developing.

This under-prediction tendency in 2012 is in contrast to the results from 2011, when the genesis forecasts were closer to the mark. For example, the 59 forecasts for a 30% chance of development resulted in a 31% "hit" rate of the storm actually developing in 2011. So far in 2013, there seems to be a tendency to under-predict again. For example, 48 hours before Tropical Storm Barry developed, NHC was carrying just a 10% chance of development."



link to full post with additional info/graphics: http://www.wunderground.com/blog/JeffMa ... rynum=2468

[artist]

I think he is not figuring in the qualifier that states 'within 48 hours' is he?

[jinftl]

I believe he did - there was a note in the actual blog post below the picture i attached which referenced it:

NHC did predictions on new formation of a tropical depression or tropical storm (cyclogenesis) beginning in 2010. The forecasts are expressed in the Tropical Weather Outlook in 10% probability increments, and in terms of categories (“low”, “medium”, or “high”) for a tropical cyclone forming within a 48-hour period. These genesis forecasts had a low (under-forecast) bias in the Atlantic basin during 2012 (top). For example, for cases where a 30% chance of formation was given, the actual percentage of storms that formed was 50%.

[RL3AO]

Dr. Master's didn't do the math, it was a part of the annual NHC Verification Report, which he linked too.

http://www.nhc.noaa.gov/verification/pd ... n_2012.pdf

Its on page 52.

[artist]

thanks to both of you. Very interesting...

[BigB0882]

So basically, if the NHC gives it a 70% chance or higher then it will almost surely form. They have never missed one of them yet, as matter of fact.

[jinftl]

so it would seem - once the NHC reaches that 'red' threshold of development in the next 48 hours, the wheels seem to be pretty much in motion at that point. I don't have the data, but it seems the time period for when a system is assigned a red (70% or greater) threat of development in the next 48 hours, the actual time until advisories are initiated as a td or ts often can be a matter of hours - a 2pm TWO red seems to turn to advisories by 5pm or 11pm that same day - maybe the next morning the latest for instance.

So basically, if the NHC gives it a 70% chance or higher then it will almost surely form. They have never missed one of them yet, as matter of fact.

[RL3AO]

So basically, if the NHC gives it a 70% chance or higher then it will almost surely form. They have never missed one of them yet, as matter of fact.

That was just for last year. I believe if you look at previous seasons they've had 80 and 90 percent systems not develop.

[HurricaneBelle]

That was just for last year. I believe if you look at previous seasons they've had 80 and 90 percent systems not develop.

Yeah, I recall the infamous system near Louisiana a few years back that went from 0 or 10 to 80 (courtesy of Stewart) and back again within the space of 3-4 outlooks.

[jinftl]

2011 was the first season the NHC used this new system - the actual % developing were a bit over and under with those predictions - maybe they needed to get the kinks out and there is a learning curve with any new approach. Still decent overall though.

That was just for last year. I believe if you look at previous seasons they've had 80 and 90 percent systems not develop.

Yeah, I recall the infamous system near Louisiana a few years back that went from 0 or 10 to 80 (courtesy of Stewart) and back again within the space of 3-4 outlooks.

[CrazyC83]

It can also be looked at from a Probability of Precipitation perspective.

When the POP is 0%, nothing is said. In this case, it would be worth mentioning if it is near land for potential land impacts only. Most NWS offices will also not make mention at 10% either.

A 20% area means a slight chance for precipitation - could be seen the same way for development. Also 30% to 50% means a chance exists (same with a code orange from NHC).

Once at 60% and 70%, it mentions precipitation is likely, and it appears development becomes likely at that stage as well. 80% and 90% also are assumed that it will happen in wording as well.

Some thoughts:
* They should not draw areas with less than 20% probability, except at dissipation, when land is not threatened. 0% and 10% should still be used near land though if impacts are expected or occurring even if genesis is not expected (i.e. flooding rains).

* Also, if a storm is far from land, a probability higher than 70% should not be used, except when the situation is "holding" - i.e. genesis likely taken place but waiting additional data (i.e. a visible satellite or persistence).

* When a probability is higher than 70% and a system is near land, watches and warnings should be issued if necessary just as if it was an operational tropical cyclone. In that case, it would be assumed that a Hurricane Hunters aircraft is on its way or initial development is occurring.