Poor Performance of the Global Models During the 2020 Season

[BYG Jacob]

Figured I would start a discussion for some of the horrendous accuracy of models this year. Constant phantom storms in the EPAC, complete misses on even having a storm, and most recently a freakin hurricane 3 days before they form, and then drastically underdoing how intense they are. The models have sucked in 2020, and I'm wondering why.

[SconnieCane]

(Lack of) aircraft data due to the reduction in commercial flights during the COVID-19 pandemic.

However, I'd noticed the models seeming to struggle even in the last years of the 2010s. I feel like the early part of the decade was the heyday for NWP (particularly the EURO and its coup on Sandy, or the 4/27//11 tornado outbreak (enabling SPC to nearly perfectly place the area of highest risk on their outlooks compared to where the most numerous and violent tornadoes actually ended up tracking). Tweaks made since then to improve the models seem to have often had the opposite effect, at least in my subjective opinion.

[EquusStorm]

Just a stark reminder that models are tools not rules and have to be interpreted with a modicum of knowledge and care instead of taken verbatim... essential tools, absolutely, but gotta be careful leaning too hard without really analyzing

[SconnieCane]

An addendum to my previous post. There have been some recent significant Atlantic tropical cyclones where the global models did quite well. Redeveloping Harvey for example, where after a few runs of burying it into Mexico, they caught onto an eventual Texas landfall resulting in Biblical rainfall totals with quite a bit of lead time.

Irma's crucial last few days were also handled rather well, with consensus (naturally with some outlier models/runs) building on a WNW track that would hook sharply to the right somewhere near the Bahamas or Florida Straights. It just so happened that exactly where that turn took place made a world of difference in terms of impacts. It also was apparent with several days (at least as long if not longer than the hurricane watch period) of lead time that Puerto Rico was in Maria's crosshairs.

The 500mb pattern in September 2018 was modeled well enough and with enough consistency that it was evident soon after genesis that Florence had a much higher likelihood of making an eventual CONUS landfall than most cyclones forming where it did. Likewise Michael, other than some outliers taking it to near Tampa Bay had a pretty strong early consensus on a landfall near Panama City (intensity was another matter...).

My comment about the models struggling has more to do with land-based severe convective weather. It seems every spring for the last 4-5, there are several significant severe weather/tornado outbreaks apparent on the global models in the medium range that fail to materialize. Even the vaunted Euro has had issues with this starting in about 2017. I know phantom outbreaks just like phantom TCs are going to show up now and then, but for ALL of them to be misses?...

[St0rmTh0r]

I'm not ringing the doom bell and I respect the hard work that goes into building and running these models. However, as most can see this has been a hyperactive July by all standards. A Hurricane in the Gulf was missed by all models. They are struggling with the I storm and there developing TD right behind that. A tropical storm is spinning off Africa as I type. The +NAO
is setting up in August as we can see also. This is going to be a major problem if the models can't pick up on the storms heading into what is more than likely a hyperactive peak. It needs to get fixed soon or there will be a lot of chaos

[tolakram]

The only difference I see is we don't have the old GFS spinning up every storm, the euro has never been good at cyclogenesis, only catching sure things in ideal conditions.

[FireRat]

It seems that it would be a very good idea to take an old school approach this year, and observe areas which have activity in the present and near future like meteorologists of the 20th century did, and then wait for a TD to form. Once this occurs, then we ought to take the models into account seriously for track and potential intensity.

This year cyclogenesis has been almost impossible to foresee because almost every disturbance is finding favorable conditions to develop. I would actually add, that this year probably would've been a good one for the old GFS, because most phantom storms would actually have spun up! I suspect that in this Gung-Ho of a year in the Atlantic, the lack of phantom systems due to corrections of the models, have and will continue to cause us to miss the formation of several storms up until they almost become tropical storms. 2020 is definitely a good year to use our eyes to try and predict storm formation, no pun intended!

[Hammy]

The only difference I see is we don't have the old GFS spinning up every storm, the euro has never been good at cyclogenesis, only catching sure things in ideal conditions.

Why does it seem like it has such a tendency to spin up every cloud into a hurricane in the EPAC but fails to catch systems in the Atlantic--are there different dynamics it's not picking up?

[BYG Jacob]

Bump because the models have completely failed with Joesephine

[BYG Jacob]

Can I bump this now for the Euro and GFS failing on our 2 invests, or do I have to wait until they're both hurricanes?

[tolakram]

I'm not sure what the definition of failure is. A professional reading the model output can still use them effectively, the latest tropical tidbit video explained this well.

Failure to us is when a model doesn't perfectly portray genesis from a wave to a tropical storm to a hurricane. I'm not sure that counts. In other words, while some years are better than others in this regard I'm not sure it matters since the globals are almost always wrong with intensity anyway.

[BYG Jacob]

Bumping this for whiffing on Nana

[CrazyC83]

Even with Laura, it was the HMON and HWRF that sniffed it correctly, while the globals kept going back and forth and greatly underestimating the intensity. Human forecasters are way ahead of models this year.

[BYG Jacob]

Even with Laura, it was the HMON and HWRF that sniffed it correctly, while the globals kept going back and forth and greatly underestimating the intensity. Human forecasters are way ahead of models this year.

Its like models think we're in a Nino instead of cool-neutral leaning into Nina

[USTropics]

Some genesis verification charts for this season:

GFS


CMC


UKMET


Consensus TC genesis verification


source: http://moe.met.fsu.edu/modelgen/index.php

[toad strangler]

Bumping this for whiffing on Nana

Sad

[Spacecoast]

Some genesis verification charts for this season:

GFS
https://i.imgur.com/6HUpUG6.png

CMC
https://i.imgur.com/HoBx8nq.png

UKMET
https://i.imgur.com/x1DTFKl.png

Consensus TC genesis verification
https://i.imgur.com/453ruDX.png

source: http://moe.met.fsu.edu/modelgen/index.php

The UKMET looks like the overall winner there for genesis verification.

If I am reading this right, the GFS seems to have more phantom X's than the others.
Has FSU (or anyone else) verified GFS-Para for this season, or previous seasons?

[USTropics]

Some genesis verification charts for this season:

GFS
https://i.imgur.com/6HUpUG6.png

CMC
https://i.imgur.com/HoBx8nq.png

UKMET
https://i.imgur.com/x1DTFKl.png

Consensus TC genesis verification
https://i.imgur.com/453ruDX.png

source: http://moe.met.fsu.edu/modelgen/index.php

The UKMET looks like the overall winner there for genesis verification.

If I am reading this right, the GFS seems to have more phantom X's than the others.
Has FSU (or anyone else) verified GFS-Para for this season, or previous seasons?

Yes they have an in-depth reanalysis for 2019 (and some systems from 2018/2020) using the new GFSv16 (GFS-para) vs. GFSv15 (operational). Here are some highlights:

For 500mb heights over NA, GFSv16 has a slightly better score:


Both models have observed cold bias, however GFSv16 has shown nominal improvement:


EC 2m temp bias:


As far as track goes, GFSv16 definitely had some significant improvements over the current operational GFS. It had a better lead time with the eventual track of Dorian:


Intensity/track error comparison for Dorian:


The evolution of Olga over CONUS was also a big win for the GFSv16:


As far as genesis is concerned, there were definite improvements in detecting systems in multiple basins:


A 2018 reanalysis also improved on genesis lead time and track for Michael:



Juliette (EPAC) intensity/track error:


For 2020, Isaias was also correctly forecasted by the GFSv16:


The reanalysis shows GFSv16 had better track, intensity, and lead time forecasts for Michael, Dorian, Juliette, Lorena, Olga, and Vicky compared to GFSv15. There were increased false positives (e.g. phantom storms), but there was improvement to predicted TC threat and intensity. GFSv15 performed better with Barbara, Kiko, and Lorenzo (GFSv16 was too slow with Barbara/right bias as well as Lorenzo's early recurvature).

source: https://www.emc.ncep.noaa.gov/users/meg/gfsv16/

[USTropics]

Some genesis verification charts for this season:

GFS
https://i.imgur.com/6HUpUG6.png

CMC
https://i.imgur.com/HoBx8nq.png

UKMET
https://i.imgur.com/x1DTFKl.png

Consensus TC genesis verification
https://i.imgur.com/453ruDX.png

source: http://moe.met.fsu.edu/modelgen/index.php

The UKMET looks like the overall winner there for genesis verification.

If I am reading this right, the GFS seems to have more phantom X's than the others.
Has FSU (or anyone else) verified GFS-Para for this season, or previous seasons?

The GFS has underpredicted essentially every forecast probability percentage this season. The UKMET has been a bit more mixed signals (like large delta swings at 30-40% probability forecasts), but verifies well at all other forecast probability %.

[tolakram]

Just to save it. Sunday 12Z runs of where Nana is now.









here's a signature, it's very small, but it's there.

GFS-para did pretty darn well ... for 2020.