Model Wars

[Andy_L]

Good morning

as a long time reader here at storm2K (over 10 years now) I wanted to point out something that, to me anyway, is getting worse and makes things unpleasant on the tropical boards.

Its beginning to feel like, to post, you have to hang your hat on a specific model and defend that model to the death. If "your" model fails to verify then you sulk...and when it does verify, you have to gloat about it. Am I the only person that sees how ridiculous this is and how childish it makes you seem?

I LOVE coming here to read and learn about tropical weather, PLEASEEEE don't ruin things here

Cheers
Andy

[Kalrany]

100% agree. It is getting to the point that I have been looking to see if there is an ignore button for some posters, but then I know I would lose some great discussions when they are not model-harping. But it is very very very frustrating and significantly off-putting. I now cringe when I even see the standard nicknames for the models.

[tolakram]

It's an issue every year to some extent.

The GFS frustration is growing though, I know I'm pretty angry that we had to fly numerous planes and gather a ton of data just so the GFS could agree with the Euro. That is always going to cause a bit of frustration, even more so if you're in the path of the storm and trying to plan ahead. I live in Cincinnati and not in a flood prone area so I never have to plan, or have much warning, that a disaster is coming. I can't imagine having to make a decision to stay or go, much less having to deal with so much uncertainty in the forecast.

So at times there will be some venting from some folks, but generally I don't think we let it go on too long.

[WeatherEmperor]

I think we should maybe Sticky this thread and just use it to vent our frustrations or model war all we want. I really mean that. I think its ok to do that so long as we keep all of that in this Thread and not in threads for each different storm. Seriously i think we should just let it all come out in this thread. I agree with Tolakram in a sense that how many times have we seen a model such as the Gfs for example stick to a particular solution for like 10+ consecutive runs...and then all of a sudden do a flip flop and show something else? I mean really this happens over and over and over and it is frustrating it really is. I think we should just vent but try to do it in a respectful way this way we can leave the threads for the individual storms and their respective model threads clean from these model wars. Does that sound fair?


Sent from my iPhone6 using Tapatalk

[JtSmarts]

This has been most frustrating year of model watching that I can remember, and I'm also tired of our American models constantly being behind the curve. I remember vividly how the Euro first started to become revered as the "king" back in 2003 when it nailed Isabel and things are only slowly changing. I remember our old friend Derek Ortt had a very colorful nick-name for the GFS and it still sticks lol.

[Andy_L]

while I understand the frustrations...its not the fact that any particular model performed well or another one underperforms...I think that's the nature of the beast with something LIKE models. Its the attitude that is being used in the posts about it. truly comes across like school kids arguing about things "mine is better than yours" attitude...

I think a HEALTHY discussion about model accuracy isn't a bad thing, maybe someone will touch on something that will make it back to the programmers and in turn will be a part in helping to make them better. But the gloating and attitude because 'this' one verifies, or 'that' one fails is where I have a bone to pick.

as I said, I don't post a lot (I have a low count lol) but I come here to read and learn a LOT...even in the off season. Its a fantastic wealth of knowledge. I just find, this year in particular, I cringe when I click on model threads because I know that I have to weed through garbage to get the important parts

cheers
Andy

[Kalrany]

while I understand the frustrations...its not the fact that any particular model performed well or another one underperforms...I think that's the nature of the beast with something LIKE models. Its the attitude that is being used in the posts about it. truly comes across like school kids arguing about things "mine is better than yours" attitude...

I think a HEALTHY discussion about model accuracy isn't a bad thing, maybe someone will touch on something that will make it back to the programmers and in turn will be a part in helping to make them better. But the gloating and attitude because 'this' one verifies, or 'that' one fails is where I have a bone to pick.

as I said, I don't post a lot (I have a low count lol) but I come here to read and learn a LOT...even in the off season. Its a fantastic wealth of knowledge. I just find, this year in particular, I cringe when I click on model threads because I know that I have to weed through garbage to get the important parts

cheers
Andy

+1

[RL3AO]

Most people cannot begin to grasp the complexities of a weather model. You can have Ph.D. in meteorology and look at the math that goes into a global model and have their head spin. Its so easy to sit here and say "GFS sucks" and not even begin to understand how hard it now is to make a model 1% better. Some of these models use 5th and 6th order derivatives to calculate things like friction and boundary layer motion. The Euro is consistently slightly better than the GFS...but the GFS is very very very good. The ECMWF has more resources than NCEP going into improving its medium range forecast and it shows.

[CFLHurricane]

I agree this thread should be stickied so people can use it to vent model frustration here. As a casual observer, I find them to be distracting and just plain boring.

And for the record, I'm not a fan of any model... So :p.....

[Andy_L]

FANTASTIC example of a healthy discussion about models. much better than "KING ***** (insert your fav model here) predicted this 7 MONTHS ago and its right!!! best model EVER!!



btw I DO understand the level of mathematics involved and the amount of computing power required to crunch those numbers...its scary

Most people cannot begin to grasp the complexities of a weather model. You can have Ph.D. in meteorology and look at the math that goes into a global model and have their head spin. Its so easy to sit here and say "GFS sucks" and not even begin to understand how hard it now is to make a model 1% better. Some of these models use 5th and 6th order derivatives to calculate things like friction and boundary layer motion. The Euro is consistently slightly better than the GFS...but the GFS is very very very good. The ECMWF has more resources than NCEP going into improving its medium range forecast and it shows.

[tolakram]

Article about the latest model issues. Just because it's printed doesn't mean it's accurate, but interesting nonetheless.

http://www.nytimes.com/2015/10/03/upshot/hurricane-joaquin-forecast-european-model-leads-pack-again.html?_r=2

[caneman]

I dislike the model wars immensely. There really shouldn't be any. Per the nhc. model accuracy is near 50 50 for the euro and the gfs from 2012 - 2014. It's petty to claim one as a so called king and erroneous as well and falsely leads newbies to believe they should listen to one over another, when in fact, the nhc gets best results and accuracy by using a blend.

[terstorm1012]

Most people cannot begin to grasp the complexities of a weather model. You can have Ph.D. in meteorology and look at the math that goes into a global model and have their head spin. Its so easy to sit here and say "GFS sucks" and not even begin to understand how hard it now is to make a model 1% better. Some of these models use 5th and 6th order derivatives to calculate things like friction and boundary layer motion. The Euro is consistently slightly better than the GFS...but the GFS is very very very good. The ECMWF has more resources than NCEP going into improving its medium range forecast and it shows.

as the quip attributed to George Box goes: "all models are wrong, but they are sometimes useful." My own variation is they're always useful because they're always telling you something.

this is a great post. I find the model fan clubs a bit distracting to be honest, and I find the overreliance by some on our admittedly very good models a bit distracting too. Sometimes it is just helpful to look at a surface plot and the water vapor imagery. There are a few meteorology programs that I'm aware of that still make students physically draw out maps as part of coursework (they may all do this, I don't know) so they get that hands-on experience with the weather in a sense. I didn't last in my met program--at the time the math flabbergasted me--but I've been hand-drawing weather maps since childhood. I still do on occasion.

[RL3AO]

Most people cannot begin to grasp the complexities of a weather model. You can have Ph.D. in meteorology and look at the math that goes into a global model and have their head spin. Its so easy to sit here and say "GFS sucks" and not even begin to understand how hard it now is to make a model 1% better. Some of these models use 5th and 6th order derivatives to calculate things like friction and boundary layer motion. The Euro is consistently slightly better than the GFS...but the GFS is very very very good. The ECMWF has more resources than NCEP going into improving its medium range forecast and it shows.

as the quip attributed to George Box goes: "all models are wrong, but they are sometimes useful." My own variation is they're always useful because they're always telling you something.

this is a great post. I find the model fan clubs a bit distracting to be honest, and I find the overreliance by some on our admittedly very good models a bit distracting too. Sometimes it is just helpful to look at a surface plot and the water vapor imagery. There are a few meteorology programs that I'm aware of that still make students physically draw out maps as part of coursework (they may all do this, I don't know) so they get that hands-on experience with the weather in a sense. I didn't last in my met program--at the time the math flabbergasted me--but I've been hand-drawing weather maps since childhood. I still do on occasion.

Contouring and map analysis is still a big part of our forecasting, mesoscale, and synoptic class here at St. Cloud State. It is very important to take the concepts from math (dynamics) and apply it to a real map.

[CrazyC83]

Had all those planes and balloons not been increased, could we still have split decisions now? We'd be at Hurricane Watch time (at least) for the Carolina coast with a reliable model saying out to sea. That would be an emergency manager's worst nightmare, and evacuations cost tens or hundreds of millions of dollars.

[gatorcane]

The real issue with the GFS is the data, not the actual model itself. I believe that there was some research done that showed when the GFS was provided the ECMWF-data, it performed similarly to the ECMWF. There is a reason why the ECMWF data is not free and is available only with a hefty-license.

So if that is the case, the effort with improving the GFS should really be with improving the NCEP data fed into the model right?

[WeatherEmperor]

What I wanna know is how come the Gfdl is now so bad??? I mean in the late 90's and early 2000's this model was a super star and now it barely gets anything right. Also it was supposed to be replaced by the Hwrf in 2008 yet is still being used. I heard they stopped development in the Gfdl model itself since 2007 which means it doesnt get upgrades anymore? Could somebody shed some light on this???


Sent from my iPhone6 using Tapatalk

[Yellow Evan]

What I wanna know is how come the Gfdl is now so bad??? I mean in the late 90's and early 2000's this model was a super star and now it barely gets anything right. Also it was supposed to be replaced by the Hwrf in 2008 yet is still being used. I heard they stopped development in the Gfdl model itself since 2007 which means it doesnt get upgrades anymore? Could somebody shed some light on this???


Sent from my iPhone6 using Tapatalk

Pretty sure the GFDL gets upgrades. It seems, though, that the HWRF has surpassed the GFDL in recent years and gets more upgrades than the GFDL.

[Hammy]

I'll chime in after keeping notes over the last few years--the Euro in 2013-14 generally outperformed the GFS with one or two exceptions, and as far as intensity goes, probably 80-90% of the time when it showed something over a series of runs, it happened. This year the GFS has performed less than well, as usual, but has been ahead of the Euro on a few occasions dissipating storms the Euro didn't. The Euro on the other hand, I have never seen so many phantom storms (or phantom majors especially) from that model--half the storms it showed never formed and of the 4-5 major hurricanes it showed, only Joaquin actually played out. So this year I've tended to go with tracks where there was a greater consensus, as they have all performed horribly this year.

Except with Erika, which I got right a day or two before the models/official track did

[RL3AO]

The real issue with the GFS is the data, not the actual model itself. I believe that there was some research done that showed when the GFS was provided the ECMWF-data, it performed similarly to the ECMWF. There is a reason why the ECMWF data is not free and is available only with a hefty-license.

So if that is the case, the effort with improving the GFS should really be with improving the NCEP data fed into the model right?

When the GFS is initialized with ECMWF data it performs on par with it. The Euro has much better data assimilation and initialization. Period. Thats why its a better model. The European model as a 4D-Var assimilation scheme vs the GFS 3D-var. The extra dimension (time) gives the Euro more of a "rolling initialization" instead of the GFS's single snapshot. Last I heard NCEP was upgrading the GFS to some sort of hybrid 4D-Var data assimilation this year or next year. I don't really know a ton about modeling but its an area I'm working on improving in.