Where do you get computer-readable track data?
Moderator: S2k Moderators
Forum rules
The posts in this forum are NOT official forecasts and should not be used as such. They are just the opinion of the poster and may or may not be backed by sound meteorological data. They are NOT endorsed by any professional institution or STORM2K. For official information, please refer to products from the National Hurricane Center and National Weather Service.
-
- Tropical Low
- Posts: 21
- Joined: Sat Sep 02, 2017 9:01 pm
Where do you get computer-readable track data?
Long-time lurker around here who finally registered to ask a question. I'm working on a tool for the community that I think will be pretty well-received and have most of the code done, but I'm not sure where to get NHC data in a form that I can easily read via code.
I've seen people posting preliminary best-track data in the HURDAT format, as in:
20120930, 1800, , HU, 37.2N, 38.9W, 80, 978, 110, 90, 70, 80, 50, 40, 40, 40, 30, 25, 20, 25,
That would work well in the interim, since it's very easy to read, but I have no idea where to find it for current storms (only for post-analysis). I'd like the forecast points as well; I was able to find this page of GIS data, but that ends up being a lot _more_ than I need and I'd rather not fool around with a bunch of XML if I don't have to.
Ideally, what I'm looking for is a URL I can have the program access to read all the track data on a particular system so far, including a current forecast, in a code-friendly format. I only need positions, intensities, and optionally the type of cyclone (as in the third HURDAT column), though I can do without that last one.
I've seen people posting preliminary best-track data in the HURDAT format, as in:
20120930, 1800, , HU, 37.2N, 38.9W, 80, 978, 110, 90, 70, 80, 50, 40, 40, 40, 30, 25, 20, 25,
That would work well in the interim, since it's very easy to read, but I have no idea where to find it for current storms (only for post-analysis). I'd like the forecast points as well; I was able to find this page of GIS data, but that ends up being a lot _more_ than I need and I'd rather not fool around with a bunch of XML if I don't have to.
Ideally, what I'm looking for is a URL I can have the program access to read all the track data on a particular system so far, including a current forecast, in a code-friendly format. I only need positions, intensities, and optionally the type of cyclone (as in the third HURDAT column), though I can do without that last one.
1 likes
- Extratropical94
- Professional-Met
- Posts: 3535
- Age: 29
- Joined: Wed Oct 20, 2010 6:36 am
- Location: Hamburg, Germany
- Contact:
Re: Where do you get computer-readable track data?
ApproximateKnowledge wrote:Long-time lurker around here who finally registered to ask a question. I'm working on a tool for the community that I think will be pretty well-received and have most of the code done, but I'm not sure where to get NHC data in a form that I can easily read via code.
I've seen people posting preliminary best-track data in the HURDAT format, as in:
20120930, 1800, , HU, 37.2N, 38.9W, 80, 978, 110, 90, 70, 80, 50, 40, 40, 40, 30, 25, 20, 25,
That would work well in the interim, since it's very easy to read, but I have no idea where to find it for current storms (only for post-analysis). I'd like the forecast points as well; I was able to find this page of GIS data, but that ends up being a lot _more_ than I need and I'd rather not fool around with a bunch of XML if I don't have to.
Ideally, what I'm looking for is a URL I can have the program access to read all the track data on a particular system so far, including a current forecast, in a code-friendly format. I only need positions, intensities, and optionally the type of cyclone (as in the third HURDAT column), though I can do without that last one.
The NHC FTP server has a folder with best track data of all NATL and EPAC systems within the current year in .dat file format.
It's not exactly the HURDAT2 format where all wind radii are in one line, but there are three separate entries for TS force, 50 knot and HU force wind radii for each 6 hour point. So that would require you to pick the data fields you need for your code from the file. It looks like this
AL, 11, 2017090312, , BEST, 0, 178N, 479W, 100, 969, HU, 34, NEQ, 70, 60, 40, 70, 1013, 220, 15, 120, 0, L, 0, , 0, 0, IRMA, D, 12, NEQ, 400, 120, 100, 360, genesis-num, 023,
AL, 11, 2017090312, , BEST, 0, 178N, 479W, 100, 969, HU, 50, NEQ, 40, 20, 20, 30, 1013, 220, 15, 120, 0, L, 0, , 0, 0, IRMA, D, 12, NEQ, 400, 120, 100, 360, genesis-num, 023,
AL, 11, 2017090312, , BEST, 0, 178N, 479W, 100, 969, HU, 64, NEQ, 20, 10, 10, 15, 1013, 220, 15, 120, 0, L, 0, , 0, 0, IRMA, D, 12, NEQ, 400, 120, 100, 360, genesis-num, 023,
The files are available at http://ftp.nhc.noaa.gov/atcf/btk/
Hope this helps
0 likes
54° 11' 59'' N, 9° 9' 20'' E
Boomer Sooner!
Go Broncos! Go Cards! Go Niners!
- Daniel
Boomer Sooner!
Go Broncos! Go Cards! Go Niners!
- Daniel
Re: Where do you get computer-readable track data?
To add a bit, in case you want best track data for invests or storms in WPAC/SHEM (but not Indian Ocean) they are available at http://ftp.emc.ncep.noaa.gov/wd20vxt/hwrf-init/decks/?C=M;O=D
There are also track files of another format, like this:
which can be found on the Navy website in the respective basin and storm's folders here: https://nrlmry.navy.mil/tcdat/tc17/
and these include every storm including Indian Ocean storms but not invests.
Also, if you want data for just the latest cycle but for all active storms, you can find it here: https://nrlmry.navy.mil/tcdat/sectors/sector_file
There are also track files of another format, like this:
11L IRMA 170903 1200 17.8N 47.9W ATL 100 969
which can be found on the Navy website in the respective basin and storm's folders here: https://nrlmry.navy.mil/tcdat/tc17/
and these include every storm including Indian Ocean storms but not invests.
Also, if you want data for just the latest cycle but for all active storms, you can find it here: https://nrlmry.navy.mil/tcdat/sectors/sector_file
1 likes
DISCLAIMER: I am not a meteorologist. Any posts I made, unless copied from some official agency, are only my opinion and is unofficial and often exaggerated. Please refer to your official RSMC for official products and warnings.
- Extratropical94
- Professional-Met
- Posts: 3535
- Age: 29
- Joined: Wed Oct 20, 2010 6:36 am
- Location: Hamburg, Germany
- Contact:
Re: Where do you get computer-readable track data?
Krit-tonkla wrote:To add a bit, in case you want best track data for invests or storms in WPAC/SHEM (but not Indian Ocean) they are available at http://ftp.emc.ncep.noaa.gov/wd20vxt/hwrf-init/decks/?C=M;O=D
[...]
Good you mentioned that, I completely forgot about that EMC link. That also includes all the EPAC and ATL storm files from the NHC FTP site and files from previous years, so it's probably easier to use your link.
0 likes
54° 11' 59'' N, 9° 9' 20'' E
Boomer Sooner!
Go Broncos! Go Cards! Go Niners!
- Daniel
Boomer Sooner!
Go Broncos! Go Cards! Go Niners!
- Daniel
-
- Tropical Low
- Posts: 21
- Joined: Sat Sep 02, 2017 9:01 pm
Re: Where do you get computer-readable track data?
Extratropical94 wrote:The NHC FTP server has a folder with best track data of all NATL and EPAC systems within the current year in .dat file format.
It's not exactly the HURDAT2 format where all wind radii are in one line, but there are three separate entries for TS force, 50 knot and HU force wind radii for each 6 hour point. So that would require you to pick the data fields you need for your code from the file. It looks like this
AL, 11, 2017090312, , BEST, 0, 178N, 479W, 100, 969, HU, 34, NEQ, 70, 60, 40, 70, 1013, 220, 15, 120, 0, L, 0, , 0, 0, IRMA, D, 12, NEQ, 400, 120, 100, 360, genesis-num, 023,
AL, 11, 2017090312, , BEST, 0, 178N, 479W, 100, 969, HU, 50, NEQ, 40, 20, 20, 30, 1013, 220, 15, 120, 0, L, 0, , 0, 0, IRMA, D, 12, NEQ, 400, 120, 100, 360, genesis-num, 023,
AL, 11, 2017090312, , BEST, 0, 178N, 479W, 100, 969, HU, 64, NEQ, 20, 10, 10, 15, 1013, 220, 15, 120, 0, L, 0, , 0, 0, IRMA, D, 12, NEQ, 400, 120, 100, 360, genesis-num, 023,
The files are available at http://ftp.nhc.noaa.gov/atcf/btk/
Hope this helps
It does! That's a good start for the track data itself (though no forecast cone). Is there documentation for the line format?
Re: other posts - for now I'm only looking at ATL data, but later on I might expand worldwide. Wouldn't take much work to adapt.
0 likes
- Extratropical94
- Professional-Met
- Posts: 3535
- Age: 29
- Joined: Wed Oct 20, 2010 6:36 am
- Location: Hamburg, Germany
- Contact:
Re: Where do you get computer-readable track data?
ApproximateKnowledge wrote:Extratropical94 wrote:[...]
It does! That's a good start for the track data itself (though no forecast cone). Is there documentation for the line format?
Re: other posts - for now I'm only looking at ATL data, but later on I might expand worldwide. Wouldn't take much work to adapt.
There is, have a look at this document
https://www.nrlmry.navy.mil/atcf_web/do ... abdeck.txt
0 likes
54° 11' 59'' N, 9° 9' 20'' E
Boomer Sooner!
Go Broncos! Go Cards! Go Niners!
- Daniel
Boomer Sooner!
Go Broncos! Go Cards! Go Niners!
- Daniel
-
- Tropical Low
- Posts: 21
- Joined: Sat Sep 02, 2017 9:01 pm
Re: Where do you get computer-readable track data?
Okay. That takes care of track data. How about NHC forecast points?
0 likes
- Extratropical94
- Professional-Met
- Posts: 3535
- Age: 29
- Joined: Wed Oct 20, 2010 6:36 am
- Location: Hamburg, Germany
- Contact:
Re: Where do you get computer-readable track data?
ApproximateKnowledge wrote:Okay. That takes care of track data. How about NHC forecast points?
As far as I know, the forecast points are only included in the NHC forecast advisories and the discussions. The latter ones are a bit easier to process because all the forecast points and intensities are in one block. If you want forecast data for wind radii too then you'll need to use the forecast advisories though.
Unfortunately I don't know about the structure of the URLs of those products on the NHC page, but there is an alternate way to get them from the NWS FTP server.
The discussions are uploaded here
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt41.knhc.tcd.at1.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt42.knhc.tcd.at2.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt43.knhc.tcd.at3.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt44.knhc.tcd.at4.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt45.knhc.tcd.at5.txt
For each new storm, the header number (41-45) is increased by 1. After 45 the number 41 is used again (for example Franklin = 42, Gert = 43, Harvey = 44, Ten = 45, Irma = 41 and so on).
So your code would need to find out which header to use for which storm.
The forecast advisories are located at
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt21.knhc.tcm.at1.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt22.knhc.tcm.at2.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt23.knhc.tcm.at3.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt24.knhc.tcm.at4.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt25.knhc.tcm.at5.txt
That's a little more complicated but at least the URLs don't change.
If you want to include other data from NHC documents, you can find those by looking up the header of each product on the NHC page (it's that 4 letter-2 digit code at the beginning of every product) and navigating to that document on the NWS FTP server: http://tgftp.nws.noaa.gov/data/raw/
0 likes
54° 11' 59'' N, 9° 9' 20'' E
Boomer Sooner!
Go Broncos! Go Cards! Go Niners!
- Daniel
Boomer Sooner!
Go Broncos! Go Cards! Go Niners!
- Daniel
-
- Tropical Low
- Posts: 21
- Joined: Sat Sep 02, 2017 9:01 pm
Re: Where do you get computer-readable track data?
That's probably workable. I'd rather search for a known string than screw around with XML, and I'm pretty sure I can pull what I need out of the forecast/advisories if I can figure out the structure for URLs. Discussions will be enough if I can't do that. Thanks a ton!
1 likes
- Extratropical94
- Professional-Met
- Posts: 3535
- Age: 29
- Joined: Wed Oct 20, 2010 6:36 am
- Location: Hamburg, Germany
- Contact:
Re: Where do you get computer-readable track data?
No problem. Good luck with your project.
0 likes
54° 11' 59'' N, 9° 9' 20'' E
Boomer Sooner!
Go Broncos! Go Cards! Go Niners!
- Daniel
Boomer Sooner!
Go Broncos! Go Cards! Go Niners!
- Daniel
-
- Tropical Low
- Posts: 21
- Joined: Sat Sep 02, 2017 9:01 pm
Re: Where do you get computer-readable track data?
Expect to see it around here sometime in the next few weeks. I'm pretty excited about it
3 likes
-
- Tropical Low
- Posts: 21
- Joined: Sat Sep 02, 2017 9:01 pm
Re: Where do you get computer-readable track data?
Extratropical94 wrote:ApproximateKnowledge wrote:Okay. That takes care of track data. How about NHC forecast points?
As far as I know, the forecast points are only included in the NHC forecast advisories and the discussions. The latter ones are a bit easier to process because all the forecast points and intensities are in one block. If you want forecast data for wind radii too then you'll need to use the forecast advisories though.
Unfortunately I don't know about the structure of the URLs of those products on the NHC page, but there is an alternate way to get them from the NWS FTP server.
The discussions are uploaded here
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt41.knhc.tcd.at1.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt42.knhc.tcd.at2.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt43.knhc.tcd.at3.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt44.knhc.tcd.at4.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt45.knhc.tcd.at5.txt
For each new storm, the header number (41-45) is increased by 1. After 45 the number 41 is used again (for example Franklin = 42, Gert = 43, Harvey = 44, Ten = 45, Irma = 41 and so on).
So your code would need to find out which header to use for which storm.
The forecast advisories are located at
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt21.knhc.tcm.at1.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt22.knhc.tcm.at2.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt23.knhc.tcm.at3.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt24.knhc.tcm.at4.txt
http://tgftp.nws.noaa.gov/data/raw/wt/wtnt25.knhc.tcm.at5.txt
That's a little more complicated but at least the URLs don't change.
If you want to include other data from NHC documents, you can find those by looking up the header of each product on the NHC page (it's that 4 letter-2 digit code at the beginning of every product) and navigating to that document on the NWS FTP server: http://tgftp.nws.noaa.gov/data/raw/
Okay so I've been playing with these and it looks like those are only the _current_ discussions and forecast/advisories. Is there a way to get all of them for a storm?
0 likes
- Extratropical94
- Professional-Met
- Posts: 3535
- Age: 29
- Joined: Wed Oct 20, 2010 6:36 am
- Location: Hamburg, Germany
- Contact:
Re: Where do you get computer-readable track data?
ApproximateKnowledge wrote:Extratropical94 wrote:ApproximateKnowledge wrote:Okay. That takes care of track data. How about NHC forecast points?
[...]
Okay so I've been playing with these and it looks like those are only the _current_ discussions and forecast/advisories. Is there a way to get all of them for a storm?
I'm not sure if they are archived on the same server. The NHC has an advisory archive on their website though.
http://www.nhc.noaa.gov/archive/2017/al11/al112017.discus.026.shtml?
http://www.nhc.noaa.gov/archive/2017/al11/al112017.fstadv.026.shtml?
Only the storm number (and year) and advisory number have to be changed in the URL, but it might be more complicated to extract the text from all the html code than from a page with only the advisory text on it. (Or maybe it is not, I don't know I'm not a programmer )
0 likes
54° 11' 59'' N, 9° 9' 20'' E
Boomer Sooner!
Go Broncos! Go Cards! Go Niners!
- Daniel
Boomer Sooner!
Go Broncos! Go Cards! Go Niners!
- Daniel
Who is online
Users browsing this forum: No registered users and 102 guests