Friday 31 August 2018

gdal - "Error 4: xxxxx.dat: not recognized as a supported file format."


I am trying to convert a .dat file to a .tif file using gdal_translate. The .dat file comes with an associated .Hdr file of the same name with the metadata (these are geospatial data from SNODAS). Here are the instructions:




  1. Unzip and untar all data if you have not already: gunzip *.gz should do the trick if you work in a linux environment. Then, tar -zxvf *.tar.




  2. Using a text editor, create an ENVI header file with the following information for masked data¹ (files beginning with 'us'):





ENVI samples = 6935 lines = 3351 bands = 1 header offset = 0 file type = ENVI Standard data type = 2 interleave = bsq byte order = 1



  1. Save the header file using the exact same file name as the data file you are converting, but with a .hdr extension.


For example, the "us_ssmv01025SlL01T0024TTNATS2004010105DP001.dat" filename is used to create "us_ssmv01025SlL01T0024TTNATS2004010105DP001.hdr".



  1. Ensure your working directory contains both the .hdr file and the .dat file, then issue the following commands:



GeoTIFF


gdal_translate -of GTiff -a_srs '+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs' -a_nodata -9999 -a_ullr  -124.73333333 52.87500000 -66.94166667 24.95000000  

A common error that can be thrown after issuing the 'gdal_translate ...' command is Error 4: not recognized as a supported file format. Depending on your system, GDAL might be confused between the .hdr you created and the .Hdr which came with the data. If this occurs, try storing the .Hdr files outside of the working directory and trying again.


¹For unmasked data, you will need to open the .Hdr files delivered with the data and adjust the ENVI header and bounding coordinates in the GDAL string accordingly.


Appendix 1. Conversion example for unmasked SNODAS data.


gdal_translate -of GTiff -a_srs '+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs' -a_nodata -9999 -a_ullr -130.516666666661 58.2333333333310 -62.2499999999975 24.0999999999990 34.dat 34.tif

and header file should be:


ENVI

samples=8192
lines=4096
bands=1
header offset=0
file type=ENVI Standard
data type=2
interleave=bsq
byte order=1

So, here is my input code:



(gdal30) Lauras-iMac:SWE Casey$ gdal_translate -of GTiff -a_srs '+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs' -a_nodata -9999 -a_ullr  -124.73333333 52.87500000 -66.94166667 24.95000000 us_ssmv11034tS__T0001TTNATS2018120805HP001.dat us_ssmv11034tS__T0001TTNATS2018120805HP001.tif

And here is the returned error:


ERROR 4: `us_ssmv11034tS__T0001TTNATS2018120805HP001.dat' not recognized as a supported file format.

Now, per the warning, I do in fact have the .Hdr file outside of the cwd, and only the .dat file and the .hdr file located within:


(gdal30) Lauras-iMac:SWE Casey$ ls
us_ssmv11034tS__T0001TTNATS2018120805HP001.dat us_ssmv11034tS__T0001TTNATS2018120805HP001.hdr

Additionally, here is an overview of what I'm running:



(gdal30) Lauras-iMac:SWE Casey$ conda --version
conda 4.6.14
(gdal30) Lauras-iMac:SWE Casey$ gdalinfo --version
GDAL 3.0.0, released 2019/05/05
(gdal30) Lauras-iMac:SWE Casey$ python --version
Python 3.7.3

I performed this action pretty easily about a month ago, and have since then had a software update and am running MacOS Mojave 10.14.5. I was actually performing the task in a virtual environment in the terminal, but now have Anaconda installed. I have tried many different versions of python and gdal, completely wiped conda/python/gdal from my HD, all to no avail.


Also, gdal_translate does work for other actions, such as converting a geotif to a jpg:


(gdal30) Lauras-iMac:SNODAS_20181201 Casey$ gdal_translate -of JPEG -co QUALITY=70 -co PROGRESSIVE=ON -outsize 1400 0 -r bilinear CANYrelief1-geo.tif CANYrelief1-geo.jpg

Input file size is 2800, 2800
0...10...20...30...40...50...60...70...80...90...100 - done.

I'm a bit of a rookie and have two full days put into trying to figure this out. It's hard to understand how this worked a month ago with no problems but now refuses to work.




Here is the return from gdalinfo --formats :


GenBin -raster- (rov): Generic Binary (.hdr Labelled)
ENVI -raster- (rw+v): ENVI .hdr Labelled
EHdr -raster- (rw+v): ESRI .hdr Labelled


This is to confirm the Envi .hdr Labelled...


And...


(gdal30) Lauras-iMac:SWE Casey$ ls
34.dat 34.hdr
(gdal30) Lauras-iMac:SWE Casey$ gdalinfo 34.dat
ERROR 4: `34.dat' not recognized as a supported file format.
gdalinfo failed - unable to open '34.dat'.

Also, here is a link of the data source: ftp://sidads.colorado.edu/DATASETS/NOAA/G02158/masked/2018/12_Dec/


Each .tar file is a single day. The file contains 8 pairs, a .dat and a .Hdr file. For anyone who wishes to give it a try, just take this download and follow the given instructions.



My thought thus far is that GDAL changed, and that I'm not finding the appropriate steps to fix this.




No comments:

Post a Comment

arcpy - Changing output name when exporting data driven pages to JPG?

Is there a way to save the output JPG, changing the output file name to the page name, instead of page number? I mean changing the script fo...