Tuesday 17 November 2015

hdf files in GDAL throwing an error in R gdalinfo


I am having issues using some modis hdf files in R. Working on Windows 10. I had a previous question where gdal wasn't working but I think I have resolved that. I tested with existing tifs and they work, however these modis files are not.


library(dismo)
library(MODISTools)
library (raster)
library(sp)

library(ggmap)
library(rgdal)
library(rgeos)
library(RCurl)
library(stringr)
library(gdalUtils)
require(gdalUtils)
hdf_dir <- list.files(workd, pattern="hdf$", full.names=FALSE) #create

list of names with hdf extension

fhdf <-list.files(workd, pattern="hdf$", full.names=TRUE)
#create loop using gdal_translate
#test with one file
infile <- hdf_dir[1]
#"MOD13A3.A2010001.h11v04.005.2010043091608.hdf"
namefi <- substring(hdf_dir[1],1,nchar(hdf_dir[1])-4)
outfile <- paste0(out_tiff,"\\", namefi,".tif")
gdal_info <- GDALinfo(infile)



Error in .local(.Object, ...) : `C:\Users\R\MOD13A3.A2010001.h11v04.005.2010043091608.hdf' not recognised as a supported file format.



And:


gdal_translate(infile, namefi, sd_index=1)


Error in split1[[1]] : subscript out of bounds In addition: Warning message: running command '"C:\Program Files (x86)\GDAL\gdalinfo.exe" "MOD13A3.A2010001.h11v04.005.2010043091608.hdf"' had status 1



In the cmd line gdalinfo --formats gives:


 C:\Program Files (x86)\GDAL>gdalinfo --format hdf4

Format Details:
Short Name: HDF4
Long Name: Hierarchical Data Format Release 4
Extension: hdf
Help Topic: frmt_hdf4.html
Supports: Subdatasets

C:\Program Files (x86)\GDAL>gdalinfo --formats
Supported Formats:
BAG (ro): Bathymetry Attributed Grid

FITS (rw+): Flexible Image Transport System
GMT (rw): GMT NetCDF Grid Format
HDF4 (ros): Hierarchical Data Format Release 4
HDF4Image (rw+): HDF4 Dataset
HDF5 (ros): Hierarchical Data Format Release 5
HDF5Image (ro): HDF5 Dataset
netCDF (rw+s): Network Common Data Format
VRT (rw+v): Virtual Raster
GTiff (rw+vs): GeoTIFF
NITF (rw+vs): National Imagery Transmission Format

RPFTOC (rovs): Raster Product Format TOC format
ECRGTOC (rovs): ECRG TOC format
HFA (rw+v): Erdas Imagine Images (.img)
SAR_CEOS (rov): CEOS SAR Image
CEOS (rov): CEOS Image
JAXAPALSAR (rov): JAXA PALSAR Product Reader (Level 1.1/1.5)
GFF (rov): Ground-based SAR Applications Testbed File Format (.gff)
ELAS (rw+v): ELAS
AIG (rov): Arc/Info Binary Grid
AAIGrid (rwv): Arc/Info ASCII Grid

GRASSASCIIGrid (rov): GRASS ASCII Grid
SDTS (rov): SDTS Raster
DTED (rwv): DTED Elevation Raster
PNG (rwv): Portable Network Graphics
JPEG (rwv): JPEG JFIF
MEM (rw+): In Memory Raster
JDEM (rov): Japanese DEM (.mem)
GIF (rwv): Graphics Interchange Format (.gif)
BIGGIF (rov): Graphics Interchange Format (.gif)
ESAT (rov): Envisat Image Format

BSB (rov): Maptech BSB Nautical Charts
XPM (rwv): X11 PixMap Format
BMP (rw+v): MS Windows Device Independent Bitmap
DIMAP (rov): SPOT DIMAP
AirSAR (ro): AirSAR Polarimetric Image
RS2 (ros): RadarSat 2 XML Product
PCIDSK (rw+v): PCIDSK Database File
PCRaster (rw): PCRaster Raster File
ILWIS (rw+v): ILWIS Raster Map
SGI (rw+): SGI Image File Format 1.0

SRTMHGT (rwv): SRTMHGT File Format
Leveller (rw+): Leveller heightfield
Terragen (rw+): Terragen heightfield
ISIS3 (rov): USGS Astrogeology ISIS cube (Version 3)
ISIS2 (rw+v): USGS Astrogeology ISIS cube (Version 2)
PDS (rov): NASA Planetary Data System
TIL (rov): EarthWatch .TIL
ERS (rw+v): ERMapper .ers Labelled
JP2OpenJPEG (rwv): JPEG-2000 driver based on OpenJPEG library
L1B (rovs): NOAA Polar Orbiter Level 1b Data Set

FIT (rwv): FIT Image
GRIB (rov): GRIdded Binary (.grb)
RMF (rw+v): Raster Matrix Format
WCS (rovs): OGC Web Coverage Service
WMS (rwvs): OGC Web Map Service
MSGN (ro): EUMETSAT Archive native (.nat)
RST (rw+v): Idrisi Raster A.1
INGR (rw+v): Intergraph Raster
GSAG (rwv): Golden Software ASCII Grid (.grd)
GSBG (rw+v): Golden Software Binary Grid (.grd)

GS7BG (rw+v): Golden Software 7 Binary Grid (.grd)
COSAR (ro): COSAR Annotated Binary Matrix (TerraSAR-X)
TSX (rov): TerraSAR-X Product
COASP (ro): DRDC COASP SAR Processor Raster
R (rwv): R Object Data Store
MAP (rov): OziExplorer .MAP
PNM (rw+v): Portable Pixmap Format (netpbm)
DOQ1 (rov): USGS DOQ (Old Style)
DOQ2 (rov): USGS DOQ (New Style)
ENVI (rw+v): ENVI .hdr Labelled

EHdr (rw+v): ESRI .hdr Labelled
GenBin (rov): Generic Binary (.hdr Labelled)
PAux (rw+): PCI .aux Labelled
MFF (rw+): Vexcel MFF Raster
MFF2 (rw+): Vexcel MFF2 (HKV) Raster
FujiBAS (ro): Fuji BAS Scanner Image
GSC (rov): GSC Geogrid
FAST (rov): EOSAT FAST Format
BT (rw+v): VTP .bt (Binary Terrain) 1.3 Format
LAN (rw+v): Erdas .LAN/.GIS

CPG (ro): Convair PolGASP
IDA (rw+): Image Data and Analysis
NDF (rov): NLAPS Data Format
EIR (rov): Erdas Imagine Raw
DIPEx (rov): DIPEx
LCP (rwv): FARSITE v.4 Landscape File (.lcp)
GTX (rw+v): NOAA Vertical Datum .GTX
LOSLAS (rov): NADCON .los/.las Datum Grid Shift
NTv2 (rw+vs): NTv2 Datum Grid Shift
CTable2 (rw+v): CTable2 Datum Grid Shift

ACE2 (rov): ACE2
SNODAS (rov): Snow Data Assimilation System
KRO (rw+v): KOLOR Raw
ARG (rwv): Azavea Raster Grid format
RIK (ro): Swedish Grid RIK (.rik)
USGSDEM (rwv): USGS Optional ASCII DEM (and CDED)
GXF (ro): GeoSoft Grid Exchange Format
HTTP (ro): HTTP Fetching Wrapper
NWT_GRD (rov): Northwood Numeric Grid Format .grd/.tab
NWT_GRC (rov): Northwood Classified Grid Format .grc/.tab

ADRG (rw+vs): ARC Digitized Raster Graphics
SRP (rovs): Standard Raster Product (ASRP/USRP)
BLX (rw): Magellan topo (.blx)
Rasterlite (rws): Rasterlite
PostGISRaster (rws): PostGIS Raster driver
SAGA (rw+v): SAGA GIS Binary Grid (.sdat)
KMLSUPEROVERLAY (rwv): Kml Super Overlay
XYZ (rwv): ASCII Gridded XYZ
HF2 (rwv): HF2/HFZ heightfield raster
PDF (rw): Geospatial PDF

OZI (rov): OziExplorer Image File
CTG (rov): USGS LULC Composite Theme Grid
E00GRID (rov): Arc/Info Export E00 GRID
ZMap (rwv): ZMap Plus Grid
NGSGEOID (rov): NOAA NGS Geoid Height Grids
MBTiles (rov): MBTiles
IRIS (rov): IRIS data (.PPI, .CAPPi etc)

I can't figure out why this isn't working or what else I can try?



Answer




After much searching, I think I no longer have any errors when running these tools. First, I deleted a few gdal installs on my computer and kept the 32-bit version. Then I followed the steps outlined here and set my environment paths to where my GDAL install was located. This fixed the error in rgdal but not when using gdalUtils. To get gdalUtils to work I had to set the gdal installation in R where hdf was supported. Here is an example of my working code:


HDFd <- "C:\\Users\\L\\R\\hdfs"
out_tiff <- "C:\\Users\\L\\R\\Tiffs"
hdf_dir <- list.files(HDFd, pattern="hdf$", full.names=FALSE) #create list of names with hdf extension
#Create full directory name of files
fhdf <-list.files(HDFd, pattern="hdf$", full.names=TRUE)
#create loop using gdal_translate
#test with one file
#use full path
infile <- fhdf_dir[1]

#"MOD13A3.A2010001.h11v04.005.2010043091608.hdf"
gdal_chooseInstallation(hasDrivers=c("HDF4","HDF5"))
# Get the version of this installation:
getOption("gdalUtils_gdalPath")[[
gdal_chooseInstallation(hasDrivers=c("HDF4","HDF5"))]]$version

namefi <- substring(hdf_dir[1],1,nchar(hdf_dir[1])-4)
outfile <- paste0(out_tiff,"\\", namefi,".tif")
sds <- get_subdatasets(fhdf[1])
#Use full directory name

gdal_translate(fhdf[1], outfile, sd_index=1)
#For loop to convert all to TIFF in directory
for (i in 1:length(fhdf)){
name <-substring(hdf_dir[i],1,nchar(hdf_dir[i])-4)
newfile <- paste0(out_tiff,"\\", name,".tif")
print(name)
gdal_translate(fhdf[i],newfile,sd_index=1)

}

No comments:

Post a Comment

arcpy - Changing output name when exporting data driven pages to JPG?

Is there a way to save the output JPG, changing the output file name to the page name, instead of page number? I mean changing the script fo...