raster - Get GDAL up and running with ArcSDE enabled

I want to migrate rasters from ArcSDE 9.2 to Postgresql.

I have access to a ArcSDE server with several large raster datasets (mosaics) in ArcSDE 9.2. I can use ArcCatalog, ArcMap and ArcToolbox to view and manage the datasets. If I use the Export Raster tool, it fails, since the generated tif is larger that 4Gb.

Unfortunately, I'm not able to compile GDAL with SDE support. I have access to the SDE dlls installed on the server, but I don't have the ArcSDE Development Kit to compile GDAL with SDE support. I was able to compile GDAL, but not with SDE support.

I would like to split the raster mosaics. How can I split the mosaic in several smaller rasters?

Answer

I was able to export all data (vector and raster) from ArcSDE using GDAL. To have ArcSDE support in GDAL it is not necessary to compile GDAL. If you have access to an ArcSDE installation, that's enough. You will use dlls that are already installed on the server. You don’t need any media from ESRI, like ArcSDE Development Kit.

To this data transfer, I used a free Windows 7 virtual machine from Microsoft to run GDAL with SDE support.

Get GDAL up and running with ArcSDE enabled

• Download MS4W (select ms4w_3.0.6 from the archives, not the last version)



• Unzip it to c:\


• Copy gdal_SDE.dll and ogr_SDE.dll from c:\ms4w\gdalplugins\ignored\sde-9.2 to c:\ms4w\gdalplugins

If you install MS4W on the same server where ArcSDE is installed, this is enough. It you are using another machine, you also need:

• Copy your c:\arcgis\ArcSDE folder from the server running ArcSDE to the same location on your local machine (the entire folder zipped is just 32Mb)


• Update you environment variables:
  
  
  
  • Add to PATH C:\arcgis\ArcSDE\sqlexe\bin;C:\ms4w\tools\gdal-ogr
  
  
  • Create SDEHOME with C:\arcgis\ArcSDE\sqlexe\
  
  
  
  
  

Check SDE support

Open the command prompt.

C:\Users\IEUser>cd c:\ms4w
C:\ms4w>setenv.bat
C:\ms4w>gdalinfo --formats
Supported Formats:
  SDE (ro): ESRI ArcSDE

  VRT (rw+v): Virtual Raster
  GTiff (rw+v): GeoTIFF
  NITF (rw+v): National Imagery Transmission Format
  (...)

C:\ms4w>ogrinfo --formats
Supported Formats:
  -> "SDE" (read/write)
  -> "ESRI Shapefile" (read/write)
  -> "MapInfo File" (read/write)

  -> "UK .NTF" (readonly)
  -> "SDTS" (readonly)
  -> "TIGER" (read/write)
  (...)

Test with your own ArcSDE data

C:\ms4w>ogrinfo -ro -so -al SDE:192.168.100.32,5159,FINAL,sdeuser,sdepassword,final.SDE.REDMUNICIPAL,sde.DEFAULT
INFO: Open of `SDE:192.168.100.32,5159,FINAL,sdeuser,sdepassword,final.SDE.REDEMUNICIPAL,sd
.DEFAULT'
      using driver `SDE' successful.


Layer name: FINAL.SDE.REDEMUNICIPAL
Geometry: Unknown (any)
Feature Count: 32974
Extent: (-42781.579992, 113467.000009) - (-31999.999950, 129415.940056)
Layer SRS WKT:
PROJCS["Datum_73_Hayford_Gauss_IPCC",
    GEOGCS["GCS_Datum_73",
    (...)


C:\ms4w>gdalinfo SDE:192.168.100.32,5158,raster,sdeuser,sdepassword,raster.SDE.REN
Driver: SDE/ESRI ArcSDE
Files: none associated
Size is 6693, 8639
Coordinate System is:
PROJCS["Datum_73_Hayford_Gauss_IPCC",
    GEOGCS["GCS_Datum_73",
        DATUM["Datum_73",
            SPHEROID["International_1924",6378388.0,297.0]],
        PRIMEM["Greenwich",0.0],

        UNIT["Degree",0.0174532925199433]],
    PROJECTION["Transverse_Mercator"],
    (...)

Use GDAL to get data from ArcSDE

For my use case, I’ve transferred vector data directly from ArcSDE to Postgis with:

ogr2ogr -append -f "PostgreSQL" PG:"host=192.168.100.82 user=geobox dbname=final password=geobox active_schema=teste" SDE:192.168.100.32,5159,FINAL,sdeuser,sdepassword,final.SDE.VIALARGA,sde.DEFAULT -nln vialarga -s_srs EPSG:27493 -t_srs EPSG:3763

To transfer raster data, I’ve created tif files, for smaller raster datasets.

gdal_translate -of GTiff -co PROFILE=BASELINE -co TFW=YES -co COMPRESS=LZW -co PREDICTOR=2 -a_srs EPSG:27493 SDE:192.168.100.32,5158,raster,sdeuser,sdepassword,raster.SDE.RAN ran27493.tif

For the larger datasets, I’ve sliced the images into tiles. I use the same gdal_translate as above with an additional -srcwin parameter generated by small python script from the question Splitting raster into smaller chunks using GDAL?