arcgis desktop - Want to use GDAL in Arc10 toolbox via subprocess call

I'm attempting to launch a version Python 2.5 that supports GDAL from an ArcGIS10 toolbox script. I'm doing this because I'm having loads of trouble importing the GDAL library directly into Arc, and I'm writing an application that will need to be backwards compatiable with Arc 9.3.

Here's what I'm attempting to do. The main.py file in the toolbox:

import subprocess

p = subprocess.Popen(['C:\OSGeo4W\gdal_python_exec.bat', 'X:\\local\\import_tests.py'])

gdal_python_exec.bat is a windows batch script that fires up the 2.5 version of Python I want while also setting up some environment variables:

@echo off
set OSGEO4W_ROOT=C:\OSGeo4W
PATH=%OSGEO4W_ROOT%\bin;%PATH%
for %%f in (%OSGEO4W_ROOT%\etc\ini\*.bat) do call %%f
@echo on

@C:\OSGeo4W\bin\python.exe %1

Next, import_tests.py tries to import gdal:

try:
    from osgeo import gdal
    raw_input('Imported! (Press enter)')
except Exception, e:
    print(e)
    raw_input('Failed! (Press enter)')

When I run main.py at a DOS command line as C:\Python26\ArcGIS10.0\python.exe gdal_arc_test.py things work fine. However, if I take the same script and add it as a 'toolbox' inside of Arc launch it from there, I get a "DLL not found" for the GDAL lib inside the import_tests.py file!

How can this happen when subprocess is the module that's launching a different Python interpreter? Any ideas on what could be happening? It feels like the version of Python launched from Arc's tool box does something very strange, but I would have thought the subprocess call would separate that stuff out.

P.S. I can verify that the os.environ['PATH'] variables are the same in both calls.

Answer

if the py2.5 script is relatively stable, meaning you don't need to change it much or often, try turning it into a self contained executable and call that instead. There are two python-to-exe programs I see mentioned often: py2exe and pyinstaller. I've personally had the best luck with pyinstaller, though I never used it in a gis-inside-gis fashion such as you are attempting. There are some other py-to-exe's discussed here on Stack Overflow.

However responding to the problem which is driving you to use 2.5: I used python 2.6 with Arcgis 9.3 for a long time to good effect using the recipe at Can I use python 2.6 with ArcGIS 9.3?.