arcgis 10.0 - Network Analyst: arcpy's 'Solve' running out of memory

I've run OD cost matrix analysis using GUI tools n ArcGIS 10.

My goal is relatively simple - find closest 50 neighbors for each origin point.

The dataset I'm working with is relatively large:

• network dataset consists of ~ 1.16 million elements


• there are ~ 100k origins


• and ~ 1.3 million destinations.

Origins are part of the destinations dataset.

The limits I set for the geoprocessing tool are: default_number_destinations_to_find set to 51 (since first 'neighbor' will be the same, overlapping point) and default_cutoff = 50000 (since I know from previous analyses that this value cannot be larger than that).

In order to save memory (I think) I go with output_path_shape set to NO_LINES - I do not need graphical representation of OD matrix - just tabular data (correct me if I'm wrong here).

It takes a while to process this task (~ 24h), but nonetheless I get the results. So far so good.

In order to automate this task I tried putting all this steps into python script and came up with:

try:

    import arcpy
    from arcpy import env

    # Check out any necessary licenses

    arcpy.CheckOutExtension("Network")

    # set environment settings
    env.workspace = "D:\\GIS\\04-buildings.gdb"
    env.overwriteOutput = True

    # Local variables:
    NW_FIX_ND = "D:\\GIS\\02-network-restrict.gdb\\NW_FIX\\NW_FIX_ND"
    DESTINATIONS = "buildings\\destinations"
    ORIGINS = "buildings\\origins_1"

    OD_COST_MATRIX = "OD Cost Matrix"
    LINES = "OD Cost Matrix\\Lines"
    bn_1 = "results\\bn_1"

    # Make OD Cost Matrix Layer
    arcpy.MakeODCostMatrixLayer_na(NW_FIX_ND, "OD Cost Matrix", "Length", "50000", "51", "Length", "ALLOW_UTURNS", "restriction", "NO_HIERARCHY", "", "NO_LINES")

    # Add Destinations
    arcpy.AddLocations_na(OD_COST_MATRIX, "Destinations", DESTINATIONS, "Name h_gebaeude_id #;CurbApproach # 0", "5000 Meters", "", "TLM_STRASSE_RESTR_FIX SHAPE;NW_FIX_ND_Junctions NONE", "MATCH_TO_CLOSEST", "APPEND", "NO_SNAP", "5 Meters", "EXCLUDE", "TLM_STRASSE_RESTR_FIX #;NW_FIX_ND_Junctions #")


    # Add Origins
    arcpy.AddLocations_na(OD_COST_MATRIX, "Origins", ORIGINS, "Name h_gebaeude_id #;TargetDestinationCount # 51;CurbApproach # 0;Cutoff_Length # #", "5000 Meters", "", "TLM_STRASSE_RESTR_FIX SHAPE;NW_FIX_ND_Junctions NONE", "MATCH_TO_CLOSEST", "CLEAR", "NO_SNAP", "5 Meters", "EXCLUDE", "TLM_STRASSE_RESTR_FIX #;NW_FIX_ND_Junctions #")

    # Solve
    arcpy.Solve_na(OD_COST_MATRIX, "SKIP", "TERMINATE")

    # Select OD Lines
    arcpy.SelectData_management(OD_COST_MATRIX, "Lines")

    # Select

    arcpy.Select_analysis(LINES, bn_1, "")

    print "Completed successfully"

except Exception as e:

    # If an error occurred, print line number and error message
    import traceback, sys
    tb = sys.exc_info()[2]
    print "An error occured on line %i" % tb.tb_lineno

    print str(e)

However, when I attempt to run this script I get the following error:

An error occured on line 39
ERROR 030024: Solve returned a failure.
Out of memory.
Failed to execute (Solve).

My questions here - why does arcpy fail when GUI could do the job? My reasoning was that by running the task in python and avoiding the graphical overhead - it should even work faster? Any ideas on what I could do to make arcpy handle this problem?

Answer

Unfortunately I've never managed to get to test @user27148 solution on ArcGIS 10.0.

Currently I'm working on the similar problem and can now confirm that with version 10.5 and 64-bit background geoprocessing installed I was able to run very similar analysis on even larger files.

Before installation of 64-bit I was getting similar 'out of memory' error. Running Python script against 64-bit resolved the issue.