Wednesday, 26 December 2018

performance - Understanding why arcpy.CreateScratchName() gets slower with more file geodatabases in folder?


In my ArcGIS Pro 1.4.1 application, I noticed that a short section of ArcPy code that creates a scratch name, and then uses that to create a file geodatabase (always in the same folder with the same root name), goes from taking a few seconds the first time it is run to taking 2 minutes around the 30th time.


I have isolated that it is CreateScratchname rather than CreateFileGDB that is slowing down by running this code:


import arcpy,time,os

workingFolder = r"C:\Temp\TestFolder"
for i in range(100):
scratchName = arcpy.CreateScratchName("Working",".gdb","Workspace",workingFolder)
baseName = os.path.basename(scratchName.split(".")[0])
workingGDB = r"{0}\{1}.gdb".format(workingFolder,baseName)

start = time.clock()
arcpy.CreateFileGDB_management(workingFolder,baseName)
elapsed = (time.clock() - start)
print("Creating working geodatabase {0} took {1} seconds".format(workingGDB,int(round(elapsed))))

to create this output where you will see that the time for CreateFileGDB is consistently 1 second whereas the time for CreateScratchName is 3 seconds for the first time, then climbs from 1 second to 4-5 seconds by the 25th iteration:


Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul  5 2016, 11:41:13) [MSC v.1900 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>>
======== RESTART: C:\CompilationPlots\Scripts\CreateWorkingGDBtest.py ========

Creating scratchname C:\Temp\TestFolder\Working0.gdb took 4 seconds
Creating working geodatabase C:\Temp\TestFolder\Working0.gdb took 3 seconds
Creating scratchname C:\Temp\TestFolder\Working1.gdb took 0 seconds
Creating working geodatabase C:\Temp\TestFolder\Working1.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working2.gdb took 0 seconds
Creating working geodatabase C:\Temp\TestFolder\Working2.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working3.gdb took 1 seconds
Creating working geodatabase C:\Temp\TestFolder\Working3.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working4.gdb took 1 seconds
Creating working geodatabase C:\Temp\TestFolder\Working4.gdb took 1 seconds

Creating scratchname C:\Temp\TestFolder\Working5.gdb took 1 seconds
Creating working geodatabase C:\Temp\TestFolder\Working5.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working6.gdb took 1 seconds
Creating working geodatabase C:\Temp\TestFolder\Working6.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working7.gdb took 1 seconds
Creating working geodatabase C:\Temp\TestFolder\Working7.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working8.gdb took 1 seconds
Creating working geodatabase C:\Temp\TestFolder\Working8.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working9.gdb took 2 seconds
Creating working geodatabase C:\Temp\TestFolder\Working9.gdb took 1 seconds

Creating scratchname C:\Temp\TestFolder\Working10.gdb took 2 seconds
Creating working geodatabase C:\Temp\TestFolder\Working10.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working11.gdb took 3 seconds
Creating working geodatabase C:\Temp\TestFolder\Working11.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working12.gdb took 2 seconds
Creating working geodatabase C:\Temp\TestFolder\Working12.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working13.gdb took 2 seconds
Creating working geodatabase C:\Temp\TestFolder\Working13.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working14.gdb took 2 seconds
Creating working geodatabase C:\Temp\TestFolder\Working14.gdb took 1 seconds

Creating scratchname C:\Temp\TestFolder\Working15.gdb took 2 seconds
Creating working geodatabase C:\Temp\TestFolder\Working15.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working16.gdb took 3 seconds
Creating working geodatabase C:\Temp\TestFolder\Working16.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working17.gdb took 3 seconds
Creating working geodatabase C:\Temp\TestFolder\Working17.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working18.gdb took 3 seconds
Creating working geodatabase C:\Temp\TestFolder\Working18.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working19.gdb took 3 seconds
Creating working geodatabase C:\Temp\TestFolder\Working19.gdb took 1 seconds

Creating scratchname C:\Temp\TestFolder\Working20.gdb took 3 seconds
Creating working geodatabase C:\Temp\TestFolder\Working20.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working21.gdb took 3 seconds
Creating working geodatabase C:\Temp\TestFolder\Working21.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working22.gdb took 4 seconds
Creating working geodatabase C:\Temp\TestFolder\Working22.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working23.gdb took 5 seconds
Creating working geodatabase C:\Temp\TestFolder\Working23.gdb took 1 seconds
Creating scratchname C:\Temp\TestFolder\Working24.gdb took 4 seconds
Creating working geodatabase C:\Temp\TestFolder\Working24.gdb took 1 seconds

>>>

The above would not be such a concern except that in my application I go on to fill the working geodatabase with about 50 feature classes and tables at each iteration, and this seems to make CreateScratchName run even slower.


Is the slowing down of CreateScratchName demonstrated in my code above, and anecdotally exacerbated by the size of each file geodatabase once they have been filled, to be expected?




No comments:

Post a Comment

arcpy - Changing output name when exporting data driven pages to JPG?

Is there a way to save the output JPG, changing the output file name to the page name, instead of page number? I mean changing the script fo...