My script takes separate 3D feature classes, calculates a field, appends to a target feature class, and then deletes the originals. This is a huge data processing task of over 4700 mass point feature classes appending to a single one. I know it was a big mistake to put so many feature classes into SDE, but given they're in there now, I've got to deal with them there (yes, the sheer fc count is bringing SDE performance to its knees--hence my urgency).
Early on it worked great, in that the merged result showed valid z-values. I was able to build a terrain and it looked great. I'd been running this in the background for weeks, and have just noticed z-values of 0 in the output, where the originals were not 0! Oh no!!! I'm certainly glad I still have the file-based originals!
Pseudocode:
An in_memory feature class is created from a z-aware template
For each fc in the feature dataset:
It is appended to the empty in_memory fc
The TILE_ID field is calculated using a portion of the name of the source fc
It is then appended to the target SDE fc
The in_memory feature class is emptied
Each processed fc is then deleted
When I initially posted, I was curious if it was my use of in_memory that caused this trouble. I had not initially been using it, and only switched to it because performance was far too slow otherwise, and WAY faster in_memory. To find where trouble started, I ran the Add XY Coordinates tool, which proved the trouble started with in_memory use. At that time I had also switched from CalculateField_management to da.UpdateCursor, so that's likely the bigger reason for the performance enhancement. I will now change my code to avoid in_memory.
No comments:
Post a Comment