Saturday, 12 December 2015

arcgis desktop - Set missing pipe network data (age, size) based on nearest neighboring or connecting pipes


I have pipe network data (polylines) with missing age and size attributes. For pipe age, I have 10779 pipes with 1034 of them having NODATA values. For pipe size, I have 10779 pipes with 738 of them having NODATA values.


What tool can I use to set the missing values based on the nearest (or connecting) pipe attribute information?


EDIT 1: In my mind, I feel like I could set up something like this pseudo code:


for each pipe
if the pipe[age]==NODATA
find the pipe_nearest
pipe[age] = pipe_nearest[age]
if the pipe[size]==NODATA

find the pipe_nearest
pipe[size] = pipe_nearest[size]

And then it would be all done. I just can't figure out how to determine the polyline nearest the current polyline.


Edit 2: Attempting to use FelixIP's solution. First off, I got access to another copy of the data set with the pipes clearly connected to each other - I have from and to node fields (FNODE_1, and TNODE_1), and the pipes are clearly snapped to each other. I have copied my initial layer twice, creating has_X and Missing_X layers for both age (YEAR_) and diameter (DIAM) information.


enter image description here


Starting with diameter first, I adjusted the script for the names of my own layers, so it reads:


def fillGap (d,fr):
mxd = arcpy.mapping.MapDocument("CURRENT")
lyr=arcpy.mapping.ListLayers(mxd,"Has_Diam")[0]

q='"TNODE_1"=%s%s%s' %(r"'",fr,"'")
tbl=arcpy.da.TableToNumPyArray(lyr,"DIAM",q)
if len(tbl)==0:return d
return tbl[0][0]

fillGap (!DIAM!,!FNODE_1!)

but for some reason I am getting this error on execution:


enter image description here


I am trying to sort it out, but since I don't completely the Python code that is used in calculating the expression, it is proving to be a little difficult.




Answer



The method below works for pipes with ends snapped to other pipe ends, I am using shapefile as an example.


Add 2 text fields (FROMN, TON), 50 characters long to your table and run this script on field FROMN


def truncate(f, n):
s = '{}'.format(f)
i, p, d = s.partition('.')
return '.'.join([i, (d+'0'*n)[:n]])
def fromto(shp,ft):
p=shp.firstPoint
if ft>0:p=shp.lastPoint

return truncate(p.X,2)+"-"+truncate(p.Y,2)

Using


fromto( !Shape!,0 )

Run the same script on field TON, using


fromto( !Shape!,1 )

Your table will look similar to this. enter image description here


Note that missing diameter has value of -1 in my example.



Create clones of your pipes layer in table of content using copy, paste. Call one layer MISSING and use definition query


"DIAMETER" = -1


Call one layer AVAILABLE and use definition query


"DIAMETER" <> -1


Your table of content will look like that:


enter image description here


Note number of records in MISSING layer and run this script on field DIAMETER of MISSING layer:


def fillGap (d,fr):
mxd = arcpy.mapping.MapDocument("CURRENT")
lyr=arcpy.mapping.ListLayers(mxd,"AVAILABLE")[0]

q='"TON"=%s%s%s' %(r"'",fr,"'")
tbl=arcpy.da.TableToNumPyArray(lyr,"DIAMETER",q)
if len(tbl)==0:return d
return tbl[0][0]

fillGap (!DIAMETER!,!FROMN!)

enter image description here


enter image description here


Close table and open it again. You’ll notice reduction of records in MISSING table. Run script again and again until no reduction occurs. This is filling gaps in downstream direction. In general it is a good idea to find maximum of IN diameters, sort tbl in script in descending order, but it only applicable for topologically correct network, i.e. line direction = flow direction



What script does is saving you time on joining MISSING and AVAILABLE tables using FROMN and TON fields accordingly.


If after multiple runs you still have records in MISSING layer, you have to fill gaps in upstream direction. To do so, replace query line in script


q='"FROMN"=%s%s%s' %(r"'",fr,"'")

and run it using


fillGap (!DIAMETER!, !ton! )

If your pipes aren’t snapped at the ends, let me know I’ll update my answer.


In the meantime try this:


enter image description here



Output table will look like this


enter image description here


It is the answer to your question:


for each (missing) pipe find nearest pipe with available size


isn’t it?


Very fast and very dirty approach is to join it to MISSING table (PIPE_ID to PIPE_ID) and calculate DIAMETER in MISSING table using DIAMETER_1 And this is equivalent to


if the pipe[size]==NODATA: pipe[size] = pipe_nearest[size]


No comments:

Post a Comment

arcpy - Changing output name when exporting data driven pages to JPG?

Is there a way to save the output JPG, changing the output file name to the page name, instead of page number? I mean changing the script fo...