Tuesday, 29 September 2015

polyline creation - Creating arcs between two points in ArcPy?


I've been working on an python script automating the visualization of live point data. I have a table with two coordinate pairs. I'd like to connect these two point with an arc of a circle or similar (parabola??).


I've been able to connect the two points with a straight line, but it gives me a boring visual.


One of the major hurdles is my license level: ArcView.



Anyone have any idea how I can generate a set of points that represent the path of a line between these two points?


I can then use the points to polyline command.



Answer



It seems the most common problem with these types of "flow maps" is that when many lines are included, they collide to such a great extent that it makes it difficult to discern any non-obvious pattern (when reciprocal flows are considered it happens to an even greater extent). Also the long lines tend to dominate the graphic, although it is quite possible the distribution of flows is predominately over short spaces (for instance a host of different distributions between places tend to be similar to Levy flights). I suppose this isn't necessarily a bad thing (the long lines may be more intrinsically interesting that the short lines for many phenomenon), but I don't think we want to lose the forest for the trees so to speak.


Although I don't doubt I've missed some potential "solutions" that have been proposed, I will try to sum up some of the ways individuals have tried to solve the problem in work that I have come across.


Distorting the Lines


If you peruse some of the other threads on the sight you will see some examples of how people have dealt with this problem. In particular, the lines are distorted so they don't overlap with each other or other objects on the map. Whuber's answer on another similar question (already mentioned in a comment) is an example of this. A presentation by some researchers at Stanford demonstrates this same idea (Phan et al., 2005). Thanks for that presentation goes to dslamb for this answer on another thread (and all of the answers to that thread will be on interest for your question as well). I particularly find it interesting that one of the cardinal examples of this is the old immigration map by Minard is an example of a desirable output (circa 1864!).


Given your particular use case (small number of nodes and lines), this seems sufficient. The other "solutions" I present are more intended to visualize data with many lines and many origins-destinations (although I assume they will be useful summaries for the community in general, so I continue on regardless).


Using Alpha Blending, Color, and Line Width/Height


The maps I listed in that same thread previously noted, Representation of network flows are examples of these. The facebook friends is a good case of adjusting the alpha level of lines, so it takes many more flows to represent a darker (or brighter in that case) connection between the two places. This also demphasizes the longer lines because they tend to happen more infrequently. Similar logic comes from Value-by-Alpha maps for polygon areas (Roth et al., 2010) that have been mentioned on this forum before.



The other map I present in that same answer uses color, and a non-traditional 3d perspective arcing lines (Ratti et al., 2010). The authors used a clustering criteria to group homogenous areas together and color code them (so by definition the areas within the color have more similar flow patterns than between colors). The clustering criteria in and of itself could be interesting to identify patterns in the data, although it seems a likely problem with this, as Andrew Gelman has mentioned, is that it tells you pretty much what you already know, that places nearer to each other tend to have more connections.


Lastly, in this category I include techniques that weight the lines (similar to alpha blending) using either the line width, or in the case of the 3d perspective line height, to convey the volume of the flow. See the page on Tobler's flow mapping software page for some examples in 2d (and the other article I mentioned is an example in 3d using line heights). Also on that page Tobler has a very useful article describing the problems with flowmapping and their historical application (Tobler, 1987).


Another example in 3d is this answer by a mankoff on this site. This post on the Sociological images blog shows a useful way in a flow diagram to distinguish between in-flows and out-flows (although again it works because the number of nodes and relatively small, and the nodes on the network can be layed out in an arbitrary way to reduce overplotting). Those same types of arrows (and a few others using hashings) are also in (Tobler, 1987).


In the end though line width and color don't really solve the over-plotting problem. The arcs in 3d help somewhat, although with more complicated flow patterns I think they will have limited utility. IMO alpha blending seems to be the most useful in a wide variety of situations of these three, but color and line width could/should be used in conjunction with line distortion mentioned above.


Data Reduction


I group two types of techniques here, 1) using small multiple maps (i.e. many maps with inherently less objects to visualize so overplotting is reduced), or 2) other graphical representions, that are not lines, but represent some of the flows via density or choropleth maps. Examples of these can be found in (Corcoran et al., 2009; Rae, 2009; Wood et al., 2010) (thanks to iant for the Rae reference). These tend to reduce the amount of visual information presented by either presenting a series of small multiple maps (or just a smaller area), or use a choropleth mapping scheme to represent some statistic (examples could be number of in-flows, number of outflows, direction of the flows, average distance of the flows). If you have point level data you could represent these statistics through kernal density raster maps, or aggregate them into quadrats.


When information is reduced like this, overplotting isn't as much a problem. A very cool interactive online example is this migration map by Forbes magazine. You can only see one county at a time, but the reduction of information makes it much easier to parse the lines (and the difference between in-flows and out-flows). A recent post on the ESRI mapping blog also uses a similar technique with the small multiples (they also choose a particular projection for the world map to have "pretty looking" lines, and make good use of color to further highlight different international origins). In that example it works out pretty well because the end destination is the same for all flows, but if flows could be reciprocal it probably wouldn't work out as well.


Using Other Non-map Representations of Flows


Others on this site have suggested using alternative diagrams to the actual map to represent the flows (just mapping the origins and destinations in some other manner than their actual geographic location). Examples of these are either cicular visualizations (such as that produced by Circos), arc diagrams (see this example on Protovis, these are also called kriskograms (Xiao & Chun, 2009)), or matrix heat maps (here is another example from the Protovis website). Another option would be to use some type of automated network layout to identify patterns in the flows (such as that capable by Graphviz). Besides Graphviz it appears Gephi, the NetworkX python library, and some R libraries are popular tools as well (see this answer on the stats site).


The libraries I cite are pretty cool as they have developed interactive visualizations as well. Here is an example with a similar style to the circular graphics (although not circular!). Here is another interactive visualization using some of the line distortion techniques discussed earlier, network placement (that appears similar to circular Dorling cartograms) as well as other useful statistical summaries (I saw both of those examples originally on the information aesthetics blog).




Some other resources I think are useful are the software and articles coming from the Spatial Data Mining and Visual Analytics Lab. Also the crime travel demand modeling in the CrimeStat program is a gentle introduction to applicable regression techniques for such flow data. Either of these tools may allow you to identify interesting correlations in the flow patterns to other geographic information. Another place to perhaps recieve some useful inspiration for either graphically displaying the data or statistical analysis would be a recent issue of in the Journal of Computational and Graphical Statistics, Volume 20 Issue 2, on examining flight arrival/departure statistics for commercial carriers in the US from 1987 to 2008 (if you are interested in handling big data this would be worthwhile to examine as well). All the articles are free and they have associated posters with each paper.


In the end, the data and the medium will dictate how well some of these techniques work in reducing the visual clutter that comes along with flow data. I hope this is a useful place though to find ideas on how to deal with this visualization problem though. If you further refine your question into what you want to accomplish, then others can give useful feedback into actual programmatic implementations (if something is not already available).





*note, links to ungated pdf documents are included when I could find one


-

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)

arcpy - Changing output name when exporting data driven pages to JPG?

Is there a way to save the output JPG, changing the output file name to the page name, instead of page number? I mean changing the script fo...