Thursday, 14 November 2019

openstreetmap - Query one overpass api process multiple times


I using a local instance of the overpass api, because I need to make a lot of queries (over 100k). The solution I'm using right now is to call bin/osm3s_query for every single query.


This is very slow, because osm3s_query has to start, answer the query and shut down for every single query.


Is there a way of having some kind of daemon waiting for my queries?


I guess if osm3s_query doesn't have to start and shutdown for every single query, this should accelerate things a lot. The "daemon" for running a web server seems to do exactly what I'm doing right now (firing up one osm3s_query process for every query). Therefore I couldn't find anything helpful in the documentation



Answer



The best option in your case is probably to fetch all traffic light at once according to your gps track's bounding box. Cache that result locally in whatever language you're processing the Overpass API result and you're probably all set. Don't worry about the size, even a 300km by 300km area just takes some seconds to run.


Right now, there's no option in stock Overpass API to handle several requests in a row. Apache calls the interpreter binary via CGI for each single query. Usually, that's not that much of a concern, as most queries take more time, so start up costs are negligible.


BTW: There's also a FastCGI prototype for Overpass API on Github to reduce some of the startup costs. However, even with the support of FastCGI, each query usually has some specific fix cost in the range of maybe 50+ milliseconds, and as data needs to be fetched from disk (resp. buffer cache) and decompressed for each single query, you're better off with fewer but larger queries.


No comments:

Post a Comment

arcpy - Changing output name when exporting data driven pages to JPG?

Is there a way to save the output JPG, changing the output file name to the page name, instead of page number? I mean changing the script fo...