Friday, 7 October 2016

postgresql - osm2pgsql --append super slow (node 2k/s vs 1500k/s for --create)


I'm working with multiple pbf files (one per administrative region). I thought I would import them one by one in postgis, with:


osm2pgsql --proj 4326 --hstore-all --database gis first.pbf --style custom.style --slim --flat-nodes osm2pgsql.flatnodes --create
for file in other_pbfs; do
osm2pgsql --proj 4326 --hstore-all --database gis $file.pbf --style custom.style --slim --flat-nodes osm2pgsql.flatnodes --append
done

However, the first import (--create) is going quite fast:




Processing: Node(262182k 1691.5k/s) Way(38130k 125.43k/s) Relation(0 0.00/s)



While the second one (--append) is going… super slow:



Processing: Node(13k 2.5k/s) Way(0 0.00/s) Relation(0 0.00/s)



If I merge these two PBFs before import (osmium merge *.pbf -o output.pbf), it goes fast again but I'd like to remove that unnecessary step if possible?


I tried with different formats (.pbf, .osm, …) but there is no changes in speed results.




No comments:

Post a Comment

arcpy - Changing output name when exporting data driven pages to JPG?

Is there a way to save the output JPG, changing the output file name to the page name, instead of page number? I mean changing the script fo...