Monday, 16 March 2015

postgresql - Update database via osmosis and osm2pgsql too slow


I got a box with the current hardware config:


CPU: Intel Xeon E3-1220 V2 3.1 GHz
RAM: 10gb
Hard Disk: Western Digital WD10EZRX SATA Hard Drive

OS: Centos 6.5 64 bit

DB: PostgreSQL 9.3.2

and PostgreSQL got following config


shared_buffers = 1024MB
work_mem = 256MB
maintenance_work_mem = 4096MB
fsync = off
synchronous_commit = off
checkpoint_segments = 100


I imported the entire OSM Europe data into PGSQL DB with the following command:


osm2pgsql -c -d gps -U gps --cache 8000 --number-processes 4 --slim europe-latest.osm.bz2

and it took 6 days to complete... it's a lot but it's ok...


now i want to keep my DB updated with the combo of osmosis + osm2pgsql cmds:


osmosis --read-replication-interval workingDirectory=/osmosisworkingdir/ --simplify-change --write-xml-change - | \
osm2pgsql --append -d gps -U gps --cache 8000 --number-processes 4 -s -

Well... this takes MORE than one day to just align single day changes! That makes impossible to keep my local DB aligned!


Am i doing something wrong? Should i upgrade my RAM? Why updating is SOOOOOOOOOO damn heavier than load data for the first time?





No comments:

Post a Comment

arcpy - Changing output name when exporting data driven pages to JPG?

Is there a way to save the output JPG, changing the output file name to the page name, instead of page number? I mean changing the script fo...