« OsmExtract output » : différence entre les versions
Aller à la navigation
Aller à la recherche
Contenu ajouté Contenu supprimé
Aucun résumé des modifications |
Aucun résumé des modifications |
||
Ligne 1 : | Ligne 1 : | ||
'''This run was on MMU (may 2017) with various optimizing choices.''' |
'''This run was on MMU (may 2017) with various optimizing choices.''' |
||
'''To be compared with [http://wiki.openstreetmap.org/wiki/Osm2pgsql/benchmarks this].''' |
|||
'''It took nearly 96 hours (nearly 4 days):''' |
'''It took nearly 96 hours (nearly 4 days):''' |
Version du 27 mai 2017 à 11:26
This run was on MMU (may 2017) with various optimizing choices.
To be compared with this.
It took nearly 96 hours (nearly 4 days):
>> osmExtract This script converts OSM data into postgres DB Supposed to run on MMU (execution time ~1 hour) interactive because asking password see documentation in wiki page GeoJava -rw-r--r-- 1 pge pge 35G Mar 17 11:55 /media/Shared/server.geo/planet-170306.osm.pbf Tue May 23 09:25:36 CEST 2017 OSM -> postgres gis database : start osm2pgsql SVN <span style="color:#008000;">version 0.88.1</span> (64bit id space) Password:qwert Using built-in tag processing pipeline Using projection SRS 4326 (Latlong) Setting up table: planet_osm_point Setting up table: planet_osm_line Setting up table: planet_osm_polygon Setting up table: planet_osm_roads Allocating memory for dense node cache Allocating dense node cache in one big chunk Allocating memory for sparse node cache Sharing dense sparse Node-cache: cache=8000MB, maxblocks=1024000*8192, allocation method=11 Mid: loading persistent node cache from /media/Shared/server.geo/FlatNodes Allocated space for persistent node cache file Maximum node in persistent node cache: 0 Mid: pgsql, scale=10000000 cache=8000 Setting up table: planet_osm_nodes Setting up table: planet_osm_ways Setting up table: planet_osm_rels Reading in file: /media/Shared/server.geo/planet-170306.osm.pbf Processing: Node(3780985k 1694.7k/s) Way(398714k 15.50k/s) Relation(4165320 17.96/s) Standard exception processing relation id=6244046: TopologyException: side location conflict at -75.702342200000004 41.350785100000003 Processing: <span style="color:#008000;">Node(3780985k 1694.7k/s) Way(398714k 15.50k/s) Relation(4839260 18.54/s)</span> parse time: 288955s Node stats: total(3780985254), max(4719536321) in 2231s Way stats: total(398714532), max(478741177) in 25727s Relation stats: total(4839264), max(7037743) in 260997s Maximum node in persistent node cache: 4719640575 Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Committing transaction for planet_osm_roads Mid: loading persistent node cache from /media/Shared/server.geo/FlatNodes Maximum node in persistent node cache: 4719640575 Setting up table: planet_osm_nodes Setting up table: planet_osm_ways Setting up table: planet_osm_rels Using built-in tag processing pipeline Mid: loading persistent node cache from /media/Shared/server.geo/FlatNodes Maximum node in persistent node cache: 4719640575 Setting up table: planet_osm_nodes Setting up table: planet_osm_ways Setting up table: planet_osm_rels Using built-in tag processing pipeline Mid: loading persistent node cache from /media/Shared/server.geo/FlatNodes Maximum node in persistent node cache: 4719640575 Setting up table: planet_osm_nodes Setting up table: planet_osm_ways Setting up table: planet_osm_rels Using built-in tag processing pipeline Mid: loading persistent node cache from /media/Shared/server.geo/FlatNodes Maximum node in persistent node cache: 4719640575 Setting up table: planet_osm_nodes Setting up table: planet_osm_ways Setting up table: planet_osm_rels Using built-in tag processing pipeline Going over pending ways... 15229821 ways are pending Using 4 helper-processes Finished processing 15229821 ways in 17608 sec 15229821 Pending ways took 17608s at a rate of 864.94/s Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Committing transaction for planet_osm_roads Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Committing transaction for planet_osm_roads Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Committing transaction for planet_osm_roads Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Committing transaction for planet_osm_roads Going over pending relations... 0 relations are pending Using 4 helper-processes Finished processing 0 relations in 0 sec Committing transaction for planet_osm_point WARNING: there is no transaction in progress Committing transaction for planet_osm_line WARNING: there is no transaction in progress Committing transaction for planet_osm_polygon WARNING: there is no transaction in progress Committing transaction for planet_osm_roads WARNING: there is no transaction in progress Committing transaction for planet_osm_point WARNING: there is no transaction in progress Committing transaction for planet_osm_line WARNING: there is no transaction in progress Committing transaction for planet_osm_polygon WARNING: there is no transaction in progress Committing transaction for planet_osm_roads WARNING: there is no transaction in progress Committing transaction for planet_osm_point WARNING: there is no transaction in progress Committing transaction for planet_osm_line WARNING: there is no transaction in progress Committing transaction for planet_osm_polygon WARNING: there is no transaction in progress Committing transaction for planet_osm_roads WARNING: there is no transaction in progress Committing transaction for planet_osm_point WARNING: there is no transaction in progress Committing transaction for planet_osm_line WARNING: there is no transaction in progress Committing transaction for planet_osm_polygon WARNING: there is no transaction in progress Committing transaction for planet_osm_roads WARNING: there is no transaction in progress Sorting data and creating indexes for planet_osm_point Stopping table: planet_osm_nodes Sorting data and creating indexes for planet_osm_line Stopping table: planet_osm_ways Building index on table: planet_osm_ways Stopping table: planet_osm_rels Building index on table: planet_osm_rels Sorting data and creating indexes for planet_osm_polygon Sorting data and creating indexes for planet_osm_roads Stopped table: planet_osm_nodes in 0s Stopped table: planet_osm_rels in 4352s Copying planet_osm_point to cluster by geometry finished Creating geometry index on planet_osm_point Copying planet_osm_roads to cluster by geometry finished Creating geometry index on planet_osm_roads Creating osm_id index on planet_osm_point Creating osm_id index on planet_osm_roads Creating indexes on planet_osm_roads finished Creating indexes on planet_osm_point finished All indexes on planet_osm_roads created in 21368s Completed planet_osm_roads All indexes on planet_osm_point created in 21513s Completed planet_osm_point Copying planet_osm_polygon to cluster by geometry finished Creating geometry index on planet_osm_polygon /home/pge/Scripts/osmExtract: line 26: 22466 Killed osm2pgsql --hstore --create -l -d gis --slim -C $CACHESIZE -U pge -W --flat-nodes $FLATNODES --number-processes 4 --style $STYLEFILE $OSMFILE <span style="color:#008000;">real 5735m45.130s user 634m55.836s sys 63m50.672s</span> OSM -> postgres gis database : done Sat May 27 09:01:21 CEST 2017