OSMデータの取り込みツールとプログラミングについて

 OSMデータの取り込みツールとプログラミングについて

OpenStreetMapのデータを取り込むツールはいくつかありますが、それぞれの特徴についてちゃんと把握するのは難しいと思います。
今回はosm2pgsql, imposm3, osm2pgroutingなどの現在使われているデータベースへのインポートツールについての解説と、OSMのデータを扱うためのライブラリやプログラミング言語の実装(C/C++, Python, Rust, JavaScript)について紹介し、OSMをデータを扱うためにどのような仕組みを使うと良いかというヒントになる発表を行います。

9987516c273a7874b12b742e7aef828f?s=128

Taro Matsuzawa aka. btm

August 06, 2016
Tweet

Transcript

  1. OSM import tools and programming OSM のデー タ取り込みツー ルとプログラミング Taro

    Matsuzawa @smellman Georepublic Japan / OSMFJ / OSGeo.JP / jus
  2. about me GIS programmer at Georepublic I don’t know “Geography”

    well. Specialty: Python / Ruby / Web browser / Unix OS Hobby: Breakcore
  3. Topics File format OSM import tools Programming with OSM data

    (.osm, .pbf )
  4. File format .osm, .osc (XML file) All nodes contain the

    location in the WGS84 reference system. Simplest format and readable for human. .osc is OSM Change file. .pbf (Protocol Bu ff er) Binary format and half of the size of gzipped osm file. .proto file defines format and many programming language supports Protocol Bu ff er. Some columns are delta‑encoded. .o5m, .o5c Design for small file size and fast processing. .osh History file includes all changes.
  5. Compare Human reaable Process speed file size osm yes too

    slow biggest pbf no fast small o5m no fast small pbf is smaller than o5m but 7zip compressed o5m is smaller than pbf.
  6. osmconvert osmconvert use to convert and process OpenStreetMap file. Supports

    .osm, .osc, .osh, .o5m, .o5c and .pbf files. Clip area with longiute and latitude, or Polygon. # i n s t a l l $ w g e t ‐ O ‐ h t t p : / / m . m . i 2 4 . c c / o s m c o n v e r t . c | c c ‐ x c ‐ ‐ l z ‐ O 3 ‐ o o s m c o n v e r t # c o n v e r t $ . / o s m c o n v e r t n o r w a y . p b f > n o r w a y . o s m # c l i p a r e a w i t h l o n g i t u d e a n d l a t i t u d e $ . / o s m c o n v e r t g e r m a n y . p b f ‐ b = 1 0 . 5 , 4 9 , 1 1 . 5 , 5 0 ‐ o = n u e r n b e r g . o 5 m # c l i p a r e a w i t h P o l y g o n . $ . / o s m c o n v e r t g e r m a n y . p b f ‐ B = h a m b u r g . p o l y ‐ o = h a m b u r g . p b f
  7. osm import tools osm2pgsql imposm3 ogr2ogr osm2pgrouting

  8. osm2pgsql Most famous tool to import osm data to postgresql.

    Written in C++ and libosmium. OpenStreetMap.org default style depends osm2pgsql. https://github.com/gravitystorm/openstreetmap‑ carto/blob/master/openstreetmap‑carto.style Control import tags with style file.
  9. example # T h i s i s t h

    e . s t y l e f i l e f o r O p e n S t r e e t M a p C a r t o , w h i c h i s c u r r e n t l y # t h e s a m e a s t h e u p s t r e a m o s m 2 p g s q l s t y l e # p h s t o r e i s u s e d i n s t e a d o f p o l y g o n , n o c o l u m n t o p r e s e r v e c o m p a t i b i l i t y # w i t h o l d e r o s m 2 p g s q l v e r s i o n s # O s m T y p e T a g D a t a T y p e F l a g s n o d e , w a y a c c e s s t e x t l i n e a r n o d e , w a y a d d r : h o u s e n a m e t e x t l i n e a r n o d e , w a y a d d r : h o u s e n u m b e r t e x t l i n e a r n o d e , w a y a d d r : i n t e r p o l a t i o n t e x t l i n e a r n o d e , w a y a d m i n _ l e v e l t e x t l i n e a r n o d e , w a y a e r i a l w a y t e x t l i n e a r n o d e , w a y a e r o w a y t e x t p o l y g o n n o d e , w a y a m e n i t y t e x t p o l y g o n n o d e , w a y a r e a t e x t p o l y g o n # h a r d c o d e d s u p p o r t f o r a r e a = 1 / y e s = > p o l y g o n i s n o d e , w a y b a r r i e r t e x t l i n e a r n o d e , w a y b i c y c l e t e x t l i n e a r n o d e , w a y b r a n d t e x t l i n e a r n o d e , w a y b r i d g e t e x t l i n e a r n o d e , w a y b o u n d a r y t e x t l i n e a r
  10. Imposm 3 Alternative osm import tool. Imposm 3 is written

    in Go. Imposm 2 is written in Python and configurations/mappings are not compatible. High performance and firster than osm2pgsql. Create table structrue with YAML/JSON file. We can design postgresql table structure for rendering server.
  11. Notice: Imposm 3 Current master can't import japan pbf. Use

    osm2vectortiles/imposm3 instead of master branch. $ g i t c l o n e h t t p s : / / g i t h u b . c o m / o s m 2 v e c t o r t i l e s / i m p o s m 3 \ $ G O P A T H / s r c / g i t h u b . c o m / o m n i s c a l e / i m p o s m 3 $ c d $ G O P A T H / s r c / g i t h u b . c o m / o m n i s c a l e / i m p o s m 3 $ g i t r e s e t ‐ ‐ h a r d 0 6 0 f 4 1 1 1 f c a f 4 5 9 e 8 b 0 9 a 0 3 5 1 8 3 6 d f e 8 e b 4 d c 9 4 3 $ g o g e t $ g o i n s t a l l
  12. osm2pgsql vs Imposm 3: table structures osm2pgsql controls tags with

    style file but it can't control table structures. Imposm 3 controls table structures and tags in json or yaml file (mapping file).
  13. osm2pgsql table structures o s m _ j p _

    o s m 2 p g s q l = # \ d L i s t o f r e l a t i o n s S c h e m a | N a m e | T y p e | O w n e r ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ + ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ + ‐ ‐ ‐ ‐ ‐ ‐ ‐ + ‐ ‐ ‐ ‐ ‐ ‐ ‐ p u b l i c | g e o g r a p h y _ c o l u m n s | v i e w | b t m p u b l i c | g e o m e t r y _ c o l u m n s | v i e w | b t m p u b l i c | p l a n e t _ o s m _ l i n e | t a b l e | o s m p u b l i c | p l a n e t _ o s m _ p o i n t | t a b l e | o s m p u b l i c | p l a n e t _ o s m _ p o l y g o n | t a b l e | o s m p u b l i c | p l a n e t _ o s m _ r o a d s | t a b l e | o s m p u b l i c | r a s t e r _ c o l u m n s | v i e w | b t m p u b l i c | r a s t e r _ o v e r v i e w s | v i e w | b t m p u b l i c | s p a t i a l _ r e f _ s y s | t a b l e | b t m ( 9 r o w s )
  14. osm2pgsql table structures (2) o s m _ j p

    _ o s m 2 p g s q l = # \ d p l a n e t _ o s m _ p o l y g o n T a b l e " p u b l i c . p l a n e t _ o s m _ p o l y g o n " C o l u m n | T y p e | M o d i f i e r s ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ + ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ + ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ o s m _ i d | b i g i n t | a c c e s s | t e x t | a d d r : h o u s e n a m e | t e x t | a d d r : h o u s e n u m b e r | t e x t | a d d r : i n t e r p o l a t i o n | t e x t | a d m i n _ l e v e l | t e x t | a e r i a l w a y | t e x t | a e r o w a y | t e x t | a m e n i t y | t e x t | a r e a | t e x t | b a r r i e r | t e x t | b i c y c l e | t e x t | b r a n d | t e x t | b r i d g e | t e x t | b o u n d a r y | t e x t | b u i l d i n g | t e x t | . . .
  15. imposm3 table structure (1) Design in mapping file b u

    i l d i n g s : f i e l d s : ‐ n a m e : o s m _ i d t y p e : i d ‐ n a m e : g e o m e t r y t y p e : g e o m e t r y ‐ k e y : n a m e n a m e : n a m e t y p e : s t r i n g ‐ n a m e : t y p e t y p e : m a p p i n g _ v a l u e m a p p i n g : b u i l d i n g : ‐ _ _ a n y _ _ t y p e : p o l y g o n
  16. imposm3 table structure (2) o s m _ j p

    _ i m p o s m 3 = # \ d L i s t o f r e l a t i o n s S c h e m a | N a m e | T y p e | O w n e r ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ + ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ + ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ + ‐ ‐ ‐ ‐ ‐ ‐ ‐ p u b l i c | g e o g r a p h y _ c o l u m n s | v i e w | b t m p u b l i c | g e o m e t r y _ c o l u m n s | v i e w | b t m p u b l i c | o s m _ a d m i n | t a b l e | o s m p u b l i c | o s m _ a d m i n _ i d _ s e q | s e q u e n c e | o s m p u b l i c | o s m _ a e r o w a y s | t a b l e | o s m p u b l i c | o s m _ a e r o w a y s _ i d _ s e q | s e q u e n c e | o s m p u b l i c | o s m _ a m e n i t i e s | t a b l e | o s m p u b l i c | o s m _ a m e n i t i e s _ i d _ s e q | s e q u e n c e | o s m p u b l i c | o s m _ b a r r i e r p o i n t s | t a b l e | o s m p u b l i c | o s m _ b a r r i e r p o i n t s _ i d _ s e q | s e q u e n c e | o s m p u b l i c | o s m _ b a r r i e r w a y s | t a b l e | o s m p u b l i c | o s m _ b a r r i e r w a y s _ i d _ s e q | s e q u e n c e | o s m p u b l i c | o s m _ b u i l d i n g s | t a b l e | o s m p u b l i c | o s m _ b u i l d i n g s _ i d _ s e q | s e q u e n c e | o s m . . .
  17. imposm3 table structure (3) # \ d o s m

    _ b u i l d i n g s T a b l e " p u b l i c . o s m _ b u i l d i n g s " C o l u m n | T y p e | M o d i f i e r s ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ + ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ + ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ ‐ i d | i n t e g e r | n o t n u l l d e f a u l t n e x t v a l ( ' o s m _ b u i l d i n g s _ i d _ s e q ' : : r e g c l a s o s m _ i d | b i g i n t | n a m e | c h a r a c t e r v a r y i n g | t y p e | c h a r a c t e r v a r y i n g | g e o m e t r y | g e o m e t r y ( G e o m e t r y , 3 8 5 7 ) | I n d e x e s : " o s m _ b u i l d i n g s _ p k e y " P R I M A R Y K E Y , b t r e e ( i d ) " o s m _ b u i l d i n g s _ g e o m " g i s t ( g e o m e t r y ) " o s m _ b u i l d i n g s _ g e o m _ g e o h a s h " b t r e e ( s t _ g e o h a s h ( s t _ t r a n s f o r m ( s t _ s e t s r i d ( b o x 2 d ( g e o m e t r y ) : : g e " o s m _ b u i l d i n g s _ o s m _ i d _ i d x " b t r e e ( o s m _ i d )
  18. query in OSM‑bright osm2pgsql ( S E L E C

    T w a y , b u i l d i n g A S t y p e F R O M p l a n e t _ o s m _ p o l y g o n W H E R E b u i l d i n g N O T I N ( ' ' , ' 0 ' , ' f a l s e ' , ' n o ' ) O R D E R B Y S T _ Y M i n ( S T _ E n v e l o p e ( w a y ) ) D E S C ) A S d a t a imposm3 ( S E L E C T g e o m e t r y , t y p e , n a m e , a r e a F R O M o s m _ b u i l d i n g s O R D E R B Y S T _ Y M i n ( S T _ E n v e l o p e ( g e o m e t r y ) ) D E S C ) A S d a t a
  19. benchmark osm2pgsql vs imposm3 osm2pgsql $ / u s r

    / b i n / t i m e . / o s m 2 p g s q l ‐ U o s m ‐ d o s m _ j p _ o s m 2 p g s q l \ ‐ S . . / d e f a u l t . s t y l e ‐ c ‐ C 2 0 0 0 ‐ ‐ f l a t ‐ n o d e s / t m p / f l a t n o d e \ ~ / d e v e l o p / o s m / d a t a / j a p a n ‐ 1 6 0 4 1 7 . o s m . p b f . . . O s m 2 p g s q l t o o k 1 0 5 7 s o v e r a l l 1 0 5 8 . 6 7 r e a l 9 8 5 . 2 9 u s e r 6 7 . 8 4 s y s imposm3 $ / u s r / b i n / t i m e i m p o s m 3 i m p o r t ‐ c o n n e c t i o n \ p o s t g i s : / / o s m : o s m @ l o c a l h o s t / o s m _ j p _ i m p o s m 3 ‐ m a p p i n g \ $ G O P A T H / s r c / g i t h u b . c o m / o m n i s c a l e / i m p o s m 3 / e x a m p l e ‐ m a p p i n g . y m l \ ‐ o p t i m i z e ‐ r e a d j a p a n ‐ 1 6 0 4 1 7 . o s m . p b f ‐ w r i t e . . . [ A u g 4 2 0 : 3 2 : 3 5 ] [ I N F O ] I m p o s m t o o k : 3 1 m 1 2 . 9 6 2 1 5 0 4 6 2 s 1 8 7 3 . 1 5 r e a l 2 1 3 2 . 9 7 u s e r 2 9 5 . 1 3 s y s
  20. ogr2ogr ogr2ogr converts simple features data between file formats. ogr2ogr

    supports .osm file and pbf file. o g r 2 o g r ‐ ‐ c o n f i g O S M _ U S E _ C U S T O M _ I N D E X I N G N O \ ‐ s k i p f a i l u r e s ‐ f " E S R I S h a p e f i l e " \ s e n d a g a y a _ s h a p e f i l e s e n d a g a y a . o s m
  21. osm2pgrouting Import tool for osm data to pgRouting database. Builds

    the routing network topology automatically.
  22. Programming for OSM files.

  23. Libraries for programmer osmium (C++, Python, Node.js) libosmpbfreader (C++) osmpbfreader‑rs

    (Rust) rust‑osm‑reader (Rust)
  24. osmium C++ library for osm data. http://osmcode.org/libosmium/ It provide C++11

    header library. Easy to contain your C++ product with cmake. osmium provide python binding (PyOsmium) and node.js binding (Node‑Osmium). Supports too many OpenStreetMap file format. Old versions are still used in many products. https://github.com/joto/osmium If you find the product use old version, it's pull‑request chance (or death road).
  25. concept osmium provide simple interfaces. open many format (.osm, .osh,

    .osc, .pbf, ...) OSM Entity classes (Node, Way, Relation, Changesets, ...) Also provide iterator for tag. Basic Types (OSM id, version, user, Location, Segment(connection between two location), Box) process each OSM Entities in Handler
  26. Handler Need to implement to process OSM Entities. i m

    p o r t o s m i u m a s o c l a s s F i l e S t a t s H a n d l e r ( o . S i m p l e H a n d l e r ) : d e f _ _ i n i t _ _ ( s e l f ) : o . S i m p l e H a n d l e r . _ _ i n i t _ _ ( s e l f ) s e l f . n o d e s = 0 s e l f . w a y s = 0 s e l f . r e l s = 0 d e f n o d e ( s e l f , n ) : s e l f . n o d e s + = 1 d e f w a y ( s e l f , w ) : s e l f . w a y s + = 1 d e f r e l a t i o n ( s e l f , r ) : s e l f . r e l s + = 1
  27. libosmpbfreader and osmpbfreader‑rs libosmpbfreader is C++ library to read osm's

    pbf file. https://github.com/CanalTP/libosmpbfreader This library supports only osm id, location and tags. Can't make user stats. osmpbfreader‑rs is Rust library and same concept as libosmpbfreader. https://github.com/TeXitoi/osmpbfreader‑rs It's use iterator and match.
  28. l e t r = s t d : :

    f s : : F i l e : : o p e n ( & s t d : : p a t h : : P a t h : : n e w ( f i l e n a m e ) ) . u n w r a p ( ) ; l e t m u t p b f = o s m p b f r e a d e r : : O s m P b f R e a d e r : : n e w ( r ) ; l e t m u t n b _ n o d e s = 0 ; l e t m u t s u m _ l o n = 0 . ; l e t m u t s u m _ l a t = 0 . ; l e t m u t n b _ w a y s = 0 ; l e t m u t n b _ w a y _ n o d e s = 0 ; l e t m u t n b _ r e l s = 0 ; l e t m u t n b _ r e l _ r e f s = 0 ;
  29. f o r o b j i n p b

    f . i t e r ( ) { i f ! f i l t e r ( o b j . t a g s ( ) ) { c o n t i n u e ; } i n f o ! ( " { : ? } " , o b j ) ; m a t c h o b j { o s m p b f r e a d e r : : O s m O b j : : N o d e ( n o d e ) = > { n b _ n o d e s + = 1 ; s u m _ l o n + = n o d e . l o n ; s u m _ l a t + = n o d e . l a t ; } o s m p b f r e a d e r : : O s m O b j : : W a y ( w a y ) = > { n b _ w a y s + = 1 ; n b _ w a y _ n o d e s + = w a y . n o d e s . l e n ( ) ; } o s m p b f r e a d e r : : O s m O b j : : R e l a t i o n ( r e l ) = > { n b _ r e l s + = 1 ; n b _ r e l _ r e f s + = r e l . r e f s . l e n ( ) ; } } }
  30. rust‑osm‑reader (pbf‑reader) rust‑osm‑reader is Rust implementation and uses multithreaded approach

    for reading data. https://gitlab.com/oleksandromelchuk/rust‑osm‑reader This library also supports only osm id, location and tags. But I hacked to get user name. use Concurrency and match Package name has problem, pbf is not used only OpenStreeetMap.
  31. l e t ( m u t n o d

    e _ t x , n o d e _ r x ) = m p s c : : c h a n n e l : : < P B F D a t a > ( ) ; l e t h = t h r e a d : : s p a w n ( m o v e | | { r e t u r n r e a d _ p b f ( & " s r c / s a m p l e . p b f " . t o _ s t r i n g ( ) , 1 0 , & m u t n o d e _ t x ) ; } ) ; l e t m u t n o d e s _ c o u n t = 0 ; l e t m u t t a g s _ c o u n t = 0 ; / / i n a l l e n t i t i e s l e t m u t w a y _ c o u n t = 0 ; l e t m u t r e l _ c o u n t = 0 ; l e t m u t i n f o _ c o u n t = 0 ; l e t m u t s t r i n g s _ c o u n t = 0 ; l e t m u t t o t a l _ s t r i n g s _ c o u n t = 0 ; l e t m u t s t r i n g s _ i n d e x = H a s h M a p : : n e w ( ) ; l o o p { m a t c h n o d e _ r x . r e c v ( ) { O k ( p b f d a t a ) = > {
  32. m a t c h p b f d a

    t a { P B F D a t a : : N o d e s S e t ( s e t ) = > { f o r ( i d , n o d e ) i n s e t { n o d e s _ c o u n t = n o d e s _ c o u n t + 1 ; t a g s _ c o u n t = t a g s _ c o u n t + n o d e . t a g s . t a g s . l e n ( ) ; i f ! n o d e . t a g s . t a g s . i s _ e m p t y ( ) { m a t c h s t r i n g s _ i n d e x . g e t ( & n o d e . t a g s . s t r i n g _ t a b l e _ i d ) { S o m e ( s m a p ) = > { l e t k v s = n o d e . t a g s . g e t _ k e y s _ v a l s ( s m a p ) ; a s s e r t _ e q ! ( k v s . l e n ( ) , n o d e . t a g s . t a g s . l e n ( ) ) ; f o r ( k , v ) i n k v s { p r i n t l n ! ( " N o d e i d { } t a g s [ { } : { } ] " , i d , k , v ) ; } } N o n e = > { p r i n t l n ! ( " E r r o r " ) ; } } } } } P B F D a t a : : W a y s S e t ( s e t ) = > { / / . . .
  33. osmium(C++) benchmark % / u s r / b i

    n / t i m e . / e x a m p l e s / o s m i u m _ c o u n t \ . . / . . / . . / . . / d a t a / t a i w a n ‐ l a t e s t . o s m . p b f N o d e s : 5 3 4 2 2 6 0 W a y s : 5 6 3 7 8 6 R e l a t i o n s : 1 4 0 5 9 0 . 5 8 r e a l 2 . 2 3 u s e r 0 . 2 1 s y s % / u s r / b i n / t i m e . / e x a m p l e s / o s m i u m _ c o u n t \ . . / . . / . . / . . / d a t a / j a p a n ‐ 1 6 0 4 1 7 . o s m . p b f N o d e s : 1 1 4 8 2 5 5 9 0 W a y s : 1 1 6 6 0 6 2 2 R e l a t i o n s : 3 2 4 5 1 1 2 . 3 3 r e a l 4 6 . 3 7 u s e r 4 . 3 5 s y s
  34. pyosmium benchmark % / u s r / b i

    n / t i m e p y t h o n e x a m p l e s / o s m _ f i l e _ s t a t s . p y \ . . / . . / . . / d a t a / t a i w a n ‐ l a t e s t . o s m . p b f N o d e s : 5 3 4 2 2 6 0 W a y s : 5 6 3 7 8 6 R e l a t i o n s : 1 4 0 5 9 4 . 1 3 r e a l 5 . 4 2 u s e r 0 . 1 6 s y s % / u s r / b i n / t i m e p y t h o n e x a m p l e s / o s m _ f i l e _ s t a t s . p y \ . . / . . / . . / d a t a / j a p a n ‐ 1 6 0 4 1 7 . o s m . p b f N o d e s : 1 1 4 8 2 5 5 9 0 W a y s : 1 1 6 6 0 6 2 2 R e l a t i o n s : 3 2 4 5 1 8 9 . 9 3 r e a l 1 1 6 . 4 7 u s e r 2 . 5 6 s y s
  35. node‑osmium benchmark % / u s r / b i

    n / t i m e n o d e i n d e x . j s \ . . / . . / d a t a / t a i w a n ‐ l a t e s t . o s m . p b f N o d e s : 5 3 4 2 2 6 0 W a y s : 5 6 3 7 8 6 R e l a t i o n s : 1 4 0 5 9 8 . 7 2 r e a l 9 . 6 1 u s e r 0 . 6 6 s y s % / u s r / b i n / t i m e n o d e i n d e x . j s \ . . / . . / d a t a / j a p a n ‐ 1 6 0 4 1 7 . o s m . p b f C o m m a n d t e r m i n a t e d a b n o r m a l l y .
  36. libosmpbfreader benchmark % / u s r / b i

    n / t i m e . / e x a m p l e _ c o u n t e r \ . . / . . / . . / d a t a / t a i w a n ‐ l a t e s t . o s m . p b f [ I N F O ] R e a d i n g t h e f i l e . . / . . / . . / d a t a / t a i w a n ‐ l a t e s t . o s m . p b f [ I N F O ] W e f i n i s h e d r e a d i n g t h e f i l e W e r e a d 5 3 4 2 2 6 0 n o d e s , 5 6 3 7 8 6 w a y s a n d 1 4 0 5 9 r e l a t i o n s 3 . 7 4 r e a l 3 . 5 9 u s e r 0 . 1 2 s y s % / u s r / b i n / t i m e . / e x a m p l e _ c o u n t e r \ . . / . . / . . / d a t a / j a p a n ‐ 1 6 0 4 1 7 . o s m . p b f [ I N F O ] R e a d i n g t h e f i l e . . / . . / . . / d a t a / j a p a n ‐ 1 6 0 4 1 7 . o s m . p b f [ I N F O ] W e f i n i s h e d r e a d i n g t h e f i l e W e r e a d 1 1 4 8 2 5 5 9 0 n o d e s , 1 1 6 6 0 6 2 2 w a y s a n d 3 2 4 5 1 r e l a t i o n s 7 6 . 3 9 r e a l 7 3 . 4 8 u s e r 2 . 3 2 s y s
  37. osmpbfreader‑rs benchmark % / u s r / b i

    n / t i m e c a r g o r u n ‐ ‐ r e l e a s e \ \ . . / . . / d a t a / t a i w a n ‐ l a t e s t . o s m . p b f R u n n i n g ` t a r g e t / r e l e a s e / o s m p b f r e a d e r ‐ r s ‐ t e s t . . / . . / d a t a / t a i w a n ‐ l a t e s t . o s m . p b f ` c o u n t i n g o b j e c t s . . . 5 3 4 2 2 6 0 n o d e s , 5 6 3 7 8 6 w a y s , 1 4 0 5 9 r e l a t i o n s 4 . 1 3 r e a l 4 . 0 1 u s e r 0 . 0 8 s y s % / u s r / b i n / t i m e c a r g o r u n ‐ ‐ r e l e a s e \ \ . . / . . / d a t a / j a p a n ‐ 1 6 0 4 1 7 . o s m . p b f R u n n i n g ` t a r g e t / r e l e a s e / o s m p b f r e a d e r ‐ r s ‐ t e s t . . / . . / d a t a / j a p a n ‐ 1 6 0 4 1 7 . o s m . p b f ` c o u n t i n g o b j e c t s . . . 1 1 4 8 2 5 5 9 0 n o d e s , 1 1 6 6 0 6 2 2 w a y s , 3 2 4 5 1 r e l a t i o n s 8 8 . 8 5 r e a l 8 6 . 9 1 u s e r 1 . 4 4 s y s
  38. rust‑osm‑reader(0.1.2) benchmark % / u s r / b i

    n / t i m e c a r g o r u n ‐ ‐ r e l e a s e \ . . / . . / d a t a / t a i w a n ‐ l a t e s t . o s m . p b f . . . 5 3 4 2 2 6 0 n o d e s , 5 6 3 7 8 6 w a y s , 1 4 0 5 9 r e l a t i o n s D O N E ! ! ! ! 2 . 2 0 r e a l 7 . 4 4 u s e r 0 . 5 3 s y s % / u s r / b i n / t i m e c a r g o r u n ‐ ‐ r e l e a s e \ . . / . . / d a t a / j a p a n ‐ 1 6 0 4 1 7 . o s m . p b f . . . 1 1 4 8 2 5 5 9 0 n o d e s , 1 1 6 6 0 6 2 2 w a y s , 3 2 4 5 1 r e l a t i o n s D O N E ! ! ! ! 6 1 . 0 8 r e a l 1 6 8 . 3 6 u s e r 2 0 . 6 4 s y s
  39. rust‑osm‑reader(master) benchmark % / u s r / b i

    n / t i m e c a r g o r u n ‐ ‐ r e l e a s e ‐ ‐ e x a m p l e b e n c h m a r k \ . . / . . / d a t a / t a i w a n ‐ l a t e s t . o s m . p b f R u n n i n g ` t a r g e t / r e l e a s e / e x a m p l e s / b e n c h m a r k . . / . . / d a t a / t a i w a n ‐ l a t e s t . o s m . p b f ` . . . 5 3 4 2 2 6 0 n o d e s , 5 6 3 7 8 6 w a y s , 1 4 0 5 9 r e l a t i o n s D O N E ! ! ! ! 1 . 0 0 r e a l 5 . 2 5 u s e r 0 . 4 0 s y s % / u s r / b i n / t i m e c a r g o r u n ‐ ‐ r e l e a s e ‐ ‐ e x a m p l e b e n c h m a r k . . / . . / d a t a / j a p a n ‐ 1 6 0 4 1 7 . o s m . p b f R u n n i n g ` t a r g e t / r e l e a s e / e x a m p l e s / b e n c h m a r k . . / . . / d a t a / j a p a n ‐ 1 6 0 4 1 7 . o s m . p b f ` . . . 1 1 4 8 2 5 5 9 0 n o d e s , 1 1 6 6 0 6 2 2 w a y s , 3 2 4 5 1 r e l a t i o n s D O N E ! ! ! ! 1 6 . 4 0 r e a l 1 1 0 . 2 6 u s e r 5 . 8 0 s y s
  40. benchmark result Program Taiwan Japan osmium (C++) 0.58 12.33 rust‑osm‑reader

    master (Rust) 1.00 16.40 rust‑osm‑reader 0.1.2 (Rust) 2.20 61.08 libosmpbfreader (C++) 3.74 76.39 osmpbfreader‑rs (Rust) 4.13 88.85 pyosmium (Python) 4.13 89.93 node‑osmium (Node) 8.72 none
  41. summary osmium has good interface and better speed. If you

    are C++11/Python programmer, osmium is best solution in now. If you are Javascript programmer, let's hack node‑osmium. Rust is hot programming language in OSM. Let's try Rust.