Content integration at Bookmate Evgeny Li exAspArk

No content

• 3 million users • 600 publishers • 1000 new uploaded books per day

Simplified architecture Publisher Metadata XML,CSV Book EPUB, FB2 Cover JPEG, PNG Bookmate Source Clients

Metadata http://www.editeur.org/83/Overview ONIX (ONline Information eXchange) is an XML-based international standard for representing and communicating book industry product information in electronic form.

How many people in Russia use ONIX http://25.media.tumblr.com/tumblr_m784a1326G1rb3v0jo1_250.gif

ONIX sample                  20131202T192459          42          EUR          ...                      03          01                        15              9781824302123                                                  1                  A01                  David  Herbert  Lawrence                                              10                  FIC000000                            ...                                                    11                  00                  20101028                                              06                                        AS  CA  GU  MP  PH  PR  US  VI                                              ...                    ...

Specification is about 500 pages long http://1.bp.blogspot.com/-6-PaY1EWEK0/UnZt-TRK5QI/AAAAAAAAOnU/hSTQA3YNNxQ/s1600/chandler-bing-book.gif

• ONIX 2.1 – https://github.com/yob/onix (not maintained) • ONIX 3.0 – https://github.com/immateriel/im_onix RubyGems

Gems use the same names for modules & classes https://media.giphy.com/media/ToMjGpnXBTw7vnokxhu/giphy.gif

RubyGems • ONIX 2.1 – https://github.com/exAspArk/onix2 (fork) • ONIX 3.0 – https://github.com/immateriel/im_onix

• Easy to start • Open source • No ubiquitous language • Extra data abstractions https://en.wikipedia.org/wiki/Domain-driven_design

ONIX 2.1 Book metadata ONIX 3.0 onix2 im_onix onix2 adapter onix3 adapter Gems Application XML

Simple Made Easy http://www.infoq.com/presentations/Simple-Made-Easy Simple Easy

ONIX 2.1 Book metadata ONIX 3.0 onix2 im_onix onix2 adapter onix3 adapter Gems Application XML Why to use this extra layer?

ONIX 2.1 Book metadata ONIX 3.0 onix2 adapter onix3 adapter Application XML

https://github.com/exAspArk/better_struct

BetterStruct example BetterStruct.new(nil)  ==  BetterStruct.new(nil).no_undefined_method_error   #  =>  true  

BetterStruct example BetterStruct.new(nil)  ==  BetterStruct.new(nil).no_undefined_method_error   #  =>  true   BetterStruct.new(nil)  ==  BetterStruct.new(nil).maybe.monad   #  =>  true  

BetterStruct example BetterStruct.new(nil)  ==  BetterStruct.new(nil).no_undefined_method_error   #  =>  true   BetterStruct.new(nil)  ==  BetterStruct.new(nil).maybe.monad   #  =>  true   like_open_struct  =  BetterStruct.new({  title:  "Fifty  Shades  of  Grey"  })   like_open_struct.title.value   #  =>  "Fifty  Shades  of  Grey"  

BetterStruct example BetterStruct.new(nil)  ==  BetterStruct.new(nil).no_undefined_method_error   #  =>  true   BetterStruct.new(nil)  ==  BetterStruct.new(nil).maybe.monad   #  =>  true   like_open_struct  =  BetterStruct.new({  title:  "Fifty  Shades  of  Grey"  })   like_open_struct.title.value   #  =>  "Fifty  Shades  of  Grey"   like_open_struct.title.sub("Fifty",  "No").first.value   #  =>  "N"

ONIX example                                                                        Atlas  Shrugged                                                  

class  ONIX::V3::ProductAdapter      def  initialize(product)          @product  =  BetterStruct.new(product)      end      def  title          @product.descriptive_detail.title_detail.title_element.title_text.value      end   end                                                                        Atlas  Shrugged                                                   ONIX example

class  ONIX::V3::ProductAdapter      def  initialize(product)          @product  =  BetterStruct.new(product)      end      def  title          @product.descriptive_detail.title_detail.title_element.title_text.value      end   end   ONIX example xml  =  "Atlas  Shrugged TitleText>"  

class  ONIX::V3::ProductAdapter      def  initialize(product)          @product  =  BetterStruct.new(product)      end      def  title          @product.descriptive_detail.title_detail.title_element.title_text.value      end   end   ONIX example https://github.com/savonrb/nori xml  =  "Atlas  Shrugged TitleText>"   product  =  Nori.new.parse(xml)   #  =>  {  "DescriptiveDetail"  =>  {  "TitleDetail"  =>  {  "TitleElement"  =>  {  "TitleText"  =>   "Atlas  Shrugged"  }}}}  

class  ONIX::V3::ProductAdapter      def  initialize(product)          @product  =  BetterStruct.new(product)      end      def  title          @product.descriptive_detail.title_detail.title_element.title_text.value      end   end   ONIX example xml  =  "Atlas  Shrugged TitleText>"   product  =  Nori.new.parse(xml)   #  =>  {  "DescriptiveDetail"  =>  {  "TitleDetail"  =>  {  "TitleElement"  =>  {  "TitleText"  =>   "Atlas  Shrugged"  }}}}   product_adapter  =  ONIX::V3::ProductAdapter.new(product)   product_adapter.title   #  =>  "Atlas  Shrugged" https://github.com/savonrb/nori

product   #  =>  {  "DescriptiveDetail"  =>  {  "TitleDetail"  =>  {  "TitleElement"  =>  {  "TitleText"  =>   "Atlas  Shrugged"  }}}}   BetterStruct.new(product).descriptive_detail.title_detail.title_element.title_text.value   ONIX example

product   #  =>  {  "DescriptiveDetail"  =>  {  "TitleDetail"  =>  {  "TitleElement"  =>  {  "TitleText"  =>   "Atlas  Shrugged"  }}}}   BetterStruct.new(product).descriptive_detail.title_detail.title_element.title_text.value   product["DescriptiveDetail"]["TitleDetail"]["TitleElement"]["TitleText"]   ONIX example

product   #  =>  {  "DescriptiveDetail"  =>  {  "TitleDetail"  =>  {  "TitleElement"  =>  {  "TitleText"  =>   "Atlas  Shrugged"  }}}}   BetterStruct.new(product).descriptive_detail.title_detail.title_element.title_text.value   product["DescriptiveDetail"]  &&        product["DescriptiveDetail"]["TitleDetail"]  &&        product["DescriptiveDetail"]["TitleDetail"]["TitleElement"]  &&      product["DescriptiveDetail"]["TitleDetail"]["TitleElement"]["TitleText"]   ONIX example

product   #  =>  {  "DescriptiveDetail"  =>  {  "TitleDetail"  =>  {  "TitleElement"  =>  {  "TitleText"  =>   "Atlas  Shrugged"  }}}}   BetterStruct.new(product).descriptive_detail.title_detail.title_element.title_text.value   product["DescriptiveDetail"]  &&        product["DescriptiveDetail"]["TitleDetail"]  &&        product["DescriptiveDetail"]["TitleDetail"]["TitleElement"]  &&      product["DescriptiveDetail"]["TitleDetail"]["TitleElement"]["TitleText"]   product[“DescriptiveDetail”].      try(:[],  “TitleDetail").      try(:[],  “TitleElement").      try(:[],  "TitleText")   ONIX example

product   #  =>  {  "DescriptiveDetail"  =>  {  "TitleDetail"  =>  {  "TitleElement"  =>  {  "TitleText"  =>   "Atlas  Shrugged"  }}}}   BetterStruct.new(product).descriptive_detail.title_detail.title_element.title_text.value   product["DescriptiveDetail"]  &&        product["DescriptiveDetail"]["TitleDetail"]  &&        product["DescriptiveDetail"]["TitleDetail"]["TitleElement"]  &&      product["DescriptiveDetail"]["TitleDetail"]["TitleElement"]["TitleText"]   product[“DescriptiveDetail”].      try(:[],  “TitleDetail").      try(:[],  “TitleElement").      try(:[],  "TitleText")   product["DescriptiveDetail"]["TitleDetail"]["TitleElement"]["TitleText"]  rescue  nil ONIX example

CSV example Título completo Año publicación Nombre 1 eISBN epub Programa de transición 2008 Trotsky, León 9781449262123 filepath  =  "spanish_publisher.csv"   books  =  CSV.parse(File.read(filepath),  headers:  true)   book_data  =  BetterStruct.new(books.first.to_h)  

CSV example Título completo Año publicación Nombre 1 eISBN epub Programa de transición 2008 Trotsky, León 9781449262123 filepath  =  "spanish_publisher.csv"   books  =  CSV.parse(File.read(filepath),  headers:  true)   book_data  =  BetterStruct.new(books.first.to_h)   book_data.titulo_completo.value   #  =>  "Programa  de  transición"  

CSV example Título completo Año publicación Nombre 1 eISBN epub Programa de transición 2008 Trotsky, León 9781449262123 filepath  =  "spanish_publisher.csv"   books  =  CSV.parse(File.read(filepath),  headers:  true)   book_data  =  BetterStruct.new(books.first.to_h)   book_data.titulo_completo.value   #  =>  "Programa  de  transición"   book_data.ano_publicacion.value   #  =>  "2008"  

CSV example Título completo Año publicación Nombre 1 eISBN epub Programa de transición 2008 Trotsky, León 9781449262123 filepath  =  "spanish_publisher.csv"   books  =  CSV.parse(File.read(filepath),  headers:  true)   book_data  =  BetterStruct.new(books.first.to_h)   book_data.titulo_completo.value   #  =>  "Programa  de  transición"   book_data.ano_publicacion.value   #  =>  "2008"   book_data.nombre_1.split(",  ").reverse.join("  ").value   #  =>  "León  Trotsky"  

CSV example Título completo Año publicación Nombre 1 eISBN epub Programa de transición 2008 Trotsky, León 9781449262123 filepath  =  "spanish_publisher.csv"   books  =  CSV.parse(File.read(filepath),  headers:  true)   book_data  =  BetterStruct.new(books.first.to_h)   book_data.titulo_completo.value   #  =>  "Programa  de  transición"   book_data.ano_publicacion.value   #  =>  "2008"   book_data.nombre_1.split(",  ").reverse.join("  ").value   #  =>  "León  Trotsky"   book_data.e_isbn_epub.value   #  =>  "9781449262123"

Conclusion Don’t ruin information behind a micro-language, i.e. a class with information-specific methods Rich Hickey

Thank you!