yamabiko使ってみました。#TechLunch

 yamabiko使ってみました。#TechLunch

yamabiko使ってみました。
2014/01/8 (水) 12:00-13:00 @ Livesense TechLunch
発表者:原田 貴義

E60aa4f80303f3f386898546ddb3686a?s=128

Livesense Inc.

April 21, 2014
Tweet

Transcript

  1. yamabikoΛ͞ΘͬͯΈ·ͨ͠ ͸Βͩɹ͔ͨΑ͠

  2. ॳΊʹ yamabiko͕࡞ΒΕͨͷͰࢼ͠ʹ࢖ͬͯΈ·ͨ͠ɻ

  3. yamabiko໋໊ͷલ y-ken͞Μͱ੮͕ྡͳͷͰϦϦʔεલʹ࿩͕͋Γ·ͨ͠ɻ y-ken > h-taka͞Μ͍͍ͷग़དྷ·ͨ͠ʂ y-ken > ੌ͍ͷग़དྷ·ͨ͠ʂ h-taka <

    ΄ʔ ͱ͍͏ձ࿩Λ΄΅ຖ೔͍ͯ͠·ͨ͠ɻ (͜ͷ͜ΖԿͷ͜ͱ͔ͩͬ͞ͺΓΘ͔͍ͬͯ·ͤΜɻ)
  4. yamabiko໋໊ͷޙ y-ken͞Μͱ੮͕ྡͳͷͰ y-ken > ໊͍͍લͰ͢Ͷʂʂ y-ken > ໘ന͍ͷ͕ग़དྷ·ͨ͠ʂ h-taka <

    ΄ʔ ͱ͍͏ձ࿩Λ΄΅ຖ೔੝Γ্͕Γ·ͨ͠ɻ (͜ͷ͜ΖԿͷ͜ͱ͔ͩͬ͞ͺΓΘ͔͍ͬͯ·ͤΜɻ)
  5. yamabiko໋໊ͷޙ y-ken͞Μͱ੮͕ྡͳͷͰ y-ken > ͔ͤͬ͘࡞ͬͨͷͰࣾ಺ͷਓʹ΋৮ͬͯཉ͍͠Ͱ͢ Ͷɻ y-ken > Ͷɻ y-ken

    > Ͷ!! h-taka < ... ͱ͍͏͜ͱͰ೥຤೥࢝ʹগ͠৮ͬͯΈ·ͨ͠ɻ
  6. ͱ͍͏͜ͱͰ Yamabiko͞ΜΛΠϯετʔϧͯ͠؆୯ʹࢼ͠·ͨ͠ɻ ͍ͭͰͳͷͰrails͔ΒσʔλΛѻ͑Δςετ΋͠·ͨ͠ɻ

  7. ຊ೔ͷྲྀΕ •  Yamabikoશମ૾ •  ElasticsearchΠϯετʔϧ •  fluent-plugin-mysql-replicatorΠϯετʔϧ •  yamabikoΠϯετʔϧ • 

    replicator_managerςʔϒϧ௥Ճ •  ͓·͚ʹrails-tire͔Β࢖༻͢Δɻ
  8. શମ૾ ৄ͘͠͸ͪ͜Βɻ https://confluence.livesense.jp/pages/viewpage.action?pageId=117455003 MySQL͔ΒElasticsearch΁ͷඇಉظϨϓϦέʔγϣϯΛߦ͏ϛυϧ΢ΣΞ

  9. ElasticsearchΠϯετʔϧ Elasticsearchμ΢ϯϩʔυ ղౚ͠·͢ɻ ࡉ͔͍ઃఆΛ͠·͢ɻ /etc/init.d/elasticsearch start

  10. Elasticsearch։࢝͠·͢ɻ /etc/init.d/elasticsearch start Starting ElasticSearch... Waiting for ElasticSearch....... running: PID:15742

  11. Elasticsearchؔ࿈ͦͷଞ •  Elasticsearch-headΛΠϯετʔϧ͠·͢ɻ

  12. fluent-plugin-mysql-replicatorΠϯετʔϧ sudo gem install fluent-plugin-mysql-replicator sudo /usr/lib64/fluent/ruby/bin/fluent-gem install / fluent-plugin-mysql-replicator

  13. yamabikoΠϯετʔϧ ҎԼͷrpmΛμ΢ϯϩʔυ͢Δɻ https://github.com/y-ken/yamabiko/releases/download/2013.12.13/yamabiko- libyaml-0.1.4-1.el6.x86_64.rpm https://github.com/y-ken/yamabiko/releases/download/2013.12.13/ yamabiko-1.1.18-0.el6.x86_64.rpm rpmΛΠϯετʔϧ͢Δɻ sudo yum install

    yamabiko-libyaml-0.1.4-1.el6.x86_64.rpm yamabiko-1.1.18-0.el6.x86_64.rpm
  14. yamabikoΠϯετʔϧ Τϥʔ͕ൃੜ͢Δɻ sudo yum install yamabiko-1.1.18-0.el6.x86_64.rpm࣮ߦ࣌ʹҎԼͷΤϥʔ͕ൃੜɻ Error: Package: yamabiko-1.1.18-0.el6.x86_64 (/yamabiko-1.1.18-0.el6.x86_64)

    Requires: mysql-devel
  15. yamabikoΠϯετʔϧ Τϥʔ͕ൃੜ͢Δɻ ࢦఆ͢Δύοέʔδ໊͕ҟͳΔͨΊɺrpmΛrebuild͠·ͨ͠ɻ sudo yum install rpmrebuild rpmrebuild -e -p

    yamabiko-1.1.18-0.el6.x86_64.rpm > Requires: mysql-devel < Requires: MySQL-devel rebuildͨ͠rpmΛ࠶౓Πϯετʔϧ্ͯ͠ख͘ग़དྷ·ͨ͠ɻ
  16. replicator_managerςʔϒϧ௥Ճ ࠓճ͸MySQLʹYamabiko؅ཧDBͱؔ࿈tableΛ࢖༻͠· ͢ɻ DB: replicator_manager TB: settings : ಉظ͢Δ಺༰Λ௥Ճ͠·͢ɻ hash_tables

    : ಉظ݁Ռ͕௥Ճ͞Ε·͢ɻ
  17. yamabiko.confͷઃఆมߋ MySQLͷ઀ଓઌΛมߋ͠·͢ɻ มߋ఺͸ҎԼͷͱ͓ΓͰ͢ɻ > tag hosts.host.${event}.${primary_key} < tag replicator.${name}.${event}.${primary_key} >

    <match replicator.**> < <match hosts.**> ͱ͍͏ײ͡ͰσϞ͠·͢ɻ
  18. ͓·͚ʹrails-tire͔Β࢖༻͢Δɻ railsͷमਖ਼఺(scaffoldΛ࢖༻͍ͯ͠·͢ɻ) model < include Tire::Model::Search < include Tire::Model::Callbacks <

    def self.search(params) < লུ < end controller > model.all < model.search(params)
  19. ͓·͚ʹrails-tire͔Β࢖༻͢Δɻ Ͱ΋̎

  20. ·ͱΊ ɾෳ਺ઃఆ͕ඞཁͳͷͰͳΕΔ·Ͱେมɻ ɾDB΍ςʔϒϧΛෳ਺࢖༻ग़དྷΔͷͰ֦ு͠΍͘͢؅ཧ͠΍ͦ͢͏ ɾ৘ใ͕ฐࣾconfluenceͷํ͕ଟ͍ɻ ɾಉظ͢Δ࣌ؒ͸୹͍ҹ৅ɻ ɾ୯७ʹΠϯετʔϧͯ͠࢖ͬͯΈͨͷʹमਖ਼ґཔΛग़͍ͯͨ͠ɻ