yamabiko使ってみました。#TechLunch

Transcript

1. yamabikoΛ͞ΘͬͯΈ·ͨ͠ ͸Βͩɹ͔ͨΑ͠

2. ॳΊʹ yamabiko͕࡞ΒΕͨͷͰࢼ͠ʹ࢖ͬͯΈ·ͨ͠ɻ

3. yamabiko໋໊ͷલ y-ken͞Μͱ੮͕ྡͳͷͰϦϦʔεલʹ࿩͕͋Γ·ͨ͠ɻ y-ken > h-taka͞Μ͍͍ͷग़དྷ·ͨ͠ʂ y-ken > ੌ͍ͷग़དྷ·ͨ͠ʂ h-taka <

   ΄ʔ ͱ͍͏ձ࿩Λ΄΅ຖ೔͍ͯ͠·ͨ͠ɻ (͜ͷ͜ΖԿͷ͜ͱ͔ͩͬ͞ͺΓΘ͔͍ͬͯ·ͤΜɻ)

4. yamabiko໋໊ͷޙ y-ken͞Μͱ੮͕ྡͳͷͰ y-ken > ໊͍͍લͰ͢Ͷʂʂ y-ken > ໘ന͍ͷ͕ग़དྷ·ͨ͠ʂ h-taka <

   ΄ʔ ͱ͍͏ձ࿩Λ΄΅ຖ೔੝Γ্͕Γ·ͨ͠ɻ (͜ͷ͜ΖԿͷ͜ͱ͔ͩͬ͞ͺΓΘ͔͍ͬͯ·ͤΜɻ)

5. yamabiko໋໊ͷޙ y-ken͞Μͱ੮͕ྡͳͷͰ y-ken > ͔ͤͬ͘࡞ͬͨͷͰࣾ಺ͷਓʹ΋৮ͬͯཉ͍͠Ͱ͢ Ͷɻ y-ken > Ͷɻ y-ken

   > Ͷ!! h-taka < ... ͱ͍͏͜ͱͰ೥຤೥࢝ʹগ͠৮ͬͯΈ·ͨ͠ɻ

6. ͱ͍͏͜ͱͰ Yamabiko͞ΜΛΠϯετʔϧͯ͠؆୯ʹࢼ͠·ͨ͠ɻ ͍ͭͰͳͷͰrails͔ΒσʔλΛѻ͑Δςετ΋͠·ͨ͠ɻ

7. ຊ೔ͷྲྀΕ •  Yamabikoશମ૾ •  ElasticsearchΠϯετʔϧ •  fluent-plugin-mysql-replicatorΠϯετʔϧ •  yamabikoΠϯετʔϧ • 

   replicator_managerςʔϒϧ௥Ճ •  ͓·͚ʹrails-tire͔Β࢖༻͢Δɻ

8. શମ૾ ৄ͘͠͸ͪ͜Βɻ https://confluence.livesense.jp/pages/viewpage.action?pageId=117455003 MySQL͔ΒElasticsearch΁ͷඇಉظϨϓϦέʔγϣϯΛߦ͏ϛυϧ΢ΣΞ

9. ElasticsearchΠϯετʔϧ Elasticsearchμ΢ϯϩʔυ ղౚ͠·͢ɻ ࡉ͔͍ઃఆΛ͠·͢ɻ /etc/init.d/elasticsearch start

10. Elasticsearch։࢝͠·͢ɻ /etc/init.d/elasticsearch start Starting ElasticSearch... Waiting for ElasticSearch....... running: PID:15742

11. Elasticsearchؔ࿈ͦͷଞ •  Elasticsearch-headΛΠϯετʔϧ͠·͢ɻ

12. fluent-plugin-mysql-replicatorΠϯετʔϧ sudo gem install fluent-plugin-mysql-replicator sudo /usr/lib64/fluent/ruby/bin/fluent-gem install / fluent-plugin-mysql-replicator

13. yamabikoΠϯετʔϧ ҎԼͷrpmΛμ΢ϯϩʔυ͢Δɻ https://github.com/y-ken/yamabiko/releases/download/2013.12.13/yamabiko- libyaml-0.1.4-1.el6.x86_64.rpm https://github.com/y-ken/yamabiko/releases/download/2013.12.13/ yamabiko-1.1.18-0.el6.x86_64.rpm rpmΛΠϯετʔϧ͢Δɻ sudo yum install

    yamabiko-libyaml-0.1.4-1.el6.x86_64.rpm yamabiko-1.1.18-0.el6.x86_64.rpm

14. yamabikoΠϯετʔϧ Τϥʔ͕ൃੜ͢Δɻ sudo yum install yamabiko-1.1.18-0.el6.x86_64.rpm࣮ߦ࣌ʹҎԼͷΤϥʔ͕ൃੜɻ Error: Package: yamabiko-1.1.18-0.el6.x86_64 (/yamabiko-1.1.18-0.el6.x86_64)

    Requires: mysql-devel

15. yamabikoΠϯετʔϧ Τϥʔ͕ൃੜ͢Δɻ ࢦఆ͢Δύοέʔδ໊͕ҟͳΔͨΊɺrpmΛrebuild͠·ͨ͠ɻ sudo yum install rpmrebuild rpmrebuild -e -p

    yamabiko-1.1.18-0.el6.x86_64.rpm > Requires: mysql-devel < Requires: MySQL-devel rebuildͨ͠rpmΛ࠶౓Πϯετʔϧ্ͯ͠ख͘ग़དྷ·ͨ͠ɻ

16. replicator_managerςʔϒϧ௥Ճ ࠓճ͸MySQLʹYamabiko؅ཧDBͱؔ࿈tableΛ࢖༻͠· ͢ɻ DB: replicator_manager TB: settings : ಉظ͢Δ಺༰Λ௥Ճ͠·͢ɻ hash_tables

    : ಉظ݁Ռ͕௥Ճ͞Ε·͢ɻ

17. yamabiko.confͷઃఆมߋ MySQLͷ઀ଓઌΛมߋ͠·͢ɻ มߋ఺͸ҎԼͷͱ͓ΓͰ͢ɻ > tag hosts.host.${event}.${primary_key} < tag replicator.${name}.${event}.${primary_key} >

    <match replicator.**> < <match hosts.**> ͱ͍͏ײ͡ͰσϞ͠·͢ɻ

18. ͓·͚ʹrails-tire͔Β࢖༻͢Δɻ railsͷमਖ਼఺(scaffoldΛ࢖༻͍ͯ͠·͢ɻ) model < include Tire::Model::Search < include Tire::Model::Callbacks <

    def self.search(params) < লུ < end controller > model.all < model.search(params)

19. ͓·͚ʹrails-tire͔Β࢖༻͢Δɻ Ͱ΋̎

20. ·ͱΊ ɾෳ਺ઃఆ͕ඞཁͳͷͰͳΕΔ·Ͱେมɻ ɾDB΍ςʔϒϧΛෳ਺࢖༻ग़དྷΔͷͰ֦ு͠΍͘͢؅ཧ͠΍ͦ͢͏ ɾ৘ใ͕ฐࣾconfluenceͷํ͕ଟ͍ɻ ɾಉظ͢Δ࣌ؒ͸୹͍ҹ৅ɻ ɾ୯७ʹΠϯετʔϧͯ͠࢖ͬͯΈͨͷʹमਖ਼ґཔΛग़͍ͯͨ͠ɻ