yamabiko使ってみました。#TechLunch

yamabiko使ってみました。
2014/01/8 (水) 12:00-13:00 @ Livesense TechLunch
発表者：原田 貴義

Transcript

1. yamabikoΛ͞ΘͬͯΈ·ͨ͠ ͸Βͩɹ͔ͨΑ͠
2. ॳΊʹ yamabiko͕࡞ΒΕͨͷͰࢼ͠ʹ࢖ͬͯΈ·ͨ͠ɻ
3. yamabiko໋໊ͷલ y-ken͞Μͱ੮͕ྡͳͷͰϦϦʔεલʹ࿩͕͋Γ·ͨ͠ɻ y-ken > h-taka͞Μ͍͍ͷग़དྷ·ͨ͠ʂ y-ken > ੌ͍ͷग़དྷ·ͨ͠ʂ h-taka < ΄ʔ ͱ͍͏ձ࿩Λ΄΅ຖ೔͍ͯ͠·ͨ͠ɻ (͜ͷ͜ΖԿͷ͜ͱ͔ͩͬ͞ͺΓΘ͔͍ͬͯ·ͤΜɻ)
4. yamabiko໋໊ͷޙ y-ken͞Μͱ੮͕ྡͳͷͰ y-ken > ໊͍͍લͰ͢Ͷʂʂ y-ken > ໘ന͍ͷ͕ग़དྷ·ͨ͠ʂ h-taka < ΄ʔ ͱ͍͏ձ࿩Λ΄΅ຖ೔੝Γ্͕Γ·ͨ͠ɻ (͜ͷ͜ΖԿͷ͜ͱ͔ͩͬ͞ͺΓΘ͔͍ͬͯ·ͤΜɻ)
5. yamabiko໋໊ͷޙ y-ken͞Μͱ੮͕ྡͳͷͰ y-ken > ͔ͤͬ͘࡞ͬͨͷͰࣾ಺ͷਓʹ΋৮ͬͯཉ͍͠Ͱ͢ Ͷɻ y-ken > Ͷɻ y-ken > Ͷ!! h-taka < ... ͱ͍͏͜ͱͰ೥຤೥࢝ʹগ͠৮ͬͯΈ·ͨ͠ɻ
6. ͱ͍͏͜ͱͰ Yamabiko͞ΜΛΠϯετʔϧͯ͠؆୯ʹࢼ͠·ͨ͠ɻ ͍ͭͰͳͷͰrails͔ΒσʔλΛѻ͑Δςετ΋͠·ͨ͠ɻ
7. ຊ೔ͷྲྀΕ •  Yamabikoશମ૾ •  ElasticsearchΠϯετʔϧ •  fluent-plugin-mysql-replicatorΠϯετʔϧ •  yamabikoΠϯετʔϧ •  replicator_managerςʔϒϧ௥Ճ •  ͓·͚ʹrails-tire͔Β࢖༻͢Δɻ
8. શମ૾ ৄ͘͠͸ͪ͜Βɻ https://confluence.livesense.jp/pages/viewpage.action?pageId=117455003 MySQL͔ΒElasticsearch΁ͷඇಉظϨϓϦέʔγϣϯΛߦ͏ϛυϧ΢ΣΞ
9. ElasticsearchΠϯετʔϧ Elasticsearchμ΢ϯϩʔυ ղౚ͠·͢ɻ ࡉ͔͍ઃఆΛ͠·͢ɻ /etc/init.d/elasticsearch start
10. Elasticsearch։࢝͠·͢ɻ /etc/init.d/elasticsearch start Starting ElasticSearch... Waiting for ElasticSearch....... running: PID:15742
11. Elasticsearchؔ࿈ͦͷଞ •  Elasticsearch-headΛΠϯετʔϧ͠·͢ɻ
12. fluent-plugin-mysql-replicatorΠϯετʔϧ sudo gem install fluent-plugin-mysql-replicator sudo /usr/lib64/fluent/ruby/bin/fluent-gem install / fluent-plugin-mysql-replicator
13. yamabikoΠϯετʔϧ ҎԼͷrpmΛμ΢ϯϩʔυ͢Δɻ https://github.com/y-ken/yamabiko/releases/download/2013.12.13/yamabiko- libyaml-0.1.4-1.el6.x86_64.rpm https://github.com/y-ken/yamabiko/releases/download/2013.12.13/ yamabiko-1.1.18-0.el6.x86_64.rpm rpmΛΠϯετʔϧ͢Δɻ sudo yum install yamabiko-libyaml-0.1.4-1.el6.x86_64.rpm yamabiko-1.1.18-0.el6.x86_64.rpm
14. yamabikoΠϯετʔϧ Τϥʔ͕ൃੜ͢Δɻ sudo yum install yamabiko-1.1.18-0.el6.x86_64.rpm࣮ߦ࣌ʹҎԼͷΤϥʔ͕ൃੜɻ Error: Package: yamabiko-1.1.18-0.el6.x86_64 (/yamabiko-1.1.18-0.el6.x86_64) Requires: mysql-devel
15. yamabikoΠϯετʔϧ Τϥʔ͕ൃੜ͢Δɻ ࢦఆ͢Δύοέʔδ໊͕ҟͳΔͨΊɺrpmΛrebuild͠·ͨ͠ɻ sudo yum install rpmrebuild rpmrebuild -e -p yamabiko-1.1.18-0.el6.x86_64.rpm > Requires: mysql-devel < Requires: MySQL-devel rebuildͨ͠rpmΛ࠶౓Πϯετʔϧ্ͯ͠ख͘ग़དྷ·ͨ͠ɻ
16. replicator_managerςʔϒϧ௥Ճ ࠓճ͸MySQLʹYamabiko؅ཧDBͱؔ࿈tableΛ࢖༻͠· ͢ɻ DB: replicator_manager TB: settings : ಉظ͢Δ಺༰Λ௥Ճ͠·͢ɻ hash_tables : ಉظ݁Ռ͕௥Ճ͞Ε·͢ɻ
17. yamabiko.confͷઃఆมߋ MySQLͷ઀ଓઌΛมߋ͠·͢ɻ มߋ఺͸ҎԼͷͱ͓ΓͰ͢ɻ > tag hosts.host.${event}.${primary_key} < tag replicator.${name}.${event}.${primary_key} > <match replicator.**> < <match hosts.**> ͱ͍͏ײ͡ͰσϞ͠·͢ɻ
18. ͓·͚ʹrails-tire͔Β࢖༻͢Δɻ railsͷमਖ਼఺(scaffoldΛ࢖༻͍ͯ͠·͢ɻ) model < include Tire::Model::Search < include Tire::Model::Callbacks < def self.search(params) < লུ < end controller > model.all < model.search(params)
19. ͓·͚ʹrails-tire͔Β࢖༻͢Δɻ Ͱ΋̎
20. ·ͱΊ ɾෳ਺ઃఆ͕ඞཁͳͷͰͳΕΔ·Ͱେมɻ ɾDB΍ςʔϒϧΛෳ਺࢖༻ग़དྷΔͷͰ֦ு͠΍͘͢؅ཧ͠΍ͦ͢͏ ɾ৘ใ͕ฐࣾconfluenceͷํ͕ଟ͍ɻ ɾಉظ͢Δ࣌ؒ͸୹͍ҹ৅ɻ ɾ୯७ʹΠϯετʔϧͯ͠࢖ͬͯΈͨͷʹमਖ਼ґཔΛग़͍ͯͨ͠ɻ