とある業務オペレーション自動化の話

B8668563cbe87543c303dbb818cf9f02?s=47 onigra
October 09, 2015

 とある業務オペレーション自動化の話

StackOverflow DevDays in Japan 2015.10.09

B8668563cbe87543c303dbb818cf9f02?s=128

onigra

October 09, 2015
Tweet

Transcript

  1. ͱ͋Δۀ຿ΦϖϨʔγϣϯ ࣗಈԽͷ࿩ StackOverflow DevDays 2015.10.09 @onigra_

  2. onigra Yudai Suzuki / ླ໦༤େ onigra_ tenso, inc. / tensoגࣜձࣾ

  3. • Web Application Developer • DevOps • Rubyist • My

    favorite episode of Joel on Software: My First BillG Review
  4. tensoגࣜձࣾʹ͍ͭͯ

  5. Cross Border E-Commerce
 ӽڥECαʔϏε

  6. We buy at Japanese EC on your behalf and ship

    the items to your address overseas.
  7. Monolithic PHP Application

  8. Monolithic PHP Application Micro Ruby Application Micro Ruby Application Micro

    Ruby Application Micro Ruby Application
  9. ։ൃͷܦҢ

  10. ։ൃͷܦҢ • αʔϏε։࢝౰ॳʹ࡞ΒΕͨɺ୅ཧߪೖͰ࢖ ༻͞ΕΔγεςϜ͕͋ͬͨ • αʔϏε͕֦େ͠ɺࣗ෼͕ೖࣾͯ͠࡞Γ௚͢ ͜ͱʹͳͬͨ

  11. Architecture

  12. Application • Ruby (Ruby on Rails) • Capybara • PhantomJS

    (Poltergeist) • Sidekiq
  13. Infrastructure • AWS • Ansible • Tsudura (Original Packer Clone)

    • Capistrano • Blue-Green Deployment
  14. None
  15. Blue-Green Deployment • Auto Scaling Groupʹର͢ΔELBͷ෇͚֎͕͠Ͱ͖ΔΑ͏ʹͳͬͨͷͰಋ ೖ • ௨ৗͷDeploy͸Capistranoɺϛυϧ΢ΣΞ౳ͷΞοϓσʔτΛߦͬͨࡍ ʹ͸BG

    Deploy • PackerͰAnsible͕࢖͍ͮΒ͔ͬͨΓɺLaunch Config࡞੒΍Auto Scaling Groupͷ੾ସΛࣗಈͰ΍Γ͔ͨͬͨͷͰࣗ࡞ͷPacker clone cliΛ࡞੒ • http://github.com/onigra/tsudura • https://speakerdeck.com/onigra/about-tsudura
  16. Job͕΍͍ͬͯΔ͜ͱ • νΣοΫΞ΢τʹඞཁͳ৘ใΛࣗಈೖྗ • ෳ਺ͷ߲໨ͷத͔Βબ୒ͷ൑அ͕ඞཁͳ΋ͷ ͸ɺͪ͜ΒͰܾΊͨ༏ઌॱҐΛ΋ͱʹબ୒͠ ͨΓ͢Δ • ࣗࣾ؅ཧը໘ʹऔಘͨ͠৘ใΛࣗಈೖྗ

  17. SidekiqͰJobΛ
 ฒྻ࣮ߦ͢Δࡍʹ • δϣϒ͝ͱʹηογϣϯ͕෼͔ΕΔΑ͏ʹ͢Δ • ෼͚ͳ͍ͱಉ࣮࣌ߦ࣌ʹΤϥʔ • CookieϑΝΠϧΛRunnerͷΠϯελϯε͝ͱʹ TempfileͰ࡞੒ •

    RunnerͷΠϯελϯε͝ͱʹCapybara::SessionΛ ࡞੒
  18. class WorkerSession attr_reader :session def initialize tempfile = Tempfile.new(prefix, path)

    Capybara.register_driver :poltergeist do |app| Capybara::Poltergeist::Driver.new( app, phantomjs_options: [ "--cookies-file=#{tempfile.path}", ] ) end @session = Capybara::Session.new(:poltergeist) end end
  19. class ExampleRunner def initialize(param) @param = param @capybara = WorkerSession.new

    end def run login access_to_checkout select_shipping_method select_payment_method submit ensure logout delete_session end end
  20. class ExampleWorker include Sidekiq::Worker sidekiq_options queue: :example def perform(param) ExampleRunner.new(param).run

    end end
  21. ։ൃ։͔࢝ΒϦϦʔε·Ͱ

  22. ։ൃ։͔࢝ΒϦϦʔε·Ͱ • ։ൃ։͔࢝Β2ि͙ؒΒ͍ͰϦϦʔε • جຊ1ਓͰ։ൃɺΠϯϑϥɺӡ༻ • ϑΝʔετϦϦʔε͸Ұ෦ػೳͷΈ • ϑΝʔετϦϦʔεޙɺগͣͭ͠ଞͷਓʹख఻ͬͯ΋ΒͬͨΓ •

    ܧଓతʹϦϦʔεΛߦ͍ɺ࠷ऴతʹۀ຿ଆ͕΍Γ͍ͨ͜ͱΛશ෦࣮ݱɻͦ ͷܭըΛͪΌΜͱۀ຿୲౰ऀͱѲͬͨ • CIͱσϓϩΠͷ࢓૊ΈΛ͔ͬ͠Γ੔උͯͨ͠ͷͰɺҰਓͰ΋͏·͍ͬͨ͘
  23. ӡ༻͕͸͡·ͬͯ

  24. PhantomJS͕Ϋϥογϡ͢Δ

  25. None
  26. ϦϦʔε౰ॳ • ࣮ߦ݅਺: 600-900/Day • ࣦഊ݅਺: 10-100/Day • PhantomJSͷΫϥογϡ͸Auto HealingͰޡ

    ຐԽͭͭ͠ӡ༻
  27. ৽ػೳϦϦʔε

  28. ৽ػೳϦϦʔεલ • ࣮ߦ݅਺: 1200-2000/day • ࣦഊ݅਺: 100-200/day

  29. ϦϦʔεޙ • ࣮ߦ݅਺: 5000-6000/day • ࣦഊ݅਺: 600-1500/day • Ұ൪ଟ͍࣌Ͱ12000/5000

  30. ϝϞϦϦʔΫʹΑΔCrashଟൃ • δϣϒ͕ϑϦʔζͨ͠ΠϯελϯεΛ௵ͯ͠ ͸ཱͯΔӡ༻ͷ೔ʑ • ΠϯελϯεαΠζ্͛ͨΓɺΠϯελϯε ਺૿΍ͯ͠΋େ͖ͳޮՌ͸ແ͠

  31. (❨›°□°)❩›︵ᵲᴸᵲ

  32. ֘౰ػೳͷ࠶ઃܭ • ແବʹ࣮ߦͯ͠͠·ͬͯΔδϣϒ͕ଟ͍ • ঢ়ଶΛݟͯɺແବͳδϣϒ͸࣮ߦ͞Εͳ͍Α ͏ʹͨ͠ • ݟͯݟ͵;Γͯͨ͠ϝϞϦϦʔΫʹຊ֨తʹ ޲͖߹͏

  33. ϝϞϦϦʔΫରࡦ

  34. https://github.com/teampoltergeist/ poltergeist#memory-leak PoltergeistͷREADME

  35. session.driver.quit • ηογϣϯ࡞੒࣌ʹىಈͨ͠PhantomJSͷϓ ϩηεΛKILL͢Δϝιου͕༻ҙ͞Εͯͨ • ݁ہ΍ͬͨͷ͸͜ΕΛRunner#runϝιουͷ ensureʹೖΕ͚ͨͩ

  36. None
  37. None
  38. None
  39. class ExampleRunner def initialize(param) @param = param @capybara = WorkerSession.new

    end def run login access_to_checkout select_shipping_method select_payment_method submit ensure logout delete_session end end ͜͜΋ବ໨
  40. νϡʔχϯάͷ݁Ռ • ॲཧ݅਺: 5000-7000/day • ࣦഊ݅਺: 20-50/day • Ϋϥογϡൃੜ͠ͳ͘ͳͬͨ •

    ϏϏͬͯಉ࣮࣌ߦ਺Λগͳ͍͕ͯͨ͘͠ɺ૿ ΍ͯ͠΋໰୊ͳ͘ͳͬͨ
  41. None
  42. ݱࡏ๊͍͑ͯΔ՝୊

  43. CI͕͏·͍͔͘ͳ͘ͳͬͨ • CI͸CircleCI • ๺ถ͔ΒͷΞΫηεʹͳΔͷͰɺςετ಺ͰԿ౓΋Login܁ Γฦ͢ͱɺαʔϏεʹΑͬͯ͸Captureڬ·Εͯ٧Ή • ։ൃ౰ॳ͸େৎ෉ͩͬͨʢςετέʔεগͳ͔͔ͬͨΒʁʣ • ࠃ಺ͷαʔόʹDrone.ioཱͯΔͱ͔ɺࠃ಺ͷCIαʔϏε

    (SideCI)࢖͏ͱ͔ݕ౼த
  44. ౰વ͕ͩɺ֎෦αʔϏεͷ
 ϚʔΫΞοϓ͕มΘͬͨΒେม • ͦΕΛݕ஌͢ΔͷΛؚΊͯɺ1೔ʹ਺ճςετ ࣮ߦͯͨ͠Μ͚ͩͲ… • ͱΓ͋͑ͣࣗ෼ͷϚγϯͰ1೔ʹԿճ͔࣮ߦ͠ ͯΔ • ࠓͷॴൃੜ͍ͯ͠ͳ͍

  45. ࠷ޙʹ

  46. ࠷ޙʹ • ͜Μͳײ͡Ͱ೔ʑ੒௕தͷαʔϏεΛεέʔ ϧͤ͞ΔͨΊʹؤு͍ͬͯ·͢ • ͦΜͳঢ়گͳͷͰɺฐࣾΤϯδχΞืूதͰ ͢ • ಛʹΠϯϑϥΤϯδχΞืूதͰ͢ʂ

  47. Do you have any question?