安定したFeature Specを助ける3つの処方箋

この発表では、弊社プロダクトにおいてテスト環境を改善するために実施した施策をfeature specに着目して紹介します。我々のチームはこの1年でFactoryデータに構文チェックを導入し、ドライバを置き換え、System specを用いることによりさらにデータの取り扱いを改善しました。これらによる余計なコードの削減、成功率の改善といった効果も発表します。

Transcript

1. ҆ఆͨ͠
 'FBUVSF
   4QFD
   Λॿ͚Δ
   ͭͷॲํᝦ ϦϯΧʔζגࣜձࣾ ླ໦ ཽଠ 1
2. ࣗݾ঺հ • 1989೥9݄22೔ ࣛࣇౡ࢈·Ε ౦ژɾࡳຈҭͪ • ϦϯΧʔζגࣜձࣾ Linkers for BankνʔϜ • Railsྺ3೥ (2016೥10݄ʙ) • FࣾܥSIerग़਎ (͜ͷ࿩࢝·Δͱ௕͍) • લ࡞(7/17 Otemachi.rb)→
 https://speakerdeck.com/strviola/que-ren-hua-mian-besutopurakuteisu 2
3. Feature Spec 3
4. "ॲํᝦ" •ChromeDriver •default wait (Ծ) •System Spec 4
5. ChromeDriver 5
6. Feature Spec (Capybara) ͷߏ੒ཁૉ • server: RailsΞϓϦέʔγϣϯͱ઀ଓ • rspec: MatcherΦϒδΣΫτɾϝιου(have_title, ...)Λఏڙ • selector: HTMLཁૉΛऔಘ • node: HTMLཁૉΛѻ͏ • driver: ϒϥ΢βͷڍಈΛ࠶ݱ (ಛʹJavaScript) • Poltergeist.js • ChromeDriver 6
7. FEATURE SPEC MAP Rails RSpec MVC DB rspec-rails rspec-core Capybara server rspec node selector driver Driver Poltergeist.js 7
8. Poltergeist αϙʔτͷྺ࢙ • ͔ͭͯ།Ұͷબ୒ࢶɻฐࣾͰ΋࢖͍ͬͯͨ • 2017೥ɺ Headless Chrome (beta) ొ৔ • 2018೥5݄ࠒ Stable એݴ • ͜ΕΛػʹ Phantom.js ͷϝϯςφʔ͕ҾୀΛએݴ • "I don't see any future in developing PhantomJS.
 Developing PhantomJS 2 and 2.5 as a single developer is a bloody hell." • Phantom.jsʹڧ͘ґଘ͍ͯͨ͠Poltergeist΋αϙʔτऴྃ • ChromeDriver͕ೖΕସΘΔΑ͏ʹීٴ • PoltergeistΑΓ҆ఆɺ͍ܰ 8
9. ChromeDriver Λ࢖͏ͨΊʹ • gem 'webdrivers' • chromedriver-helper ʹͳ͍ͬͯΔࢿྉ͸ݹ͍ͷͰ஫ҙ • spec/rails_helper.rb ʹىಈઃఆΛॻ͘ • ΍΍هड़ଟ͍ • ࠷ۙͦ͜·Ͱ͠ͳ͍͍ͯ͘Β͍͠ (ະணख) • CIͰճ͢৔߹ChromeͷΠϯετʔϧ͕ඞཁ 9
10. ChromeDriver ઃఆ෦෼ Capybara.register_driver :selenium_chrome_headless do |app| options = %w[headless disable-gpu window-size=1280,1200 lang=ja no-sandbox disable-dev-shm-usage disable-setuid-sandbox no-cache disable-infobars --enable-features=NetworkService,NetworkServiceInProcess] driver = Capybara::Selenium::Driver.new( app, browser: :chrome, options: Selenium::WebDriver::Chrome::Options.new(args: options), desired_capabilities: Selenium::WebDriver::Remote::Capabilities.chrome( login_prefs: { browser: 'ALL' }, chrome_options: { args: options })) bridge = driver.browser.send(:bridge) path = "session/#{bridge.session_id}/chromium/send_command" bridge.http.call( :post, path, cmd: 'Page.setDownloadBehavior', params: { behavior: 'allow', downloadPath: DownloadHelper::PATH.to_s }) driver end ىಈΦϓγϣϯ ϒϥ΢βઃఆ HTTP઀ଓઃఆ 10
11. ChromeDriverͷ·ͱΊ • 2017೥Ҏલ͔ΒFeature SpecΛӡ༻͍ͯ͠Δ৔߹͸
 ੾Γସ͕͑ඞཁ͔΋ • Poltergeist͸Ή͠Ζ࢖͏ͱةͳ͍ • SystemSpec(ޙड़)ͱͷ݉Ͷ߹͍΋͋ΓChromeDriver͸࣮࣭ެࣜ 11
12. default wait (Ծ) 12
13. Ͳ͜ͷ͜ͱͩ • Capybara͸σϑΥϧτͰཁૉͷग़ݱΛ଴ͭ • Capybara.default_max_wait_time = 5 # example • ݹ͍ࢿྉͩͱɺཁૉͷग़ݱΛ଴ͭͨΊʹಠࣗͷϝιουΛ
 ఆ͍ٛͯͨ͠Γ͢Δ • ex: https://artsy.github.io/blog/2012/02/03/reliably-testing-asynchronous-ui-w-slash-rspec-and-capybara/ • ͜ͷ࣌୅ͷϝιου͕࢒͍ͬͯΔͱͭΒ͍ 13
14. Ξχϝʔγϣϯ ΫϦοΫ ৳ͼΔ
15. ݹ͍ϫʔΫΞϥ΢ϯυ def wait_for_update(update_command, sec: 10, times: 1) times -= 1 previous_status = update_command.call yield if block_given? Timeout.timeout(sec) do loop do current_status = update_command.call break unless previous_status == current_status sleep(0.1) end end true rescue Timeout::Error times > 0 ? retry : false end ίϚϯυ(proc)Λ࣮ߦ(ʂʁ) ίϚϯυ(proc)Λ࣮ߦ ࣮ߦ݁Ռ͕ҧ͑͹୤ग़ 15
16. ݹ͍ϫʔΫΞϥ΢ϯυ # ͜Μͳײ͡ͷϝιου͕ݸ͙Β͍͋ͬͨ def wait_for_page_text_update(sec: 10, times: 1, &block) update_command = if capybara_support_evaluate_script? proc { page.evaluate_script('document.documentElement.innerText') } else proc { page.text } end wait_for_update(update_command, sec: sec, times: times, &block) end # def wait_for_accept_confirm_dialog # def wait_for_delivery_mail_subject # def wait_for_html_update # etc... ςΩετΛදࣔ ͖ͬ͞ͷϝιουʹ౉͢ 16
17. ݹ͍ϫʔΫΞϥ΢ϯυ # ࢖͍ํʜࢀߟ·Ͱʹ RSpec.feature '**** test' do scenario 'post ****' do login_as account, scope: :account visit foo_bar_path click_on 'Animation trigger element' wait_for_page_text_update do click_on 'Some awesome link' end end end 17
18. default waitͷ·ͱΊ • ςΩετΛ଴ͭͷ͸ expect ͰOK • expect(page).to have_link 'Some awesome link'
 click_on 'Some awesome link' • ςετ΋Ͱ͖ΔͷͰແବʹ͸ͳΒͳ͍ • wait_for଒͸ (͓ͦΒ͘) Poltergeist ࣌୅ͷԠٸાஔ • ༉அ͢ΔͱࣅͨΑ͏ͳϝιου͕ྔ࢈͞Εଓ͚Δ • ૉ௚ʹςετॻ͍ͯೝ஌ࢿݯͷઅ໿ 18
19. SystemSpec 19
20. Feature Spec ࠷େͷ՝୊ • Rails serverͱRSpec͕ผͷϓϩηεͰ࣮ߦ͞ΕΔ • DBͱͷίωΫγϣϯ͕ಠཱɺτϥϯβΫγϣϯ͕ڞ༗͞Εͳ͍ • ςετέʔε͝ͱʹDBΛॳظԽ͢Δ໨తͰ
 DatabaseCleaner (DatabaseRewinder) ͳͲ֎෦ͷGem͕ඞཁ • Headless Chromeͷઃఆ͕͠ΜͲ͍(ઌड़) • ଞʹ΋΢νͷνʔϜಛ༗ͷࣄ৘ׂ͕͕͋ͬͨѪ 20
21. SystemSpec • Rails5.1ͷ৽ػೳ: SystemTest • RailsެࣜͷE2EςετϑϨʔϜϫʔΫ • Minitest • SystemSpec: SystemTestΛRSpec͔Β࢖͏ • Կ͕ҧ͏? • ݁Ռతʹ͸ಉ͡ػೳΛఏڙ • ςετͱRails͕ಉ͡ϓϩηεͰ࣮ߦͰ͖Δ • Rails૊ΈࠐΈͷDBϩʔϧόοΫ͕࢖͑Δ
22. FEATURE SPEC MAP Rails RSpec MVC DB rspec-rails rspec-core Capybara server rspec node selector driver Driver ChromeDriver 22
23. SYSTEM SPEC MAP Rails RSpec MVC DB rspec-rails rspec-core Capybara server rspec node selector driver Driver ChromeDriver 23
24. ಋೖͷͨΊʹͨ͜͠ͱ • type: :feature Λ type: :system ʹஔ͖׵͑ • feature Λ describe ʹஔ͖׵͑
 require 'rails_helper'
 # RSpec.feature 'create ****', type: :feature do
 RSpec.describe 'create ****', type: :system do
 # ...
 end • feature ͩͱ (چ) feature spec ʹͳͬͯ͠·͏ • let/given, before/background ͸໰Θͳ͍ɻݱঢ়ࠞࡏ • DatabaseRewinder Λফ͢ • ChromeDriverͷઃఆΛγϯϓϧʹ͍ͨ͠(ະணख) • ϝϯόʔΛઆಘ (ॏཁ) 24
25. SystemSpecͷ·ͱΊ • DatabaseCleaner (DatabaseRewinder) ͸ Rails 5.1 Ҏ্ͳΒ
 ࢖Θͳ͍͍ͯ͘ • Feature Spec ͷΤΠϦΞε͸ࠓޙফ͍͔͑ͯ͘΋ • it/scenario, let/given, before/background • ͨͩ͠ feature ͚ͩ͸ҧ͏ • ౰໘྆ํ࢖͑ΔͷͰνʔϜͰ߹ҙऔ͓ͬͯ͜͏ • σʔλ͕ѻ͍΍͘͢ͳΓೝ஌ࢿݯͷઅ໿ 25
26. ิ଍: ೝ஌ࢿݯͱ͸ • ਓؒͷ೴͸ʮ൑அ͢Δʯͱ͍͏͜ͱࣗମʹճ਺੍ݶ͕͋Δ
 ͱ͍͏৺ཧ্ֶͷઆɻ໷ʹ৸Δͱճ෮͢Δ • ଞʹʮણࡉͳ࡞ۀʯʮ׳Ε͍ͯͳ͍࡞ۀʯͳͲͰ΋ݮগ • ೝ஌ࢿݯͷઅ໿͸ڧྗͳϥΠϑϋοΫ • Steve Jobs͸ͳ͍ͥͭ΋ࠇγϟπʹδʔύϯͩͬͨͷ͔ ೔ʑͨ͘͞Μͷਓͱձ͍ɺ༷ʑͳҙࢥܾఆΛߦ͏൴Β͸ɺීஈ͔Βେ͖ͳܾ அΛഭΒΕ͍ͯ·͢ɻͦͷͨΊձࣾͷܦӦ΍੓࣏ʹؔΘΔॏେͳܾஅΛ͢Δͱ ͖ʹ೴͕ർΕͳ͍Α͏ɺແବͳܾஅΛ͠ͳ͍Α͏ʹ͍ͯ͠Δͷͩͦ͏Ͱ͢ɻ ແବͳܾஅͱ͸͜͜Ͱ͸෰બͼͷ͜ͱΛࢦ͠·͢ɻ ग़య: தౡ૱ ʮͳͥɺ͋ͳͨͷ࢓ࣄ͸ऴΘΒͳ͍ͷ͔ɹεϐʔυ͸࠷ڧͷ෢ثͰ͋Δʯ 2016೥, จڹࣾ 26
27. ৹൑ • ౰νʔϜͰͷऔΓ૊Έ • ChromeDriverҠߦ: 2018೥10݄ • default wait (wait_for଒࡟আ): 2019೥6݄ • System SpecҠߦ: 2019೥7݄ 27
28. ৹൑ • ౷ܭ • GitLab APIΛ࢖͍௚ۙ8000ճ෼ͷCI݁ՌΛऔಘ • ूܭظؒ 2018/6/13ʙ2019/10/28 (݁Ռతʹ) • ͜ΕΛ400ճ͝ͱʹ෼ׂ • ͏ͪdevelop branchͰ࣮ߦ͞Εͨ݁ՌΛநग़(શମͷ1ʙ2ׂ) • success / (success + failed) ΛάϥϑԽ • ౰વ੒ޭ཰100%͕๬·͍͠ ݁Ռ΍͍͔ʹ…ʂʁ 28
29. ޮՌ: ੒ޭ཰ͷਪҠ • ײ֮తʹҰக • લ൒ͷ཰ͷ௿͞ • 2݄͔Β͠͹Β࣌ؒ͘΋ ΍ͨΒ͔͔ͬͨ • λΠϜΞ΢τ1h→2hʹ ͨ͠Γ… • ࢪࡦͷޮՌ • 2౓ͷVࣈճ෮ • ͑͑࿩΍ͱࢥͬͨͷʹͳΜͰ࠷ޙԼ͕ͬͱ ΜͶΜ 29      ΧςΰϦ࣠                     TVDDFTT ChromeDriverҠߦ wait_for... ࡟আ SystemSpecҠߦ
30. ࿩͞ͳ͔ͬͨࣄ • GitLab CIͱAWS CodeBuildʹ෼཭͍ͯͨ͠࿩ • ʮϚελʔσʔλʯͱDatabase Rewinder • ʮϚελʔσʔλʯΛฤू͢ΔγφϦΦ • Database Rewinderʹݕ஌͞Εͳ͍INSERT • rspec-retry • ࢖͏ͱΤϥʔʹͳΔFactoryBot • RSpec JUnit Formatter • Docker ImageͰςετ؀ڥߏங 30
31. ·ͱΊ • Rails E2Eςετ ྩ࿨ͷৗࣝ • ChromeDriver • default wait • SystemSpec • ೝ஌ࢿݯΛઅ໿ͯ͠շదͳRailsϥΠϑ 31
32. ग़య • titusfortner/webdrivers: Keep your Selenium WebDrivers updated automatically
 https://github.com/titusfortner/webdrivers • Phantom.js͕ϝϯςφϯεऴྃʹͳΔ݅Λௐ΂ͯΈͨ - kytikenͷϒϩά
 http://kytiken.hatenablog.com/entry/2017/11/29/175814 • capybara/README.md at master · teamcapybara/capybara
 https://github.com/teamcapybara/capybara/blob/master/README.md • RSpec 3.7 ͕ϦϦʔε͞Ε·ͨ͠ʂ
 http://rspec.info/ja/blog/2017/10/rspec-3-7-has-been-released/ • ͳͥɺ͋ͳͨͷ࢓ࣄ͸ऴΘΒͳ͍ͷ͔ εϐʔυ͸࠷େͷ෢ثͰ͋Δ
 https://www.amazon.co.jp/dp/4905073413/ref=cm_sw_r_tw_dp_U_x_wcRQDb4B1PQDT • API Docs | GitLab
 https://docs.gitlab.com/ce/api/README.html • NARKOZ/gitlab: Ruby wrapper and CLI for the GitLab REST API
 https://github.com/NARKOZ/gitlab 32