安定したFeature Specを助ける3つの処方箋

この発表では、弊社プロダクトにおいてテスト環境を改善するために実施した施策をfeature specに着目して紹介します。我々のチームはこの1年でFactoryデータに構文チェックを導入し、ドライバを置き換え、System specを用いることによりさらにデータの取り扱いを改善しました。これらによる余計なコードの削減、成功率の改善といった効果も発表します。

Transcript

1. ҆ఆͨ͠

   'FBUVSF4QFD
   Λॿ͚Δ
   ͭͷॲํᝦ
   ϦϯΧʔζגࣜձࣾ
   ླ໦ ཽଠ
   1
   

   View Slide

2. ࣗݾ঺հ
   • 1989೥9݄22೔ ࣛࣇౡ࢈·Ε ౦ژɾࡳຈҭͪ
   • ϦϯΧʔζגࣜձࣾ Linkers for BankνʔϜ
   • Railsྺ3೥ (2016೥10݄ʙ)
   • FࣾܥSIerग़਎ (͜ͷ࿩࢝·Δͱ௕͍)
   • લ࡞(7/17 Otemachi.rb)→

   https://speakerdeck.com/strviola/que-ren-hua-mian-besutopurakuteisu
   2
   

   View Slide

3. Feature Spec
   3
   

   View Slide

4. "ॲํᝦ"
   •ChromeDriver
   •default wait (Ծ)
   •System Spec
   4
   

   View Slide

5. ChromeDriver
   5
   

   View Slide

6. Feature Spec (Capybara) ͷߏ੒ཁૉ
   • server: RailsΞϓϦέʔγϣϯͱ઀ଓ
   • rspec: MatcherΦϒδΣΫτɾϝιου(have_title, ...)Λఏڙ
   • selector: HTMLཁૉΛऔಘ
   • node: HTMLཁૉΛѻ͏
   • driver: ϒϥ΢βͷڍಈΛ࠶ݱ (ಛʹJavaScript)
   • Poltergeist.js
   • ChromeDriver
   6
   

   View Slide

7. FEATURE SPEC MAP
   Rails
   RSpec
   MVC DB
   rspec-rails rspec-core
   Capybara
   server rspec
   node
   selector
   driver
   Driver
   Poltergeist.js
   7
   

   View Slide

8. Poltergeist αϙʔτͷྺ࢙
   • ͔ͭͯ།Ұͷબ୒ࢶɻฐࣾͰ΋࢖͍ͬͯͨ
   • 2017೥ɺ Headless Chrome (beta) ొ৔
   • 2018೥5݄ࠒ Stable એݴ
   • ͜ΕΛػʹ Phantom.js ͷϝϯςφʔ͕ҾୀΛએݴ
   • "I don't see any future in developing PhantomJS.

   Developing PhantomJS 2 and 2.5 as a single developer is a bloody hell."
   • Phantom.jsʹڧ͘ґଘ͍ͯͨ͠Poltergeist΋αϙʔτऴྃ
   • ChromeDriver͕ೖΕସΘΔΑ͏ʹීٴ
   • PoltergeistΑΓ҆ఆɺ͍ܰ
   8
   

   View Slide

9. ChromeDriver Λ࢖͏ͨΊʹ
   • gem 'webdrivers'
   • chromedriver-helper ʹͳ͍ͬͯΔࢿྉ͸ݹ͍ͷͰ஫ҙ
   • spec/rails_helper.rb ʹىಈઃఆΛॻ͘
   • ΍΍هड़ଟ͍
   • ࠷ۙͦ͜·Ͱ͠ͳ͍͍ͯ͘Β͍͠ (ະணख)
   • CIͰճ͢৔߹ChromeͷΠϯετʔϧ͕ඞཁ
   9
   

   View Slide

10. ChromeDriver ઃఆ෦෼
    Capybara.register_driver :selenium_chrome_headless do |app|
    options = %w[headless disable-gpu window-size=1280,1200 lang=ja no-sandbox disable-dev-shm-usage
    disable-setuid-sandbox no-cache disable-infobars
    --enable-features=NetworkService,NetworkServiceInProcess]
    driver = Capybara::Selenium::Driver.new(
    app,
    browser: :chrome,
    options: Selenium::WebDriver::Chrome::Options.new(args: options),
    desired_capabilities: Selenium::WebDriver::Remote::Capabilities.chrome(
    login_prefs: { browser: 'ALL' },
    chrome_options: { args: options }))
    bridge = driver.browser.send(:bridge)
    path = "session/#{bridge.session_id}/chromium/send_command"
    bridge.http.call(
    :post, path,
    cmd: 'Page.setDownloadBehavior',
    params: {
    behavior: 'allow',
    downloadPath: DownloadHelper::PATH.to_s
    })
    driver
    end
    ىಈΦϓγϣϯ
    ϒϥ΢βઃఆ
    HTTP઀ଓઃఆ
    10
    

    View Slide

11. ChromeDriverͷ·ͱΊ
    • 2017೥Ҏલ͔ΒFeature SpecΛӡ༻͍ͯ͠Δ৔߹͸

    ੾Γସ͕͑ඞཁ͔΋
    • Poltergeist͸Ή͠Ζ࢖͏ͱةͳ͍
    • SystemSpec(ޙड़)ͱͷ݉Ͷ߹͍΋͋ΓChromeDriver͸࣮࣭ެࣜ
    11
    

    View Slide

12. default wait (Ծ)
    12
    

    View Slide

13. Ͳ͜ͷ͜ͱͩ
    • Capybara͸σϑΥϧτͰཁૉͷग़ݱΛ଴ͭ
    • Capybara.default_max_wait_time = 5 # example
    • ݹ͍ࢿྉͩͱɺཁૉͷग़ݱΛ଴ͭͨΊʹಠࣗͷϝιουΛ

    ఆ͍ٛͯͨ͠Γ͢Δ
    • ex: https://artsy.github.io/blog/2012/02/03/reliably-testing-asynchronous-ui-w-slash-rspec-and-capybara/
    • ͜ͷ࣌୅ͷϝιου͕࢒͍ͬͯΔͱͭΒ͍
    13
    

    View Slide

14. Ξχϝʔγϣϯ
    ΫϦοΫ
    ৳ͼΔ
    

    View Slide

15. ݹ͍ϫʔΫΞϥ΢ϯυ
    def wait_for_update(update_command, sec: 10, times: 1)
    times -= 1
    previous_status = update_command.call
    yield if block_given?
    Timeout.timeout(sec) do
    loop do
    current_status = update_command.call
    break unless previous_status == current_status
    sleep(0.1)
    end
    end
    true
    rescue Timeout::Error
    times > 0 ? retry : false
    end
    ίϚϯυ(proc)Λ࣮ߦ(ʂʁ)
    ίϚϯυ(proc)Λ࣮ߦ
    ࣮ߦ݁Ռ͕ҧ͑͹୤ग़
    15
    

    View Slide

16. ݹ͍ϫʔΫΞϥ΢ϯυ
    # ͜Μͳײ͡ͷϝιου͕ݸ͙Β͍͋ͬͨ
    def wait_for_page_text_update(sec: 10, times: 1, &block)
    update_command =
    if capybara_support_evaluate_script?
    proc { page.evaluate_script('document.documentElement.innerText') }
    else
    proc { page.text }
    end
    wait_for_update(update_command, sec: sec, times: times, &block)
    end
    # def wait_for_accept_confirm_dialog
    # def wait_for_delivery_mail_subject
    # def wait_for_html_update
    # etc...
    ςΩετΛදࣔ
    ͖ͬ͞ͷϝιουʹ౉͢
    16
    

    View Slide

17. ݹ͍ϫʔΫΞϥ΢ϯυ
    # ࢖͍ํʜࢀߟ·Ͱʹ
    RSpec.feature '**** test' do
    scenario 'post ****' do
    login_as account, scope: :account
    visit foo_bar_path
    click_on 'Animation trigger element'
    wait_for_page_text_update do
    click_on 'Some awesome link'
    end
    end
    end
    17
    

    View Slide

18. default waitͷ·ͱΊ
    • ςΩετΛ଴ͭͷ͸ expect ͰOK
    • expect(page).to have_link 'Some awesome link'

    click_on 'Some awesome link'
    • ςετ΋Ͱ͖ΔͷͰແବʹ͸ͳΒͳ͍
    • wait_for଒͸ (͓ͦΒ͘) Poltergeist ࣌୅ͷԠٸાஔ
    • ༉அ͢ΔͱࣅͨΑ͏ͳϝιου͕ྔ࢈͞Εଓ͚Δ
    • ૉ௚ʹςετॻ͍ͯೝ஌ࢿݯͷઅ໿
    18
    

    View Slide

19. SystemSpec
    19
    

    View Slide

20. Feature Spec ࠷େͷ՝୊
    • Rails serverͱRSpec͕ผͷϓϩηεͰ࣮ߦ͞ΕΔ
    • DBͱͷίωΫγϣϯ͕ಠཱɺτϥϯβΫγϣϯ͕ڞ༗͞Εͳ͍
    • ςετέʔε͝ͱʹDBΛॳظԽ͢Δ໨తͰ

    DatabaseCleaner (DatabaseRewinder) ͳͲ֎෦ͷGem͕ඞཁ
    • Headless Chromeͷઃఆ͕͠ΜͲ͍(ઌड़)
    • ଞʹ΋΢νͷνʔϜಛ༗ͷࣄ৘ׂ͕͕͋ͬͨѪ
    20
    

    View Slide

21. SystemSpec
    • Rails5.1ͷ৽ػೳ: SystemTest
    • RailsެࣜͷE2EςετϑϨʔϜϫʔΫ
    • Minitest
    • SystemSpec: SystemTestΛRSpec͔Β࢖͏
    • Կ͕ҧ͏?
    • ݁Ռతʹ͸ಉ͡ػೳΛఏڙ
    • ςετͱRails͕ಉ͡ϓϩηεͰ࣮ߦͰ͖Δ
    • Rails૊ΈࠐΈͷDBϩʔϧόοΫ͕࢖͑Δ
    

    View Slide

22. FEATURE SPEC MAP
    Rails
    RSpec
    MVC DB
    rspec-rails rspec-core
    Capybara
    server rspec
    node
    selector
    driver
    Driver
    ChromeDriver
    22
    

    View Slide

23. SYSTEM SPEC MAP
    Rails
    RSpec
    MVC DB
    rspec-rails rspec-core
    Capybara
    server
    rspec
    node
    selector
    driver
    Driver
    ChromeDriver
    23
    

    View Slide

24. ಋೖͷͨΊʹͨ͜͠ͱ
    • type: :feature Λ type: :system ʹஔ͖׵͑
    • feature Λ describe ʹஔ͖׵͑

    require 'rails_helper'

    # RSpec.feature 'create ****', type: :feature do

    RSpec.describe 'create ****', type: :system do

    # ...

    end
    • feature ͩͱ (چ) feature spec ʹͳͬͯ͠·͏
    • let/given, before/background ͸໰Θͳ͍ɻݱঢ়ࠞࡏ
    • DatabaseRewinder Λফ͢
    • ChromeDriverͷઃఆΛγϯϓϧʹ͍ͨ͠(ະணख)
    • ϝϯόʔΛઆಘ (ॏཁ)
    24
    

    View Slide

25. SystemSpecͷ·ͱΊ
    • DatabaseCleaner (DatabaseRewinder) ͸ Rails 5.1 Ҏ্ͳΒ

    ࢖Θͳ͍͍ͯ͘
    • Feature Spec ͷΤΠϦΞε͸ࠓޙফ͍͔͑ͯ͘΋
    • it/scenario, let/given, before/background
    • ͨͩ͠ feature ͚ͩ͸ҧ͏
    • ౰໘྆ํ࢖͑ΔͷͰνʔϜͰ߹ҙऔ͓ͬͯ͜͏
    • σʔλ͕ѻ͍΍͘͢ͳΓೝ஌ࢿݯͷઅ໿
    25
    

    View Slide

26. ิ଍: ೝ஌ࢿݯͱ͸
    • ਓؒͷ೴͸ʮ൑அ͢Δʯͱ͍͏͜ͱࣗମʹճ਺੍ݶ͕͋Δ

    ͱ͍͏৺ཧ্ֶͷઆɻ໷ʹ৸Δͱճ෮͢Δ
    • ଞʹʮણࡉͳ࡞ۀʯʮ׳Ε͍ͯͳ͍࡞ۀʯͳͲͰ΋ݮগ
    • ೝ஌ࢿݯͷઅ໿͸ڧྗͳϥΠϑϋοΫ
    • Steve Jobs͸ͳ͍ͥͭ΋ࠇγϟπʹδʔύϯͩͬͨͷ͔
    ೔ʑͨ͘͞Μͷਓͱձ͍ɺ༷ʑͳҙࢥܾఆΛߦ͏൴Β͸ɺීஈ͔Βେ͖ͳܾ
    அΛഭΒΕ͍ͯ·͢ɻͦͷͨΊձࣾͷܦӦ΍੓࣏ʹؔΘΔॏେͳܾஅΛ͢Δͱ
    ͖ʹ೴͕ർΕͳ͍Α͏ɺແବͳܾஅΛ͠ͳ͍Α͏ʹ͍ͯ͠Δͷͩͦ͏Ͱ͢ɻ
    ແବͳܾஅͱ͸͜͜Ͱ͸෰બͼͷ͜ͱΛࢦ͠·͢ɻ
    ग़య: தౡ૱
    ʮͳͥɺ͋ͳͨͷ࢓ࣄ͸ऴΘΒͳ͍ͷ͔ɹεϐʔυ͸࠷ڧͷ෢ثͰ͋Δʯ
    2016೥, จڹࣾ
    26
    

    View Slide

27. ৹൑
    • ౰νʔϜͰͷऔΓ૊Έ
    • ChromeDriverҠߦ: 2018೥10݄
    • default wait (wait_for଒࡟আ): 2019೥6݄
    • System SpecҠߦ: 2019೥7݄
    27
    

    View Slide

28. ৹൑
    • ౷ܭ
    • GitLab APIΛ࢖͍௚ۙ8000ճ෼ͷCI݁ՌΛऔಘ
    • ूܭظؒ 2018/6/13ʙ2019/10/28 (݁Ռతʹ)
    • ͜ΕΛ400ճ͝ͱʹ෼ׂ
    • ͏ͪdevelop branchͰ࣮ߦ͞Εͨ݁ՌΛநग़(શମͷ1ʙ2ׂ)
    • success / (success + failed) ΛάϥϑԽ
    • ౰વ੒ޭ཰100%͕๬·͍͠
    ݁Ռ΍͍͔ʹ…ʂʁ
    28
    

    View Slide

29. ޮՌ: ੒ޭ཰ͷਪҠ
    • ײ֮తʹҰக
    • લ൒ͷ཰ͷ௿͞
    • 2݄͔Β͠͹Β࣌ؒ͘΋
    ΍ͨΒ͔͔ͬͨ
    • λΠϜΞ΢τ1h→2hʹ
    ͨ͠Γ…
    • ࢪࡦͷޮՌ
    • 2౓ͷVࣈճ෮
    • ͑͑࿩΍ͱࢥͬͨͷʹͳΜͰ࠷ޙԼ͕ͬͱ
    ΜͶΜ
    29
    
    
    
    
    
    ΧςΰϦ࣠
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    TVDDFTT
    ChromeDriverҠߦ
    wait_for... ࡟আ
    SystemSpecҠߦ
    

    View Slide

30. ࿩͞ͳ͔ͬͨࣄ
    • GitLab CIͱAWS CodeBuildʹ෼཭͍ͯͨ͠࿩
    • ʮϚελʔσʔλʯͱDatabase Rewinder
    • ʮϚελʔσʔλʯΛฤू͢ΔγφϦΦ
    • Database Rewinderʹݕ஌͞Εͳ͍INSERT
    • rspec-retry
    • ࢖͏ͱΤϥʔʹͳΔFactoryBot
    • RSpec JUnit Formatter
    • Docker ImageͰςετ؀ڥߏங
    30
    

    View Slide

31. ·ͱΊ
    • Rails E2Eςετ ྩ࿨ͷৗࣝ
    • ChromeDriver
    • default wait
    • SystemSpec
    • ೝ஌ࢿݯΛઅ໿ͯ͠շదͳRailsϥΠϑ
    31
    

    View Slide

32. ग़య
    • titusfortner/webdrivers: Keep your Selenium WebDrivers updated automatically

    https://github.com/titusfortner/webdrivers
    • Phantom.js͕ϝϯςφϯεऴྃʹͳΔ݅Λௐ΂ͯΈͨ - kytikenͷϒϩά

    http://kytiken.hatenablog.com/entry/2017/11/29/175814
    • capybara/README.md at master · teamcapybara/capybara

    https://github.com/teamcapybara/capybara/blob/master/README.md
    • RSpec 3.7 ͕ϦϦʔε͞Ε·ͨ͠ʂ

    http://rspec.info/ja/blog/2017/10/rspec-3-7-has-been-released/
    • ͳͥɺ͋ͳͨͷ࢓ࣄ͸ऴΘΒͳ͍ͷ͔ εϐʔυ͸࠷େͷ෢ثͰ͋Δ

    https://www.amazon.co.jp/dp/4905073413/ref=cm_sw_r_tw_dp_U_x_wcRQDb4B1PQDT
    • API Docs | GitLab

    https://docs.gitlab.com/ce/api/README.html
    • NARKOZ/gitlab: Ruby wrapper and CLI for the GitLab REST API

    https://github.com/NARKOZ/gitlab
    32
    

    View Slide