Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Magento Test Automation - My Journey

Avatar for Fabian Schmengler Fabian Schmengler
May 23, 2017
Technology
0
310

Magento Test Automation - My Journey

Meet Magento Germany 2017

Slides as HTML: https://www.schmengler-se.de/slides/magento-test-automation-my-journey/
Avatar for Fabian Schmengler

Fabian Schmengler

May 23, 2017
Tweet

More Decks by Fabian Schmengler

See All by Fabian Schmengler
Transforming a PHP Team into PWA Superheroes
Avatar for Fabian Schmengler schmengler
0
160
9 Common Misconceptions about TDD
Avatar for Fabian Schmengler schmengler
1
410
Dealing with Testing Fatigue #mm18pl
Avatar for Fabian Schmengler schmengler
0
380
Dealing with Testing Fatigue
Avatar for Fabian Schmengler schmengler
0
770
Porting a complex extension to Magento 2 (Mage Titans)
Avatar for Fabian Schmengler schmengler
1
580
Porting a complex extension to Magento 2
Avatar for Fabian Schmengler schmengler
2
1.5k
Porting a complex extension to Magento 2
Avatar for Fabian Schmengler schmengler
0
150
[DE] Usability für Programmierer
Avatar for Fabian Schmengler schmengler
0
640
[DE] integer_net Projektstrukturen
Avatar for Fabian Schmengler schmengler
0
140

Other Decks in Technology

See All in Technology
Google Cloud Next 2025 Recap マーケティング施策の運用及び開発を支援するAIの活用 / Use of AI to support operation and development of marketing campaign
Avatar for Atsushi Yoshikawa atsushiyoshikawa
0
220
Gateway H2 モジュールで
スマートホーム入門
Avatar for MinoruInachi minoruinachi
0
140
Part1 GitHubってなんだろう？その２
Avatar for tomokusaba tomokusaba
2
760
newmo の創業を支える Software Architecture と Platform Engineering
Avatar for Yuki Ito 110y
5
530
Software Architecture in an AI-Driven World
Avatar for AGAWA Koji atty303
23
11k
250510 StepFunctionのテスト自動化始めました vol.1
Avatar for Takumi Abe east_takumi
1
230
Ninno LT
Avatar for Yasunobu Kawaguchi kawaguti
PRO
1
120
Why Platform Engineering? - マルチプロダクト・少人数 SRE の壁を越える挑戦 -
Avatar for 株式会社ヌーラボ nulabinc
PRO
5
440
UIパフォーマンス最適化: AIを活用して100倍の速度向上を実現した事例
Avatar for kinocoboy kinocoboy2
0
260
dbtとリバースETLでデータ連携の複雑さに立ち向かう
Avatar for Toru Morooka morookacube
0
840
LLM アプリケーションのためのクラウドセキュリティ - CSPM の実装ポイント-
Avatar for KINTOテクノロジーズ Osaka Tech Lab osakatechlab
0
420
クラウドネイティブ環境の脅威モデリング
Avatar for Kyohei Mizumoto kyohmizu
2
410

Featured

See All Featured
Helping Users Find Their Own Way: Creating Modern Search Experiences
Avatar for Dan Newman danielanewman
29
2.6k
Rails Girls Zürich Keynote
Avatar for Gregor Martynus gr2m
94
13k
Faster Mobile Websites
Avatar for Dean Hume deanohume
307
31k
Become a Pro
Avatar for Speaker Deck speakerdeck
PRO
28
5.3k
Building Adaptive Systems
Avatar for Chris Keathley keathley
41
2.5k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
Avatar for Eileen M. Uchitelle eileencodes
137
33k
Statistics for Hackers
Avatar for Jake VanderPlas jakevdp
799
220k
Testing 201, or: Great Expectations
Avatar for Joseph Mastey jmmastey
42
7.5k
The Illustrated Children's Guide to Kubernetes
Avatar for Chris Short chrisshort
48
49k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
Avatar for Tammy Everts tammyeverts
52
2.5k
Fashionably flexible responsive web design (full day workshop)
Avatar for Andy Clarke malarkey
407
66k
Docker and Python
Avatar for Tania Allard trallard
44
3.4k

Transcript

  1. None

  2. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 . Magento Test Automation: My Journey @fschmengler

  3. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 . Magento Test Automation: My Journey @fschmengler

  4. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 .. Magento Test Automation: My Journey @fschmengler

  5. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ... Magento Test Automation: My Journey @fschmengler

  6. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 .... SimpleTest? PHPUnit? Ibuildings MageTest? EcomDev PHPUnit? Magento Test Automation: My Journey @fschmengler

  7. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ..... Magento Test Automation: My Journey @fschmengler

  8. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ...... foo_bar? Foo_Bar? fooBar? foobar? bar? foo/bar? Magento Test Automation: My Journey @fschmengler

  9. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ....... TEST ALL THE THINGS! public function testObserver() { $this->assertEventObserverDefined( 'adminhtml', 'core_block_abstract_prepare_layout_before', SGH_OrderPrintPopups_Model_Observer_Block::MODEL, 'onCoreBlockAbstractPrepareLayoutBefore', 'orderprintpopups' ); } Magento Test Automation: My Journey @fschmengler

  10. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ........ TEST ALL THE THINGS! /** * @param Mage_Core_Block_Abstract $block * @return Varien_Event_Observer */ protected function _mockEventObserver($block, $otherArgs = array()) { $args = array_merge(array('block' => $block), $otherArgs); $observer = new Varien_Event_Observer(); $observer->setEvent(new Varien_Event($args)); return $observer; } Magento Test Automation: My Journey @fschmengler

  11. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ......... TEST ALL THE THINGS! $this->assertLayoutHandleLoaded( 'orderprintpopups_warensendung_index', 'module layout handle loaded' ); $this->assertLayoutBlockRendered( SGH_OrderPrintPopups_Block_Popup_Warensendung::ALIAS, 'popup block rendered' ); Magento Test Automation: My Journey @fschmengler

  12. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 .......... TEST ALL THE THINGS! $route = 'orderprintpopups/warensendung/barcodestatic'; $this->assertResponseBodyQueryRegex( '#sgh_warensendung', '#<img [^>]*src="[^"]+' . $route . '#' ); Magento Test Automation: My Journey @fschmengler

  13. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 .......... TEST ALL THE THINGS! /** * getter tests */ test("getter", function() { equal(this.block.getName(), "foo", "getName()"); equal(this.block.getContent(), "lorem ipsum falleri fallera", "getContent()"); equal(this.block.getPlaceholder(), "{foo}", "getPlaceholder()"); }); Magento Test Automation: My Journey @fschmengler

  14. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............ What lies behind the code Hours of debugging Hidden depenencies to global state Bugs of the testing frameworks Magento Test Automation: My Journey @fschmengler

  15. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............S A story of skipped tests $this->markTestSkipped('fatal error???'); Magento Test Automation: My Journey @fschmengler

  16. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............S A story of skipped tests $this->markTestSkipped('fatal error???'); $this->markTestSkipped('conflict with fixture and existing db'); Magento Test Automation: My Journey @fschmengler

  17. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............S A story of skipped tests $this->markTestSkipped('fatal error???'); $this->markTestSkipped('conflict with fixture and existing db'); $this->markTestSkipped('"out of stock", although fixture seems ok'); //FIXME Magento Test Automation: My Journey @fschmengler

  18. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............S A story of skipped tests $this->markTestSkipped('fatal error???'); $this->markTestSkipped('conflict with fixture and existing db'); $this->markTestSkipped('"out of stock", although fixture seems ok'); //FIXME $this->markTestSkipped( 'The new comparator classes in PHPUnit 3.6 cannot handle recursive comparision with $maxDepth parameter YET.' ); Magento Test Automation: My Journey @fschmengler

  19. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............S A story of skipped tests $this->markTestSkipped('fatal error???'); $this->markTestSkipped('conflict with fixture and existing db'); $this->markTestSkipped('"out of stock", although fixture seems ok'); //FIXME $this->markTestSkipped( 'The new comparator classes in PHPUnit 3.6 cannot handle recursive comparision with $maxDepth parameter YET.' ); $this->markTestSkipped( 'expectation + fixture changed. reason for missing prefix unknown '. 'but only occurs in test environment!' ); Magento Test Automation: My Journey @fschmengler

  20. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SS A story of skipped tests $this->markTestSkipped( 'Das klappt schon wieder alles nicht mit Adminlogin+Dispatch' ); Magento Test Automation: My Journey @fschmengler

  21. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SS A story of skipped tests $this->markTestSkipped( 'Das klappt schon wieder alles nicht mit Adminlogin+Dispatch' ); $this->markTestSkipped('Untestbar in der Form. Mglw. ein Bug in EcomDev_PHPUnit der zu leerem orig_data führt. Leider funktioniert @runInSeparateProcess nicht mit Magento.') Magento Test Automation: My Journey @fschmengler

  22. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSS Magento Test Automation: My Journey @fschmengler

  23. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSF Magento 1.6 Update Magento Test Automation: My Journey @fschmengler

  24. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF Magento Test Automation: My Journey @fschmengler

  25. 06/2011 12/2011 06/2012 12/2012 06/2013 12/2013 06/2014 0 50 100

    150 200 250 300 350 400 450 Product Description Blocks one of the first modules LLOC LLOC (Tests) 2011 > 2012 > 2013 > 2014 > 2015 > 2016 > 2017 ............SSSFF. Magento Test Automation: My Journey @fschmengler

  26. 01/2012 07/2012 12/2012 0 50 100 150 200 250 300

    350 Payment one year into the project... LLOC LLOC (Tests) 2011 > 2012 > 2013 > 2014 > 2015 > 2016 > 2017 ............SSSFF...S Magento Test Automation: My Journey @fschmengler

  27. 01/2012 07/2012 12/2012 0 500 1000 1500 2000 2500 3000

    3500 Payment Symfony Service LLOC LLOC (Tests) 2011 > 2012 > 2013 > 2014 > 2015 > 2016 > 2017 ............SSSFF...S. Magento Test Automation: My Journey @fschmengler

  28. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.. Diploma Thesis on Magento Development Magento Test Automation: My Journey @fschmengler

  29. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S... Research & Experiments Built-in tests Magento TAF for Smoke Test Suite Abstraction of automated acceptance tests + ZiBreve Magento Test Automation: My Journey @fschmengler

  30. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.... Selenium IDE > FitNesse Magento Test Automation: My Journey @fschmengler

  31. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S..... Agency project with Selenium Automated Acceptance Tests Selenium IDE Tests for every feature Partially written by QA Magento Test Automation: My Journey @fschmengler

  32. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S Magento Test Automation: My Journey @fschmengler

  33. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S. Agency project with EcomDev_PHPUnit Automated Unit/Integration T ests Tests mandatory, coverage monitored Sophisticated setup Still typical problems Magento Test Automation: My Journey @fschmengler

  34. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.F DEADLINE IS COMING Magento Test Automation: My Journey @fschmengler

  35. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF M1 Integration Tests: Typical Problems Tests altering global state (singletons, registry) Magento Test Automation: My Journey @fschmengler

  36. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF M1 Integration Tests: Typical Problems Tests altering global state (singletons, registry) Tests depending on existing data, others delete it and use their own Magento Test Automation: My Journey @fschmengler

  37. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF M1 Integration Tests: Typical Problems Tests altering global state (singletons, registry) Tests depending on existing data, others delete it and use their own Tests not well isolated, not running in DB transaction Magento Test Automation: My Journey @fschmengler

  38. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF M1 Integration Tests: Typical Problems Tests altering global state (singletons, registry) Tests depending on existing data, others delete it and use their own Tests not well isolated, not running in DB transaction Complicated fixture setup Magento Test Automation: My Journey @fschmengler

  39. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF. THIS WEIRD UNIT TESTING TRICK BLEW MY MIND! Decouple business logic from Magento Use plain PHPUnit Magento Test Automation: My Journey @fschmengler

  40. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF.. Magento Test Automation: My Journey @fschmengler

  41. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF... Bad news we still need higher level tests Magento Test Automation: My Journey @fschmengler

  42. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF... Bad news we still need higher level tests Good news less of them Magento Test Automation: My Journey @fschmengler

  43. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF.... Magento Test Automation: My Journey @fschmengler

  44. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF..... Experiments Acceptance and integration test framework Magento Test Automation: My Journey @fschmengler

  45. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF..... Experiments Acceptance and integration test framework Acceptance test framework "THE SELENIUM-BASED TESTING FRAMEWORK FOR PEOPLE WHO HATE TESTING" Magento Test Automation: My Journey @fschmengler

  46. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF..... Experiments Acceptance and integration test framework Acceptance test framework "THE SELENIUM-BASED TESTING FRAMEWORK FOR PEOPLE WHO HATE TESTING" Test framework Codeception Magento Test Automation: My Journey @fschmengler

  47. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF...... A new era: Magento 2 Magento Test Automation: My Journey @fschmengler

  48. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF....... Magento 2 Extension Reused refactored lib from Magento 1 extension Integration tests for Magento 2 specific code Magento Test Automation: My Journey @fschmengler

  49. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........ Magento 2 Projects Great integration test framework Magento Test Automation: My Journey @fschmengler

  50. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........ Magento 2 Projects Great integration test framework Easier to start with Magento Test Automation: My Journey @fschmengler

  51. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........ Magento 2 Projects Great integration test framework Easier to start with More reliable Magento Test Automation: My Journey @fschmengler

  52. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S Magento 2 Projects Still frustrating sometimes Magento Test Automation: My Journey @fschmengler

  53. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S Magento 2 Projects Still frustrating sometimes Failures after minor version update Magento Test Automation: My Journey @fschmengler

  54. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S Magento 2 Projects Still frustrating sometimes Failures after minor version update Test runs not completely isolated Magento Test Automation: My Journey @fschmengler

  55. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S Magento 2 Projects Still frustrating sometimes Failures after minor version update Test runs not completely isolated Decoupled approach still works best Magento Test Automation: My Journey @fschmengler

  56. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S. TDD Katas: Back to the roots Regular practice Small tasks, no real project Goal: Make TDD my default habit https://www.schmengler-se.de/katas Magento Test Automation: My Journey @fschmengler

  57. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S.. How I would start today / Recommendations 1. For a quick win, create smoke test suite with Codeception Magento Test Automation: My Journey @fschmengler

  58. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S.. How I would start today / Recommendations 1. For a quick win, create smoke test suite with Codeception 2. Katas / small self contained projects to learn TDD Magento Test Automation: My Journey @fschmengler

  59. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S.. How I would start today / Recommendations 1. For a quick win, create smoke test suite with Codeception 2. Katas / small self contained projects to learn TDD 3. Start to build (small) parts of your code independent from Magento - "TDD" those parts Magento Test Automation: My Journey @fschmengler

  60. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S.. How I would start today / Recommendations 1. For a quick win, create smoke test suite with Codeception 2. Katas / small self contained projects to learn TDD 3. Start to build (small) parts of your code independent from Magento - "TDD" those parts 4. Try to add simple integration tests (in M1 with Xtest) but skip if it's getting complicated Magento Test Automation: My Journey @fschmengler

  61. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S... Mistakes I won't make again Try to sell tests as separate part of project quote or estimation Magento Test Automation: My Journey @fschmengler

  62. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S... Mistakes I won't make again Try to sell tests as separate part of project quote or estimation Try to teach "unit testing" and "testing Magento" at once Magento Test Automation: My Journey @fschmengler

  63. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S... Time: 40 minutes OK (42 slides, 0 cat pictures) Magento Test Automation: My Journey @fschmengler

  64. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S... Time: 40 minutes OK (42 slides, 0 cat pictures) Thank you! Magento Test Automation: My Journey @fschmengler

  65. 2011 > 2012 > 2013 > 2014 > 2015 >

    2016 > 2017 ............SSSFF...S.....S.FF........S... Time: 40 minutes OK (42 slides, 0 cat pictures) Thank you! Questions? Magento Test Automation: My Journey @fschmengler

  66. Magento Test Automation: My Journey @fschmengler

  67. Announcement: MageTestFest NOVEMBER 17TH 2017 MAGENTO. SOFTWARE TESTING. PARTY. https://magetestfest.nl/

    by Yireo. Magento Test Automation: My Journey @fschmengler

  68. Links http://xtest-mage.com/ https://magiumlib.com/ http://codeception.com/ Image sources Marvin: The Hitchhiker's Guide

    to the Galaxy (2005 movie) Angry German: https://www.youtube.com/watch?v=-_xUIDRxdmc Black metal cat: https://twitter.com/evilbmcats Agile testing pyramid: http://www.agilecoachjournal.com Magento Test Automation: My Journey @fschmengler