Chefs Plate
● We're hiring!
○ Front-End Developer (x2)
○ Data Engineer
○ Product Owner - Mobile
○ IT Manager
○ IT Administrator
○ Plus many more! Check out
https://www.chefsplate.com/careers
Slide 3
Slide 3 text
EFFECTIVE
TESTING FOR
PHP
APPLICATIONS
Presented by David Chang
Twitter: @davidchchang
Slide 4
Slide 4 text
“
Software and cathedrals are much the
same: first we build them, then we pray.
- Anonymous
Slide 5
Slide 5 text
Not exactly the best way to test code
Slide 6
Slide 6 text
“
Weeks of programming can save you
hours of planning (and writing tests).
- Anonymous
ON THE MENU
Agenda:
◦ Why is Testing Important? ✔
◦ Types of Testing
◦ PHP Testing Tools
◦ Testing guidelines
◦ Integrating with CI
◦ CI Action Plan
Slide 11
Slide 11 text
TYPES OF TESTING
What are they are why are they useful?
1
Slide 12
Slide 12 text
IT'S EASY TO GET LOST
Unit Testing
Integration Testing
Systems Testing
Acceptance Testing
Functional Testing
Usability Testing
Stress/Load Testing
Security Testing
Smoke Testing
Regression Testing
Accessibility Testing
Automated Testing
White-box Testing
Black-box Testing
API Testing
Static Testing
Dynamic Testing
Code Coverage
Non-Functional Testing
Model-Based Testing
Exploratory Testing
Ad Hoc Testing Visual Testing
Grey-box Testing
GUI Testing
Sanity Testing
Slide 13
Slide 13 text
MAIN LEVELS OF TESTING
UNIT TESTING
INTEGRATION TESTING
SYSTEMS TESTING
(USER) ACCEPTANCE
TESTING
Slide 14
Slide 14 text
◦ Tests the smallest testable part of
the code, usually individual
functions
◦ Usually tested in isolation, with the
help of method stubs, mocks and
test harnesses
◦ Objective: isolate a unit of code
and validate its correctness.
UNIT TESTING
Slide 15
Slide 15 text
EXAMPLE OF A UNIT TEST
Slide 16
Slide 16 text
EXAMPLE OF A UNIT TEST
Slide 17
Slide 17 text
UNIT TESTING: PROS AND CONS
Cons:
◦ Setting up realistic or
practical test cases
not always easy
◦ Like documentation,
difficult to constantly
maintain for accuracy
Pros:
◦ Early detection of
problems
◦ Allows developers to
think through
various conditions,
providing more
structure and
improving code
design
◦ Great help during
refactors
Slide 18
Slide 18 text
◦ Larger tests are grouped into
modules and those are tested as a
group
◦ Objective: reveal issues between
integrated components of a
system
◦ Can help identify gaps that were
not caught in unit testing
◦ Commonly referred to as
functional or framework tests
INTEGRATION TESTING
Slide 19
Slide 19 text
EXAMPLE OF AN INTEGRATION TEST
Slide 20
Slide 20 text
◦ Was the postal code validated
when it was submitted?
◦ Did we return the appropriate error
if it was invalid?
EXAMPLE OF AN INTEGRATION TEST
Slide 21
Slide 21 text
NO INTEGRATION TESTS
Slide 22
Slide 22 text
◦ End-to-end testing of a particular
feature
◦ Objective: ensure your
feature/application is ready for
production
SYSTEMS TESTING
Slide 23
Slide 23 text
Examples:
◦ When submitting the customer
email, was the email persisted
correctly in the database or the
email service provider?
◦ Was the customer added to the
correct mailing list?
◦ Did we protect against spammers?
Did we introduce any vulnerabilities
or performance issues?
SYSTEMS TESTING
Slide 24
Slide 24 text
◦ Performed by product
owner/client
◦ Objective: confirm that
requirements for project were met
◦ Example:
▫ Marketing team to confirm the email
was correctly added to the right mailing
list
▫ They might also confirm that the email
has the correct fields embedded
(USER) ACCEPTANCE TESTING
Slide 25
Slide 25 text
MAIN LEVELS OF TESTING
UNIT TESTING
INTEGRATION TESTING
SYSTEMS TESTING
(USER) ACCEPTANCE
TESTING
Slide 26
Slide 26 text
PHP TESTING TOOLS
What are the most common tools and when should they be
used?
2
Slide 27
Slide 27 text
TDD vs BDD
◦ TDD: Test-Driven Development
◦ BDD: Behaviour-Driven
Development
◦ Both write unit tests before coding.
Main difference:
◦ TDD (e.g. PhpUnit) uses PHP code
for tests
◦ BDD (e.g. Behat) uses
human-readable sentences
Slide 28
Slide 28 text
EXAMPLE OF A UNIT TEST IN PHPUNIT
Slide 29
Slide 29 text
What's wrong with PhpUnit?
Nothing really. It can be used to test
anything.
However, because it is so generalized,
more specialized tools can help to
enforce good coding practices.
WHAT ABOUT PHPUNIT?
Slide 30
Slide 30 text
◦ Multi-purpose: can handle unit,
functional, API & acceptance tests
◦ Superset of PhpUnit
◦ Write test code in PHP
Slide 31
Slide 31 text
Codeception as a superset of PhpUnit
Slide 32
Slide 32 text
BDD acceptance testing with Codeception
Slide 33
Slide 33 text
◦ BDD tool that allows business
people to write software rules
without being a developer
◦ Behat is the PHP implementation of
Ruby's Cucumber
◦ Uses Gherkin syntax instead of
PHP; can take longer to onboard
for PHP developers
Slide 34
Slide 34 text
BEHAT & STORYBDD
Slide 35
Slide 35 text
BEHAT & STORYBDD
Slide 36
Slide 36 text
GTA PHP hosting a meetup about testing Behat with Docker
Slide 37
Slide 37 text
“
PhpSpec, which I maintain, is a tool for
designing classes that work well. It's
best used to drive the design of a clean,
decoupled, isolated domain model
without getting too involved in
infrastructure.
- Ciaran McNulty
Slide 38
Slide 38 text
More like SpecBDD compared to
Behat's StoryBDD
Slide 39
Slide 39 text
PHPSPEC & SPECBDD
Slide 40
Slide 40 text
PHPSPEC & SPECBDD
Slide 41
Slide 41 text
PHPSPEC & SPECBDD
Slide 42
Slide 42 text
◦ Great tool for testing web
application UIs
◦ Not just limited to PHP
Slide 43
Slide 43 text
Use whichever tool works for you.
THE FINAL VERDICT
Slide 44
Slide 44 text
TESTING GUIDELINES
How do we know we're testing the right things?
3
Slide 45
Slide 45 text
UNIT TEST COVERAGE
Shouldn't we just focus on getting unit test coverage to 100%?
3*
Slide 46
Slide 46 text
3
TYPES OF CODE COVERAGE
Once you have Xdebug installed, you
can use PhpUnit to determine code
coverage:
◦ Line coverage
◦ Function/method coverage
◦ Class/trait coverage
◦ Branch coverage
◦ Path coverage
◦ Change Risk Anti-Patterns (CRAP)
Index
Slide 47
Slide 47 text
3
UNIT TEST COVERAGE REPORT
Slide 48
Slide 48 text
100% code coverage is a little bit misleading though...
Slide 49
Slide 49 text
“
No amount of testing can prove a
software right, a single test can prove a
software wrong.
- Amir Ghahrai, testingexcellence.com
Slide 50
Slide 50 text
SOME BAD THINGS TO TEST
◦ Testing a built-in library function
(E.g. Math.round or array_unshift)
◦ Testing that when a button is
clicked, that the button was clicked
◦ Individual getters and setters
Slide 51
Slide 51 text
WHAT NEEDS TO BE TESTED?
Some ideas:
◦ Start with the risky and complex
elements first
◦ Test components that are
commonly used
◦ Testing assumptions about
evolving APIs
Slide 52
Slide 52 text
Source: Steve Sanderson, Selective Unit Testing – Costs and Benefits
Slide 53
Slide 53 text
◦ Middleware & router
◦ Authorization/authentication
◦ Error handling (how our API
handles thrown exceptions)
◦ Payment & email, etc.
◦ Data layer & graceful degradation
◦ Localizations
◦ Expected response formats
WHAT ARE HIGH IMPACT AREAS?
Slide 54
Slide 54 text
HOW TO MAKE TESTS SCALABLE?
When you have many tests or they
are taking a long time:
◦ De-couple from database
◦ Mock expensive or difficult to
reproduce calls (esp. third party)
◦ Run tests in parallel
Slide 55
Slide 55 text
UNIT TESTING FUNCTION CALLS WITH MOCKERY
Slide 56
Slide 56 text
PARALLEL UNIT TESTING WITH PARATEST
Slide 57
Slide 57 text
Paratest helps speed up performance
significantly.
Source: @zizacozizaco
PARALLEL UNIT TESTING WITH PARATEST
Slide 58
Slide 58 text
But be careful:
◦ Certain tests with dependencies
(e.g. CRUD tests, queuing jobs)
may be harder to run in parallel
PARALLEL UNIT TESTING WITH PARATEST
Slide 59
Slide 59 text
INTEGRATING TESTS WITHIN CI
How testing fits into the bigger picture
4
Slide 60
Slide 60 text
CI has one primary purpose:
To improve the quality and speed of
software delivery.
PURPOSE OF CONTINUOUS INTEGRATION
Slide 61
Slide 61 text
Jenkins in Action
Source: National Instruments, Getting Started with TestStand and
Jenkins
Slide 62
Slide 62 text
◦ Don't run unit tests during CI
▫ Should be tested during development
◦ Focus on integration and systems
tests (e.g. security, performance),
and core suite of critical
components
PRIORITIES FOR CI TESTS
Slide 63
Slide 63 text
CONTINUOUS INTEGRATION IN 8 (EASY) STEPS
1. Convert integration tests into unit
tests (or create unit tests)
▫ We need to make it clear where
functionality is failing, which is not that
clear with integration tests
▫ Remove DB dependencies
2. Automatically run unit tests upon
commit
▫ Forbid pushes that do not pass tests
3. Fix all failing unit tests
Slide 64
Slide 64 text
CONTINUOUS INTEGRATION IN 8 (EASY) STEPS
4. Identify areas for parallelization
(say that three times fast)
▫ E.g. jobs need to be tested immediately
▫ Certain CRUD operations need to be
tested in sequence
5. Create integration test suite
▫ Define test data set needed
▫ Should comprise of critical (high priority)
unit tests as well
▫ Staging/production security checks (e.g.
uptime robot)
Slide 65
Slide 65 text
CONTINUOUS INTEGRATION IN 8 (EASY) STEPS
6. Run integration tests during build
process (within CI framework)
▫ Whoever breaks the build should be
notified and assigned immediately.
▫ Policy should be: Can't go home unless
issue is resolved; can roll back.
▫ Blow it up in Slack if needed.
7. Work with DevOps teams to build
out systems tests
▫ 7b) Hire DevOps team
8. Communicate plan and create
development tickets
Slide 66
Slide 66 text
SonarQube for PHP
Slide 67
Slide 67 text
Thanks!
ANY QUESTIONS?
Slide 68
Slide 68 text
CREDITS
Special thanks to SlidesCarnival for releasing
this presentation template for free
See you next month at:
Chefs Plate
Wednesday, December 13, 2017
Talk:
Performance Tuning your PHP Applications
Presented by David Chang (@davidchchang)
Slide 69
Slide 69 text
BOOKS TO READ
Continuous Delivery:
Reliable Software
Releases through Build,
Test, and Deployment
Automation
Jez Humble, David Farley