Designing Functional Tests

A presentation by @stuherbert  for @GanbaroDigital To Catch Problems Before
Your Customers Do Designing Functional Tests

@GanbaroDigital Who does your testing?

https://ﬂic.kr/p/dgkjrY

https://ﬂic.kr/p/6YZ2se

https://ﬂic.kr/p/7xMwVL

@GanbaroDigital Who Tests Your App? • Developers • Testers /
QA team • … or your customer?

@GanbaroDigital How does the customer feel when they are the
only tester?

https://ﬂic.kr/p/83Z8Kp

@GanbaroDigital It’s amazing how many ﬁrms rely on their customers
to do all of the testing.

@GanbaroDigital It’s worrying how many ﬁrms rely on their customers
to do all of the testing.

@GanbaroDigital Not the hallmark of   any reputable profession!

@GanbaroDigital How does functional testing address this?

@GanbaroDigital To answer that, let’s do a walkthrough of what
you’re shipping.

Your App User Network

Your App User Database Network

@GanbaroDigital A Typical Web Application • Single application • Persistent
database • Accessed via a web browser

@GanbaroDigital Busier websites end up with multiple app nodes.

Your App User Database Network Network Your App Your App

@GanbaroDigital This is a simplistic view of your app.

Your App User Database Network Network Your App Your App

Logic User Database Network Logic Logic Network UI & API
UI & API UI & API Local Storage Local Storage Local Storage

@GanbaroDigital Break Down Your App • User interface & API
• Processing logic • Anything stored on the local machine

@GanbaroDigital Doesn’t BDD testing already address this?

@GanbaroDigital Here is your app:

@GanbaroDigital And here is your app  according to BDD testing:

User UI & API

@GanbaroDigital Here is your app again:

@GanbaroDigital and that’s without getting into enterprise architectures,

@GanbaroDigital and that’s without getting into enterprise architectures, security infrastructure,

big data processing,

big data processing, or web-scale approaches.

@GanbaroDigital There’s quite a gap between your app, and your
app according to BDD tests.

@GanbaroDigital The effectiveness of BDD testing is inversely proportional to
the number of moving parts in your app’s architecture.

@GanbaroDigital BDD testing cares that   something can be done.
Functional tests care that   something has been done   accurately & completely.

@GanbaroDigital Both types of test are important.

@GanbaroDigital How do functional tests achieve that?

@GanbaroDigital To answer that, let’s break down   your application
architecture.

Logic Database Network Logic Logic Network UI & API UI
& API UI & API Local Storage Local Storage Local Storage User

& API UI & API Local Storage Local Storage Local Storage Actor

@GanbaroDigital Functional tests cover all supported operations

@GanbaroDigital Functional tests cover all supported operations, regardless of who
or what performs the operation.

@GanbaroDigital Actors interact with UI & API to exercise app
logic.

@GanbaroDigital Functional tests perform   the same interaction that actors
do,  in the same way that actors do.

@GanbaroDigital It is important that functional tests do not cheat.

@GanbaroDigital When functional tests cheat, the customer becomes the tester.

@GanbaroDigital For example:   if your actor uses a web
browser, the functional test   must use a web browser.

@GanbaroDigital Simulated web browsers cannot reproduce 100% of browser quirks.

@GanbaroDigital The more client-side JavaScript, the more of these quirks
you’ll run into.

@GanbaroDigital How can a simulated browser pick up on layout
quirks?

@GanbaroDigital Use a real browser, take screenshots, record a video
of the test.

@GanbaroDigital

@GanbaroDigital For example:   if your actor uses   a
speciﬁc client library, the functional test must use the same client library.

@GanbaroDigital Use a layered approach to your functional tests. Often
also known as isolate to eliminate.

@GanbaroDigital Use direct API calls   to prove that your
API works   at all.

@GanbaroDigital Use the client libraries to prove   that your
actors can use the API too.

@GanbaroDigital Functional tests are testing state machine transitions.

@GanbaroDigital Functional tests look at the data that persists after
an actor has performed an operation.

@GanbaroDigital … or that nothing has changed.

@GanbaroDigital Check for all the data   that should have
been created.

@GanbaroDigital Check for all the data   that should have
been changed.

@GanbaroDigital For example: if your UI says payment has been
taken, has the purchase been stored in your database?

@GanbaroDigital For example: can your actor   use the service
that they have just paid for?

@GanbaroDigital Bringing it all together:

@GanbaroDigital Functional Tests • Start from a deﬁned state •
Perform one operation • Use the same tools as the actor • Result in another deﬁned state • Prove that the target state is achieved

@GanbaroDigital … and they’re   incredibly strict about it.

@GanbaroDigital Catch problems where it is cheapest to do so.

@GanbaroDigital How can we describe that?

@GanbaroDigital Functional Tests • Start from a deﬁned state •
Perform one operation • Use the same tools as the actor • Result in another deﬁned state • Prove that the target state is achieved

@GanbaroDigital Wording A Functional Test • “Given <starting conditions>” •
“Using <specific tool>” • “As <specific actor>” • “When I do <specific operation>” • “A <long list of things> happen” • “And I can now do   <a list of other things>”

@GanbaroDigital What would that look like as test code?

@GanbaroDigital As A Code Flow • Test setup • Pre-test
inspection • Action(s) • Post-test inspection • Test tear down

@GanbaroDigital Revisit: Who does your testing?

@GanbaroDigital Today’s Test Approach Unit tests BDD tests Developers Testers

@GanbaroDigital Functional tests can help plug that gap

@GanbaroDigital A Better Approach Unit tests Functional tests BDD tests
Testers Developers Developers Testers

@GanbaroDigital This is not a new approach.

@GanbaroDigital This is how we used to test things before
the web came along.

https://ﬂic.kr/p/8eWNo

@GanbaroDigital New Practices New Year

Would you like to   know more? We can help.
Stuart Herbert ~ @stuherbert Founder @GanbaroDigital

Designing Functional Tests

Designing Functional Tests

More Decks by Stuart Herbert

Other Decks in Programming

Featured

Transcript