Introduction to API Testing Automation by Postman

by konifar

Slide 1

Slide 1 text

Introduction to API Testing Automation by Postman STAR 2021 12/11 (Sat) @konifar

Slide 2

Slide 2 text

Agenda 1. Postman as a testing tool 2. CLI tool: Newman & GitHub Actions 3. Automation practices 4. Cons of Postman and alternative tool

Slide 3

Slide 3 text

Agenda 1. Postman as a testing tool 2. CLI tool: Newman & GitHub Actions 3. Automation practices 4. Cons of Postman and alternative tool ● What are Collection and Request? ● How to write test code?

Slide 4

Slide 4 text

Agenda 1. Postman as a testing tool 2. CLI tool: Newman & GitHub Actions 3. Automation practices 4. Cons of Postman and alternative tool ● What is Newman? ● How to run Newman?

Slide 5

Slide 5 text

Agenda 1. Postman as a testing tool 2. CLI tool: Newman & GitHub Actions 3. Automation practices 4. Cons of Postman and alternative tool ● Switching API endpoint by environment ● Preparing before request ● Solving API dependencies

Slide 6

Slide 6 text

Agenda 1. Postman as a testing tool 2. CLI tool: Newman & GitHub Actions 3. Automation practices 4. Cons of Postman and alternative tool ● Pros and cons of Postman ● Introduction to Scenarigo

Slide 7

Slide 7 text

Sample API: Pixela https://pixe.la/ Thanks for awesome API! @a_know

Slide 8

Slide 8 text

1. Postman as a testing tool

Slide 9

Slide 9 text

Sending HTTP Request

Slide 10

Slide 10 text

Sending HTTP Request Test cases Request Details Request Result

Slide 11

Slide 11 text

Collection and Request Test cases Collection Folder Request

Slide 12

Slide 12 text

Building Request Request Details Method, URL Headers, Body…

Slide 13

Slide 13 text

Checking response Request Result

Slide 14

Slide 14 text

Verifying response

Slide 15

Slide 15 text

Test code snippets Verify Status code Verify JSON response Code snippets

Slide 16

Slide 16 text

Test results

Slide 17

Slide 17 text

2. CLI tool: Newman & GitHub Actions

Slide 18

Slide 18 text

Newman https://github.com/postmanlabs/newman ❯ npm install -g newman ❯ newman run https://www.getpostman.com/collections/18507875-41095035-c383-49ab-8553-a353132a1924 newman E2E Test ❏ POST /v1/users ↳ 200_success POST https://pixe.la/v1/users [200 OK, 518B, 118ms] ✓ Status code is 200 ✓ message is correct ✓ isSuccess is correct

Slide 19

Slide 19 text

Install Newman ❯ npm install -g newman ❯ newman run https://www.getpostman.com/collections/18507875-41095035-c383-49ab-8553-a353132a1924 newman E2E Test ❏ POST /v1/users ↳ 200_success POST https://pixe.la/v1/users [200 OK, 518B, 118ms] ✓ Status code is 200 ✓ message is correct ✓ isSuccess is correct Install via npm

Slide 20

Slide 20 text

Run Newman ❯ npm install -g newman ❯ newman run https://www.getpostman.com/collections/18507875-41095035-c383-49ab-8553-a353132a1924 newman E2E Test ❏ POST /v1/users ↳ 200_success POST https://pixe.la/v1/users [200 OK, 518B, 118ms] ✓ Status code is 200 ✓ message is correct ✓ isSuccess is correct Run requests in specific collection

Slide 21

Slide 21 text

Slide 22

Slide 22 text

# With API Key ❯ newman run https://www.getpostman.com/collections/{collection_id}?apikey={api_key} # Specify folder ❯ newman run https://www.getpostman.com/collections/{collection_id} --folder '{folder_name}' With API Key

Slide 23

Slide 23 text

Slide 24

Slide 24 text

GitHub Actions workflow jobs: run: runs-on: ubuntu-latest steps: - uses: actions/checkout@v1 - uses: actions/setup-node@v1 - run: sudo npm install -g newman - name: Run newman run: newman run https://www.getpostman.com/collections/$POSTMAN_COLLECTION_ID env: POSTMAN_COLLECTION_ID: ${{ secrets.POSTMAN_COLLECTION_ID }} https://github.com/konifar/stac2021-newman-example/pull/1

Slide 25

Slide 25 text

Using newman-action jobs: run: runs-on: ubuntu-latest steps: - uses: actions/checkout@v1 - uses: matt-ball/newman-action@master with: apiKey: ${{ secrets.POSTMAN_API_KEY }} collection: ${{ secrets.POSTMAN_COLLECTION_ID }} environment: ${{ secrets.POSTMAN_ENVIRONMENT_ID }} https://github.com/matt-ball/newman-action

Slide 26

Slide 26 text

3. Automation practices

Slide 27

Slide 27 text

3 Practices 1. Preparing to run repeatedly 2. Generating data before request 3. Switching API URL by environment

Slide 28

Slide 28 text

1. Preparing to run repeatedly

Slide 29

Slide 29 text

1. Preparing to run repeatedly Failed when the request runs again because the user already exists.

Slide 30

Slide 30 text

Solution1 Solution1: Create different user every time.

Slide 31

Slide 31 text

Use dynamic variables https://learning.postman.com/docs/writing-scripts/script-references/variables-list/ Set unique username by using timestamp.

Slide 32

Slide 32 text

Solution1 cons https://learning.postman.com/docs/writing-scripts/script-references/variables-list/ Many test users are created😰

Slide 33

Slide 33 text

Solution2 Create another request which delete user in same Collection and run it before the same user is created.

Slide 34

Slide 34 text

Run all requests in Collection

Slide 35

Slide 35 text

Solution2 cons Depends on run order😰

Slide 36

Slide 36 text

Solution3 Call user delete API in Pre-request Script I prefer this way.

Slide 37

Slide 37 text

2. Generating data before request

Slide 38

Slide 38 text

2. Generating data before request ● Base64 encoding ● Create random parameter ● etc…

Slide 39

Slide 39 text

Base64 encoding in Pre-request Script // Get env variables const clientId = pm.environment.get('VALID_CLIENT_ID'); const clientSecret = pm.environment.get('VALID_CLIENT_SECRET'); // Base64 encoding const base64encoded = CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(`${clientId}:${clientSecret}`)); console.log(`base64encoded: ${base64encoded}`); // Set value to collection variables pm.collectionVariables.set("base64encoded", base64encoded);

Slide 40

Slide 40 text

Get environment variables // Get env variables const clientId = pm.environment.get('VALID_CLIENT_ID'); const clientSecret = pm.environment.get('VALID_CLIENT_SECRET'); // Base64 encoding const base64encoded = CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(`${clientId}:${clientSecret}`)); console.log(`base64encoded: ${base64encoded}`); // Set value to collection variables pm.collectionVariables.set("base64encoded", base64encoded); Defined env variables in Postman

Slide 41

Slide 41 text

Base64 encoding // Get env variables const clientId = pm.environment.get('VALID_CLIENT_ID'); const clientSecret = pm.environment.get('VALID_CLIENT_SECRET'); // Base64 encoding const base64encoded = CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(`${clientId}:${clientSecret}`)); console.log(`base64encoded: ${base64encoded}`); // Set value to collection variables pm.collectionVariables.set("base64encoded", base64encoded);

Slide 42

Slide 42 text

Collection variables // Get env variables const clientId = pm.environment.get('VALID_CLIENT_ID'); const clientSecret = pm.environment.get('VALID_CLIENT_SECRET'); // Base64 encoding const base64encoded = CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(`${clientId}:${clientSecret}`)); console.log(`base64encoded: ${base64encoded}`); // Set value to collection variables pm.collectionVariables.set("base64encoded", base64encoded);

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Slide 45

Slide 45 text

3. Switching API URL by environment

Slide 46

Slide 46 text

Depends on test environment Different URLs for staging and prod Different usernames and tokens for each environment

Slide 47

Slide 47 text

Define Environment Define variables for each environments. I prefer upper snake case style. https://learning.postman.com/docs/sending-requests/managing-environments/

Slide 48

Slide 48 text

Use Environment in request

Slide 49

Slide 49 text

Use Environment in script

Slide 50

Slide 50 text

4. Cons of Postman and alternative tool

Slide 51

Slide 51 text

Pros of Postman 1. Simple GUI. No coding skills required. 2. Easy to setup CI by Newman. 3. Apply JavaScript code.

Slide 52

Slide 52 text

Cons of Postman 1. Can not reuse code in Pre-request scripts. 2. Difficult to maintain with several members. 3. Pricing.

Slide 53

Slide 53 text

Cons of Postman 1. Can not reuse code in Pre-request scripts. 2. Difficult to maintain with several members. 3. Pricing. We may have to write many same code for authentication request in Pre-requests

Slide 54

Slide 54 text

Cons of Postman 1. Can not reuse code in Pre-request scripts. 2. Difficult to maintain with several members. 3. Pricing. ● Postman has some features like Fork, GitHub integration. ● But it's hard to setup and difficult to review each other. ● I prefer GitHub Pull Request driven development 🙂

Slide 55

Slide 55 text

Cons of Postman 1. Can not reuse code in Pre-request scripts. 2. Difficult to maintain with several members. 3. Pricing. ● free for up to 3 users. ● $12 per user/month on Basic plan. ● Not so expensive but it depends on the budget. https://www.postman.com/pricing/

Slide 56

Slide 56 text

Scenarigo ● A scenario based API testing tool written in Go. ● Easy to write test scenarios as YAML file. ● Enable to customize by writing Go plugin. https://github.com/zoncoen/scenarigo Thanks for awesome tool! @zoncoen

Slide 57

Slide 57 text

Create User scenario yaml steps: - include: '../include/delete_user.yml' - protocol: http request: method: POST url: '{{env.SCENARIGO_BASE_URL}}/v1/users' body: token: '{{env.SCENARIGO_TEST_USER_TOKEN}}' username: '{{env.SCENARIGO_TEST_USER_NAME}}' agreeTermsOfService: 'yes' notMinor: 'yes' expect: code: 200 body: message: "Success. Let's visit {{env.SCENARIGO_BASE_URL}}/@{{env.SCENARIGO_TEST_USER_NAME}} , it is your profile page!" isSuccess: true

Slide 58

Slide 58 text

Slide 59

Slide 59 text

Slide 60

Slide 60 text

Slide 61

Slide 61 text

Run scenarigo ❯ scenarigo run ok scenarios/include/delete_user.yml 0.415s --- FAIL: scenarios/post_v1_users/200_success.yml (0.19s) … 10 | agreeTermsOfService: 'yes' 11 | notMinor: 'yes' 12 | expect: > 13 | code: 500 ^ … expected 500 but got OK FAIL FAIL scenarios/post_v1_users/200_success.yml 0.201s

Slide 62

Slide 62 text

Postman & Scenarigo Postman Scenarigo Manage scenarios in… Postman GUI GitHub, GitLab, BitBucket… We can review in Pull Request! Define Request in… Postman GUI request block in YAML file Define Environment in… Postman Environment System Environment Generate data before request Pre-request Script Go Plugin Request before request as... Pre-request Script Include another YAML file We can reuse it! Define Test in… Tests expect block in YAML file https://github.com/konifar/stac2021-newman-example/pull/5/files

Slide 63

Slide 63 text

Slide 64

Slide 64 text

Summary

Slide 65

Slide 65 text

Postman is awesome! ● Simple GUI makes easy to set up API tests for especially non-developers. ● Newman makes easy to run tests on CI.

Slide 66

Slide 66 text

But Scenarigo is better for us! ● We prefer Pull Request based development style ● So we decided to migrate Postman to Scenarigo ● Scenarigo covers major features of Postman

Slide 67

Slide 67 text

Thanks for watching!

Slide 68

Slide 68 text

Appendix ● Sample code to run Newman and Scenarigo on GitHub Actions ○ https://github.com/konifar/stac2021-newman-example ● Sample API: Pixela ○ https://pixe.la/ ● Postman official docs ○ https://learning.postman.com/docs ● Newman ○ https://github.com/postmanlabs/newman ● Scenarigo ○ https://github.com/zoncoen/scenarigo