Introduction to API Testing Automation by Postman

Transcript

1. Introduction to API Testing Automation by Postman STAR 2021 12/11

   (Sat) @konifar

2. Agenda 1. Postman as a testing tool 2. CLI tool:

   Newman & GitHub Actions 3. Automation practices 4. Cons of Postman and alternative tool

3. Agenda 1. Postman as a testing tool 2. CLI tool:

   Newman & GitHub Actions 3. Automation practices 4. Cons of Postman and alternative tool • What are Collection and Request? • How to write test code?

4. Agenda 1. Postman as a testing tool 2. CLI tool:

   Newman & GitHub Actions 3. Automation practices 4. Cons of Postman and alternative tool • What is Newman? • How to run Newman?

5. Agenda 1. Postman as a testing tool 2. CLI tool:

   Newman & GitHub Actions 3. Automation practices 4. Cons of Postman and alternative tool • Switching API endpoint by environment • Preparing before request • Solving API dependencies

6. Agenda 1. Postman as a testing tool 2. CLI tool:

   Newman & GitHub Actions 3. Automation practices 4. Cons of Postman and alternative tool • Pros and cons of Postman • Introduction to Scenarigo

7. Sample API: Pixela https://pixe.la/ Thanks for awesome API! @a_know

8. 1. Postman as a testing tool

9. Sending HTTP Request

10. Sending HTTP Request Test cases Request Details Request Result

11. Collection and Request Test cases Collection Folder Request

12. Building Request Request Details Method, URL Headers, Body…

13. Checking response Request Result

14. Verifying response

15. Test code snippets Verify Status code Verify JSON response Code

    snippets

16. Test results

17. 2. CLI tool: Newman & GitHub Actions

18. Newman https://github.com/postmanlabs/newman ❯ npm install -g newman ❯ newman run

    https://www.getpostman.com/collections/18507875-41095035-c383-49ab-8553-a353132a1924 newman E2E Test ❏ POST /v1/users ↳ 200_success POST https://pixe.la/v1/users [200 OK, 518B, 118ms] ✓ Status code is 200 ✓ message is correct ✓ isSuccess is correct

19. Install Newman ❯ npm install -g newman ❯ newman run

    https://www.getpostman.com/collections/18507875-41095035-c383-49ab-8553-a353132a1924 newman E2E Test ❏ POST /v1/users ↳ 200_success POST https://pixe.la/v1/users [200 OK, 518B, 118ms] ✓ Status code is 200 ✓ message is correct ✓ isSuccess is correct Install via npm

20. Run Newman ❯ npm install -g newman ❯ newman run

    https://www.getpostman.com/collections/18507875-41095035-c383-49ab-8553-a353132a1924 newman E2E Test ❏ POST /v1/users ↳ 200_success POST https://pixe.la/v1/users [200 OK, 518B, 118ms] ✓ Status code is 200 ✓ message is correct ✓ isSuccess is correct Run requests in specific collection

21. Run Newman ❯ npm install -g newman ❯ newman run

    https://www.getpostman.com/collections/18507875-41095035-c383-49ab-8553-a353132a1924 newman E2E Test ❏ POST /v1/users ↳ 200_success POST https://pixe.la/v1/users [200 OK, 518B, 118ms] ✓ Status code is 200 ✓ message is correct ✓ isSuccess is correct Show results. All tests are passed!

22. # With API Key ❯ newman run https://www.getpostman.com/collections/{collection_id}?apikey={api_key} # Specify

    folder ❯ newman run https://www.getpostman.com/collections/{collection_id} --folder '{folder_name}' With API Key

23. # With API Key ❯ newman run https://www.getpostman.com/collections/{collection_id}?apikey={api_key} # Specify

    folder ❯ newman run https://www.getpostman.com/collections/{collection_id} --folder '{folder_name}' Specify folder

24. GitHub Actions workflow jobs: run: runs-on: ubuntu-latest steps: - uses:

    actions/checkout@v1 - uses: actions/setup-node@v1 - run: sudo npm install -g newman - name: Run newman run: newman run https://www.getpostman.com/collections/$POSTMAN_COLLECTION_ID env: POSTMAN_COLLECTION_ID: ${{ secrets.POSTMAN_COLLECTION_ID }} https://github.com/konifar/stac2021-newman-example/pull/1

25. Using newman-action jobs: run: runs-on: ubuntu-latest steps: - uses: actions/checkout@v1

    - uses: matt-ball/newman-action@master with: apiKey: ${{ secrets.POSTMAN_API_KEY }} collection: ${{ secrets.POSTMAN_COLLECTION_ID }} environment: ${{ secrets.POSTMAN_ENVIRONMENT_ID }} https://github.com/matt-ball/newman-action

26. 3. Automation practices

27. 3 Practices 1. Preparing to run repeatedly 2. Generating data

    before request 3. Switching API URL by environment

28. 1. Preparing to run repeatedly

29. 1. Preparing to run repeatedly Failed when the request runs

    again because the user already exists.

30. Solution1 Solution1: Create different user every time.

31. Use dynamic variables https://learning.postman.com/docs/writing-scripts/script-references/variables-list/ Set unique username by using timestamp.

32. Solution1 cons https://learning.postman.com/docs/writing-scripts/script-references/variables-list/ Many test users are created😰

33. Solution2 Create another request which delete user in same Collection

    and run it before the same user is created.

34. Run all requests in Collection

35. Solution2 cons Depends on run order😰

36. Solution3 Call user delete API in Pre-request Script I prefer

    this way.

37. 2. Generating data before request

38. 2. Generating data before request • Base64 encoding • Create

    random parameter • etc…

39. Base64 encoding in Pre-request Script // Get env variables const

    clientId = pm.environment.get('VALID_CLIENT_ID'); const clientSecret = pm.environment.get('VALID_CLIENT_SECRET'); // Base64 encoding const base64encoded = CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(`${clientId}:${clientSecret}`)); console.log(`base64encoded: ${base64encoded}`); // Set value to collection variables pm.collectionVariables.set("base64encoded", base64encoded);

40. Get environment variables // Get env variables const clientId =

    pm.environment.get('VALID_CLIENT_ID'); const clientSecret = pm.environment.get('VALID_CLIENT_SECRET'); // Base64 encoding const base64encoded = CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(`${clientId}:${clientSecret}`)); console.log(`base64encoded: ${base64encoded}`); // Set value to collection variables pm.collectionVariables.set("base64encoded", base64encoded); Defined env variables in Postman

41. Base64 encoding // Get env variables const clientId = pm.environment.get('VALID_CLIENT_ID');

    const clientSecret = pm.environment.get('VALID_CLIENT_SECRET'); // Base64 encoding const base64encoded = CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(`${clientId}:${clientSecret}`)); console.log(`base64encoded: ${base64encoded}`); // Set value to collection variables pm.collectionVariables.set("base64encoded", base64encoded);

42. Collection variables // Get env variables const clientId = pm.environment.get('VALID_CLIENT_ID');

    const clientSecret = pm.environment.get('VALID_CLIENT_SECRET'); // Base64 encoding const base64encoded = CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(`${clientId}:${clientSecret}`)); console.log(`base64encoded: ${base64encoded}`); // Set value to collection variables pm.collectionVariables.set("base64encoded", base64encoded);

43. Collection variables // Get env variables const clientId = pm.environment.get('VALID_CLIENT_ID');

    const clientSecret = pm.environment.get('VALID_CLIENT_SECRET'); // Base64 encoding const base64encoded = CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(`${clientId}:${clientSecret}`)); console.log(`base64encoded: ${base64encoded}`); // Set value to collection variables pm.collectionVariables.set("base64encoded", base64encoded); Check collection variable is set

44. Collection variables // Get env variables const clientId = pm.environment.get('VALID_CLIENT_ID');

    const clientSecret = pm.environment.get('VALID_CLIENT_SECRET'); // Base64 encoding const base64encoded = CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(`${clientId}:${clientSecret}`)); console.log(`base64encoded: ${base64encoded}`); // Set value to collection variables pm.collectionVariables.set("base64encoded", base64encoded); Use variable in request

45. 3. Switching API URL by environment

46. Depends on test environment Different URLs for staging and prod

    Different usernames and tokens for each environment

47. Define Environment Define variables for each environments. I prefer upper

    snake case style. https://learning.postman.com/docs/sending-requests/managing-environments/

48. Use Environment in request

49. Use Environment in script

50. 4. Cons of Postman and alternative tool

51. Pros of Postman 1. Simple GUI. No coding skills required.

    2. Easy to setup CI by Newman. 3. Apply JavaScript code.

52. Cons of Postman 1. Can not reuse code in Pre-request

    scripts. 2. Difficult to maintain with several members. 3. Pricing.

53. Cons of Postman 1. Can not reuse code in Pre-request

    scripts. 2. Difficult to maintain with several members. 3. Pricing. We may have to write many same code for authentication request in Pre-requests

54. Cons of Postman 1. Can not reuse code in Pre-request

    scripts. 2. Difficult to maintain with several members. 3. Pricing. • Postman has some features like Fork, GitHub integration. • But it's hard to setup and difficult to review each other. • I prefer GitHub Pull Request driven development 🙂

55. Cons of Postman 1. Can not reuse code in Pre-request

    scripts. 2. Difficult to maintain with several members. 3. Pricing. • free for up to 3 users. • $12 per user/month on Basic plan. • Not so expensive but it depends on the budget. https://www.postman.com/pricing/

56. Scenarigo • A scenario based API testing tool written in

    Go. • Easy to write test scenarios as YAML file. • Enable to customize by writing Go plugin. https://github.com/zoncoen/scenarigo Thanks for awesome tool! @zoncoen

57. Create User scenario yaml steps: - include: '../include/delete_user.yml' - protocol:

    http request: method: POST url: '{{env.SCENARIGO_BASE_URL}}/v1/users' body: token: '{{env.SCENARIGO_TEST_USER_TOKEN}}' username: '{{env.SCENARIGO_TEST_USER_NAME}}' agreeTermsOfService: 'yes' notMinor: 'yes' expect: code: 200 body: message: "Success. Let's visit {{env.SCENARIGO_BASE_URL}}/@{{env.SCENARIGO_TEST_USER_NAME}} , it is your profile page!" isSuccess: true

58. Create User scenario yaml steps: - include: '../include/delete_user.yml' - protocol:

    http request: method: POST url: '{{env.SCENARIGO_BASE_URL}}/v1/users' body: token: '{{env.SCENARIGO_TEST_USER_TOKEN}}' username: '{{env.SCENARIGO_TEST_USER_NAME}}' agreeTermsOfService: 'yes' notMinor: 'yes' expect: code: 200 body: message: "Success. Let's visit {{env.SCENARIGO_BASE_URL}}/@{{env.SCENARIGO_TEST_USER_NAME}} , it is your profile page!" isSuccess: true Define delete user request in other yaml file. Same feature to Postman Pre-request Script.

59. Create User scenario yaml steps: - include: '../include/delete_user.yml' - protocol:

    http request: method: POST url: '{{env.SCENARIGO_BASE_URL}}/v1/users' body: token: '{{env.SCENARIGO_TEST_USER_TOKEN}}' username: '{{env.SCENARIGO_TEST_USER_NAME}}' agreeTermsOfService: 'yes' notMinor: 'yes' expect: code: 200 body: message: "Success. Let's visit {{env.SCENARIGO_BASE_URL}}/@{{env.SCENARIGO_TEST_USER_NAME}} , it is your profile page!" isSuccess: true Define create user request. Same feature to Postman request. Environment variables are available.

60. Create User scenario yaml steps: - include: '../include/delete_user.yml' - protocol:

    http request: method: POST url: '{{env.SCENARIGO_BASE_URL}}/v1/users' body: token: '{{env.SCENARIGO_TEST_USER_TOKEN}}' username: '{{env.SCENARIGO_TEST_USER_NAME}}' agreeTermsOfService: 'yes' notMinor: 'yes' expect: code: 200 body: message: "Success. Let's visit {{env.SCENARIGO_BASE_URL}}/@{{env.SCENARIGO_TEST_USER_NAME}} , it is your profile page!" isSuccess: true Verify response values here. Same feature to Postman tests.

61. Run scenarigo ❯ scenarigo run ok scenarios/include/delete_user.yml 0.415s --- FAIL:

    scenarios/post_v1_users/200_success.yml (0.19s) … 10 | agreeTermsOfService: 'yes' 11 | notMinor: 'yes' 12 | expect: > 13 | code: 500 ^ … expected 500 but got OK FAIL FAIL scenarios/post_v1_users/200_success.yml 0.201s

62. Postman & Scenarigo Postman Scenarigo Manage scenarios in… Postman GUI

    GitHub, GitLab, BitBucket… We can review in Pull Request! Define Request in… Postman GUI request block in YAML file Define Environment in… Postman Environment System Environment Generate data before request Pre-request Script Go Plugin Request before request as... Pre-request Script Include another YAML file We can reuse it! Define Test in… Tests expect block in YAML file https://github.com/konifar/stac2021-newman-example/pull/5/files

63. Postman & Scenarigo Postman Scenarigo Manage scenarios in… Postman GUI

    GitHub, GitLab, BitBucket… We can review in Pull Request! Define Request in… Postman GUI request block in YAML file Define Environment in… Postman Environment System Environment Generate data before request Pre-request Script Go Plugin Request before request as... Pre-request Script Include another YAML file We can reuse it! Define Test in… Tests expect block in YAML file https://github.com/konifar/stac2021-newman-example/pull/5/files QA Team needs to learn YAML, Go, Git and GitHub. (We did it !)

64. Summary

65. Postman is awesome! • Simple GUI makes easy to set

    up API tests for especially non-developers. • Newman makes easy to run tests on CI.

66. But Scenarigo is better for us! • We prefer Pull

    Request based development style • So we decided to migrate Postman to Scenarigo • Scenarigo covers major features of Postman

67. Thanks for watching!

68. Appendix • Sample code to run Newman and Scenarigo on

    GitHub Actions ◦ https://github.com/konifar/stac2021-newman-example • Sample API: Pixela ◦ https://pixe.la/ • Postman official docs ◦ https://learning.postman.com/docs • Newman ◦ https://github.com/postmanlabs/newman • Scenarigo ◦ https://github.com/zoncoen/scenarigo