Service Virtualization with NPM modules

Transcript

1. APIs Dynamic Service Mocks and Tests with NodeJS Newman & Express modules Prepared and Presented by Harish Anand Thiru Kumareshan, for Clivarsity Demo Session on 1st March 2019
2. Topics Covered Introduction To API/Service Mocking, – What is an API, how to define? And Introduction to Swagger – What is API Mocking? and What advantages can it give? Mocking APIs – How to Mock APIs, – Any Ready-made tools? – What advantages and disadvantages Ready-made tools do have? Dynamic API Mocks with NodeJs Express Module – Why to go For Custom API Stubs, – Introduction to NodeJs, advantages/disadvantages of the npm modules – Creating a Sample Mock with the custom API framework Developing Automated Testsuite with Mocks before actual API – Introduction and How to On Postman CLI – Adding newman to Dynamic Mock project – Scheduling with Cron Jobs and Defining Reporting styles and Channel Deploying “Service Mocks with Automated Tests Package” & CI – Versioning with Git – Dockizing the Node Package, – Introduction to Google App Engine (GAE), Google Kubernetes Engine (GKE), Google Cloud Build (GCB) – Introduction and Sample CI pipelines
3. What is an API? A simple Google search can get you the following- “a set of functions and procedures allowing the creation of applications that access the features or data of an operating system, application, or other service.” But what does that mean!! Introduction To API/Service Mocking
4. For understanding the same, first we see what a physical interface is, again from google ;-) – a point where two systems, subjects, organizations, etc. meet and interact. – a device or program enabling a user to communicate with a computer. Introduction To API/Service Mocking
5. Why and How to define an API As we just got from the previous image, each internal component inside a laptop is exposed with an unique/shared Interface, so is not required for a company which makes the laptop to dump all hardware in one unit, instead it can build the core which shares the port for the users to add their necessary hardware to it. Eg., Laptops doesn’t restrict user from using only its inbuilt monitor but can allow user to add any compatible display device by exposing it display port. This port exposing creates an ecosystem, where the usability and quality of the laptop/device can be improved with future inventions! But there is a catch too!! As specifies though laptops expose a display port, but it won’t accept any display device, it can detect and work only with the compatible/predefined display devices. Introduction To API/Service Mocking
6. And why there is this compatibility issue and why can’t the display port accepts any display device? The answer may simple but is also vague as is dependents on many factor ranging from – Security to Performance. And also to design a peripheral for another device, one need to know how the signals are to be transferred, and to which internal cores the signals to be sent, basically they need to know the complete architecture/operations of the actual device (laptop) to develop its peripheral device(External LCD Monitor). To overcome these kinds of issues and to keep the ecosystem go smooth the standards were introduced. By considering the same display port example, the standards we have like VGA, DVI, miniDVI, HDMI, Thunderbolt, etc., So anyone can design/manufacture the display device for the laptop without needing to know the laptop’s internal architecture or how it works, as long as they stick to the laptop’s display port standards. Introduction To API/Service Mocking
7. Same way for an application to expose its api, it need to convey its consuming application what it needs, to share what it can. Introduction To API/Service Mocking
8. And again the next issue, the compatibility (what data need to be sent it and in what format) for accessing the application to get a desired output is major problem (though recent services has an option to get the discovery service is again tough to access the discovery service without a known definition) eg., one application may accept rest call with Input as XML, JSON or form objects, etc., or it may accepts in rpc calls, etc., So this makes tough for anyone to develop an application that consume the api, and again like hardware one can’t have a general standards implied on the Specific software (This shouldn’t confused with standards like REST/RPC/SOAP, etc.,). To avoid these kind of issues, is always a good practice to define the APIs and what it can do, So anyone can develop any consuming application and need not to worry on how data is been prepared or how is responded back, as long as he knows the API definition which states what needs to inputted and in what format to application and what he can get back for the input. Introduction To API/Service Mocking
9. Introduction to Swagger There are many ways to define an API, including a word document, WSDLs, etc., One of such tools for defining REST interface are Swagger and OAS, these has a set of standards to define the API elements, for eg., the API should basically have Request path, header content, body content, Response code and a Response content. Swagger and/or OAS has the set of standard rules in creating/forming those API elements, so to maintain the compatibility throughout the software lifecycle. Eg., swagger api definition yaml file. Reference https://github.com/swagger-api, https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md Introduction To API/Service Mocking swagger.yaml
10. What is API Mocking? and What advantages can it give? As we now know what is an API, now we can see what API Mocking is? And why we need to mock an API? Service Mocking is quite called with different names like, stubbing, Service Virtualisation, etc., but all point to one thing, and for understanding the same we refer the same diagram again, but in a different perspective, Here, we can see there are two APIs which are talking to each other, and also with its own Front ends. Now lets say the API1 is connected to car and which collects all drive stats from the car, and API2 which get the GPS and acceleration data from the car via API1. Now, for the API2 has to design/developed/work it expects/needs API1 in place and constant data from the same, and it will fail to work if no data received from the API1 Introduction To API/Service Mocking
11. From this scenario, we can clearly see the dependencies among the systems. Though the API1 is already defined, the API2 can start its development phase. But for the regression testing on the API2s each release require its dependent API to be up and running and also makes easy for the debugging the application in development phase without making the real calls Advantages: Ease of Parallel development, Easy to debug the developing application, without making real calls to the connecting systems, Independent Regression Tests/ API Releases, enabling CICD, End to Performance test and effort estimation only on the API been developed, etc., Introduction To API/Service Mocking
12. Mocking/Stubbing APIs- how to and When to? Mocking/Stubbing APIs, as name suggest we will be sending a dummy values for the data in the response, for example considering the previous API which receives the GPS and acceleration data from the car, we know what values are expected from the car but we don’t know what exact values that we will get in a given time. For this scenario we can simply create a dummy API which can send a random value for GPS and Acceleration at any given time (in same data format as in the API definition ie., if the GPS co-ordinates are expected to be in a float then the values be like 40.730862 is generated randomly) like below, {GPS: {co_ordinates: { Latitude: 40.730862, Longitude: -73.987156 } }, Acceleration: {x:123, y:2244, z=-12323} } And for when to Mock? – there are no any hard rules on the same but usually is prescribed to create the stubs are created before/during the development phase which will help in both development and regression tests activities and also based on the needs the same is extended and upgraded on the later regressions Mocking APIs
13. Are there any tools readily available for API Mocks? Answer is yes! There are lots and lots of tools available both Paid and Open Source Few popular API Mock engines/software/frameworks are, Mocking APIs
14. There are lot of pros in using the readily available Mocking frameworks, quite few notable are, 1. Straight solution, simple mapping, 2. Simple and pre-defined standards, 3. Readily available library for different response formats, 4. Can be quick to market, Though is having great pros, there are few pain points in using the ready solutions like, 1. Will not be having full/native support for all type of calls eg., calls like RPC/DB Mocks 2. Will not be easily/straight solution to be deployable as standalones (mostly open-sourced tools), 3. Though deployable, size of the deploying application is huge as need to compile and build the whole mocking tool, 4. Mostly static or Limited/No Customizability, 5. Need to stick to the Tool limitations/Standards (Like defining predefined response in files), etc., Mocking APIs
15. Why to go for Custom build API Stubs? Yes, from the previous slide we could notice that, though the ready made tools provides lots and lots of features and looks efficient, but its pain points make them as an inefficient solution for long runs, specially the customizability and deployablity affects its long-run usage. And though few readily available frameworks like Wiremock are addressing those issues, still those has issues like defining static responses/JSON files for each response, and though those frameworks can be altered to support the full flexibility but the effort for the same will again look like a separate development task. Hence the need for custom build API stubs, which should address the issues of the readily available tools and also should carry their benefits. The custom build APIs can follow any environment/framework/language as far as it can serve the needs, Dynamic API Mocks with NodeJs Express Module
16. The custom build APIs can follow any environment/framework/language as far as it can serve the needs likes, 1. Should be easily customizable, 2. Easy to adapt/anyone can contribute and collaborate, 3. Easy to update/maintain for new CRs 4. Should be easily deployable as standalones, 5. Should support Integrations/can provide/expose API for E2E test simulations, 6. Should be robust and reliable, 7. Should not have any tool dependency (Pointing operation dependency, Not the build dependency ;-)). Dynamic API Mocks with NodeJs Express Module
17. Introduction to NodeJs, advantages/disadvantages of the npm modules This presentation we prefer to explain the custom API mock building with npm modules NodeJs in framework/environment. Why NodeJs? - Open ecosystem easy to contribute and Rich ecosystem, lots of collaborators/Contributors (though declining recent years) even big companies like Netflix, Paypal, etc., - supports for native Java, JavaScript, Groovy, - Can easily migrate to GO - Easy to create package/dockerize and can deploy in most servers including clouds Disadvantages: -Reliability- Though npm applies strict security and audit on the packages, there case from past where modules went missing crashes all packages Dynamic API Mocks with NodeJs Express Module
18. Creating a Sample Mock with the custom API framework In this section, we will see how to create a npm package, and how to add the npm modules like Npm Express, requests etc., to create a basic server, which will respond back the defined response from a selected element in a available responses JSON object. Dynamic API Mocks with NodeJs Express Module
19. In this section, we will see how to create a test collection from Postman, and how to integrate it to a node package using “neman” module, and also we will see how we select different styles of reporting including CLI, JSON, Junit, HTML and custom reports. We will also see dockerize the package and how to have scheduled test runs with cron Jobs running inside the docker. Developing Automated Testsuite with Mocks before actual API
20. In this section, we will see how to have version control on the test package using GIT/SVN. We will explore few cloud operations and terminologies for how to deploy the test solution in a Cloud server like Google Cloud platform and will see how to create a docker image for test package and how to call/add the image the cloud build steps in CI verification phase. Deploying “Service Mocks with Automated Tests Package” & CI