Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Developing, Testing and Deploying Microservices

Developing, Testing and Deploying Microservices

Presentation by Tobby Kuyinu on developing, testing and deploying microservices on forLoop Hangout conference.

forLoop

May 30, 2016
Tweet

More Decks by forLoop

Other Decks in Programming

Transcript

  1. Developing microservices ➔ Components ◆ Programming Language ◆ Data storage

    ◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits
  2. Developing microservices ➔ Components ◆ Programming Language ◆ Data storage

    ◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits
  3. Developing microservices ➔ Components ◆ Programming Language ◆ Data storage

    ◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits
  4. Components Libraries, packages and frameworks Libraries and packages - Helps

    separate concerns (e.g. ORMs) - Helps devs focus on core functionality - You’ll write some yourself if need be Frameworks - Essentially needed for their ‘out of the box API’ features - For languages that usually have heavy frameworks, it’s best to find a micro version
  5. Developing microservices ➔ Components ◆ Programming Language ◆ Data storage

    ◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits
  6. But really . . . the interface is just an

    api as you know it. You know, restful ;)
  7. Developing microservices ➔ Components ◆ Programming Language ◆ Data storage

    ◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits
  8. Standards & best practices Project structure ➔ Readme.md (absolutely compulsory)

    ➔ App ◆ Config • Config.js, di.js, db.js ◆ Controllers ◆ Lib • Oauth_helper.js, errors.js ◆ Models ◆ Repositories ◆ Routes ◆ Services ◆ Validations
  9. Standards & best practices Project structure ➔ Readme.md (absolutely compulsory)

    ➔ App ◆ Config • Config.js, di.js, db.js ◆ Controllers ◆ Lib • Oauth_helper.js, errors.js ◆ Models ◆ Repositories ◆ Routes ◆ Services ◆ Validations ➔ *Bin (custom scripts? Docker?)
  10. Standards & best practices Project structure ➔ Readme.md (absolutely compulsory)

    ➔ App ◆ Config • Config.js, di.js, db.js ◆ Controllers ◆ Lib • Oauth_helper.js, errors.js ◆ Models ◆ Repositories ◆ Routes ◆ Services ◆ Validations ➔ *Bin (custom scripts? Docker?) ➔ *Migrations/Schema (or equivalent)
  11. Standards & best practices Project structure ➔ Readme.md (absolutely compulsory)

    ➔ App ◆ Config • Config.js, di.js, db.js ◆ Controllers ◆ Lib • Oauth_helper.js, errors.js ◆ Models ◆ Repositories ◆ Routes ◆ Services ◆ Validations ➔ *Bin (custom scripts? Docker?) ➔ *Migrations/Schema (or equivalent) ➔ Test ◆ Helpers ◆ Regression ◆ unit
  12. Developing microservices ➔ Components ◆ Programming Language ◆ Data storage

    ◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits
  13. Standards & best practices Coding style Proper documentation and (preferably)

    an autodoc builder or generator /** * @method: properDocs * @param {intelligence} commonSense It only makes sense to document properly * @param {awesomeness} goodDev Good devs document their codes * @return {awesomeness} goodCode - It’s impossible to go wrong with a * properly documented code base. Moreso, * new devs can catch up easily. **/ properDocs: (commonSense, goodDev) => { }
  14. Standards & best practices Coding style Properly structured source code

    files according to responsibility Services - should contain business logic and straightforward functional tasks related to an entity. - should abstract data manipulation and retrieval but should not directly implement the data store. If a library isn’t being used, then create repositories for that
  15. Standards & best practices Coding style Properly structured source code

    files according to responsibility Models - should do exactly what the name suggests: model entities and represent them as entities with interfaces that can be used by the services - is sometimes simplified by an ORM module/library (e.g. bookshelf.js) or your repositories if no library is used
  16. Standards & best practices Coding style Properly structured source code

    files according to responsibility Controllers - should always be the entry point - should simply co-ordinate a sequence of activities to be done by the services and decide what point to return an API response
  17. Standards & best practices Coding style Properly structured source code

    files according to responsibility - Service: function getUserData(userId) { userModel.getUser(userId) .then(userDetails => { If (userDetails) { userModel.getUserAddresses(userId) .then(addresses => { resolve({user: userDetails, addresses: addresses}); }); } else { reject(new Error(‘User not found’)); } }); }
  18. Standards & best practices Coding style Properly structured source code

    files according to responsibility - Model: function getUser(userId) { Return this.repository.fetchUserById(userId) .then(userData => { resolve((userData.length() > 0)? userData[0] : userData); }); } Function getUserAddresses(userId) { Return this.repository.fetchUserAddressByUserId(userId) .then(addressData => { resolve(addressData); }); }
  19. Standards & best practices Coding style Properly structured source code

    files according to responsibility - Repository: function fetchUserById(userId) { Return mysql.select(‘*’) .from(‘users’) .where({user_id: userId}); } Function fetchUserAddressByUserId(userId) { Return mysql.select(‘*’) .from(‘addresses’) .where({user_id: userId}); } See how easy it’ll be to switch the data store technology if the need arises?
  20. Standards & best practices Coding style Always favor configurables, using

    a helper file for constants where necessary - You want to do this: - Function getSomeData(page, limit) { let constants = require(‘app/lib/constants’); limit = min(limit, constants.pagination.MAX_ALLOWED_LIMIT); let opts = { page: page, offset: (page - 1) * limit, limit: limit } . . . } - Rather than this: limit = min(limit, 50);
  21. Developing microservices ➔ Components ◆ Programming Language ◆ Data storage

    ◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits
  22. Standards & best practices Best practices Have a base service

    to extend Use linting tools to ensure code quality
  23. Standards & best practices Best practices Have a base service

    to extend Use linting tools to ensure code quality Write unit and regression tests
  24. Standards & best practices Best practices Have a base service

    to extend Use linting tools to ensure code quality Write unit and regression tests Protect your API (validate as much as you can)
  25. Standards & best practices Best practices Have a base service

    to extend Use linting tools to ensure code quality Write unit and regression tests Protect your API (validate as much as you can) Dockerize your service
  26. Developing microservices ➔ Components ◆ Programming Language ◆ Data storage

    ◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits
  27. Standards & best practices Boundaries and limits A microservice must

    remain micro Must be 100% responsible for it’s data
  28. Standards & best practices Boundaries and limits A microservice must

    remain micro Must be 100% responsible for it’s data Must not be dependent on another service
  29. Standards & best practices Boundaries and limits A microservice must

    remain micro Must be 100% responsible for it’s data Must not be dependent on another service Should preferably not be responsible for caching
  30. Standards & best practices Boundaries and limits A microservice must

    remain micro Must be 100% responsible for it’s data Must not be dependent on another service Should preferably not be responsible for caching Must not have any unrelated models hanging around
  31. Developing microservices ➔ Components ◆ Programming Language ◆ Data storage

    ◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits
  32. testing coverage Against whatever you were told, test coverage isn’t

    a measure of quality. It’s simply to help find untested code
  33. testing Environments Option 1: When running locally, stub out supporting

    technologies fakeSaveToDB(id, data) { Return ; } mockery.registerMock(database.saveToDB, fakeSaveToDB);
  34. deploying patterns Multiple services per instance Pros: - Efficient resource

    utilisation - Relatively fast deployment and startup Cons: - No isolation of services - One service screws all - Complex deployment
  35. deploying patterns Service instance per host (VM) Pros: - Services

    run in isolation - Fixed amount of CPU and memory - Easy deployments Cons: - Inefficient resource utilisation - Expensive - Slow deployment for new versions - Heavy lifting overhead
  36. deploying patterns Service instance per host (Container) Pros: - Services

    run in isolation - Fixed amount of CPU and memory - Encapsulates tech used to implement - Easy deployments Cons: - Not as mature as VMs - Not as secure as VMs - Heavy lifting overhead - Provisioning costs
  37. deploying patterns - worthy mention Serverless deployment - E.g. AWS

    Lambda - Package your service as a zip file and upload - Supply a metadata (and functions that handle ‘events’) - Automatically runs enough instances to handle requests - Services must be stateless - Limited to only a few languages