Share
Embed
Developing, Testing and Deploying Microservices
Presentation by Tobby Kuyinu on developing, testing and deploying microservices on forLoop Hangout conference.
forLoop
May 30, 2016
Transcript
78 slides
-
Developing microservices ➔ Components ◆ Programming Language ◆ Data storage
◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits -
Developing microservices ➔ Components ◆ Programming Language ◆ Data storage
◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits -
Developing microservices ➔ Components ◆ Programming Language ◆ Data storage
◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits -
Components Libraries, packages and frameworks Libraries and packages - Helps
separate concerns (e.g. ORMs) - Helps devs focus on core functionality - You’ll write some yourself if need be Frameworks - Essentially needed for their ‘out of the box API’ features - For languages that usually have heavy frameworks, it’s best to find a micro version -
Developing microservices ➔ Components ◆ Programming Language ◆ Data storage
◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits -
But really . . . the interface is just an
api as you know it. You know, restful ;) -
Developing microservices ➔ Components ◆ Programming Language ◆ Data storage
◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits -
Standards & best practices Project structure - Nodejs use-case but
applies to any stack -
Standards & best practices Project structure ➔ Readme.md (absolutely compulsory)
➔ App ◆ Config • Config.js, di.js, db.js ◆ Controllers ◆ Lib • Oauth_helper.js, errors.js ◆ Models ◆ Repositories ◆ Routes ◆ Services ◆ Validations -
Standards & best practices Project structure ➔ Readme.md (absolutely compulsory)
➔ App ◆ Config • Config.js, di.js, db.js ◆ Controllers ◆ Lib • Oauth_helper.js, errors.js ◆ Models ◆ Repositories ◆ Routes ◆ Services ◆ Validations ➔ *Bin (custom scripts? Docker?) -
Standards & best practices Project structure ➔ Readme.md (absolutely compulsory)
➔ App ◆ Config • Config.js, di.js, db.js ◆ Controllers ◆ Lib • Oauth_helper.js, errors.js ◆ Models ◆ Repositories ◆ Routes ◆ Services ◆ Validations ➔ *Bin (custom scripts? Docker?) ➔ *Migrations/Schema (or equivalent) -
Standards & best practices Project structure ➔ Readme.md (absolutely compulsory)
➔ App ◆ Config • Config.js, di.js, db.js ◆ Controllers ◆ Lib • Oauth_helper.js, errors.js ◆ Models ◆ Repositories ◆ Routes ◆ Services ◆ Validations ➔ *Bin (custom scripts? Docker?) ➔ *Migrations/Schema (or equivalent) ➔ Test ◆ Helpers ◆ Regression ◆ unit -
Developing microservices ➔ Components ◆ Programming Language ◆ Data storage
◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits -
Standards & best practices Coding style - Nodejs use-case but
applies to any stack -
Standards & best practices Coding style Proper documentation and (preferably)
an autodoc builder or generator /** * @method: properDocs * @param {intelligence} commonSense It only makes sense to document properly * @param {awesomeness} goodDev Good devs document their codes * @return {awesomeness} goodCode - It’s impossible to go wrong with a * properly documented code base. Moreso, * new devs can catch up easily. **/ properDocs: (commonSense, goodDev) => { } -
Standards & best practices Coding style Properly structured source code
files according to responsibility Services - should contain business logic and straightforward functional tasks related to an entity. - should abstract data manipulation and retrieval but should not directly implement the data store. If a library isn’t being used, then create repositories for that -
Standards & best practices Coding style Properly structured source code
files according to responsibility Models - should do exactly what the name suggests: model entities and represent them as entities with interfaces that can be used by the services - is sometimes simplified by an ORM module/library (e.g. bookshelf.js) or your repositories if no library is used -
Standards & best practices Coding style Properly structured source code
files according to responsibility Controllers - should always be the entry point - should simply co-ordinate a sequence of activities to be done by the services and decide what point to return an API response -
Standards & best practices Coding style Properly structured source code
files according to responsibility - Service: function getUserData(userId) { userModel.getUser(userId) .then(userDetails => { If (userDetails) { userModel.getUserAddresses(userId) .then(addresses => { resolve({user: userDetails, addresses: addresses}); }); } else { reject(new Error(‘User not found’)); } }); } -
Standards & best practices Coding style Properly structured source code
files according to responsibility - Model: function getUser(userId) { Return this.repository.fetchUserById(userId) .then(userData => { resolve((userData.length() > 0)? userData[0] : userData); }); } Function getUserAddresses(userId) { Return this.repository.fetchUserAddressByUserId(userId) .then(addressData => { resolve(addressData); }); } -
Standards & best practices Coding style Properly structured source code
files according to responsibility - Repository: function fetchUserById(userId) { Return mysql.select(‘*’) .from(‘users’) .where({user_id: userId}); } Function fetchUserAddressByUserId(userId) { Return mysql.select(‘*’) .from(‘addresses’) .where({user_id: userId}); } See how easy it’ll be to switch the data store technology if the need arises? -
Standards & best practices Coding style Always favor configurables, using
a helper file for constants where necessary - You want to do this: - Function getSomeData(page, limit) { let constants = require(‘app/lib/constants’); limit = min(limit, constants.pagination.MAX_ALLOWED_LIMIT); let opts = { page: page, offset: (page - 1) * limit, limit: limit } . . . } - Rather than this: limit = min(limit, 50); -
Developing microservices ➔ Components ◆ Programming Language ◆ Data storage
◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits -
Standards & best practices Best practices Have a base service
to extend Use linting tools to ensure code quality -
Standards & best practices Best practices Have a base service
to extend Use linting tools to ensure code quality Write unit and regression tests -
Standards & best practices Best practices Have a base service
to extend Use linting tools to ensure code quality Write unit and regression tests Protect your API (validate as much as you can) -
Standards & best practices Best practices Have a base service
to extend Use linting tools to ensure code quality Write unit and regression tests Protect your API (validate as much as you can) Dockerize your service -
Developing microservices ➔ Components ◆ Programming Language ◆ Data storage
◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits -
Standards & best practices Boundaries and limits A microservice must
remain micro Must be 100% responsible for it’s data -
Standards & best practices Boundaries and limits A microservice must
remain micro Must be 100% responsible for it’s data Must not be dependent on another service -
Standards & best practices Boundaries and limits A microservice must
remain micro Must be 100% responsible for it’s data Must not be dependent on another service Should preferably not be responsible for caching -
Standards & best practices Boundaries and limits A microservice must
remain micro Must be 100% responsible for it’s data Must not be dependent on another service Should preferably not be responsible for caching Must not have any unrelated models hanging around -
Developing microservices ➔ Components ◆ Programming Language ◆ Data storage
◆ Libraries, packages and frameworks ◆ Interfaces (API) ➔ Standards & Best practices ◆ Project structure ◆ Coding style ◆ Best practices ◆ Boundaries and limits -
Testing and deploying microservices ➔ Testing ◆ Components ◆ Coverage
◆ Environments ➔ Deploying ◆ Patterns -
Testing and deploying microservices ➔ Testing ◆ Components ◆ Coverage
◆ Environments ➔ Deploying ◆ Patterns -
testing coverage Against whatever you were told, test coverage isn’t
a measure of quality. It’s simply to help find untested code -
Testing and deploying microservices ➔ Testing ◆ Components ◆ Coverage
◆ Environments ➔ Deploying ◆ Patterns -
testing Environments Option 1: When running locally, stub out supporting
technologies fakeSaveToDB(id, data) { Return ; } mockery.registerMock(database.saveToDB, fakeSaveToDB); -
Testing and deploying microservices ➔ Testing ◆ Components ◆ Coverage
◆ Environments ➔ Deploying ◆ Patterns -
deploying patterns Multiple services per instance Pros: - Efficient resource
utilisation - Relatively fast deployment and startup Cons: - No isolation of services - One service screws all - Complex deployment -
deploying patterns Service instance per host (VM) Pros: - Services
run in isolation - Fixed amount of CPU and memory - Easy deployments Cons: - Inefficient resource utilisation - Expensive - Slow deployment for new versions - Heavy lifting overhead -
deploying patterns Service instance per host (Container) Pros: - Services
run in isolation - Fixed amount of CPU and memory - Encapsulates tech used to implement - Easy deployments Cons: - Not as mature as VMs - Not as secure as VMs - Heavy lifting overhead - Provisioning costs -
deploying patterns - worthy mention Serverless deployment - E.g. AWS
Lambda - Package your service as a zip file and upload - Supply a metadata (and functions that handle ‘events’) - Automatically runs enough instances to handle requests - Services must be stateless - Limited to only a few languages -
Testing and deploying microservices ➔ Testing ◆ Components ◆ Coverage
◆ Environments ➔ Deploying ◆ Patterns