Microservices Lifecycle Management V2

Includes API Implementation Details

Transcript

1. Microservices Lifecycle Management Micheal Benedict (@micheal)
 Cloud & Data Infrastructure @Pinterest
2. About Pinterest (History) Pinterest Infrastructure Lifecycle of a job - Challenges - Solution Importance Future 1 2 3 4 5
14. MONOLITH + 
 MICROSERVICES (SERVING)
15. FENCING & OWNERSHIP Clear isolation of services & its ownership. RELIABILITY
 Failure isolation and graceful degradation SCALABILITY & EFFICIENCY Scale independently ensuring efficient use of infrastructure DEVELOPER PRODUCTIVITY Make it simple for engineers to build and launch services quickly and easily MICROSERVICES The obvious benefits
16. However…
18. Not always what it seems…
19. tldr;
20. What is the lifecycle of a service?
21. RELEASE MONITOR CREATE DEPRECATE
22. RELEASE TEST & BUILD PACKAGE MONITOR LOGS, METRICS & TRACE GRAPH & ALERTS ONCALL DEPLOY (CANARY/PROD) CREATE DEPRECATE
23. RELEASE TEST & BUILD PACKAGE MONITOR LOGS, METRICS & TRACE GRAPH & ALERTS ONCALL DEPLOY (CANARY/PROD) MANAGE CREATE DEPRECATE
24. RELEASE TEST & BUILD PACKAGE MONITOR LOGS, METRICS & TRACE GRAPH & ALERTS ONCALL DEPLOY (CANARY/PROD) MANAGE IDENTITY METADATA RESOURCE PROVISIONING CREATE DEPRECATE METER & CHARGEBACK OWNERSHIP
25. RELEASE TEST & BUILD PACKAGE MONITOR LOGS, METRICS & TRACE GRAPH & ALERTS ONCALL DEPLOY (CANARY/PROD) MANAGE IDENTITY METADATA RESOURCE PROVISIONING CREATE DEPRECATE METER & CHARGEBACK OWNERSHIP Developer’s time > everything else
26. Who owns what?
28. Logical grouping of identifiers tied to the business The dictionary OWNERSHIP ENTITY MODEL BUSINESS OWNER TEAM PROJECT 1:N 1:N JOB NAME 1:N <SCOPE, IDENTIFIERS>
 (Depends on Identity Manager) 1:N OWNERSHIP IDENTITY
29. BUSINESS OWNER TEAM / Eng Manager PROJECT 1:N 1:N JOB NAME 1:N <SCOPE, IDENTIFIERS>
 (Depends on Identity Manager) 1:N OWNERSHIP IDENTITY INFRASTRUCTURE Core Services PinAndBoard 1:N 1:N pin_writer_service 1:N <compute, pin_write_cluster> <blob, pin_media_pictures> <blob, pin_media_videos> 1:N
30. BUSINESS OWNER TEAM / Eng Manager PROJECT 1:N 1:N JOB NAME 1:N <SCOPE, IDENTIFIERS>
 (Depends on Identity Manager) 1:N OWNERSHIP IDENTITY INFRASTRUCTURE Core Services PinAndBoard 1:N 1:N pin_writer_service 1:N <compute, pin_write_cluster> <blob, pin_media_pictures> <blob, pin_media_videos> 1:N
31. Need for canonical way of identifying services across infrastructure
32. pin_write_service BLOB STORAGE _prefix=<UUID> _prefix=<UUID> COMPUTE _cluster=<UUID> KEY/VAL STORAGE _namespace=<UUID> JOB NAME <SCOPE, IDENTIFIERS> 1:N IDENTIFIER PER RESOURCE TYPE CANONICAL JOB IDENTIFIER
33. Canonical identifiers for a job Identifying a job across platform/infrastructure services. COMPUTE BLOB STORAGE KEY/VAL
 STORAGE foo_service _cluster=
 <UUID> _namespace= <UUID> IDENTITY PROVISIONING SERVICE _prefix= <UUID> IDENTITY MANAGER
34. BUSINESS OWNER TEAM / Eng Manager PROJECT 1:N JOB NAME <SCOPE, IDENTIFIERS>
 (Depends on Identity Manager) 1:N OWNERSHIP IDENTITY INFRASTRUCTURE Core Services / Manager Name PinAndBoard 1:N pin_writer_service <compute, pin_write_cluster> <blob, pin_media_pictures> <blob, pin_media_videos> 1:N Service Directory V1
35. $ curl https://identity-api/api/v1/identifiers?name=pinalytics-thrift QUERY
 Who owns identifier pinalytics-thrift?
36. $ curl https://identity-api/api/v1/identifiers?ownerUsername=manager QUERY
 Show all identifiers owned by EM=<ManagerName>
37. Where to look up job metadata/configuration?
38. Key/Val pairs tied to Jobs & Projects following an hierarchical order Source of truth for Job Metadata METADATA
 MANAGER KEY/VAL KEY/VAL BUSINESS OWNER TEAM PROJECT 1:N 1:N JOB NAME 1:N <SCOPE, IDENTIFIERS>
 (Depends on Identity & Credential Manager) 1:N OWNERSHIP IDENTITY
39. How to inventory & allocate infra resources?
40. So, what resources can I use? Inventorying and provisioning of resources across platform/infrastructure services. RESOURCE
 MANAGER Define resources to offer: - Online Compute - Storage - Batch Compute Abstract resource provisioning by providing a workflow to provision resources - Allows policies (ex: < 100 vCPU free to launch) - Tie to identity system
41. So, what resources can I use? Inventorying and provisioning of resources across platform/infrastructure services. RESOURCE
 MANAGER COMPUTE BLOB STORAGE KEY/VAL
 STORAGE foo_service CPU MEMORY DISK STORAGE IN GB GETS PUTS STORAGE IN GB WPS RPS RESOURCE PROVISIONING SERVICE INFRA (CLOUD/DC) IDENTITY PROVISIONING SERVICE
42. $ curl https://resource-api/api/v1/offerings QUERY
 Show all the available infrastructure resources
43. $ curl https://resource-api/api/v1/measures?infrastructureName=Monarch&offeringName=Spark QUERY
 Show the resources, I will be charged for, for the Spark offering in Monarch
44. How to meter utilization of resources & attribute cost?
45. METER &CHARGEBACK How much am I using? Ability to meter allocation and utilization of resources per service, per engineering team and charge them accordingly Enables Visibility & Accountability Metering across Infrastructure requires standard `schema` - ts (timestamp) - identifier - infrastructure - resource - utilization Leverage internal visibility/observability stack Unit price definition per resource can difficult.
46. $ curl https://chargeback-api/api/v1/bills/monthly?month=05&year=2017&ownerUsername=manager QUERY
 What is <manager>’s infrastructure footprint (utilization & spend) for the month of May 2017?
47. SERVICES ON COMPUTE VM (EC2) - STACKED BY SERVICE IDENTIFIERS VCPU-DAYS
48. SERVICES ON COMPUTE VM (EC2) - STACKED BY ENG MANAGERS VCPU-DAYS
49. Why is this important?
50. focus #1 Simplify E2E Dev XP What are the steps a developer is required (but not expected) to do when building, launching & managing services, batch jobs, etc.?
51. focus #2 An integrated Infra Platform What is required to build a reliable, scalable, efficient & well integrated infrastructure platform?
52. focus #3 Infra Governance Without hampering developer experience and adding opswork, What controls are required to effectively utilize & manage Infrastructure
53. Future
54. DASHBOARD (SINGLE PANE OF GLASS) METADATA RESOURCE PROVISIONING & CAPACITY BUDET, METERING & CHARGEBACK IDENTITY PROVIDER APIS & ADAPTERS REPORTING WORKFLOWS { INFRASTRUCTURE AND PLATFORM SERVICES DATACENTER / PUBLIC CLOUD INTERNAL APIS OWNERSHIP
55. Thanks!