About Pinterest (History)
Pinterest Infrastructure
Lifecycle of a job
- Challenges
- Solution
Importance
Future
1
2
3
4
5
Slide 3
Slide 3 text
No content
Slide 4
Slide 4 text
2010
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
2017
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
No content
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
No content
Slide 11
Slide 11 text
No content
Slide 12
Slide 12 text
No content
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
MONOLITH +
MICROSERVICES
(SERVING)
Slide 15
Slide 15 text
FENCING & OWNERSHIP
Clear isolation of services & its ownership.
RELIABILITY
Failure isolation and graceful degradation
SCALABILITY & EFFICIENCY
Scale independently ensuring efficient use of
infrastructure
DEVELOPER PRODUCTIVITY
Make it simple for engineers to build and
launch services quickly and easily
MICROSERVICES
The obvious benefits
Logical grouping of identifiers tied to the
business
The dictionary
OWNERSHIP
ENTITY MODEL
BUSINESS OWNER
TEAM
PROJECT
1:N
1:N
JOB NAME
1:N
(Depends on Identity
Manager)
1:N
OWNERSHIP
IDENTITY
Slide 29
Slide 29 text
BUSINESS OWNER
TEAM / Eng Manager
PROJECT
1:N
1:N
JOB NAME
1:N
(Depends on Identity
Manager)
1:N
OWNERSHIP
IDENTITY
INFRASTRUCTURE
Core Services
PinAndBoard
1:N
1:N
pin_writer_service
1:N
1:N
Slide 30
Slide 30 text
BUSINESS OWNER
TEAM / Eng Manager
PROJECT
1:N
1:N
JOB NAME
1:N
(Depends on Identity
Manager)
1:N
OWNERSHIP
IDENTITY
INFRASTRUCTURE
Core Services
PinAndBoard
1:N
1:N
pin_writer_service
1:N
1:N
Slide 31
Slide 31 text
Need for canonical way
of identifying services
across infrastructure
Slide 32
Slide 32 text
pin_write_service
BLOB
STORAGE
_prefix=
_prefix=
COMPUTE
_cluster=
KEY/VAL
STORAGE
_namespace=
JOB NAME
1:N
IDENTIFIER
PER RESOURCE TYPE
CANONICAL JOB
IDENTIFIER
Slide 33
Slide 33 text
Canonical identifiers for a job
Identifying a job across platform/infrastructure
services.
COMPUTE
BLOB
STORAGE
KEY/VAL
STORAGE
foo_service
_cluster=
_namespace=
IDENTITY PROVISIONING SERVICE
_prefix=
IDENTITY
MANAGER
Slide 34
Slide 34 text
BUSINESS OWNER
TEAM / Eng Manager
PROJECT
1:N
JOB NAME
(Depends on Identity
Manager)
1:N
OWNERSHIP
IDENTITY
INFRASTRUCTURE
Core Services /
Manager Name
PinAndBoard
1:N
pin_writer_service
1:N
Service Directory V1
Slide 35
Slide 35 text
$ curl https://identity-api/api/v1/identifiers?name=pinalytics-thrift
QUERY
Who owns identifier pinalytics-thrift?
Slide 36
Slide 36 text
$ curl https://identity-api/api/v1/identifiers?ownerUsername=manager
QUERY
Show all identifiers owned by EM=
Slide 37
Slide 37 text
Where to look up job
metadata/configuration?
Slide 38
Slide 38 text
Key/Val pairs tied to Jobs & Projects following
an hierarchical order
Source of truth for Job Metadata
METADATA
MANAGER
KEY/VAL
KEY/VAL
BUSINESS OWNER
TEAM
PROJECT
1:N
1:N
JOB NAME
1:N
(Depends on Identity &
Credential Manager)
1:N
OWNERSHIP
IDENTITY
Slide 39
Slide 39 text
How to inventory &
allocate infra resources?
Slide 40
Slide 40 text
So, what resources can I use?
Inventorying and provisioning of resources
across platform/infrastructure services.
RESOURCE
MANAGER
Define resources to offer:
- Online Compute
- Storage
- Batch Compute
Abstract resource provisioning by providing a
workflow to provision resources
- Allows policies (ex: < 100 vCPU free to launch)
- Tie to identity system
Slide 41
Slide 41 text
So, what resources can I use?
Inventorying and provisioning of resources
across platform/infrastructure services.
RESOURCE
MANAGER
COMPUTE
BLOB
STORAGE
KEY/VAL
STORAGE
foo_service
CPU
MEMORY
DISK
STORAGE IN GB
GETS
PUTS
STORAGE IN GB
WPS
RPS
RESOURCE
PROVISIONING
SERVICE
INFRA
(CLOUD/DC)
IDENTITY
PROVISIONING
SERVICE
Slide 42
Slide 42 text
$ curl https://resource-api/api/v1/offerings
QUERY
Show all the available infrastructure resources
Slide 43
Slide 43 text
$ curl https://resource-api/api/v1/measures?infrastructureName=Monarch&offeringName=Spark
QUERY
Show the resources, I will be charged for, for the
Spark offering in Monarch
Slide 44
Slide 44 text
How to meter utilization
of resources & attribute
cost?
Slide 45
Slide 45 text
METER
&CHARGEBACK
How much am I using?
Ability to meter allocation and utilization of
resources per service, per engineering team
and charge them accordingly
Enables Visibility & Accountability
Metering across Infrastructure requires
standard `schema`
- ts (timestamp)
- identifier
- infrastructure
- resource
- utilization
Leverage internal visibility/observability stack
Unit price definition per resource can difficult.
Slide 46
Slide 46 text
$ curl https://chargeback-api/api/v1/bills/monthly?month=05&year=2017&ownerUsername=manager
QUERY
What is ’s infrastructure footprint
(utilization & spend) for the month of May 2017?
Slide 47
Slide 47 text
SERVICES ON COMPUTE VM (EC2) - STACKED BY SERVICE IDENTIFIERS
VCPU-DAYS
Slide 48
Slide 48 text
SERVICES ON COMPUTE VM (EC2) - STACKED BY ENG MANAGERS
VCPU-DAYS
Slide 49
Slide 49 text
Why is this important?
Slide 50
Slide 50 text
focus #1
Simplify
E2E Dev XP
What are the steps a developer is required (but not expected) to do when
building, launching & managing services, batch jobs, etc.?
Slide 51
Slide 51 text
focus #2
An integrated
Infra Platform
What is required to build a reliable, scalable, efficient & well integrated
infrastructure platform?
Slide 52
Slide 52 text
focus #3
Infra
Governance
Without hampering developer experience and adding opswork,
What controls are required to effectively utilize & manage Infrastructure