Tweet
Tweet

Slide 1

Slide 1 text

1

Slide 2

Slide 2 text

3 Why 2.0 is fast? Core tools/software in 2.0 1 2 Dev Culture at CircleCI 3 New Features in 2.0 4 5 Agenda QA

Slide 3

Slide 3 text

4 Why 2.0 is fast? 2.0 is blazing fast

Slide 4

Slide 4 text

5 Basic Architecture API Billing Webhook Usage Job Dispatcher gRPC Server Workflows Nomad Docker Container Build Agent Test Results 1.0/2.0 Services 2.0 Services Git Push

Slide 5

Slide 5 text

6 Build Agent ● Core component of 2.0 ● Written in Go ● gRPC and Asynchronous Build Execution ● Running on Docker

Slide 6

Slide 6 text

7 ● Support Multiple Languages ● Message Streaming ● Asynchronous RPC call ● SSL/TSL communication ● Many many many features…. No More SSH Asynchronous Build Execution With gRPC High Performance RPC framework

Slide 7

Slide 7 text

8 Basic Architecture API Billing Webhook Usage Job Dispatcher gRPC Server Workflows Nomad Docker Container Build Agent Test Results 1.0/2.0 Services 2.0 Services Git Push

Slide 8

Slide 8 text

9 Build Execution: 1.0 vs 2.0 1.0 2.0

Slide 9

Slide 9 text

10 SSH is everywhere in 1.0 Host Machine Container Step 1: ssh ubuntu@container1 bundle install Step 1: Exit Code & Output Step 2: Exit Code & Output Step 2: ssh ubuntu@container1 rspec Synchronous Build Step Execution

Slide 10

Slide 10 text

11 gRPC in 2.0 Asynchronous Build Step Execution gRPC Server Build Agent Compiled Build Steps: step 1, step 2 Step 1: Exit Code & Output Step 2: Exit Code & Output

Slide 11

Slide 11 text

12 ● Pre-install packages ● No more apt-get install ● Creative Caching Custom Image & Flexible Caching

Slide 12

Slide 12 text

13 Custom Image and Caching 1.0 2.0

Slide 13

Slide 13 text

14 Other improvements ● Inference daemon is gone ● Remove extra tasks when starting container ● The performance of each step is mostly the same ● Proper resource allocation

Slide 14

Slide 14 text

15 Result: google/guava 1.0 17:43s https://circleci.com/gh/marcomorain/guava/38 2.0 7:41s https://circleci.com/gh/marcomorain/guava/48

Slide 15

Slide 15 text

16 Core Tools/Software in 2.0 Tools/software we love in 2.0

Slide 16

Slide 16 text

17 ● Leading Linux Container Platform ● Replacing LXC in CircleCI 2.0 ● Production ready ● Very active development ● Docker is not slow! Leading Linux Container Platform Docker The leading container platform

Slide 17

Slide 17 text

18 Basic Architecture API Billing Webhook Usage Job Dispatcher gRPC Server Workflows Nomad Docker Container Build Agent Test Results 1.0/2.0 Services 2.0 Services Git Push

Slide 18

Slide 18 text

19 LXC vs Docker 1.0 (LXC) 2.0 (Docker)

Slide 19

Slide 19 text

20 Things We Learned About Docker Performance Docker is not slow Active Development Always use the latest Kernel Container Isolation Cgroup is not perfect

Slide 20

Slide 20 text

21 Things We Learned About Docker Active Development ● Always use the latest kernel ● Performance and stability issues with old kernels Container Isolation ● Using cgroup doesn’t guarantee resource isolation ● We are working on better I/O performance ● https://sysdig.com/blog/container-isolation-gone-wrong/ Performance ● Docker is not slow ● I/O penalty in CoW is small ● CircleCI DB images with optimizations

Slide 21

Slide 21 text

22 Revenge of VMs ● VMs are used for machine executor in 2.0 ● VMs are getting cheaper ● Also faster: 45 ~ 60 secs to spin up a VM in GCP ● Full control over hardware resource

Slide 22

Slide 22 text

24 ● Simple Architecture ● Still beta but reliable Job Scheduler in 2.0 Nomad Easy Deploy Applications At Any Scale

Slide 23

Slide 23 text

25 Basic Architecture API Billing Webhook Usage Job Dispatcher gRPC Server Workflows Nomad Docker Container Build Agent Test Results 1.0/2.0 Services 2.0 Services Git Push

Slide 24

Slide 24 text

26 Nomad: Job Scheduler in 2.0 ● All builds are Nomad jobs ● Simple architecture ● Still pre-1.0 but stable ● Sometimes backward compatibility is broken, though...

Slide 25

Slide 25 text

27 ● Backbone for microservices in 2.0 ● Rolling updates of services Breaking monolith into microservices Kubernetes Production-Grade Container Orchestration

Slide 26

Slide 26 text

28 Breaking Monolith into Microservices CircleCI Monolith Microservices Kubernetes

Slide 27

Slide 27 text

29 Advantages and Disadvantages in Microservices Advantages Disadvantages

Slide 28

Slide 28 text

33 New Features in 2.0 Don’t miss them!

Slide 29

Slide 29 text

34 jobs: build: docker: - image:quay.io/project/image:tag auth: username: $QUAY_USERNAME password: $QUAY_PASSWORD Sample .circleci/config.yml Private Images ● Support Private Repositories ● DockerHub, GCR, ECR, Quay.io and more

Slide 30

Slide 30 text

35 ● Missing feature in 2.0 ● Filter job execution by tag workflows: version: 2 build-workflow: jobs: - build: filters: tags: ignore: /^testing-.*/ Sample .circleci/config.yml Git Tag Support

Slide 31

Slide 31 text

36 ● Creating sets of env vars in the organization page ● Share the env vars among projects workflows: version: 2 my-workflow: jobs: - run-tests: context: org-global Sample .circleci/config.yml Contexts

Slide 32

Slide 32 text

37 ● Running macOS builds in 2.0 infrastructure ● Workflow support ● We want beta customers!! jobs: build: macos: xcode: version: 8.3.3 Sample .circleci/config.yml macOS in 2.0 (beta)

Slide 33

Slide 33 text

38 ● Enterprise in 2.0 was just out! ● Only supporting AWS at the moment 2.0 in Enterprise (AWS)

Slide 34

Slide 34 text

Thank you. 39