GitOps Guide to the Galaxy (Ep 14): Exploring CI with Tekton

Now that we've looked at other continuous integration (CI) tools, let's see how Tekton stacks up. Tekton is a young yet powerful CI tool for Kubernetes and OpenShift that's optimized for building and deploying cloud-native, microservice-based applications. When it comes to CI, there are lots of options, each with a slightly different set of features, so we'll look at why you'd want to use it and dive into the features Tekton offers.

About Tekton:
Tekton is a powerful and flexible open-source framework for creating CI/CD systems, allowing developers to build, test, and deploy across cloud providers and on-premise systems.
https://tekton.dev/

Follow us:
Christian Hernandez https://twitter.com/christianh814
Chris Short https://twitter.com/ChrisShort

What is GitOps Guide to the Galaxy?
Every other Thursday at 3am ET hosts Christian Hernandez and Chris Short sit down to discuss everything in the GitOps universe, from end-to-end CICD pipelines to creating Git workflows.

Subscribe to Red Hat's YouTube channel: https://www.youtube.com/redhat/?sub_confirmation=1

About OpenShift:
Red Hat OpenShift is an open source container application platform based on the Kubernetes container orchestrator for enterprise application development and deployment.
Openshift.com https://red.ht/33Fh6uV
OpenShift on github https://github.com/openshift/
OpenShift on redhat.com https://red.ht/2o4ccDk

Presentation on YouTube: https://youtu.be/EmtvRPRyTy8
#RedHat #GitOps #Kubernetes

Transcript

1. Tekton
   High overview about the engine behind OpenShift Pipelines.
   GitOps Guide to the Galaxy
   1
   Christian Hernandez
   Cloud Platforms BU
   

   View Slide

2. GENERAL DISTRIBUTION
   Why Cloud-Native CI/CD?
   WHAT IS CI/CD?
   2
   Traditional CI/CD Cloud-Native CI/CD
   Designed for Virtual Machines Designed for Containers and Kubernetes
   Require IT Ops for CI engine maintenance Pipeline as a service with no Ops overhead
   Plugins shared across CI engine Pipelines fully isolated from each other
   Plugin dependencies with undefined update cycles Everything lifecycled as container images
   No interoperability with Kubernetes resources Native Kubernetes resources
   Admin manages persistence Platform manages persistence
   Config baked into CI engine container Configured via Kubernetes ConfigMaps
   

   View Slide

3. GENERAL DISTRIBUTION
   Why Cloud-Native CI/CD?
   WHAT IS CI/CD?
   3
   Traditional CI/CD Cloud-Native CI/CD
   Designed for Virtual Machines Designed for Containers and Kubernetes
   Require IT Ops for CI engine maintenance Pipeline as a service with no Ops overheard
   Plugins shared across CI engine Pipelines fully isolated from each other
   Plugin dependencies with undefined update cycles Everything lifecycled as container images
   No interoperability with Kubernetes resources Native Kubernetes resources
   Admin manages persistence Platform manages persistence
   Config baked into CI engine container Configured via Kubernetes ConfigMaps
   

   View Slide

4. CONFIDENTIAL Designator
   4
   OpenShift
   Pipelines
   

   View Slide

5. GENERAL DISTRIBUTION
   5
   Governed by the Continuous Delivery Foundation
   Contributions from Google, Red Hat, Cloudbees, IBM, Pivotal and many more
   An open-source project for providing a set of shared and standard
   components for building Kubernetes-style CI/CD systems
   OPENSHIFT PIPELINES
   

   View Slide

6. GENERAL DISTRIBUTION
   6
   Tekton Concepts
   TEKTON CONCEPTS
   Step
   Run commands in a container
   with volumes, env vars, etc
   Task
   A list of steps that run
   sequentially in the same pod
   Pipeline
   A graph of tasks executed in
   a certain order
   Task Run
   An invocation of a task with
   inputs and outputs
   Pipeline Run
   An invocation of a pipeline
   with inputs and outputs
   Condition
   An check that can determine
   if a task should be executed
   Pipeline Resource
   Inputs and outputs to tasks
   and pipelines (git, image, etc)
   Catalog
   A collection of reusable tasks
   Triggers
   A Tekton sub-project to start
   pipelines based on events
   

   View Slide

7. GENERAL DISTRIBUTION
   Steps
   7
   ● Run command or script in a container
   ● Kubernetes container spec
   ○ Env vars
   ○ Volumes
   ○ Config maps
   ○ Secrets
   TEKTON CONCEPTS
   - name: build
   image: maven:3.6.0-jdk-8-slim
   command: [“mvn”]
   args: [“install”]
   - name: parse-yaml
   image: python3
   script:|-
   #!/usr/bin/env python3
   ...
   

   View Slide

8. GENERAL DISTRIBUTION
   Task
   8
   Task
   Step
   Step
   Step
   Step
   TEKTON CONCEPTS
   Example Tasks: Maven Install, AWS CLI, Kubectl Deploy, Security Scan, etc
   ● Defines a unit of work to be executed
   ● A list of steps to run sequentially
   ● Step containers run in the task pod
   ● Has inputs, outputs and parameters
   ● Workspaces and results for sharing data
   ● Can run independent of pipelines
   

   View Slide

9. GENERAL DISTRIBUTION
   9
   Maven Task
   Example
   kind: Task
   metadata:
   name: maven
   spec:
   params:
   - name: goal
   type: string
   default: package
   steps:
   - name: mvn
   image: maven:3.6.0-jdk-8-slim
   command: [ mvn ]
   args: [ $(params.goal) ]
   

   View Slide

10. GENERAL DISTRIBUTION
    TaskRun
    TEKTON CONCEPTS
    10
    ● Runs a Task to completion in a pod
    ● References or embeds a Task spec
    ● Provides input to Tasks
    ○ Parameters
    ○ Resources
    ○ Service account
    ○ Workspaces
    ● Contains execution status and metadata
    TaskRun
    step status step status
    Pod
    step status
    container container container
    Task
    step step step
    refers to
    creates
    

    View Slide

11. GENERAL DISTRIBUTION
    Pipeline
    11
    ● Define Tasks execution order (graph)
    ○ Serial
    ○ Parallel
    ● Inputs and parameters
    ● Retries tasks
    ● Conditional task execution
    ● Workspaces for sharing data between tasks
    ● Reusable across projects
    Pipeline
    Task
    Task
    Task
    Task
    TEKTON CONCEPTS
    

    View Slide

12. GENERAL DISTRIBUTION
    Sharing Data Between Tasks
    TEKTON CONCEPTS
    12
    Task: results
    ● Task exposes data as variables
    ● Suitable for small pieces of data
    ● Examples: commit id and branch name
    Task: workspaces
    ● Shared volumes between tasks
    ○ Persistent volumes
    ○ Config maps
    ○ Secrets
    ● Suitable for large data
    ● Examples: code, binaries, reports
    Pipeline
    Task Task
    /workspace/myw
    PersistentVolumeClaim
    

    View Slide

13. GENERAL DISTRIBUTION
    PipelineRun
    13
    TEKTON CONCEPTS
    PipelineRun
    ● Runs a pipeline to completion
    ● References or embeds a Pipeline spec
    ● Creates TaskRuns to execute Tasks in the Pipeline
    ● TaskRun pods may get scheduled on different node
    ● Provides inputs and params to pipeline
    ● Provides volumes for declared pipeline workspaces
    PipelineRun
    TaskRun
    Pod
    Pipeline
    creates
    TaskRun
    Pod
    TaskRun
    Pod
    Task Task Task
    

    View Slide

14. GENERAL DISTRIBUTION
    Triggers
    14
    TEKTON CONCEPTS
    Run pipelines based on events like HTTP webhooks on commit, pull request, etc
    EventListener
    TriggerTemplate
    webhook
    TriggerBinding
    How to map webhook JSON
    payload to variables
    What resources to create (e.g.
    PipelineRun) using the variables
    from payload mapping
    Interceptors
    PipelineRun
    Filter or modify webhook payload
    e.g. match branch or files changes
    

    View Slide

15. GENERAL DISTRIBUTION
    Task Catalog
    TEKTON CONCEPTS
    15
    ● Catalog of reusable Tasks
    ○ Image build: buildah, kaniko, jib, buildpacks, etc
    ○ Source-to-Image: Java, Python, Go, Ruby, etc
    ○ Language specific: maven, gradle, go, …
    ○ More to come soon
    ● Import and compose pipelines
    ● Available catalogs
    ○ tektoncd/catalog
    ○ openshift/pipelines-catalog
    ● Tekton Hub in beta!
    ○ https://hub-preview.tekton.dev/
    

    View Slide

16. GENERAL DISTRIBUTION
    OpenShift Pipelines Architecture
    TEKTON CONCEPTS
    16
    PipelineResource
    Pipeline
    Task Task
    Define pipeline
    Run pipelines
    Pipeline Controllers
    (Tekton, ext, ...)
    pipeline-pod-a
    pipeline-pod-b
    PipelineRun
    TaskRun TaskRun
    pipeline-pod-c
    

    View Slide

17. Let’s Explore!
    Hacking away at it until it works.
    GitOps Happy Hour
    17
    Keyboard time!
    

    View Slide