$30 off During Our Annual Pro Sale. View Details »

Cloud Native Ambassador Day - Extending Kubernetes

Cloud Native Ambassador Day - Extending Kubernetes

Extending Kubernetes: Abstract: When somebody asks me why Kubernetes got so much traction the answer is always the same: “its extendibility”. You can run it everywhere, almost all cloud providers or bare metal providers can deliver a managed or integrated distribution of Kubernetes. This is because it acts as a pluggable API gateway to manage your infrastructure across providers. As a developer or infrastructure operator having the ability to write the best integration your company or team needs is a huge plus. During this talk, Gianluca will show you the different integration points you can use to extend Kubernetes such as shared informers, controller, kubectl plugin. Concepts are generically applicable to Kubernetes, code will mainly be in Go.

Gianluca Arbezzano

October 14, 2020
Tweet

More Decks by Gianluca Arbezzano

Other Decks in Programming

Transcript

  1. Extending Kubernetes
    The superpower behind the Kubernetes API
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  2. apiVersion: batch/v1
    kind: Job
    metadata:
    name: hello
    spec:
    template:
    # This is the pod template
    spec:
    containers:
    - name: hello
    image: busybox
    command: ['sh', '-c', 'echo "Hello, Kubernetes!" && sleep 3600']
    restartPolicy: OnFailure
    # The pod template ends here
    Example coming from kubernetes.io
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  3. Kubernetes is a declarative framework
    If you use it just as end tool you are missing its real value

    View Slide

  4. Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  5. k-proxy
    kubelet
    sched
    sched
    sched
    Control Plane
    Node
    etcd
    Kubernetes cluster
    api
    api
    api
    c-c-m
    c-c-m
    c-c-m
    c-m
    c-m
    c-m
    Node Node
    k-proxy
    kubelet kubelet
    k-proxy
    Control plane
    Scheduler
    sched
    Cloud controller
    manager
    (optional) c-c-m
    Controller
    manager c-m
    kubelet
    kubelet
    kube-proxy
    k-proxy
    (persistence store)
    etcd
    etcd
    Node
    API server
    api
    from kubernetes.io
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  6. from SlideShare: Moby CRI Containerd
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  7. How you justify Kubernetes
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  8. Play your own game.
    That's why services have
    their own API.
    Gianluca Arbezzano -
    gianarb.it -
    twitter.com/gianarb

    View Slide

  9. You know yourself, your
    team and your product.
    Build around your
    requirement
    Gianluca Arbezzano -
    gianarb.it -
    twitter.com/gianarb

    View Slide

  10. Who am I?
    Software Engineer at Equinix
    Metal (Packet)
    Open Source maintainer for
    Kubernetes, Docker,
    TestContainer
    Docker Captain and CNCF
    Ambassador
    When not coding I grow
    vegetables
    I am active on Twitter as @gianarb

    View Slide

  11. Kubernetes follows the same rule, if kubectl is not enough
    because you have an
    half way workflow that you like, or the kubectl makes your
    solution hard to
    maintain, it is OK to build something by yourself.
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  12. There different ways to extend Kubernetes, here a couple:
    . A kubectl plugin.
    . Via Client-GO or any other SDK.
    . Custom Resource Definition.
    . Aggregation layer
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  13. Let's start with kubectl
    . It is by far the most flexible and easy
    way to
    extend Kubernetes. You have to deliver an executable in your
    $PATH
    that starts
    with kubectl-*
    .
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  14. Examples:
    A binary called kubectl-ns
    can be executed as kubectl ns
    A binary called kubectl-profefe
    can be executed as kubectl profefe
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  15. kubectl-profefe
    Profefe is an open source project to do continuous profiling of
    application
    using pprof, such as Golang.
    cron job
    application
    runtime/pprof
    net/http/pprof
    pull
    Storage
    developer
    profefe-collector
    query
    push
    GET
    /debug/pprof/profile
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  16. I wrote a project called
    profefe/kube-profefe that acts as a
    bridge between profefe and Kubernetes.
    . It serves a binary called kprofefe
    , it can run as Kubernetes cronjob and it
    collects profiles targeting application running in Kubernetes.
    . kubectl-profefe
    helps you to interact with profefe and Kubernetes.
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  17. $ kubectl profefe --help
    It is a kubectl plugin that you can use to retrieve and manage profiles in Go.
    Available Commands:
    capture Capture gathers profiles for a pod or a set of them. If can filter by namespace and via label selector.
    get Display one or many resources
    help Help about any command
    load Load a profile you have locally to profefe
    Flags:
    -A, --all-namespaces If present, list the requested object(s) across all namespaces
    --as string Username to impersonate for the operation
    --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
    --cluster string The name of the kubeconfig cluster to use
    --context string The name of the kubeconfig context to use
    -f, --filename strings identifying the resource.
    -h, --help help for kubectl-profefe
    --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
    --kubeconfig string Path to the kubeconfig file to use for CLI requests.
    -n, --namespace string If present, the namespace scope for this CLI request
    -l, --selector string Selector (label query) to filter on, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
    Use "kubectl-profefe [command] --help" for more information about a command.
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  18. Links about kubectl plugins:
    "My experience with Krew to manage kubectl plugins"
    "kubectl flags in your plugin"
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  19. Client GO and other SDKs
    Kubernetes API works as any other API. There are client libraries
    that you can
    use in many languages or it works via HTTP, and all the
    languages have an HTTP
    client available.
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  20. Not all the languages are the same
    There are official supported an unofficial libraries available out
    there.
    Go, Javascript, Haskell, Python, Java, Dotnet are supported by
    the Kubernetes
    community, you can check out the actual documentation.
    All of them are in different state, Golang is well done.
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  21. kprofefe
    the binary I told you about before uses the client-go
    library to
    do a couple of things:
    Retrieve pods filtered according to label section
    or/and per namespace
    in
    order to get the right targets for profefe
    It uses the annotations for a particular pod in order to figure out where the
    pprof server runs (the right port and path)
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  22. Get pods
    // GetSelectedPods returns all the pods with the profefe annotation enabled
    // filtered by the selected labels
    func GetSelectedPods(clientset kubernetes.Interface,
    namespace string,
    listOpt metav1.ListOptions) ([]v1.Pod, error) {
    target := []v1.Pod{}
    pods, err := clientset.CoreV1().Pods(namespace).List(listOpt)
    if err != nil {
    return target, err
    }
    for _, pod := range pods.Items {
    enabled, ok := pod.Annotations[ProfefeEnabledAnnotation]
    if ok && enabled == "true" && pod.Status.Phase == v1.PodRunning {
    target = append(target, pod)
    }
    }
    return target, nil
    }
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  23. Port forward programmatically
    func PortForwardAPod(req PortForwardAPodRequest) error {
    path := fmt.Sprintf("/api/v1/namespaces/%s/pods/%s/portforward",
    req.Pod.Namespace, req.Pod.Name)
    hostIP := strings.TrimLeft(req.RestConfig.Host, "htps:/")
    transport, upgrader, err := spdy.RoundTripperFor(req.RestConfig)
    if err != nil {
    return err
    }
    dialer := spdy.NewDialer(
    upgrader,
    &http.Client{Transport: transport},
    http.MethodPost,
    &url.URL{Scheme: "https", Path: path, Host: hostIP})
    fw, err := portforward.New(
    dialer,
    []string{fmt.Sprintf("%d:%d", req.LocalPort, req.PodPort)},
    req.StopCh,
    req.ReadyCh,
    req.Streams.Out,
    req.Streams.ErrOut)
    if err != nil {
    return err
    }
    return fw.ForwardPorts()
    }
    Repository: gianarb/kube-port-forward

    View Slide

  24. Custom Resource Definition (CRD)
    Terraform has the concept of modules. A Terraform module is a
    bridge between an
    external resource and Terraform. A CRD is the same, but with
    Kubernetes.
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  25. Custom Resource Definition (CRD)
    Native resources: Pod, Services, Deployment, StatefulSet,
    Ingress...
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  26. AWS Controllers for Kubernetes: Elastic Load Balancer, S3
    bucket, CloudFormation Stack.
    Checkout: ACK
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  27. View Slide

  28. client
    api-se
    kube
    crd-cont
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  29. Why you should write a CRD?
    Your team uses Kubernetes a lot, they know how to use the
    kubectl and client-go.
    You can make available for them external services like DNS
    management or a binary/image repository with an user
    experience they already know.
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  30. Kubernetes gives you a bunch of useful features that you can
    leverage:
    . Authentication
    . Authorization
    . Audit Logs
    . Event System
    . CLI (kubectl) and client libraries
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  31. Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide

  32. Thank you
    Any question?
    Reach out to me via Twitter @gianarb.
    Gianluca Arbezzano - gianarb.it - twitter.com/gianarb

    View Slide