KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Carlos Camacho - Sept. 24th, 2020 - Red Hat 1

https://github.com/kubeinit/kubeinit 2 2 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Carlos Camacho Red Hatter Ph.D. in Computer Engineering Software Engineer Upstream OpenStack contributor Digging into the k8s ecosystem --- blog: www.anstack.com IRC (freenode): ccamacho slack (k8s): ccamacho GitHub: ccamacho Who am I?

https://github.com/kubeinit/kubeinit 3 3 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Intro

https://github.com/kubeinit/kubeinit 4 KubeInit provides Ansible playbooks and roles for the deployment and configuration of multiple Kubernetes distributions. The main goal of KubeInit is to have a fully automated way to deploy in a single command a curated list of prescribed architectures. About KubeInit KubeInit inherits some best practices from the OpenStack (TripleO) ecosystem 4 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys OpenStack is a an open source cloud operating system managing compute, storage, and networking resources throughout a datacenter using APIs.

https://github.com/kubeinit/kubeinit 5 5 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Why? Working in other research topics I had the necessity to easily deploy complex Kubernetes architectures, hitting the k8s learning curve, and having to deal with all the steps prior to the cluster deployment. Science? To be convincing, a scientific paper needs to provide evidence that the results are reproducible. - FIRE AND FORGET - This evidence might come from repeating the whole experiment independently several times, or from performing the experiment in such a way that independent data are obtained and a formal procedure of statistical inference can be applied — usually confidence intervals (CIs) or statistical significance testing.

https://github.com/kubeinit/kubeinit 6 6 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys When? I started to work on the cluster automation back in June 2018, for a research project called Pystol, where the installation/deployment project grew that much that it became its own project. Submitted to: IEEE Transactions on Cloud Computing Status: Under review Title: Chaos as a Software Product Line - A platform for improving open hybrid-cloud systems resiliency

https://github.com/kubeinit/kubeinit 7 7 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys The logo Do you remember, Fire and forget ? The logo mimics a ‘cubic’ kamehameha from the mythical Japanese anime TV series Dragon Ball. The Kamehameha is formed when the ki is concentrated into a single point for then releasing all the energy at once (our deployment command). The demo starts now!

https://github.com/kubeinit/kubeinit 8 8 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys The demo -- Steps -- Clone the repo: git clone https://github.com/Kubeinit/kubeinit.git cd kubeinit Run the playbook: time ansible-playbook \ --user root \ -v -i ./hosts/okd/inventory \ --become \ --become-user root \ ./playbooks/okd.yml

https://github.com/kubeinit/kubeinit 9 9 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys tripleo-ansible/openstack-ansible automatic documentation, roles skeleton, features distribution as roles tripleo-upgrade molecule tests architecture os-migrate linters architecture, unit tests structure, downstream CI automation tripleo-validations validations organization based on components Reduce, Reuse, and Refactor from the OpenStack ecosystem.

https://github.com/kubeinit/kubeinit 10 10 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Components

https://github.com/kubeinit/kubeinit 11 11 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Arch Infrastructure drivers k8s distribution Apache WS FreeIPA NFS HAProxy Bind9 ... Validations KubeVirt Supported services: ● Host OS: CentOS/Fedora, Debian/Ubuntu ● Infrastructure drivers: Libvirt ● Kubernetes distributions: OKD ● External services: Bind9, HAProxy, Apache WS, Validations

https://github.com/kubeinit/kubeinit 12 12 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Arch Infrastructure drivers k8s distribution Apache WS FreeIPA NFS HAProxy Bind9 ... Validations Supported services: ● Host OS: CentOS/Fedora, Debian/Ubuntu ● Infrastructure drivers: Libvirt + AWS ● Kubernetes distributions: OKD + Kubernetes ● External services: Bind9, HAProxy, Apache WS, Validations + FreeIPA + SDN + Offline deployments KubeVirt

https://github.com/kubeinit/kubeinit 13 13 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Docs docs.kubeinit.com ● Based in Sphinx (reStructuredText). ● Based in Read the Docs Sphinx Theme. ● Automatically render the docs from the roles and modules with an extension. ● Built on each merge. ● Merged in the gh-pages branch. ● Integrated in GitHub actions. ● Linted

https://github.com/kubeinit/kubeinit 14 14 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Roles ansible-playbook \ -i 'localhost,' \ role-addition.yml \ -e ansible_connection=local \ -e role_name=kubeinit-example Defaults Vars _skeleton_role_ Tasks Meta Molecule Docs

https://github.com/kubeinit/kubeinit 15 15 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys CI Unit tests EndToEnd tests Molecule tests Docs build tests Linters

https://github.com/kubeinit/kubeinit 16 16 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys CI Unit tests Molecule tests Docs build tests Linters ● Based in GitHub actions. ● They run on [push|pull_request]. ● Results in between 2 and 4 minutes. ● All the code in theory can be covered: ○ Ansible automation (functional): Molecule ○ Python modules: Unit test ○ Code style: Linters ○ Documentation: Build test

https://github.com/kubeinit/kubeinit 17 17 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys CI EndToEnd tests ● They run in an internal GitLab instance (runner). ● They run on demand by adding custom tags. ● The check pipeline runs each 15 minutes ● For security reasons they are outside GitHub ○ Users can run malicious code on PR, or when forking the code,so it runs externally triggered by tags. 1. Two simple scripts: ci/launch_e2e.py & ci/run.sh 2. A check pipeline executes launch_e2e.py each ~15 minutes (from GitLab). 3. If there is a PR with a custom tag i.e. okd-libvirt-deploy, then: a. Configure the job. b. Launch the job. c. Write back the result to the specific PR. How it works?

https://github.com/kubeinit/kubeinit 18 18 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Validations ● A role to: ○ Run pre deployment checks. ○ Run post deployment checks. ● Examples: ○ RAM availability. ○ Disk availability. ○ Endpoints check. ○ Nodes readiness. ○ Persistent storage status. ○ Test application deployments.

https://github.com/kubeinit/kubeinit 19 19 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Deep dive

https://github.com/kubeinit/kubeinit 20 20 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Net

https://github.com/kubeinit/kubeinit 21 21 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Net

https://github.com/kubeinit/kubeinit 22 22 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Bind External view DNS internal view

https://github.com/kubeinit/kubeinit 23 23 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Bind /etc/named/zones/internals/db.watata.kubeinit.local api.watata.kubeinit.local. IN A 10.0.0.100 api-int.watata.kubeinit.local. IN A 10.0.0.100 *.apps.watata.kubeinit.local. IN A 10.0.0.100 console-openshift-console.apps.watata.kubeinit.local. IN A 10.0.0.100 oauth-openshift.apps.watata.kubeinit.local. IN A 10.0.0.100

https://github.com/kubeinit/kubeinit 24 24 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys HAProxy Note: All the internal cluster traffic is routed directly ● openshift-api-server (port 6443) ● machine-config-server (port 22623) ● ingress-http (port 80) ● ingress-https (port 443)

https://github.com/kubeinit/kubeinit 25 25 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys 3rd-party Any 3rd party software should be deployed using k8s objects as additional roles. Currently integrated: KubeVirt

https://github.com/kubeinit/kubeinit 26 26 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Conclusions

https://github.com/kubeinit/kubeinit 27 27 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys Next steps/ Q&A

https://github.com/kubeinit/kubeinit 28 28 KubeInit: Bringing good practices from the OpenStack ecosystem to improve the way OKD/OpenShift deploys ● Find people interested in giving feedback, testing it, hacking it, raising issues, pushing PRs. ● Add other k8s distributions, like, plain HA kubernetes. ● Offline deployments (WIP). ● Improving testing coverage, unit tests, molecule tests. ● Roles refactor. ● Paths relative to the collection. ● Run the playbook from a container. ● Push mode -> PR mode. Next steps? Integration with other third party services, like, FreeIPA

Thank you! For more details, doubts, requests, contributions or anything you might need, please keep in touch! :) QUESTIONS ? Where to look: www.kubeinit.com docs.kubeinit.com https://github.com/kubeinit IRC (freenode): #kubeinit 29 If you liked this presentation or if you are interested in the project, please help us to give it more visibility by starring it! https://github.com/kubeinit/kubeinit blog: www.anstack.com IRC (freenode): ccamacho slack (k8s): ccamacho GitHub: ccamacho