VSHN – The DevOps Company Adrian Kosmaczewski, Developer Relations Managing Kubernetes Clusters as Cattle with Project Syn Cloud Native Romandie Meetup – Thursday, October 7th, 2021 Thanks everyone for attending this session, and thanks for inviting us to the Cloud Native Romandie Meetup today. My name is Adrian Kosmaczewski, I am Developer Relations at VSHN, the DevOps company, and I speak to you from the beautiful city of Zürich. I welcome you to this session from wherever in the world you happen to be right now. Speaker notes 1
VSHN – The DevOps Company • Pronounced ˈvɪʒn – like "vision" • The DevOps Company • Founded 2014, 46 VSHNeers located in Zürich • Switzerland’s leading DevOps, Docker & Kubernetes partner • 24/7 support • ISO 27001 certi ed • ISAE 3402 Report Type 1 veri ed • First Swiss Kubernetes Certi ed Service Provider Speaking about my team, just a few words about VSHN; that’s how you pronounce the name, and we’re "The DevOps Company". We’ve been in Zurich since 2014, we’re 46 VSHNeers (this is actually an important data point I’m going to refer to later) and we’re Switzerland’s leading DevOps, Docker & Kubernetes partner, offering 24/7 support to our customers. We’ve got a few certifications, and most importantly, we were the First Swiss Kubernetes Certified Service Provider back in 2016. Speaker notes 2
VSHN – The DevOps Company And last but not least, we run our own PaaS offering called "APPUiO", and even more important, we’ve created our own suite of tools to manage Kubernetes services called "Project Syn", which is exactly what I’m going to talk about today. Speaker notes 3
VSHN – The DevOps Company We’re partners of many companies very active in the Cloud Native space, you might recognize some of the logos on this slide. Speaker notes 4
VSHN – The DevOps Company Join Us! vshn.ch/en/jobs handbook.vshn.ch Oh, and did I mention that we’re hiring? Check out our Handbook to know more about us. Speaker notes 5
VSHN – The DevOps Company • The Evolution of Cloud Services • Introduction to Project Syn • Project Syn in Action: Live Demo! • Questions & Answers Agenda Today I’m going to very quickly review the evolution of the cloud in the past 25 years, and where we at VSHN think it is heading right now. Project Syn embodies our thoughts and ideas about that, and I’m going to explain to you what it is, how to use it, and how useful it can be. Finally, I’m going to show you a small demo of Project Syn in action. At the end of the session I’d gladly answer some questions from you. Speaker notes 6
VSHN – The DevOps Company • Perl • Java • VBScript • Objective-C (yes, WebObjects anyone?) • JavaScript (yes, Netscape Application Server!) • Lisp (only if you’re Paul Graham) Cloud Native == Web Applications We were also building "Cloud Native" applications; we just used to call them "Web Applications." They were quite monolithic; most of them built with Perl, VBScript, Java, maybe Objective-C if you were a lucky developer working with an early version of WebObjects, and even JavaScript if you were using Netscape’s own application server. Unless you were Paul Graham, in which case you were writing your web app in Lisp. Speaker notes 9
VSHN – The DevOps Company user Internet web db Servers as Pets Our servers had names. Remember that? What a weird concept nowadays! Those servers could bear boring names like "web", "db", or "assets". Speaker notes 10
VSHN – The DevOps Company user Internet Alderaan Hoth Servers as Pets If you were in a more imaginative environment, you would use names borrowed from Star Wars… Speaker notes 11
VSHN – The DevOps Company FTP We connected to those servers using some kind of remote desktop technology, or actually, way more often using FTP, or rcp if you were using BSD. If you were on the cutting edge, you might have used rsync, which was first released that year. Windows 95 users would happily use WS_FTP to handle those file tranfers. That is right: file transfers! We would sync the local folder in our local laptop with the folder in our server, and hope that we weren’t overwriting the changes from some other colleague. Speaker notes 13
VSHN – The DevOps Company CVS $ cvs checkout src/infrastructure $ cvs add new_component.c $ cvs commit -m "Added new component" $ cvs release src/infrastructure Some super advanced teams were using CVS to store changes – this is before Git, before Mercurial, before Subversion: CVS! Speaker notes 14
VSHN – The DevOps Company • Apache • IIS • Oracle • SQL Server Clusters? No! There were no clusters; you just had one Apache or IIS web server, and one database server running Oracle or SQL Server, and maybe another file server serving as CDN, and everybody was happy. Speaker notes 16
VSHN – The DevOps Company At best, you had a server rack, and hopefully it didn’t look like this. (Photo by Martijn Baudoin on Unsplash: ) Speaker notes unsplash.com/@martijnbaudoin 17
VSHN – The DevOps Company CVS → Subversion $ svn checkout http://svn.server.com/svn/project_repo $ cd project_repo/trunk/ $ svn status $ svn add new_component.c $ svn commit -m "Added new component" By that time, your CVS repository might as well have been updated to Subversion. You had to adapt your application to work in Firefox and Internet Explorer 6, which seems stuck in time. You use Prototype or jQuery to load some XML data live on the browser using this neat new technology called AJAX. Speaker notes 20
VSHN – The DevOps Company If your operations team was following the latest trends, your hardware servers might be virtualized using VMWare or some other technology. If you wanted to be in the cutting edge, maybe even running as a virtual EC2 instance on the beta EC2 program, which was launched in August of that year. The truth is that probably those virtual machines still had names, and you still connected to them directly; of course not with FTP, but most probably with scp or rsync. Speaker notes 21
VSHN – The DevOps Company Web 2.0 The spread of Web 2.0 technologies such as Ruby on Rails and Django meant that your applications required much larger runtimes to work; installing all the infrastructure required for those frameworks required lots of ssh connections and testing, to make sure your applications would run correctly. Speaker notes 22
VSHN – The DevOps Company namespace :deploy do desc 'Runs any rake task, cap deploy:rake task=db:rollback' task rake: [:set_rails_env] do on release_roles([:db]) do within release_path do with rails_env: fetch(:rails_env) do execute :rake, ENV['task'] end end end end end $ cap production deploy One of the offsprings of Ruby on Rails was a tool called "Capistrano" born in 2006, and for those who remember, it was absolutely ground-breaking: it allowed developers to specify the configuration of their application environment in a Ruby script, and then have the tool automatically sync the environment running the Ruby script on your server. But of course, Capistrano sill required you to give names to your servers. Speaker notes 23
VSHN – The DevOps Company user Internet lb web1 web2 db First Cluster This way you could have several instances of the application, all configured exactly the same, and then a quick NGINX instance could load balance requests among those instances. Neat, huh? You have got yourself your first cluster. Speaker notes 24
VSHN – The DevOps Company Much more interesting than Capistrano was the release of Puppet in 2005 (also written in Ruby, clearly the most popular language back then!) Speaker notes 25
VSHN – The DevOps Company case $operatingsystem { centos, redhat: { $service_name = 'ntpd' } debian, ubuntu: { $service_name = 'ntp' } } package { 'ntp': ensure => installed, } service { 'ntp': name => $service_name, ensure => running, enable => true, subscribe => File['ntp.conf'], } file { 'ntp.conf': path => '/etc/ntp.conf', ensure => file, require => Package['ntp'], source => "puppet:///modules/ntp/ntp.conf", } Puppet was different in many ways: the configuration files were not Ruby code, just pure text, which made them more generic, and accessible to non-developers. Puppet and Capistrano kicked-off the "configuration as code" movement that swept off the world of IT operations ever since. Speaker notes 26
VSHN – The DevOps Company So many things have happened in the meantime: GitHub, Stack Overflow, Google App Engine, the Go programming language, and Microsoft Azure were all relesed in annus mirabilis 2008. Speaker notes 29
VSHN – The DevOps Company Subversion → Git $ git clone [email protected]:akosma/component-fortune.git $ cd component-fortune $ git status $ git add new_component.c $ git commit -m "Added new component" $ git push Your good old Subversion server has been ported to Git, and this repository is probably hosted on GitHub or GitLab. Speaker notes 32
VSHN – The DevOps Company The new cool thing in 2016 is running your apps in containers (for example, using Docker Compose or Docker Swarm) or even better, in Kubernetes! This nifty container orchestrator was poised to take over the world of operations completely. Speaker notes 34
VSHN – The DevOps Company Kubernetes Cluster namespace-1 namespace-2 ingress svc1 pod1 pod2 db1 svc2 pod3 pod4 pod5 queue1 db2 user Internet Servers as Pets Cattle Very important, nobody gives names to your servers anymore. They have turned from pets into cattle. And your Kubernetes cluster is happily serving your users, and you have different namespaces for each of the applications you run in it. Your pods come and go, and only very rarely you need to know their names (unless for example if you need to kubectl logs into one of them, or something like that) Speaker notes 37
VSHN – The DevOps Company You are using GitHub actions or GitLab CI/CD pipelines to talk to your Kubernetes clusters, themselves configured through Terraform or Crossplane. Speaker notes 40
VSHN – The DevOps Company You don’t build your containers with Docker anymore, but with Podman, and you don’t store images in Docker Hub but in Quay.io or GitLab. Speaker notes 41
VSHN – The DevOps Company Thanks to AKS, EKS, GKE, Exoscale SKS, Red Hat OpenShift, and Rancher you have now various Kubernetes clusters; some for development, others for testing and quality control, and even some in the edge, running inside Raspberry Pi computers. You are running thousands of pods in hundreds of deployments, and keeping everything synchronized becomes a nightmare. IBM bought Red Hat and SUSE bought Rancher. Kubernetes is big business, measured in the hundreds of billions of dollars. And because of the current state of the world, your developers are all working from home or from Trinidad and Tobago, developing their apps and testing them locally in Minikube or K3s instances. Speaker notes 42
VSHN – The DevOps Company Clusters as Pets Cattle! But you still synchronize changes across Kubernetes clusters manually, and you even give names to those clusters. That’s cute, but as history shows, that doesn’t scale. Time for a change. Speaker notes 43
VSHN – The DevOps Company Kubernetes Has Won We at VSHN believe that Kubernetes has won. It is the de facto platform for Cloud Native applications, and the size and dynamics of its ecosystem are just exploding as we speak. These days, teams need more than just one Kubernetes cluster. A single namespace is just not enough. Teams need lots of clusters, for various reasons: testing, compliance, development, etc. And they need to bring them up and down very quickly, and to ensure lots of specific configurations and apps are synchronized across those pieces of infrastructure. Operators need to make sure that their clusters are compliant in terms of security, features, configuration, payloads, and so much more. Speaker notes 45
VSHN – The DevOps Company This is where Project Syn comes into play. Remember the beginning of this talk, when I mentioned that VSHN has only 46 VSHNeers? Well, only 30 of them are actual engineers, and they manage almost a 100'000 services in several thousand clusters spread all over the world, in almost all cloud providers you can imagine. This scale of work is simply unimaginable without strong principles and tight collaboration patterns. We built Project Syn to allow us to manage such a huge amount of services, providing 24/7 support to all of our clients. Speaker notes 46
VSHN – The DevOps Company user Internet cluster1 cluster2 cluster3 All of this means that at VSHN we breathe DevOps every day. All of our infrastructure, as well as that from our customers, is fully configured as code and stored in Git repos. Every change is tracked through GitHub pull requests and GitLab merge requests, so that many pairs of eyeballs could make sure that we are doing the right thing, and that we are doing the thing right. This also allows us to roll back changes in all security and confidence. Speaker notes 47
VSHN – The DevOps Company user Internet cluster1 cluster2 cluster3 cluster4 cluster5 cluster6 cluster7 And Project Syn extends this notion to support multiple Kubernetes clusters, anywhere in the world. Speaker notes 48
VSHN – The DevOps Company What is Project Syn? Project Syn is a set of tools helping to securely manage a eet of Kubernetes clusters. It brings a hierarchical con guration management based on GitOps principles, reusable components and an inventory of information about all Kubernetes clusters. Project Syn is a series of tools used to manage Kubernetes-at-large, or as I like to call it, "Kubernetes as Cattle". Speaker notes 49
VSHN – The DevOps Company • Multi-Cluster and Multi-Tenant Management • Full GitOps Operations • Tooling Bootstrapping • Con guration Management • Reusable Components • Default Toolset • Automatically Maintained Components • Secret Management • Service Provisioning Features Project Syn provides the following features. Speaker notes 50
VSHN – The DevOps Company • Lieutenant Operator • Lieutenant API • Steward • Commodore Main Components These are the main components of Project Syn: • Lieutenant Operator: Manages the CRDs Cluster, Tenant, GitRepo. Implements the business logic for handling all aspects around Git repositories for clusters and tenants. • Lieutenant API: REST API to expose the functionality of the Lieutenant Operator and to enforce some defaults. • Steward: In-cluster agent which bootstraps, configures and starts Argo CD (GitOps operator) and takes care of keeping the Syn cluster tools up and running. • Commodore: Configuration generator leveraging a hierarchical configuration model. Uses Kapitan under the hood, and can be extended with components. Each one of these projects is available in GitHub. Speaker notes 51
VSHN – The DevOps Company • Extension points • Written in Jsonnet • 50+ already published in GitHub • Commodore Component Hub: Commodore Components hub.syn.tools Commodore Components are the extension point of Project Syn. They allow you to customize and extend the number and variety of tasks that you can perform with Project Syn. Using Commodore Components, you can customize and apply changes to all of your clusters, ensuring conformity, coherence, alignment, and drastically reducing the workload for DevOps engineers. There are also lots of Commodore Components ready to use on GitHub; at the time of this writing, there are 50 components tagged commodore-component in the platform. Backup, security, networking, storage, you name it: there’s a Commodore Component for that! Commodore Components are written in a configuration language called Jsonnet, which extends JSON and compiles to other configuration languages such as TOML or YAML. Jsonnet is very simple to learn, and at first sight it looks like JSON with variables and reusability. We are planning the creation of a "Commodore Components Hub" to federate all of the documentation of all Commodore Components in a central location. Stay tuned for news! Speaker notes 53
VSHN – The DevOps Company Rancher Fleet CAOS ORBOS Similar Tools eet.rancher.io github.com/caos/orbos Of course, Project Syn is not alone in the race. Our two major competitors are excellent products, but even if they overlap a little, they have slightly different target use cases. The main goal of Fleet is to deploy applications to remote clusters, called the Fleet agents, whereas Project Syn focuses on generating configuration prepared for each cluster, depending on the properties of the destination cluster. Fleet implements its own GitOps model, rather than reusing existing tools like Flux or Argo CD like Project Syn does. Contrary to Project Syn, ORBOS focuses on the full lifecycle of Kubernetes clusters, whereas Project Syn doesn’t provide support for provisioning and managing Kubernetes Clusters themselves. Instead, Project Syn delegates cluster lifecycle management to specialized tooling and focuses on managing the content on a Kubernetes cluster. Speaker notes 54
VSHN – The DevOps Company 1. Minikube 2. K3s 3. Kind 4. K9s Demo Time! Show, don’t tell! This demo will show Project Syn in action. I have three Kubernetes clusters running in my own laptop in this precise moment: 1. Minikube, running the Lieutenant API and Operator & Steward 1. The Lieutenant API is exposed to the outer world using ngrok 2. K3s, running Steward 3. Kind, running Steward 1. Steward is monitoring Git repositories with all the configuration. I am using my own GitLab.com account to store those Git repositories. Speaker notes 55
VSHN – The DevOps Company In this demo, I will use Project Syn to deploy a simple application to both clusters at once. This application is specified as a Commodore Component. By compiling and pushing the catalog after adding the application, we will see it appear like magic in our clusters. We can remove it as simply, just by removing the reference to it, pushing our changes to Git, and recompiling our catalog. The Jsonnet files that make up my component are very simple, and they actually represent the structure of the YAML required to deploy my application on a cluster. They represent a Kubernetes namespace, a deployment, and a service. Once they are applied onto a cluster, the application they deploy is contained in a Docker image called "fortune-cookie-service". This Docker image encapsulates a very simple application written in Python, which returns a funny "fortune cookie of the day" type of message, and a random number. This app can be called from any web browser, or on the command line, using curl, as if it were a simple API. Speaker notes 56
VSHN – The DevOps Company • Main website: • GitHub: • Commodore Components Hub: • Chat: • Code of Conduct: • Syn Design Documents (SDDs): • Getting started: • Commodore Components tutorial: Call to Action syn.tools github.com/projectsyn hub.syn.tools community.appuio.ch/channel/projectsyn syn.tools/syn/about/code_of_conduct.html syn.tools/syn/SDDs/index.html syn.tools/syn/tutorials/getting-started.html syn.tools/tuto/index.html Project Syn is 100% open source, using the BSD 3- Clause "New" or "Revised" License. We even make our decision process open to everyone, through our Syn Design Documents or SDDs, which you can read online at the official website of Project Syn, . The demo I have shown is available online, so please check it out and try it out. It contains all the scripts required for you to verify the minimum requirements, and then to spin up your clusters with the least possible amount of work. Speaker notes syn.tools 57
VSHN – The DevOps Company • Lieutenant API: 0.8.0 (September 22nd) • Lieutenant Operator: 1.1.2 (August 25th) • Commodore: 0.10.0 (August 30th) • Steward: 0.5.0 (August 25th) Current Versions We’re actively working on Project Syn as we speak. The current versions of our tools at the time of this talk are the following: • Lieutenant API: 0.8.0 (September 22nd) • Lieutenant Operator: 1.1.2 (August 25th) • Commodore: 0.10.0 (August 30th) • Steward: 0.5.0 (August 25th) In spite of the "0.something" version numbers, Project Syn is already in production in VSHN since early 2020, and we are actively using it every day to deliver services to our customers. Please join our chat channel if you would like to use Project Syn in your infrastructure, and let us know how you are using it, and if you have any feedback we’d be glad to hear about it. Speaker notes 58
VSHN – The DevOps Company Adrian Kosmaczewski, Developer Relations – VSHN AG – Neugasse 10 – CH-8005 Zürich – +41 44 545 53 00 – – Thanks! [email protected] vshn.ch [email protected] I’d like to thank you all for having watched this presentation, and please do not hesitate to contact us if you have any questions about Project Syn. Speaker notes 59
akosma
mtsmfm
junjikoide
bonotake
yuhattor
asahihidehiko
sansantech
riru
motsuhiro131
kazumax55
ama_ch
matsuihidetoshi
mochico
hatefulcrawdad
holman
vanstee
hannesfritz
shpigford
scottboms
hursman
keathley
robhawkes
bluesmoon
ammeep
jensimmons