Pro Yearly is on sale from $80 to $50! »

What DevOps Is Really About?

What DevOps Is Really About?

This is a talk focused on various antipatterns in learning the DevOps culture, not just across companies but focused groups building on good software design.

53628fd2606be83e6343cba7917b3ef3?s=128

Bakare Emmanuel Oluwatimilehin

November 02, 2019
Tweet

Transcript

  1. None
  2. What is DevOps Really About? Bakare Emmanuel, DEIMOS ML @tiemmaBakare

  3. I’m Bakare Emmanuel • Experienced Pipeline Plumber @DEIMOSML • DevOps

    and Linux Fan Boy • You can follow me on twitter @TiemmaBakare • General Weird Guy with some humour • People call me Bakman, so there’s also that! @ T i e m m a B a k a r e 3
  4. How DevOps Started Branches of DevOps Tooling We Employ How

    Do I Famz Being One? What is This Talk About? Operating Systems Command Lines Storage Networking Virtualisation & Containers Orchestration Architecture Design
  5. How DevOps Started?

  6. Let’s start with origins. Back in 2007, a Belgian engineer

    named Patrick Debois became frustrated by the friction between developer and operations teams. As a developer and a member of the Agile community, Debois saw an opportunity to use Agile methodologies to manage infrastructure management process, much like developers manage development process. He initially described this concept as Agile Infrastructure but later coined the phrase DevOps, a portmanteau of development and operations. Source: https://www.oreilly.com/radar/theevolutionofdevops/
  7. DevOps isn’t a job, It’s a culture in which developers

    and Ops staff work with each other with mutual respect
  8. Apparently, we’re seen as gods in our foundations, the three

    eyed raven. Here’s what a DevOps Engineer Looks Like
  9. Branches Of DevOps

  10. - SysAdmins - NetOps - Cloud Ops - SecOps -

    NoOps - DataOps Branches Of DevOps - SRE - BizOps - HROps - MarketingOps - FinanceOps - Other Ops...
  11. Tooling We Employ?

  12. Tooling As opposed to alterior bias, a DevOps Engineer’s tooling

    differs based on their branch. A DevOps engineer focused on Storage might not build CNI plugins for Kubernetes to use it but he knows how to get flash and server disks hooked onto a mainframe. It’s a diverse playing field with a lot of knobs and buttons.
  13. Using Terraform or CI/CD doesn’t make you a DevOps Engineer,

    you’re still a Full Stack-OverFlow Engineer Google Much?
  14. How to famz being a DevOps Engineer?

  15. Famz - “To act in a way and appear as

    something else” To famz being a DevOps engineer these days, anyone who knows CI / CD or plays around with Docker becomes a DevOps Engineer overnight. Few know how Linux work and can explain it’s process lifecycle, or how Containers work and why you need a VM to run it on any non-UNIX system.
  16. Famz - “To act in a way and appear as

    something else” To become a DevOps engineer requires a lot of work! All this is not to scare you, but to make sure you understand the lifecycle process of becoming one. Now let’s get to the process of working towards becoming a proper DevOps engineer.
  17. How to famz being become a DevOps Engineer?

  18. Here’s what we’re covering in this chapter!

  19. Operating Systems A DevOps engineer should know his way around

    an operating system, irrespective of designation. This is because, every software you deploy can never always run on the same OS, knowing your way around the windows command interface is as crucial as writing bash scripts for Linux and OpenBSD.
  20. Operating Systems Every OS even Linux Distros work differently, some

    have features others don’t. There’s HyperV from Windows 7 but no ever looks that up. These and more others are reasons to learn the CORE of operating systems and how they work.
  21. You need to know your home before you start furnishing

    it with code.
  22. Command Lines & Scripting Sure, windows has fancy UI but

    you are meant to use the shell. Understand how to use command prompt, bash, powershell and know when a python script is what you need to accomplish the task.
  23. Storage When I say flash storage, it’s not the flash

    you use in your office but an actual SSD. When I say block storage, I mean a hard drive and not bricklayer blocks. When I say cloud storage, I mean a remote disk running on a protocol like NFS or iSCSI, not the one above you. When I say object storage, I mean things like buckets, S3, GCS These and many others are the things a DevOps engineer should know.
  24. Networking The foundations start from the OSI and scale up

    to learning things like BGP, GRE, VPC, Subnetting, IPTables, Port forwarding, Firewalling, Packet filtering, SDNs and various other things in Networking. Experience here is crucial if you see yourself going further.
  25. Virtualisation This is a way of sharing resources in parts.

    Ever heard of VMWare and Virtualbox, What about vCenter? Do you know about Hypervisors? You do know any resource can be virtualised? CPU, Disk, RAM, Networking etc. If you can use it on your computer, you can virtualise it on any server.
  26. Virtualisation / Authentication You can do pseudo identity virtualisation. We

    use things like service accounts, permissions and roles to do so by creating fake identities that can represent real people permissions. All of these and much more are how virtualisation works in the real sense of things.
  27. Containers What is a container runtime, how about sandboxing? Ever

    heard of cgroups, what about syscalls? How do containers even work, know what the OCI does? Did you know you can run VMs in Containers?, check out Kata container This and a lot more is what you learn on the way to DevOps.
  28. Orchestration Automation refers to a single task, orchestration arranges tasks

    to optimize a workflow. So orchestration is the process of “automating” automation.
  29. Orchestration and CI / CD CI - Continuous Integration (Push

    to Git, Start a Pipeline) CD - Continuous Deployment (Finish Pipeline, Push Update)
  30. Orchestration This is where we find the CI/CD Engineers, those

    who do docker builds and use Jenkins. Orchestration involves automating anything: servers, code deployments etc.
  31. Infrastructure As Code This is how we make things happen

    in orchestration. We write code to make things in this area. This is where we use things like Ansible, Puppet, Chef, Terraform.
  32. Infrastructure As Code Each has its specific application but the

    main idea is everything a DevOps engineer does should and can be automated.
  33. Architecture Design

  34. Architecture Design This is the main work of any DevOps

    Engineer, properly orchestrating any process needed to make a plan happen.
  35. What happens during an Architecture Design? Here we ask a

    lot of questions: What does the software do? Do we need to use a certain software architecture? How much load would the application get once deployed? And much more....
  36. Software Architecture This is just a way that we arrange

    things, servers, queues, things our software needs and much more.
  37. None
  38. DEVICE NOTIFICATION A separate notification service runs on the device

    which is responsible for checking a fixed list of notifications the user should get even under no internet access INTERNET PROBE NOTIFICATION SERVICE SERVER NOTIFICATION This aims to reduce the amount of code running on the device keeping only the bare minimal as more notifications increase runtime. DASHBOARD SERVER CONSUMER • Email • SMS (Twilio, SIM) • Firebase KAFKA CONTENT ON THE NEXT PAGE
  39. Aim of the Service • Checks the email, sms and

    push notification flags • Checks if the delay from the time a notification was first called has been exceeded - Ideally, two minutes to make sure it’s not a fluctuation triggering a notification • Also, makes sure the notification can again be reset after another set delay. This is an hour ideally and can be increased and decreased by us. • Email and Text messages details for the user are fetched from the DB for the server notifications and from the notifications.config file for the device side. - Reference(server): dbClient.py - Reference(device): notifications.config • Also, it diverts notifications message to sim text using the SIM 800 module in the case of no internet access. - Reference(device): SIM.py - Server side texts are handled by Twilio Logs for Notifications • The logs are text files used to hold info about the notifications called by a device. • There are currently 4 in use - Transition.log - Logs the current active source. - probeEnergy.log - This monitors the energy being drained from the inverter - activeNotifications.log - This is the log that holds the notifications that have been active - Op_window.log - For the operating window.
  40. IMPORTANT Software design isn’t something a DevOps engineer does per-se

    but it is the core that makes good software better. Knowing what to deploy and how to deploy it makes a DevOps engineer better as he can correctly outline what is needed to run with the most minimal resources applied.
  41. So what software architectures do we generally use?

  42. N-Tier Architecture (N = 1, 2, 3) This involves one,

    two, three parts of your application, all in order 1. Presentation Layer (Frontend e.g Vue) 2. Application Layer (Backend e.g Django, Flask, NodeJS) 3. Data Layer (Database, Persistence e.g Redis, Mongo, SQL)
  43. Single, 2-Tier and 3-Tier Architectures Source: https://www.guru99.com/dbms-architecture.html

  44. Monolith Says what it means, everything irrespective of the N

    number of tiers exists together. All parts of your application are bundled together and tightly integrated.
  45. Microlith (Bad Design!!!) A mix of a monolithic service with

    queues in the middle masquerading as a microservice. Pick a Monolith or a Microservice, Don’t Pick Both
  46. Microlith (Bad Design!!!) Source: https://dzone.com/articles/lagom-14-whats-new

  47. SPA - Backend & Frontend SPA - Single Page Application

    This is a form of software architecture where the backend and frontend are detached. Requests are made using one-way or streaming connections and several backend functionality can be emulated on the frontend by local caching and routing. Examples are front ends on (service workers, PWAs, web components) using backends on (websockets / http/2 / gRpc / APIs) etc
  48. Microservices This is basically a single service that does one

    thing and does it very well. So that means we can have a single service that does one thing. We could have a service for authentication, emails, transactions and whatever we feel needs to be singled out. It’s most preferred due to the fact that services can be rolled back and forward easily without directly bringing down the others. It’s also a lot more complex to orchestrate and requires fine understanding of the entire product. Directly - In the sense that an error there might bring down that service but the others that don’t need it work fine
  49. Event Driven Application This is a type of application which

    functions on a pub-sub way of receiving data. In this kind of system, we can have multiple people publishing data to a queue and many subscribers reading the same data at the same time. This allows for a many to many call to action kinda API request. We publish and subscribe by using topics or channels. Whatever you use to call them, it’s just a simple pipe. What ever you send to it can be read on the other side at about the instant it’s sent, similar to a water pipe.
  50. Event Driven Application Source: https://aws.amazon.com/pub-sub-messaging/ Multiple taps can read off

    the same pipe and many tanks can connect to one pipe.
  51. So what’s next? The best way to up your DevOps

    skills is to practice and learn. Follow up content on things you see in places like roadmap.sh and various blogs like DZone, Martin Fowler etc. The only shortcut to being a DevOps engineer is PRACTICE
  52. Thank you! Any Questions?