VSHN – The DevOps Company Adrian Kosmaczewski, Developer Relations Integrating NGINX App Protect into DevOps work ow with Git May 20th, 2021 – 16:00 CEST Welcome to this presentation about integrating NGINX App Protect in your DevOps workflows! My name is Adrian Kosmaczewski, I’m in charge of Developer Relations at VSHN, the DevOps company, and I’ll be your host tonight. Joining me are Aarno Aukia and Sergio Nuzzo from VSHN, and also Vito Colletti, Daniel Schuler, Mario Klima and Susi Wernicke from F5, the company behind NGINX. Speaker notes 1

VSHN – The DevOps Company Pronounced ˈvɪʒn – like "vision" The DevOps Company Founded 2014, 40 VSHNeers located in Zürich Switzerland’s leading DevOps, Docker & Kubernetes partner 24/7 support ISO 27001 certi ed ISAE 3402 Report Type 1 veri ed First Swiss Kubernetes Certi ed Service Provider Just a few words about VSHN; that’s how you pronounce the name, and we’re "The DevOps Company". We’ve been in Zurich since 2014, we’re 40 engineers and we’re Switzerland’s leading DevOps, Docker & Kubernetes partner, offering 24/7 support to our customers. We’ve are certified in various ways, and most importantly, we were the First Swiss Kubernetes Certified Service Provider back in 2016. Speaker notes 2

VSHN – The DevOps Company We’re partners of many companies very active in the Cloud Native space, you might recognize some of the logos on this slide. Of all these, today we’re going to talk about NGINX extensively! We love NGINX and we and many of our customers use their products every day. Speaker notes 3

VSHN – The DevOps Company Last but not least, we run our own "Platform as a Service" offering called "APPUiO". We’ve created our own suite of tools to manage lots of Kubernetes services from a central location, called "Project Syn". Last but not least, we have developed our own Kubernetes operator for backups, called K8up, which just like Project Syn is 100% open source on GitHub. Speaker notes 4

VSHN – The DevOps Company DevOps → GitOps → DevSecOps NGINX App Protect Demo Questions & Answers Agenda Today we’re going to see how security can be incorporated in DevOps workflows. Then, how NGINX App Protect can be used to secure your systems from attackers. Finally, we’re going to see a short live demo: an integration of NGINX App Protect in an Exoscale SKS Kubernetes cluster. Please feel free to ask your questions in the Q&A box at the bottom of your Zoom window, and my colleagues and I will answer them at the end of the session. Let’s get started! Speaker notes 5

VSHN – The DevOps Company DevOps → GitOps → DevSecOps Given the current state of cybersecurity threats, DevOps has become DevSecOps and teams have had to increase the resiliency of their applications on the cloud. There are, however, far too many threats out there. Speaker notes 6

VSHN – The DevOps Company 1970s – 80s 1990s 2000s – Era Mainframes Client/server Cloud Technology COBOL, DB2 C++, OOP Java, web Cycle time Years Months Weeks Cost 1M – 100M 100K – 10M 10K – 1M At risk Company Division Feature Cost Bankruptcy CIO job Negligible There has been tremendous acceleration in the past decades, and a drastic drop in cost and risk. Adapted from Adrian Cockroft, "Velocity and Volume", FlowCon, San Francisco, 2013. Speaker notes 7

VSHN – The DevOps Company 1. The Principle of Flow 2. The Principle of Feedback 3. The Principle of Continual Learning and Experimentation The Three Ways of DevOps The first principle enables fast flow of work from dev to operations, making work visible, reducing batch size, and intervals of work; building quality in, and optimizing. The second principle enables faster detection, recovery, and learning from errors and mistakes, so that feedback loops are shortened continuously. The third principle supports the generation of a high- trust based culture that supports a scientific approach to experimentation and risk taking. Speaker notes 8

VSHN – The DevOps Company If you want to know more, these are the books at the base of the pyramid. Speaker notes 9

VSHN – The DevOps Company Infrastructure as Code locals { zone = "ch-gva-2" } resource "exoscale_sks_cluster" "demo" { zone = local.zone name = "demo" version = "1.20.5" description = "Webinar demo cluster" service_level = "pro" cni = "calico" addons = ["exoscale-cloud-controller"] } resource "exoscale_security_group" "sks_nodes" { name = "sks_nodes" description = "Allows traffic between sks nodes and public pulling of logs" } DevOps engineers use a variety of tools these days to manage their infrastructure as code. In this slide, for example, we have an example of a Terraform configuration file used to generate the Exoscale SKS cluster that I’ll be using later for the demo. Speaker notes 10

VSHN – The DevOps Company GitOps When Infrastructure as Code is stored in Git repositories, we start talking about GitOps; thanks to platforms such as GitHub and GitLab, developers can trigger the execution of CI/CD pipelines automatically every time they commit code to their repository. In this example, we can see the execution of a GitLab CI/CD pipeline. Speaker notes 11

VSHN – The DevOps Company 1. The Principle of Flow Continuously and proactively harden the security of apps 2. The Principle of Feedback Keeping an eye open on threats 3. The Principle of Continual Learning and Experimentation Through retrospectives and post-mortem reports DevSecOps (or DevGitSecOps?) How do the principles of DevOps apply in DevSecOps? Very straightforwardly, as it happens; through continuous monitoring, and learning of new threats, DevSecOps teams can increase proactively the security of their applications, through a flow of new security rules added to the system. The feedback loops goes to & from & with security specialists and developers and operators, all working together as a team. Speaker notes 12

VSHN – The DevOps Company NGINX App Protect is precisely a tool that can be used in a DevSecOps strategy to provide increased security to applications running in production, even when the source code of those applications is no longer available. Speaker notes 13

VSHN – The DevOps Company Very high performance Protects off-the-box from a long list of attack vectors Works well in containers → perfect for Kubernetes NGINX Ingress Controller can auto-detect routes And automatically add WAF rules to them Security as Code → easily stored in Git repositories NGINX is very popular and well known by engineers! Advantages There are several reasons why NGINX App Protect is a compelling product to consider in your DevSecOps strategy. First of all, NGINX is a popular product among engineers; it works in containers, making it perfect for Kubernetes; it has exquisite off-the-box integration with Kubernetes through the NGINX Ingress Controller; and it can be re-configured at runtime using Kubernetes objects. Speaker notes 14

VSHN – The DevOps Company Source: "High-Performance Application Security Testing" by GigaOM High Performance In our benchmarks, NGINX App Protect outperformed ModSecurity at all tested attack rates. NGINX App Protect produced 92% lower latency than NGINX running ModSecurity (…) NGINX App Protect produced 82% lower latency than AWS WAF at 1,000 tps on the 5% bad request test. www.nginx.com/resources/library/high-performance-app-security-testing In terms of performance, GigaOM has published a report comparing NGING App Protect to other WAFs: ModSecurity AWS Web Application Firewall In those tests, NGINX consistently delivered faster results; read the complete report for more information. Speaker notes 15

VSHN – The DevOps Company Attack Type Name Description Brute Force Attack Guessing passwords Buffer Over ow Data exceeds the size of buffers Denial of Service Exhausting app resources JSON, XML parser attack Attempting parser crashes Trojan, Backdoor, Spyware Malicious code XSS, SQL Injection Not sanitizing user input Full list: docs.nginx.com/nginx-app-protect/con guration/#attack-types What can NGINX App Protect do for you? Turns out, off the box, it can do a lot. There are more than 40 attack vectors supported by default in NGINX App Protect, including all of the OWASP top 10 vulnerabilities, plus much more; the full list is available in the link below the table. DevSecOps engineers can extend the reach and capabilities of NGINX App Protect using custom signature files, and we will see them in action in the demo. Speaker notes 16

VSHN – The DevOps Company 1 CentOS & RHEL 2 Debian & Ubuntu 3 Alpine Regular Updates $ sudo yum install app-protect-attack-signatures $ sudo apt-get install app-protect-attack-signatures $ sudo apk add app-protect-attack-signatures 1 2 3 NGINX regularly updates attack signatures, depending on the new threats that appear on the Internet; users can manually update them in their own installations of NGINX App Protect by typing the commands shown in this slide. Speaker notes 17

VSHN – The DevOps Company AWS Microsoft Azure Google Cloud Availability aws.amazon.com/marketplace/pp/Nginx-Inc-NGINX-Plus-with-NGINX-App- Protect-Develo/B08LD51Q9G azuremarketplace.microsoft.com/en- us/marketplace/apps/nginxinc.nginx_plus_with_nginx_app_protect_premium console.cloud.google.com/marketplace/product/nginx-public/nginx-plus-app- protect-ubuntu1804-premium?pli=1 NGINX Plus & NGINX App Protect are available with one- click at all major cloud providers, and also from OpenShift as an operator. Speaker notes 18

VSHN – The DevOps Company User DevSecOps NGINX Kubernetes GitLab Registry Deploy an unsecure application on Kubernetes Install NGINX App Protect Add a custom signature to prevent a new threat Demo In this demo we’re going to see how to use NGINX App Protect together with the NGINX Ingress Controller, and how we can identify a new threat (principle of feedback), learn more about it (principle of continuous learning) and add it to our setup (principle of flow). Speaker notes 19

VSHN – The DevOps Company 1 Download after creating your free 30-day trial account at 2 nap-plus in the Makefile rule speci es NGINX App Protect 3 Make repo public after pushing, or add pull secret to cluster Creating a Custom Ingress Container Image 1/2 $ export REPO_URL=repository/username/image $ export TAG=latest $ git clone https://github.com/nginxinc/kubernetes-ingress/ $ cd kubernetes-ingress $ git checkout v1.11.1 $ cp ~/Downloads/nginx-repo.crt . $ cp ~/Downloads/nginx-repo.key . $ make debian-image-nap-plus PREFIX=$REPO_URL TAG=$TAG $ docker push $REPO_URL:$TAG 1 2 3 www.nginx.com/free-trial-request The simplest mechanism to install NGINX App Protect in a Kubernetes cluster is through the NGINX Ingress Controller. You can build your own NGINX Ingress Controller image using your own nginx-repo.crt and ngnix-repo.key files, which you can download when you have an NGINX Plus subscription. You can sign up for a free trial of NGINX Plus and NGINX App Protect directly on the link shown at the bottom of the slide; you will be prompted to download both files immediately after. Place them at the same location as the Makefile and create your image. In this case, we’re creating a generic Debian image. There are Makefile rules for many other configurations, including OpenShift-compatibles images based on Red Hat Enterprise Linux (these require the corresponding RHEL license keys). Speaker notes 20

VSHN – The DevOps Company 1 Same repository URL and tag as speci ed previously 2 Enables App Protect in your controller Creating a Custom Ingress Container Image 2/2 apiVersion: k8s.nginx.org/v1alpha1 kind: NginxIngressController metadata: name: my-nginx-ingress-controller spec: type: deployment image: repository: $REPO_URL tag: $TAG pullPolicy: Always serviceType: LoadBalancer nginxPlus: True appProtect: enable: True 1 2 Once your image is built and pushed to a registry, specify the repository URL and tag in this YAML file and apply it to your cluster. If your image is stored in a private registry, remember to add the required pull secret so that Kubernetes can pull images from it. For the purposes of this demo, the image is publicly available on Quay.io. Speaker notes 21

VSHN – The DevOps Company 1 Toggle this to "False" to disable App Protect 2 NGINX Ingress requires a TLS con guration to work NGINX Ingress apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: fortune-ingress annotations: kubernetes.io/ingress.class: "nginx" appprotect.f5.com/app-protect-policy: "default/dataguard-alarm" appprotect.f5.com/app-protect-enable: "True" appprotect.f5.com/app-protect-security-log-enable: "True" appprotect.f5.com/app-protect-security-log: "default/logconf" appprotect.f5.com/app-protect-security-log-destination: "syslog:server=10.107.116.170:514" spec: tls: - hosts: - nginx-vshn.info secretName: fortune-secret rules: - host: nginx-vshn.info http: paths: - path: / backend: serviceName: fortune-svc servicePort: 80 1 2 This slide shows the configuration of the NGINX Ingress for our cluster, which will "wrap" all accesses to our application, and will filter out undesirable requests, logging that information to the syslog server configured. Once we have this configured, we can remove the previous endpoint for our application; in this case we’re going to leave it to show how the same application behaves differently depending on the ingress used. Speaker notes 22

VSHN – The DevOps Company But not even NGINX’s great security team thought about one of the greatest and latest threats: the Smurfs; also known as "Die Schlümpfe" auf Deutsch, "Les Schtroumpfs" en Français, "Los Pitufos" en Español, or "I Puffi" in Italiano. They have a known tendency to use the verb "Smurf" to indicate pretty much any activity, including, of course, hacking, as shown in the picture of this slide. They have been known to infiltrate systems and whenever Clumsy Smurf does it, you can be sure there’s going to be a system failure smurfing somewhere. Source of the image: Speaker notes www.lulu-berlu.com/the-smurfs- schleich-40249-smurf-with-computer-a47195-en.html 23

VSHN – The DevOps Company Source: www.toonpool.com/user/1688/ les/brainy_2110615.jpg In this declassified picture from the CIA archives you can see Smurfs learning the arts of hacking, guided by the patient and wise hand of Brainy Smurf. Source of the image: Speaker notes www.toonpool.com/user/1688/files/brainy_2110615.jpg 24

VSHN – The DevOps Company 1 To be used in the logs 2 Signature syntax reference: Creating Custom Signatures apiVersion: appprotect.f5.com/v1beta1 kind: APUserSig metadata: name: smurfs spec: signatures: - accuracy: medium attackType: name: Other Application Activity description: Medium accuracy user defined signature with tag (Smurfs) name: smurfs_medium_attack risk: medium rule: content:"smurf"; nocase; signatureType: request systems: - name: Microsoft Windows - name: Unix/Linux tag: Smurfs 1 2 techdocs.f5.com/kb/en- us/products/big-ip_asm/manuals/product/asm-bot-and-attack- signatures-13-0-0/7.html And here we can se DevSecOps & "Security as Code" at play! This custom signature will block any request with the word "smurf" in it (noun or verb), so that no Smurf can come and break havoc in your system. The rule shown in point 2 is written using a special syntax, whose syntax is defined and documented in the page shown below. Speaker notes 25

VSHN – The DevOps Company VSHN & NGINX NGINX App Protect Documentation OWASP Top Ten Risks Learn More vshn.ch/nginx www.nginx.com/products/nginx-app-protect docs.nginx.com/nginx-app-protect/admin-guide/install owasp.org/www-project-top-ten If you want to know more about NGINX App Protect, be sure to check these links for more information. Speaker notes 26

VSHN – The DevOps Company Interested? Contact us for a 30-day trial or an individual demo! More: – Slides: Adrian Kosmaczewski, Developer Relations – VSHN AG – Neugasse 10 – CH-8005 Zürich – +41 44 545 53 00 – – Thanks! vshn.ch/nginx bit.ly/nginx-app-protect-devops [email protected] vshn.ch [email protected] We hope that this presentation and its demo will be useful to you, and please do not hesitate to contact us if you need more information, a trial account, or a personalized demo for your team. Please download the PDF with these slides from the link shown on the screen, and learn more about what we do with NGINX in our website, . Thanks for watching, and beware of Smurfs! Speaker notes VSHN.ch/nginx 27