Share
Embed
Efficient DevOps Tooling with Java and GraalVM
Ops tooling has so far been the domain of shell scripts, interpreted languages like Python or statically compile languages like Go. But with the advent of GraalVM this situation has changed significantly. But behold: it is now possible to apply the power of the Java language and its ecosystem to your DevOps tooling problems and yet get optimal performance and efficiency by using GraalVM native images. In this session we will show that versatile 12-factor CLIs and powerful Kubernetes operators can be implemented in Java super easy in no time.
M.-Leander Reimer PRO
February 09, 2021
Transcript
26 slides
-
E ffi cient DevOps Tooling with Java and GraalVM
OOP Fachforen 2021 Digital, February 9th 2021 @LeanderReimer #cloudnativenerd #qaware -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital How do you organise and enable DevOps teams for fast fl ow and high productivity? 3 -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital Too much cognitive load will become a bottleneck for fast fl ow and high productivity. • Instrinsic Cognitive Load - relates to fundamental aspects and knowledge in the problem space (e.g. used languages, APIs, frameworks) • Extraneous Cognitive Load - relates to the environment (e.g. deployment, con fi guration, console commands) • Germane Cognitive Load - relates to speci fi c aspects of the business domain (aka. „value added“ thinking) 4 https://teamtopologies.com -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital Eliminate extraneous cognitive load Minimize intrinsic cognitive load 5 -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital 6 -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital Use the right language for the job!? 7 Getty Images Liliboas Ansible Shell Scripts Ruby Python -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital 8 -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital GraalVM to the Rescue! 9 -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital GraalVM in a Nutshell • Polyglot Runtime: runs all JVM languages, R, JavaScript, NodeJS, Ruby, Python, C/C++ via LLVM with full interop • Ahead-of-time (AOT) Compilation: memory management, thread scheduling via SubstrateVM • GraalVM as a Platform: embed and extend GraalVM with Tru ff l e, implement your own language and tools 10 -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital Polyglot Mayhem • The Graal Polyglot API allows you to embed and use different languages with full bidirectional interop. • This is not the same as with the Java Scripting API (JSR 223)! 11 private static void helloPython(PolyglotMessage message) { try (Context context = Context.newBuilder().allowAllAccess(true).build()) { context.getPolyglotBindings().putMember("message", message); context.eval("python", "import polyglot\n" + "message = polyglot.import_value('message')\n" + "message['invocations'] += 1\n" + "print(message['text'])"); } } -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital 12 Code & Demos https://github.com/qaware/hands-on-graalvm https://github.com/qaware/fast-fibonacci -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital The Swiss Army Knife of Operations. 13 CLIs - The Swiss Army Knife of Operations -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital The basics of 12-factor CLI apps • Great help is essential. What version am I on? • Prefer fl ags to positional arguments. • Mind the streams. stdout is for output, stderr is for messaging. • Handle things going wrong: error code, title, how to fi x, URL, … • Be fancy: use colours, have shell completion. • Prompt if you can. • Be speedy. CLIs need to start fast. • Be clear about subcommands. 14 For complete list and info, read https://medium.com/@jdxcode/12-factor-cli-apps-dd3c227a0e46 -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital Build CLIs with Picocli and GraalVM • Native DevOps tools, CLIs or sidecar containers can now also be build using Java! Golang is still cool. • Picoli is a small framework to easily build JVM command line apps. • Support for ANSI colors, tab completion, sub commands and other 12-factor CLI app principles • In-built support for GraalVM AOT compilation to native images via the ReflectionConfigGenerator utility and annotation processor. 15 -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital 16 Code & Demos https://github.com/lreimer/microj-cli https://github.com/lreimer/microj-picocli-graalvm -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital Container Orchestration Patterns 17 Sidecar Container Extended Container Behaviour • Log Extraction / Reformatting ( fl uentd, fi le beat) • Scheduling (cron, quartz) Ambassador Container Proxy Communication • TLS Tunnel (ghostunnel, Istio) • Circuit Breaking (linked, Istio) • Request Monitoring (linked, Istio) Adapter Container Standardized Ops Interfaces • Monitoring (Prometheus) • Con fi guration (Con fi gMaps, Secrets, …) -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital Use a multi-stage Docker fi le to build Linux binary 18 FROM ghcr.io/graalvm/graalvm-ce:20.3.0 AS builder # install native-image utility RUN gu install native-image && mkdir /hands-on-graalvm # copy files content and build native application WORKDIR /hands-on-graalvm COPY . . RUN ./gradlew build -x test && ./gradlew graalNativeImage FROM gcr.io/distroless/cc-debian10:debug # copy binary and required libraries into runtime image COPY --from=builder /hands-on-graalvm/build/hands-on-graal / COPY --from=builder /opt/graalvm-ce-java11-20.3.0/lib/libsunec.so / COPY --from=debian:10.2 /usr/lib/x86_64-linux-gnu/libz* /usr/lib/x86_64-linux-gnu/ COPY --from=debian:10.2 /lib/x86_64-linux-gnu/libz* /lib/x86_64-linux-gnu/ ENTRYPOINT ["/hands-on-graal"] CMD ["Hello World from GraalVM native inside Docker."] -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital 19 Operator. - Do stuff to my Kubernetes. -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital What are operators? • Operators are codi fi ed Ops procedures! • Operators are the path towards Zero-Ops. They enable auto-updating, self-monitoring and self-healing infrastructure and applications. • The concept was coined in the Kubernetes world. It’s now been adopted and used widespread in the cloud native world. • Examples: OKD, Sealed Secrets, Kube Monkey, Weave Flux 20 -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital Kubernetes Operators in a Nutshell 21 -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital The Kill Pod Operator 22 • Super simple Chaos monkey style operator inspired by Kubemonkey • Regularly kills pods of deployments that are killpod/enabled apiVersion: apps/v1 kind: Deployment metadata: name: nginx-killpod-enabled labels: killpod/enabled: "true" killpod/application: nginx-killpod-enabled killpod/delay: "30" killpod/amount: "2" spec: ... -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital The Super Secret Operator 23 • Apply asymmetrical encrypted secrets, the operator will decrypt and manage ordinary K8s secrets under the hood • Inspired by Sealed Secrets from Bitnami https://github.com/bitnami-labs/sealed-secrets apiVersion: operators.on.hands/v1alpha1 kind: SuperSecret metadata: name: supersecret-test spec: secretData: password: eV7YoQXyZlY+y51RWXEqyu0U44EPEPwEz+fZvGo+7McOTA4wQYCdxXMANtab3aW8 ... ywqpkHYtSLvrPgFnbcuSvD2UzuUNeE2qkh6SAM1z9Lpfwi+IUZjaY34Z+RjEL5OZFPYkQ== -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital The Microservice Operator 24 apiVersion: operators.on.hands/v1alpha1 kind: Microservice metadata: name: microservice-test labels: app: nginx spec: replicas: 2 image: nginx:1.17.6 ports: - containerPort: 80 serviceType: LoadBalancer • Abstracting the usual Deployment, Service and Con fi gMap de fi nitions using a simple and uni fi ed Microservice CRD -
// OOP 2021 Digital // Efficient DevOps Tooling with Java
and GraalVM // @LeanderReimer #cloudnativenerd #qaware #OOPdigital 25 Code & Demos https://github.com/qaware/graal-operators