Build and deploy cloud native Java microservices on Kubernetes with Istio service mesh

Istio is one of the most popular services mesh solutions available these days. In this deep dive session, let us see how to build and deploy Java microservices to the cloud using Istio, Kubernetes, JHipster, and Spring Cloud.

Istio moves the responsibility of service discovery, load balancing, circuit breaking, monitoring, etc from the application to the platform(Kubernetes) thus letting the developers focus on the business code. But setting it up could be a daunting task for beginners and pros alike. In this session, we will take a deep dive into how Istio works and will learn how to use different features of istio, like load balancing, canary deployments, A/B testing, and so on.

What you will learn:

What is a service mesh, how does Istio work, and what features does it offer
How to build Java microservices quickly using JHipster
How to add Istio support to your Java microservices
Setup Istio on a Kubernetes installation (for example on Google Cloud)
Setup observability and monitoring for the Kubernetes + Istio cluster
Deploy microservices to Kubernetes
How to use different Istio features like load balancing, canary deployments, A/B testing, routing, and traffic splits
Security options for Java + Spring + Istio

Transcript

1. @deepu105
   @oktaDev
   Build and deploy cloud native
   Java microservices on Kubernetes
   with Istio service mesh
   Deepu K Sasidharan
   @deepu105 | deepu.tech
   

   View Slide

2. @deepu105
   @oktaDev
   Hi, I’m Deepu K Sasidharan
   ➔ JHipster co-lead developer
   ➔ Java Champion
   ➔ Creator of KDash, JDL Studio
   ➔ Developer Advocate @ Auth0 by Okta
   ➔ Polyglot dev, OSS aficionado, author, speaker
   @deepu105
   deepu.tech
   deepu105
   deepu05
   

   View Slide

3. @deepu105
   @oktaDev
   Cloud Native
   Microservices with a
   service mesh
   

   View Slide

4. @deepu105
   @oktaDev
   What does Istio do?
   ● Secure service-to-service communication
   ● Service discovery 
   ● Automatic load balancing 
   ● Routing, circuit breaking, retries, fail-overs, fault injection
   ● Policy layer for access control, rate limiting, A/B testing, canary rollouts,
   traffic splits, and quotas
   ● Automatic Metrics, logs, and traces 
   

   View Slide

5. @deepu105
   @oktaDev
   Istio architecture
   

   View Slide

6. @deepu105
   @oktaDev
   Prepare cluster
   

   View Slide

7. @deepu105
   @oktaDev
   Prerequisite - Kubernetes Cluster
   Create GCP Project :
   $ gcloud config set compute/region europe-west1
   $ gcloud config set compute/zone europe-west1-b
   $ gcloud projects create istio-demo - Enable billing via GUI
   $ gcloud config set project istio-demo
   $ gcloud services enable container.googleapis.com
   Create GKE Cluster :
   $ gcloud container clusters create hello-hipster \
   --num-nodes 4 \
   --machine-type n1-standard-2
   Set Credentials:
   $ gcloud container clusters get-credentials hello-hipster
   

   View Slide

8. @deepu105
   @oktaDev
   GKE Cluster
   

   View Slide

9. @deepu105
   @oktaDev
   Prerequisite - Istio
   Install Istio :
   $ cd ~/
   $ export ISTIO_VERSION=1.15.1
   $ curl -L https://istio.io/downloadIstio | sh -
   $ cd istio-$ISTIO_VERSION
   $ export PATH=$PWD/bin:$PATH
   

   View Slide

10. @deepu105
    @oktaDev
    Prerequisite - Istio
    Install Istio in GKE cluster :
    $ istioctl install --set profile=demo -y
    Get Ingress Gateway External IP:
    $ kubectl get svc istio-ingressgateway -n istio-system
    Note: The ingress IP is required later while deploying our apps
    

    View Slide

11. @deepu105
    @oktaDev
    Prerequisite - Addons
    Install Istio Addons in GKE cluster :
    $ cd istio-$ISTIO_VERSION
    $ kubectl apply -f samples/addons/grafana.yaml
    $ kubectl apply -f samples/addons/prometheus.yaml
    $ kubectl apply -f samples/addons/kiali.yaml
    $ kubectl apply -f samples/addons/extras/zipkin.yaml
    

    View Slide

12. @deepu105
    @oktaDev
    Istio pods
    

    View Slide

13. @deepu105
    @oktaDev
    Bootstrap a demo
    application with
    JHipster
    

    View Slide

14. @deepu105
    @oktaDev
    JHipster in the wild
    Most popular Rapid Application Development
    platform for Java web applications and
    microservices
    ▪ 20k+ stars, 40k projects and 600+ contributors on
    GitHub
    ▪ 2M+ installations & 20k+ app generations per month
    ▪ 300k+ overall users
    ▪ 300+ companies using JHipster
    ▪ 150+ plugins
    

    View Slide

15. @deepu105
    @oktaDev
    What can you actually do with JHipster?
    ▪ Generate simple monolith web applications
    ▪ Generate complete microservice architectures
    ▪ Generate domain model (entities)
    ▪ Generate CI/CD pipelines
    ▪ Deploy to the cloud or containers
    ▪ And lot more
    

    View Slide

16. @deepu105
    @oktaDev
    Code quality & developer experience
    ▪ Developer experience oriented
    ▪ Production grade code quality (Look at our sonar scores)
    −
    ▪ Best practices on any combinations selected
    ▪ Fully working CRUD apps out of the box
    ▪ Production grade security
    ▪ You would only have to do business logic code and UX
    design
    

    View Slide

17. @deepu105
    @oktaDev
    Let’s build
    

    View Slide

18. @deepu105
    @oktaDev
    Microservice with Istio on Kubernetes
    

    View Slide

19. @deepu105
    @oktaDev
    Install JHipster
    $ npm install -g generator-jhipster
    $ jhipster download microservice-ecommerce-store-istio-deployment.jdl
    or
    $ git clone https://github.com/oktadev/okta-java-spring-k8s-istio-microservices-example.git
    Note: I’m using JHipster version v7.9.3
    

    View Slide

20. @deepu105
    @oktaDev
    JDL
    JHipster Domain Language
    JDL : http://bit.ly/istio-jdl
    Reference : https://www.jhipster.tech/jdl/
    Studio : https://start.jhipster.tech/jdl-studio/
    Samples : https://github.com/jhipster/jdl-samples
    

    View Slide

21. @deepu105
    @oktaDev
    JDL Application
    

    View Slide

22. @deepu105
    @oktaDev
    JDL Entity & Relationship
    

    View Slide

23. @deepu105
    @oktaDev
    JDL Deployment for Kubernetes with Istio
    

    View Slide

24. @deepu105
    @oktaDev
    JHipster magic
    $ jhipster jdl jhipster-istio.jdl --fork
    

    View Slide

25. @deepu105
    @oktaDev
    Build & Push images
    $ ./gradlew bootJar -Pprod jib -Djib.to.image=deepu105/store
    $ ./gradlew bootJar -Pprod jib -Djib.to.image=deepu105/invoice
    $ ./gradlew bootJar -Pprod jib -Djib.to.image=deepu105/product
    $ ./gradlew bootJar -Pprod jib -Djib.to.image=deepu105/notification
    

    View Slide

26. @deepu105
    @oktaDev
    Istio sidecar injection
    

    View Slide

27. @deepu105
    @oktaDev
    Manual sidecar injection
    Command:
    $ istioctl kube-inject -f kubernetes/my-deployment.yaml
    Result:
    spec:
    containers:
    - image: deepu105/my-app:1.0
    ...
    - image: docker.io/istio/proxyv2:1.15.1
    name: istio-proxy
    ...
    initContainers:
    ...
    - image: docker.io/istio/proxyv2:1.15.1
    name: istio-init
    ...
    

    View Slide

28. @deepu105
    @oktaDev
    Automatic sidecar injection
    Command:
    $ kubectl label namespace istio-injection=enabled
    --overwrite=true
    # To verify
    $ kubectl get namespace -L istio-injection
    

    View Slide

29. @deepu105
    @oktaDev
    Istio Traffic Management
    

    View Slide

30. @deepu105
    @oktaDev
    Istio Virtual Services
    apiVersion: networking.istio.io/v1beta1
    kind: VirtualService
    metadata:
    name: store-virtualservice
    namespace: jhipster
    spec:
    hosts:
    - store
    http:
    - route:
    - destination:
    host: store
    subset: "v1"
    weight: 100
    retries:
    attempts: 3
    perTryTimeout: 2s
    

    View Slide

31. @deepu105
    @oktaDev
    Istio Destination Rules
    apiVersion: networking.istio.io/v1beta1
    kind: DestinationRule
    metadata:
    name: store-destinationrule
    spec:
    host: store
    trafficPolicy:
    loadBalancer:
    simple: RANDOM
    connectionPool:
    tcp:
    maxConnections: 30
    connectTimeout: 100ms
    http:
    maxRetries: 5
    outlierDetection:
    consecutive5xxErrors: 5
    interval: 30s
    baseEjectionTime: 60s
    subsets:
    - name: v1
    labels:
    version: "v1"
    

    View Slide

32. @deepu105
    @oktaDev
    Istio Gateways
    apiVersion: networking.istio.io/v1beta1
    kind: Gateway
    metadata:
    name: store-gateway
    labels:
    gateway: store-gateway
    spec:
    selector:
    istio: ingressgateway
    servers:
    - port:
    number: 80
    name: http
    protocol: HTTP
    hosts:
    - store.jhipster.35.240.66.252.nip.io
    - port:
    number: 80
    name: http2
    protocol: HTTP2
    hosts:
    - store.jhipster.35.240.66.252.nip.io
    

    View Slide

33. @deepu105
    @oktaDev
    Istio Service Entries
    apiVersion: networking.istio.io/v1alpha3
    kind: ServiceEntry
    metadata:
    name: svc-entry
    spec:
    hosts:
    - ext-svc.example.com
    ports:
    - number: 443
    name: https
    protocol: HTTPS
    location: MESH_EXTERNAL
    resolution: DNS
    

    View Slide

34. @deepu105
    @oktaDev
    Istio Sidecars
    apiVersion: networking.istio.io/v1alpha3
    kind: Sidecar
    metadata:
    name: default
    namespace: bookinfo
    spec:
    egress:
    - hosts:
    - "./*"
    - "istio-system/*"
    

    View Slide

35. @deepu105
    @oktaDev
    Break (30 mins)
    

    View Slide

36. @deepu105
    @oktaDev
    Code walkthrough
    

    View Slide

37. @deepu105
    @oktaDev
    Lets deploy
    

    View Slide

38. @deepu105
    @oktaDev
    Deploy to GKE
    $ cd kubernetes
    $ ./kubectl-apply.sh -f
    $ watch kubectl get pods -n jhipster // or use KDash 😉
    

    View Slide

39. @deepu105
    @oktaDev
    Gateway
    

    View Slide

40. @deepu105
    @oktaDev
    Telemetry
    

    View Slide

41. @deepu105
    @oktaDev
    Grafana dashboard
    

    View Slide

42. @deepu105
    @oktaDev
    Prometheus
    

    View Slide

43. @deepu105
    @oktaDev
    Zipkin tracing
    

    View Slide

44. @deepu105
    @oktaDev
    Kiali
    

    View Slide

45. @deepu105
    @oktaDev
    Request Routing and Canary
    Testing
    

    View Slide

46. @deepu105
    @oktaDev
    Traffic shift
    apiVersion: networking.istio.io/v1beta1
    kind: DestinationRule
    metadata:
    name: store-destinationrule
    namespace: jhipster
    spec:
    host: store
    trafficPolicy:
    ...
    subsets:
    - name: v1
    labels:
    version: "v1"
    - name: v2
    labels:
    version: "v2"
    apiVersion: networking.istio.io/v1beta1
    kind: VirtualService
    metadata:
    name: store-gw-virtualservice
    ..
    spec:
    hosts:
    - store.jhipster.35.240.66.252.nip.io
    gateways:
    - store-gateway
    http:
    ..
    - route:
    - destination:
    host: store
    subset: "v1"
    weight: 50
    - destination:
    host: store
    subset: "v2"
    weight: 50
    

    View Slide

47. @deepu105
    @oktaDev
    Routing
    apiVersion: networking.istio.io/v1beta1
    kind: VirtualService
    metadata:
    name: store-gw-virtualservice
    ..
    http:
    ..
    - match:
    - uri:
    prefix: /
    headers:
    user-agent:
    regex: ".*Chrome.*"
    route:
    - destination:
    host: store
    subset: "v2"
    - match:
    - uri:
    prefix: /
    route:
    - destination:
    host: store
    subset: "v1"
    

    View Slide

48. @deepu105
    @oktaDev
    Fault Injection and Circuit
    Breaking
    

    View Slide

49. @deepu105
    @oktaDev
    Fault Injection
    apiVersion: networking.istio.io/v1beta1
    kind: VirtualService
    metadata:
    name: store-gw-virtualservice
    ..
    spec:
    hosts:
    - store.jhipster.35.240.66.252.nip.io
    gateways:
    - store-gateway
    http:
    - match:
    - uri:
    prefix: /
    route:
    - destination:
    host: store
    subset: "v1"
    fault:
    delay:
    percent: 100
    fixedDelay: 10s
    apiVersion: networking.istio.io/v1beta1
    kind: VirtualService
    metadata:
    name: store-gw-virtualservice
    ..
    spec:
    hosts:
    - store.jhipster.35.240.66.252.nip.io
    gateways:
    - store-gateway
    http:
    - match:
    - uri:
    prefix: /
    route:
    - destination:
    host: store
    subset: "v1"
    fault:
    abort:
    percent: 100
    httpStatus: 503
    

    View Slide

50. @deepu105
    @oktaDev
    Circuit Breaking
    apiVersion: networking.istio.io/v1beta1
    kind: DestinationRule
    metadata:
    name: store-destinationrule
    spec:
    host: store
    trafficPolicy:
    loadBalancer:
    simple: RANDOM
    connectionPool:
    tcp:
    maxConnections: 30
    connectTimeout: 100ms
    http:
    http1MaxPendingRequests: 10
    http2MaxRequests: 100
    maxRequestsPerConnection: 10
    maxRetries: 5
    outlierDetection:
    consecutive5xxErrors: 5
    interval: 30s
    baseEjectionTime: 60s
    apiVersion: networking.istio.io/v1beta1
    kind: VirtualService
    metadata:
    name: store-virtualservice
    namespace: jhipster
    spec:
    hosts:
    - store
    http:
    - route:
    - destination:
    host: store
    subset: "v1"
    weight: 100
    retries:
    attempts: 3
    perTryTimeout: 2s
    

    View Slide

51. @deepu105
    @oktaDev
    Security
    

    View Slide

52. @deepu105
    @oktaDev
    Security components
    ● A Certificate Authority (CA) for key and certificate management
    ● The configuration API server distributes to the proxies:
    ○ authentication policies
    ○ authorization policies
    ○ secure naming information
    ● Sidecar and perimeter proxies work as Policy Enforcement Points (PEPs) to
    secure communication between clients and servers.
    ● A set of Envoy proxy extensions to manage telemetry and auditing
    

    View Slide

53. @deepu105
    @oktaDev
    Security architecture
    

    View Slide

54. @deepu105
    @oktaDev
    Authentication
    ● Peer authentication: used for service-to-service authentication to verify the
    client making the connection. Istio offers mutual TLS as a full stack solution
    for transport authentication.
    ● Request authentication: Used for end-user authentication to verify the
    credential attached to the request. Enabled using JWT validation, a custom
    authentication provider or any OpenID Connect providers, for example:
    ○ Keycloak
    ○ Okta
    ○ Auth0
    ○ Firebase Auth
    ○ Google Auth
    

    View Slide

55. @deepu105
    @oktaDev
    Oauth 2 and OIDC
    https://napo.io/posts/istio-oidc-authn--authz-with-oauth2-proxy/
    

    View Slide

56. @deepu105
    @oktaDev
    oauth2-proxy with Istio
    From https://napo.io/posts/istio-oidc-authn--authz-with-oauth2-proxy/
    

    View Slide

57. @deepu105
    @oktaDev
    Extending with WebAssembly
    

    View Slide

58. @deepu105
    @oktaDev
    Istio extension ecosystem
    ● Istio Ecosystem Wasm Extensions
    ● Proxy-Wasm ABI specification
    ● Proxy-Wasm C++ SDK
    ● Proxy-Wasm Rust SDK
    ● Proxy-Wasm AssemblyScript SDK
    ● WebAssembly Hub
    ● WebAssembly Extensions For Network Proxies (video)
    

    View Slide

59. @deepu105
    @oktaDev
    Delete GKE Cluster
    $ gcloud container clusters delete hello-hipster
    

    View Slide

60. @deepu105
    @oktaDev
    Is a service mesh worth it?
    + Kubernetes Native microservices
    + Reduced responsibilities for dev(Service discovery, security, telemetry, etc)
    + No need to write/maintain any code for some of the complex parts of a
    microservice architecture.
    + A/B testing, canary releases, and lot more
    - Complex to debug and develop locally
    - Slightly higher resource usage (CPU, Memory)
    - Higher running costs
    - Business logic related policies might be trickier
    

    View Slide

61. @deepu105
    @oktaDev
    Thank You
    Deepu K Sasidharan
    @deepu105 | deepu.tech
    https://deepu.tech/tags#java https://developer.auth0.com
    

    View Slide