How we built our Analytics platform with JHipster and OSS technologies Deepu K Sasidharan XebiaLabs JHipster 2018

2 About me Deepu K Sasidharan JHipster co-lead developer Senior product developer @ XebiaLabs OSS aficionado, author, speaker @deepu105 deepu.js.org

Have you heard about XebiaLabs?

4 About XebiaLabs dev test uat prod [Cloud] Orchestration Stack Middleware NoSQL PaaS Containers OS OS OS IaaS Network Servers DB / Storage Security SOFTWARE DEFINED DATA CENTER / CLOUD RELEASE ORCHESTRATION Backlog Management Provisioning/ Configuration Security ITSM / Service Management CMDB plan Project Management Issue Tracking ALM DEPLOYMENT AUTOMATION code SCM Code Analysis build Continuous Integration Centralized Repository test Test Tooling Test Visualization release ChatOps / Collaboration Email/ phone/ Excel operate BI / Monitoring Logging DEVOPS INTELLIGENCE

5 Release Orchestration

6 Deployment automation

Have you used JHipster?

8 About JHipster Most popular Rapid Application Development platform for Java web applications and microservices ▪ 11k+ stars on GitHub ▪ 1.3M+ installations & 20k+ app generations per month ▪ 250k+ overall users ▪ 450+ contributors & 23 core team members ▪ 250+ companies using JHipster ▪ 70+ plugins Hel s e s!

9 What can you do with JHipster ▪ Generate simple monolith web applications ▪ Generate complete microservice architectures ▪ Generate domain model (entities) ▪ Generate CI/CD pipelines ▪ Deploy to AWS, GCP, Heroku, CF ▪ Deploy to Docker, Kubernetes, Openshift, Rancher https://start.jhipster.tech/#/statistics

Demo time Let us create a microservice e-commerce application and deploy it using Docker or Kubernetes

11 The application architecture

How long would it take?

13 JDL JHipster Domain Language JDL: https://goo.gl/csHzbz Reference: https://www.jhipster.tech/jdl/ Studio: https://start.jhipster.tech/jdl-studio/

14 JDL Application

15 JDL Entity & Relationship

Start your timer

17 JHipster magic $ jhipster import-jdl app.jdl

18 Deployment or

XL Impact DevOps Intelligence and Analytics

20 ▪ Goal-based DevOps KPIs ▪ Data-driven Recommendations and Decision Support ▪ Predictive Analytics that Identify Trends and Highlight Risk ▪ One Control Panel for DevOps Impact and ROI XL Impact

21 Architecture

22 Release pipeline

23 How JHipster helped ▪ Quick bootstrap ▪ Fast POC(Easy prototyping) ▪ OOB integrations(Hazelcast, Gradle, Spring boot, ElasticSearch, K8S etc) ▪ Best practices from the start ▪ JDL entities

24 Challenges ▪ Hazelcast cache clustering with Kubernetes ▪ ElasticSearch write performance ▪ Kafka sync issues ▪ On demand Spark clusters on Kubernetes ▪ Orchestration of Infra, services and crawlers to multiple environments(single tenant architecture) ▪ ES high availability on Kubernetes(no dynamic volume resizing)

25 Solutions ▪ Hazelcast cache clustering with Kubernetes − Kubernetes discovery API ▪ ElasticSearch write performance − SSD + Memory + lot of swearing ▪ Kafka reliability issues − Moved to Google PubSub ▪ On demand Spark clusters on Kubernetes − Custom images with lot of tweaks ▪ Orchestration of Infra, services and crawlers − XL Release + XL Deploy ▪ ES high availability on Kubernetes − Still not implemented

Thank you XebiaLabs website https://xebialabs.com/ JHipster website https://jhipster.tech GitHub https://github.com/jhipster/generator-jhipster Twitter https://twitter.com/java_hipster JHipster online https://start.jhipster.tech