Heapcon: How we built our Analytics platform with JHipster and OSS technologies

This talk will focus on the experience of using JHipster in building the newest DevOps product from XebiaLabs. I will talk about the architecture, how JHipster is used and about some issues faced and how we overcome them.

Transcript

1. How we built our Analytics platform with JHipster and OSS technologies Deepu K Sasidharan XebiaLabs JHipster 2018
2. 2 About me Deepu K Sasidharan JHipster co-lead developer Senior product developer @ XebiaLabs OSS aficionado, author, speaker @deepu105 deepu.js.org
3. Have you heard about XebiaLabs?
4. 4 About XebiaLabs dev test uat prod [Cloud] Orchestration Stack Middleware NoSQL PaaS Containers OS OS OS IaaS Network Servers DB / Storage Security SOFTWARE DEFINED DATA CENTER / CLOUD RELEASE ORCHESTRATION Backlog Management Provisioning/ Configuration Security ITSM / Service Management CMDB plan Project Management Issue Tracking ALM DEPLOYMENT AUTOMATION code SCM Code Analysis build Continuous Integration Centralized Repository test Test Tooling Test Visualization release ChatOps / Collaboration Email/ phone/ Excel operate BI / Monitoring Logging DEVOPS INTELLIGENCE
5. 5 Release Orchestration
6. 6 Deployment automation
7. Have you used JHipster?
8. 8 About JHipster Most popular Rapid Application Development platform for Java web applications and microservices ▪ 11k+ stars on GitHub ▪ 1.3M+ installations & 20k+ app generations per month ▪ 250k+ overall users ▪ 450+ contributors & 23 core team members ▪ 250+ companies using JHipster ▪ 70+ plugins Hel s e s!
9. 9 What can you do with JHipster ▪ Generate simple monolith web applications ▪ Generate complete microservice architectures ▪ Generate domain model (entities) ▪ Generate CI/CD pipelines ▪ Deploy to AWS, GCP, Heroku, CF ▪ Deploy to Docker, Kubernetes, Openshift, Rancher https://start.jhipster.tech/#/statistics
10. Demo time Let us create a microservice e-commerce application and deploy it using Docker or Kubernetes
11. 11 The application architecture
12. How long would it take?
13. 13 JDL JHipster Domain Language JDL: https://goo.gl/csHzbz Reference: https://www.jhipster.tech/jdl/ Studio: https://start.jhipster.tech/jdl-studio/
14. 14 JDL Application
15. 15 JDL Entity & Relationship
16. Start your timer
17. 17 JHipster magic $ jhipster import-jdl app.jdl
18. 18 Deployment or
19. XL Impact DevOps Intelligence and Analytics
20. 20 ▪ Goal-based DevOps KPIs ▪ Data-driven Recommendations and Decision Support ▪ Predictive Analytics that Identify Trends and Highlight Risk ▪ One Control Panel for DevOps Impact and ROI XL Impact
21. 21 Architecture
22. 22 Release pipeline
23. 23 How JHipster helped ▪ Quick bootstrap ▪ Fast POC(Easy prototyping) ▪ OOB integrations(Hazelcast, Gradle, Spring boot, ElasticSearch, K8S etc) ▪ Best practices from the start ▪ JDL entities
24. 24 Challenges ▪ Hazelcast cache clustering with Kubernetes ▪ ElasticSearch write performance ▪ Kafka sync issues ▪ On demand Spark clusters on Kubernetes ▪ Orchestration of Infra, services and crawlers to multiple environments(single tenant architecture) ▪ ES high availability on Kubernetes(no dynamic volume resizing)
25. 25 Solutions ▪ Hazelcast cache clustering with Kubernetes − Kubernetes discovery API ▪ ElasticSearch write performance − SSD + Memory + lot of swearing ▪ Kafka reliability issues − Moved to Google PubSub ▪ On demand Spark clusters on Kubernetes − Custom images with lot of tweaks ▪ Orchestration of Infra, services and crawlers − XL Release + XL Deploy ▪ ES high availability on Kubernetes − Still not implemented
26. Thank you XebiaLabs website https://xebialabs.com/ JHipster website https://jhipster.tech GitHub https://github.com/jhipster/generator-jhipster Twitter https://twitter.com/java_hipster JHipster online https://start.jhipster.tech