Heapcon: How we built our Analytics platform with JHipster and OSS technologies

Heapcon: How we built our Analytics platform with JHipster and OSS technologies

This talk will focus on the experience of using JHipster in building the newest DevOps product from XebiaLabs. I will talk about the architecture, how JHipster is used and about some issues faced and how we overcome them.

7f408bc67dc9ae3b288ee92d16d3c4c2?s=128

Deepu K Sasidharan

October 18, 2018
Tweet

Transcript

  1. How we built our Analytics platform with JHipster and OSS

    technologies Deepu K Sasidharan XebiaLabs JHipster 2018
  2. 2 About me Deepu K Sasidharan JHipster co-lead developer Senior

    product developer @ XebiaLabs OSS aficionado, author, speaker @deepu105 deepu.js.org
  3. Have you heard about XebiaLabs?

  4. 4 About XebiaLabs dev test uat prod [Cloud] Orchestration Stack

    Middleware NoSQL PaaS Containers OS OS OS IaaS Network Servers DB / Storage Security SOFTWARE DEFINED DATA CENTER / CLOUD RELEASE ORCHESTRATION Backlog Management Provisioning/ Configuration Security ITSM / Service Management CMDB plan Project Management Issue Tracking ALM DEPLOYMENT AUTOMATION code SCM Code Analysis build Continuous Integration Centralized Repository test Test Tooling Test Visualization release ChatOps / Collaboration Email/ phone/ Excel operate BI / Monitoring Logging DEVOPS INTELLIGENCE
  5. 5 Release Orchestration

  6. 6 Deployment automation

  7. Have you used JHipster?

  8. 8 About JHipster Most popular Rapid Application Development platform for

    Java web applications and microservices ▪ 11k+ stars on GitHub ▪ 1.3M+ installations & 20k+ app generations per month ▪ 250k+ overall users ▪ 450+ contributors & 23 core team members ▪ 250+ companies using JHipster ▪ 70+ plugins Hel s e s!
  9. 9 What can you do with JHipster ▪ Generate simple

    monolith web applications ▪ Generate complete microservice architectures ▪ Generate domain model (entities) ▪ Generate CI/CD pipelines ▪ Deploy to AWS, GCP, Heroku, CF ▪ Deploy to Docker, Kubernetes, Openshift, Rancher https://start.jhipster.tech/#/statistics
  10. Demo time Let us create a microservice e-commerce application and

    deploy it using Docker or Kubernetes
  11. 11 The application architecture

  12. How long would it take?

  13. 13 JDL JHipster Domain Language JDL: https://goo.gl/csHzbz Reference: https://www.jhipster.tech/jdl/ Studio:

    https://start.jhipster.tech/jdl-studio/
  14. 14 JDL Application

  15. 15 JDL Entity & Relationship

  16. Start your timer

  17. 17 JHipster magic $ jhipster import-jdl app.jdl

  18. 18 Deployment or

  19. XL Impact DevOps Intelligence and Analytics

  20. 20 ▪ Goal-based DevOps KPIs ▪ Data-driven Recommendations and Decision

    Support ▪ Predictive Analytics that Identify Trends and Highlight Risk ▪ One Control Panel for DevOps Impact and ROI XL Impact
  21. 21 Architecture

  22. 22 Release pipeline

  23. 23 How JHipster helped ▪ Quick bootstrap ▪ Fast POC(Easy

    prototyping) ▪ OOB integrations(Hazelcast, Gradle, Spring boot, ElasticSearch, K8S etc) ▪ Best practices from the start ▪ JDL entities
  24. 24 Challenges ▪ Hazelcast cache clustering with Kubernetes ▪ ElasticSearch

    write performance ▪ Kafka sync issues ▪ On demand Spark clusters on Kubernetes ▪ Orchestration of Infra, services and crawlers to multiple environments(single tenant architecture) ▪ ES high availability on Kubernetes(no dynamic volume resizing)
  25. 25 Solutions ▪ Hazelcast cache clustering with Kubernetes − Kubernetes

    discovery API ▪ ElasticSearch write performance − SSD + Memory + lot of swearing ▪ Kafka reliability issues − Moved to Google PubSub ▪ On demand Spark clusters on Kubernetes − Custom images with lot of tweaks ▪ Orchestration of Infra, services and crawlers − XL Release + XL Deploy ▪ ES high availability on Kubernetes − Still not implemented
  26. Thank you XebiaLabs website https://xebialabs.com/ JHipster website https://jhipster.tech GitHub https://github.com/jhipster/generator-jhipster

    Twitter https://twitter.com/java_hipster JHipster online https://start.jhipster.tech