Java EE Monolith to Cloud-Ready Architecture: Lessons from the Drools/jBPM Ecosystem

We’ve heard a lot of hype about microservices and why we all should embrace them and design our software as suites of independently deployable services. However, it is common sense that there is a significant trade-off between this architectural style and effectively splitting existing monoliths. This session shows how the Java EE programming model significantly enables decoupling monoliths; how to distribute this programming model in a cloud-ready environment; the challenges related to this migration in the Drools/jBPM ecosystem; and how and why Red Hat has to redesign its cluster technology, file systems, CDI event distribution, web stack, and Maven as microservices.

Transcript

1. "The big question is whether you are going to
   be able to say a hearty yes to your adventure”
   Joseph Campbell
   

   View Slide

2. Java EE Monolith to
   Cloud-Ready Architecture:
   Lessons from the Drools/jBPM Ecosystem
   Eder Ignatowicz
   Sr. Software Engineer @ Red Hat
   Alex Porcelli
   Principal Software Engineer @ Red Hat
   

   View Slide

3. The Hero’s Journey
   

   View Slide

4. Business Central Usability
   Call to Adventure
   

   View Slide

5. Threshold
   

   View Slide

6. First Trial
   

   View Slide

7. Second Trial
   

   View Slide

8. Abyss
   

   View Slide

9. Supernatural aid
   

   View Slide

10. Transformation
    

    View Slide

11. Transformation
    

    View Slide

12. Transformation
    

    View Slide

13. Return
    

    View Slide

14. Rest
    

    View Slide

15. Hero’s Journey
    

    View Slide

16. Cloud readiness
    Call to Adventure
    

    View Slide

17. git repository
    BC
    Helix
    Zookeeper
    ...
    Business Central
    Cluster Nodes
    Threshold - 2016
    BC
    Helix
    Zookeeper
    git repository
    Java NIO2
    Git Implementation
    Java NIO2
    Git Implementation
    

    View Slide

18. Supernatural aid
    

    View Slide

19. FileSystem Scalability
    First Trial
    

    View Slide

20. First Trial
    NIO2 Git Implementation
    • fsService.newFileSystem(“git://examples”));
    • Internally we store one project per git repo
    • default://project
    • .niogit/project.git (physical)
    • Backed by SAN, NFS or a distributed FS
    • Advantages
    • Standard Java API (NIO2)
    • Designed to VFS (could support multiple NIO2
    implementations)
    • Perfect fit with our business needs.
    

    View Slide

21. First Trial
    NIO2 Git Implementation
    • Challenges
    • Every project is an instance of a NIO2 filesystem
    • Instantiate filesystem objects is heavyweight so, we keep all
    them in memory.
    • Multiple users, multiple projects
    • users*projects in memory
    • First trial:
    • Reduce the memory footprint
    

    View Slide

22. First Trial
    Lazy Engine for NIO2 file system
    Patent Pending (LAZY VIRTUAL FILESYSTEM INSTANTIATION AND CACHING 0816028.00258)
    

    View Slide

23. Messaging
    Second Trial
    

    View Slide

24. Second Trial
    Watch Service Events
    • All the business central git operations mapped to WatchService
    NIO2 Events
    • ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY,
    ENTRY_RENAME
    • How to replicate this event on all nodes?
    

    View Slide

25. ...
    ...
    Artemis cluster
    Business
    Central
    Cluster
    Business
    Central
    Business
    Central
    Business
    Central
    Second Trial
    ENTRY_CREATE
    Regenerated
    Watch event
    ENTRY_CREATE
    Watch event
    

    View Slide

26. Indexing
    Third Trial
    

    View Slide

27. Third Trial
    Indexing
    • What to index
    • Efficiently list all content of git repositories
    • Support different architectures
    • Support text search for large repositories
    • 10.000’s of assets
    • Third trial:
    • Implement an efficient indexing strategy for the platform
    

    View Slide

28. Third Trial
    Indexing
    • When to index
    • Watch Service listening all NIO2 Watch Events
    • How to index:
    • Index instead of list repository content
    • Plugable Engines
    • Lucene (local)
    • Infinispan (distributed in memory)
    • ElasticSearch (distributed and persisted)
    

    View Slide

29. Two new challenges appears
    Abyss
    

    View Slide

30. CDI Event Distribution
    Abyss Trial 1
    

    View Slide

31. CDI Events on Cluster
    CDI Events Cluster Propagation
    • Challenges
    • We rely on CDI event distribution for everything
    • Backend messaging
    • UI events via Errai Framework (CDI in the Browser)
    • (full web architecture on http://ederign.me - 5 pillars)
    • Becomes really hard to sync everything on cluster
    • Duplicated code
    • UI Logic that works only a single instance
    • New trial:
    • How to distribute my CDI events?
    

    View Slide

32. CDI Events on Cluster
    @Portable
    @Clustered
    public class NewProjectEvent {
    private WorkspaceProject workspaceProject;
    public NewProjectEvent(final WorkspaceProject workspaceProject) {
    this.workspaceProject = workspaceProject;
    }
    public WorkspaceProject getWorkspaceProject() {
    return workspaceProject;
    }
    }
    

    View Slide

33. CDI Events on Cluster
    public void observeAllEvents(@Observes(notifyObserver = Reception.IF_EXISTS)
    Object event,
    EventMetadata metaData) {
    if (shouldObserveThisEvent(event,metaData)) {
    broadcastOnCluster(event);
    }
    }
    

    View Slide

34. CDI Events on Cluster
    

    View Slide

35. Maven
    Abyss Trial 2
    

    View Slide

36. Build System
    Proprietary Build System
    • Challenges
    • Incremental compilation
    • In-process execution
    • Shared ClassLoader
    • Multiple users building at same time
    on the same process
    • Output of a project is a kjar (.jar) to be
    deployed into a kie-server
    • New trial:
    • How to make this build efficient and cloud
    friendly?
    

    View Slide

37. Build System
    Maven Based Build
    • How to
    • Adopt and Adapt Maven to
    • Incremental compilation
    • Low memory footprint
    • Multi-thread/Async API
    • Multi-user/multi-m2repo
    • µService/cloud native
    • Embedded & off-process
    • New trial:
    • DEV5074 - Turning Maven into a Highly Scalable, Resource-Efficient,
    Cloud-Ready Microservice
    • Moscone West - Room 2018
    • Wednesday 4pm
    

    View Slide

38. Rest
    

    View Slide

39. SaaS (Hosted Business Central)
    Call to Adventure
    

    View Slide

40. Java Hero’s
    

    View Slide

41. "The big question is whether you are going to
    be able to say a hearty yes to your adventure”
    Joseph Campbell
    

    View Slide

42. Tech Supernatural aid
    

    View Slide

43. Thank you.
    Eder Ignatowicz
    Sr. Software Engineer
    Red Hat
    Alex Porcelli
    Principal Software Engineer
    Red Hat
    

    View Slide