Integration Patterns A byte about me n Some background information about myself: n FIBer (Barcelona, 2000) and KTHer (Stockholm, 2004) n Barcelonian by birth. European by choice. Currently living in Vienna. n Experience working for small companies developing web-based software products for business customers (B2B). n Inoculated against the Not Invented Here syndrome and instead prefer to embrace a Proudly Found Elsewhere attitude! n Java (ecosystem) developer… learning Scala now! 2 Chemistry + HP48 GX --> Wittig reaction --> Software developer! Source: http://en.wikipedia.org/wiki/Wittig_reaction
Integration Patterns A bit about you? n How many of you have experience with enterprise application integration projects (in-house and B2B)? n How many of you are familiar with Spring Integration or Apache Camel? n How many of you have experience with a business process engine (jBPM, Activiti, camunda BPM)? 5
Integration Patterns Anatomy of a Pattern n A pattern n has a name (reference) n is a applicable in a given context n describes a recognizable problem n includes considerations/constraints to take into account (forces) n solution(s) n sketch illustrating the solution n other patterns to be considered n examples n diagram notation (icons) 8 Patterns are not just LEGO pieces for a solution. They almost always need to be adapted to a given context. Source: http://www.eaipatterns.com/Introduction.html
Integration Patterns Software Development Q: Is software development an individual activity? A: It depends. Q: On what? A: On the context. 10 Source: http://en.wikipedia.org/wiki/ On_the_Internet,_nobody_knows_you're_a_dog “On the Internet, nobody knows you’re a dog.” “Any fool can write code that a computer can understand. Good programmers write code that humans can understand”. -- Martin Fowler
Integration Patterns Software Development Q: Is software development an individual activity? A: It depends. Q: On what? A: On the context. 10 Source: http://en.wikipedia.org/wiki/ On_the_Internet,_nobody_knows_you're_a_dog “On the Internet, nobody knows you’re a dog.” “Any fool can write code that a computer can understand. Good programmers write code that humans can understand”. -- Martin Fowler Which code?! To do what exactly?
Integration Patterns Large Scale Enterprise Software Development 12 “The Tower of Babel” by Pieter Bruegel the Elder (1563). Kunsthistorisches Museum, Vienna.
Integration Patterns Large Scale Enterprise Software Development… 13 “The (Little) Tower of Babel” by Pieter Bruegel the Elder (1563). Museum Boijmans Van Beuningen. Rotterdam. … now on the Cloud!
Integration Patterns n Question: why? One answer: perception leads reality and not the other way around! n Words do not have meaning by themselves. We give meaning to them. n It’s then fundamental to have a common, rigorous language to communicate. 14 From “Understanding Comics: The Invisible Art”. Scott McCloud. 1993. Pages 194-195. Enterprise Software Development
n Different programming languages n Different terms/vocabulary n Different representations the same concepts BarcelonaJUG – April 22rd 2013 | Barcelona Appetizers of Enterprise Integration Patterns Enterprise Application Integration 17 Invoicing CRM HRM CRM Stock Sales SAP “O, be some other name! What's in a name? that which we call an [order] By any other name would smell as sweet;” -- William [Software Architect] Shakespeare “organizations which design systems […] are constrained to produce designs which are copies of [their own] communication structures” -- Conway’s Law* * “How Do Committees Invent?” Melvin E. Conway http://www.melconway.com/research/committees.html
Integration Patterns Integration Styles: Messaging n Pros: n Embraces asynchronous behavior n De-coupled applications (coupling is about assumptions!) n The messaging system (bus) is responsible for transferring the data n Cons: n It’s more complex than the other styles (transactions, error reporting, sequencing) 22 Source: http://www.eaipatterns.com/Messaging.html
Integration Patterns Asynchronous programming model n HTML5 / JavaScript anyone? 23 I’m too old for this s… From the movie: “Lethal Weapon” by Richard Donner. 1987.
Barcelona Appetizers of Enterprise Integration Patterns Message Channel Types 26 Source: http://eaipatterns.com/PointToPointChannel.html, http://eaipatterns.com/PublishSubscribeChannel.html There can be more receivers!
Integration Patterns Messages = Header(s) + Payload or Body n Header: information used/needed by the messaging system n Body/Payload: the actual data we are interested in transmitting n Types: n Command message: invoke an action at the receiver n Document message: pass data to the receiver (content is the concern) n Event message: notify the receiver (timing is the concern) 27
Integration Patterns Channel Adapter n “How can connect an application to the messaging system so that it can send and receive messages?” 28 Source: http://www.eaipatterns.com/ChannelAdapter.html
Integration Patterns Message Translator n How can systems using different data formats communicate with each other? n Watch out! Translation has to happen at different levels: n Transport: FTP to HTTP, … n Data representation: CSV to XML, … n Data types: and “Order” here is not an “Order” there n Data structures: HL7, CMIS, 30 Source: http://www.eaipatterns.com/MessageTranslator.html
Integration Patterns Tapas2Go: Did you notice something? n Business logic starts getting difficult to understand --> maintain! n Even more when we need to maintain a conversation (state between messages)! 36 Source: http://www.eaipatterns.com/ProcessManager.html
Integration Patterns Enter the Process Manager n Advantages: n manages state between messages (conversation) n stores intermediate results n determines next processing step(s) n with a proper notation --> easier to understand the big picture 37 Source: http://www.eaipatterns.com/ProcessManager.html
Integration Patterns Order Processing (For Real) n UML activity diagram: 38 Source: http://www.eaipatterns.com/Chapter1.html Question: How do we implement it in Java?
Integration Patterns What is a business process? n Definition: “a sequence of related, structured activities or tasks that produce a specific service or product (serve a particular goal) for a particular customer or customers.” 40 Source: http://en.wikipedia.org/wiki/Business_process
Integration Patterns When do I need a business process? n Business process modeling acid test*: 1. Long-running: hours, days, weeks, months, years, … 2. Bursty, sleeps most of the time: when awake, a flurry of activities and then finished or to sleep again! 3. Persisted state: what happens between different steps or when the application server goes down? 4. Orchestration of system or human communications: who has to do what until when? what happens afterwards? n What about communication? Common shared understanding? n What about organizational learning? n How long does it takes us to do X? n Who did what when? 41 * “Essential Business Process Modeling”, Michael Havey, O’Reilly Press http://shop.oreilly.com/product/9780596008437.do via http://stackoverflow.com/questions/4869734/how-do-you-know-when-you-need-a-bpm-solution
Integration Patterns Business Process Model and Notation 2.0 n Example: announcing a job position 42 Source: “Praxishandbuch BPMN 2.0” http://www.bpm-guide.de/2012/02/16/praxishandbuch-bpmn-2-0-in-der-3-auflage/
Integration Patterns Business Process Model and Notation 2.0 n Example: announcing a job position explained… sort of: 43 Start event Sequence flow User (human) task Pool (participant) Lane Gateway (exclusive) Call activity (sub-process) Service task End event Activities
Integration Patterns EIPs and BPMN 2.0 n Great! I just need to learn some new symbols and then … 44 From the movie: “The Matrix” by Larry and Andy Wachowski. 1999. WOAH! I know EIPs and BPMN 2.0!
Integration Patterns EIPs and BPMN 2.0 45 From the movie: “The Matrix” by Larry and Andy Wachowski. 1999. In your dreams! Mister Anderson “It is difficult, if not impossible, for anyone to learn a subject purely by reading about it, without applying the information to specific problems and thereby forcing himself to think about what has been read. Furthermore, we all learn best the things we have discovered ourselves.” -- Donald Knuth, “The Art of Computer Programming”* * Via Alexander Daniel: http://lachdrache.wordpress.com/2012/03/11/reading-•-applying-•-thinking/
start point, state changes or final states. Message: Receiving and sending messages. Timer: Cyclic timer events, points in time, time spans or timeouts. Error: Catching or throwing named errors. Cancel: Reacting to cancelled transactions or triggering cancellation. Compensation: Handling or triggering compensation. Conditional: Reacting to changed business conditions or integrating business rules. Signal: Signalling across differ- ent processes. A signal thrown can be caught multiple times. Multiple: Catching one out of a set of events. Throwing all events defined Link: Off-page connectors. Two corresponding link events equal a sequence flow. Terminate: Triggering the immediate termination of a process. Escalation: Escalating to an higher level of responsibility. Parallel Multiple: Catching all out of a set of parallel events. Start End Intermediate Catching Throwing Event Sub-Process Interrupting Top-Level Event Sub-Process Non-Interrupting Boundary Interrupting Boundary Non- Interrupting Sequence Flow defines the execution order of activities. Conditional Flow has a condition assigned that defines whether or not the flow is used. Default Flow is the default branch to be chosen if all other conditions evaluate to false. Task A Task is a unit of work, the job to be performed. When marked with a symbol it indicates a Sub-Process, an activity that can be refined. Transaction A Transaction is a set of activities that logically belong together; it might follow a specified transaction protocol. Event Sub-Process An Event Sub-Process is placed into a Process or Sub-Process. It is activated when its start event gets triggered and can interrupt the higher level process context or run in parallel (non- interrupting) depending on the start event. Call Activity A Call Activity is a wrapper for a globally defined Sub-Process or Task that is reused in the current process. Task Types Types specify the nature of the action to be performed: Send Task Receive Task User Task Manual Task Business Rule Task Service Task Script Task Markers indicate execution behavior of activities: Activity Markers Sub-Process Marker Loop Marker Parallel MI Marker Sequential MI Marker ~ Ad Hoc Marker Compensation Marker A Communication defines a set of logically related message exchanges. When marked with a symbol it indicates a Sub-Conversation, a compound conversation element. A Forked Conversation Link connects Communications and multiple Participants. A Conversation Link connects Communications and Participants. Inclusive Gateway When splitting, one or more branches are activated. All active incoming branches must complete before merging. Complex Gateway Complex merging and branching behavior that is not captured by other gateways. Exclusive Event-based Gateway (instantiate) Each occurrence of a subsequent event starts a new process instance. Parallel Event-based Gateway (instantiate) The occurrence of all subsequent events starts a new process instance. Pool (collapsed) Multi Instance Pool (collapsed) Communication Sub-Conversation Pool (collapsed) Participant B The order of message exchanges can be specified by combining message flow and sequence flow. Pool Pool Pools (Participants) and Lanes represent responsibilities for activities in a process. A pool or a lane can be an organization, a role, or a system. Lanes subdivide pools or other lanes hierarchically. Lane Task Lane Task Pool Message Flow symbolizes information flow across organizational boundaries. Message flow can be attached to pools, activities, or message events. Data Task Input Out- put Data Store A Data Object represents information flowing through the process, such as business documents, e-mails, or letters. A Data Store is a place where the process can read or write data, e.g., a database or a filing cabinet. It persists beyond the lifetime of the process instance. A Data Input is an external input for the entire process. It can be read by an activity. A Data Output is a variable available as result of the entire process. A Message is used to depict the contents of a communication between two Participants. A Collection Data Object represents a collection of information, e.g., a list of order items. Pool (Collapsed) Collaboration Diagram Pool (Expanded) Lane Lane Choreographies Choreography Diagram A Choreography Task represents an Interaction (Message Exchange) between two Participants. Choreography Task Participant A Participant B A Choreography Sub- Process contains a refined choreography with several Interactions. Multiple Participants Marker denotes a set of Participants of the same kind. Swimlanes BPMN 2.0 - Business Process Model and Notation Collection Ad-hoc Sub-Process Task Task ~ Message Start Event Message Flow Data Object Collapsed Sub-Process Event-based Gateway Escalation End Event Timer Intermediate Event Receive Task Attached Intermediate Timer Event Link Intermediate Event Manual Task End Event Data Store Link Intermediate Event Parallel Multiple Intermediate Event Text Annotation Group Multi Instance Task (Parallel) Message End Event Send Task Parallel Gateway Exclusive Gateway Attached Intermediate Error Event Signal End Event Call Activity Sub-Process Event Sub-Process Conditional Start Event Error End Event Start Event End Event Looped Sub-Process condition http://bpmb.de/poster Participant A Participant C Participant B Choreography Task Participant A Participant B Choreography Task Participant A Participant C Initiating Message Response Message Choreography Task Participant B Participant A When splitting, it routes the sequence flow to exactly one of the outgoing branches. When merging, it awaits one incoming branch to complete before triggering the outgoing flow. Exclusive Gateway Is always followed by catching events or receive tasks. Sequence flow is routed to the subsequent event/task which happens first. Event-based Gateway When used to split the sequence flow, all outgoing branches are activated simultaneously. When merging parallel branches it waits for all incoming branches to complete before triggering the outgoing flow. Parallel Gateway Choreography Sub-Process Participant A Participant C Participant B DevFest 2012 – November 10th 2012 | Vienna plexiti · pragmatic software productions BPMN 2.0: Would you like to know more? 46 And THE poster! PDF: http://www.bpmb.de/images/BPMN2_0_Poster_EN.pdf Printed: http://www.signavio.com/joomla/de/bpmn-poster.html Get THE book! German: http://www.bpm-guide.de/2012/02/16/ praxishandbuch-bpmn-2-0-in-der-3-auflage/ English: http://www.bpm-guide.de/2012/10/30/real-life- bpmn-2-0-book-now-available-in-english/ Get the BPMN 2.0 Examples at: http://www.omg.org/cgi-bin/doc?dtc/10-06-02
Integration Patterns The Job Announcement Showcase n A kick-start application for your business process-centric project! n Java EE 6 technology stack n camunda fox BPM Platform n Try online at: http://the-job-announcement.com/ n Fork, clone, modify, … at https://github.com/plexiti/the-job- announcement-fox n Executable specifications with Cucumber and Selenium at https://github.com/plexiti/the-job- announcement-exec-specs 48 Migration to camunda BPM coming real soon!
Integration Patterns What about Apache Camel?! n 51 Vincent: Do you know the funniest thing about [EIP frameworks]? Vincent: It’s the little differences. Jules: Example From the movie: “Pulp Fiction” by Quentin Tarantino. 1994.
Integration Patterns The Job Announcement / Apache Camel n “Route-centric”, i.e. where are the channels? 52 from("direct:tweets") .inOut("twitter://timeline/user") .to("log:com.[...].service.camel?level=INFO&showAll=true&multiline=true"); public void postToTwitter(Long jobAnnouncementId) { JobAnnouncement announcement = find(jobAnnouncementId); String tweet = announcement.getTwitterMessageWithLink(); log.info("About to tweet [" + tweet + "]."); // send tweet to the twitter endpoint **SYNCHRONOUSLY** (because we use the 'direct:' endpoint) CamelContext camelCtx = ctxBootStrap.getCamelContext(); ProducerTemplate producerTemplate = camelCtx.createProducerTemplate(); Status status = (Status) producerTemplate.sendBodyAndHeader("direct:tweets", ExchangePattern.InOut, tweet, "jobAnnouncementId", jobAnnouncementId); [...] } Check files TwitterPostingCamelRoute.java and JobAnnouncementServiceImpl.java on https://github.com/plexiti/the-job-announcement-fox
Integration Patterns Apache Camel n FuseSource acquired by Red Hat about a year ago --> Apache Camel integration with the rest of Red Hat’s / JBoss’ product line n Supports many more protocols/applications: http://camel.apache.org/components.html n In 2.11 the integration with Java EE/CDI has improved quite a bit: http://camel.apache.org/cdi.html n The spring-integration component bridges Camel components and Spring Integration endpoints: http://camel.apache.org/springintegration.html 53 http://camel.apache.org
Integration Patterns n In every moderate to complex software project: n The point is not to have a great model (or apply all the patterns), the point is to ship software! 55 BPMN 2.0 and EIP Patterns *”The Duct Tape Programmer” by Joel Spolsky: http://www.joelonsoftware.com/items/2009/09/23.html
Integration Patterns n In every moderate to complex software project: n The point is not to have a great model (or apply all the patterns), the point is to ship software! 55 [T]here are known knowns; there are things we know that we know. There are known unknowns; that is to say there are things that, we now know we don't know. But there are also unknown unknowns – there are things we do not know we don't know. --- United States Secretary of Defense, Donald Rumsfeld BPMN 2.0 and EIP Patterns *”The Duct Tape Programmer” by Joel Spolsky: http://www.joelonsoftware.com/items/2009/09/23.html
Integration Patterns n In every moderate to complex software project: n The point is not to have a great model (or apply all the patterns), the point is to ship software! 55 [T]here are known knowns; there are things we know that we know. There are known unknowns; that is to say there are things that, we now know we don't know. But there are also unknown unknowns – there are things we do not know we don't know. --- United States Secretary of Defense, Donald Rumsfeld BPMN 2.0 and EIP Patterns “At the end of the day, ship the fucking thing! It’s great to rewrite your code and make it cleaner and by the third time it’ll actually be pretty. But that’s not the point—you’re not here to write code; you’re here to ship products.” --- Jamie Zawinski* *”The Duct Tape Programmer” by Joel Spolsky: http://www.joelonsoftware.com/items/2009/09/23.html
Integration Patterns Wrapping up n The first electronic general-purpose digital computer: the ENIAC (1946) 56 Text Source: http://en.wikipedia.org/wiki/ENIAC Who complained about bean wiring?!
Integration Patterns Would you like to know more? n “WSO2Con 2011 - Enterprise Integration Patterns: Past, Present and Future” by Gregor Hohpe: http://www.youtube.com/watch?v=Xwi1DU6KoQ4 n Enterprise Integration Patterns Website: http://www.eaipatterns.com n “Enterprise Integration Patterns” DRAFT article by Gregor Hohpe http://www.eaipatterns.com/docs/Enterprise%20Integration%20Patterns%20-%20PLoP%20Final%20Draft%203.pdf n Spring Integration reference documentation: http://static.springsource.org/spring-integration/reference/htmlsingle/ n Spring Integration samples GitHub project: https://github.com/SpringSource/spring-integration-samples n “Practical Tips and Tricks with Spring Integration” by Oleg Zhurakousky http://www.youtube.com/watch?v=RY6dNUL8k6o 59
pragmatic software productions Answers to Discussion Questions n Q: “Do I need all THAT XML to configure Spring Integration?!” A: No. Check the Café Sample Application how to use annotations: https://github.com/SpringSource/spring-integration-samples/tree/master/applications/cafe A: Also check the Spring Integration Scala DSL http://blog.springsource.org/2012/03/05/introducing-spring-integration-scala-dsl/ A: Also check the Spring Integration Groovy DSL http://blog.springsource.org/2012/11/06/a-groovy-dsl-for-spring-integration/ 60
rafacm
oskmr
kudoas
manfredsteyer
konifar
ski
bkuhlmann
1wedge_one
kosui
jaguar_imo
philipschwarz
slephy2784
swwweet
shpigford
bluesmoon
searls
chriscoyier
lynnandtonic
colly
mza
stephaniewalter
sachag
eitanlees
![Rafael Cordones, [email protected] BarcelonaJUG, April 22rd, 2013, Barcelona Appetizers of](https://files.speakerdeck.com/presentations/9daeac4090740130a9c8529790d16483/slide_0.jpg){kind=link}
