Tweet
Tweet

Slide 1

Slide 1 text

Enterprise Integration Patterns Revisited for an Era of Big Data and Internet of Things Kai Wähner [email protected] @KaiWaehner www.kai-waehner.de LinkedIn / Xing à Please connect!

Slide 2

Slide 2 text

Key Take-Aways –  Integration is the key factor for success of business in the future! –  Enterprise Integration Patterns are used everywhere! –  Real time integration is the game changer in a Big Data and IoT world!

Slide 3

Slide 3 text

Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future

Slide 4

Slide 4 text

Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future

Slide 5

Slide 5 text

Growing IT Infrastructure Everywhere •  Applications •  Interfaces •  Technologies •  Products •  Cloud •  Mobile •  Internet of Things

Slide 6

Slide 6 text

Heterogeneity Host à Client / Server à SOA à Cloud à Big Data à IoT à ...

Slide 7

Slide 7 text

Spaghetti Solutions „point-to-point hell“

Slide 8

Slide 8 text

The Solution: Application Integration All roads lead to Rome ...

Slide 9

Slide 9 text

AmazonS3  s3  =  new  AmazonS3Client(new  Proper6esCreden6als(                  S3Sample.class.getResourceAsStream("AwsCreden6als.proper6es")));     String  bucketName  =  "my-­‐first-­‐s3-­‐bucket-­‐"  +  UUID.randomUUID();   String  key  =  "MyObjectKey";     try  {                    s3.createBucket(bucketName);          s3.putObject(new  PutObjectRequest(bucketName,  key,  createSampleFile()));            S3Object  object  =  s3.getObject(new  GetObjectRequest(bucketName,  key));                    ObjectLis6ng  objectLis6ng  =  s3.listObjects(new  ListObjectsRequest()                          .withBucketName(bucketName)                          .withPrefix("My"));                    s3.deleteObject(bucketName,  key);              s3.deleteBucket(bucketName);     }  catch  (AmazonServiceExcep6on  ase)  {        //  error  handling...   }  catch  (AmazonClientExcep6on  ace)  {        //  error  handling...         }   Writing a lot of Glue Code...

Slide 10

Slide 10 text

... vs. using Patterns, Frameworks, Tools •  Standardized Modeling •  Efficient Realization •  Automatic Testing

Slide 11

Slide 11 text

Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future

Slide 12

Slide 12 text

“A software design pattern, in the sense of a template, is a general solution to a problem in programming. A design pattern provides a reusable architectural outline that may speed the development of many computer programs.” http://en.wikipedia.org/wiki/Design_pattern What is a Design Pattern?

Slide 13

Slide 13 text

Enterprise Integration Patterns (EIP)

Slide 14

Slide 14 text

“The goal of EIPs is to document technology-independent design guidance that helps developers and architects describe and develop robust INTEGRATION solutions.” http://www.eaipatterns.com Enterprise Integration Patterns (EIP)

Slide 15

Slide 15 text

History of EIPs •  Talk from Gregor Hohpe –  WSO2 Con 2011 –  „History, present, future of EIP“ –  A lot about past (ideas, creation of book, ...) –  http://www.youtube.com/watch? v=Xwi1DU6KoQ4 •  Today‘s Talk –  Not too much about history –  Just about present and future

Slide 16

Slide 16 text

h"p://www.eaipa"erns.com   EIP Example: Splitter

Slide 17

Slide 17 text

h"p://www.eaipa"erns.com   EIP Example: Aggregator

Slide 18

Slide 18 text

Combining EIPs shows their true power! A + B + C = ABC

Slide 19

Slide 19 text

h"p://www.eaipa"erns.com   EIP Example: Composed Message Processor

Slide 20

Slide 20 text

Status Quo •  Enterprise Integration Patterns do not change •  Just technologies change •  Implementations for EIPs are already available

Slide 21

Slide 21 text

Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future

Slide 22

Slide 22 text

Enterprise Integration Patterns

Slide 23

Slide 23 text

Use Case Example

Slide 24

Slide 24 text

Modeling Tool: Microsoft Visio (Windows)

Slide 25

Slide 25 text

Modeling Tool: Omnigraffle (Mac OS X)

Slide 26

Slide 26 text

Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future

Slide 27

Slide 27 text

Complexity of Integration Integra6on   Products   Integra6on  Suite   Low High Integration Framework Alternatives for Application Integration

Slide 28

Slide 28 text

Kai Wähner - Enterprise Integration Patterns Revisited Complexity of Integration Integra6on   Products   Integra6on  Suite   Low High Integration Framework INTEGRATION Connectivity Routing Transformation When to use an Integration Framework?

Slide 29

Slide 29 text

Connectivity Integration of different Technologies

Slide 30

Slide 30 text

Many further connectors Easy to create own components HTTP   FTP   File   XSLT   EJB   JDBC Akka   TCP   SMTP   RSS   Quartz   jclouds   LDAP   JMS   MQTT   AMQP   Atom   AWS   Bean-Validation   CXF   IRC   Jetty   JMX   Lucene   Netty   RMI   SQL   Twitter MongoDB   Connectivity

Slide 31

Slide 31 text

Integration implement Integration Frameworks

Slide 32

Slide 32 text

Code Example: Apache Camel (Java DSL)

Slide 33

Slide 33 text

Code Example: Apache Camel (XML DSL)

Slide 34

Slide 34 text

Kai Wähner - Enterprise Integration Patterns Revisited Connectivity Routing Transformation INTEGRATION Tooling Monitoring Support Complexity of Integration Integra6on   Products   Integra6on  Suite   Low High Integration Framework + When to use an Enterprise Service Bus?

Slide 35

Slide 35 text

35 “Citizen Integrator / Hybrid Integration” (Gartner, Forrester) © Copyright 2000-2015 TIBCO Software Inc.

Slide 36

Slide 36 text

36 Integration Products © Copyright 2000-2015 TIBCO Software Inc. One  Integra*on   Product  does  not  fit   all  problems!  

Slide 37

Slide 37 text

Adaptive Integration Approaches Self-Service Integration for DIY Integrators (iSaaS) Business Integration Cloud Integration Integration Platform as a Service (iPaaS) Enterprise Business Users Department Audience Enterprise Integration Platform (on-premise and cloud) Enterprise Service Bus

Slide 38

Slide 38 text

Integration Concepts Decision Data Conversion Transitions Process Call Process Design Tools Integration Processes Connectors to vendor applications Connectors to technologies Connectors (e.g. JMS, java, FTP) (e.g. SAP, SF.com) Enterprise Integration Patterns

Slide 39

Slide 39 text

Enterprise Service Bus (ESB)

Slide 40

Slide 40 text

Enterprise Service Bus (ESB) Intuitive graphical design environment Streamlines time and cost of development and training Graphical Process Modeling Native Standards based XSLT Mapper Drag-n-Drop Access to Resources Fully Integrated Test Environment Adapters, Services, Processes, Deployment, Management

Slide 41

Slide 41 text

41 iPaaS for the Citizen Integrator © Copyright 2000-2015 TIBCO Software Inc.

Slide 42

Slide 42 text

iSaaS: Integration for the “Prosumer” Subscription cloud service that makes it easy to automate moving of data between apps on the Web

Slide 43

Slide 43 text

iSaaS: Integration for the “Prosumer” •  Connectors –  SaaS APIs baked in a common user interface •  Recipes –  Combinations of connectors preconfigured as “apps” –  Just configure to your accounts –  Customize as needed •  Drag and Drop ease

Slide 44

Slide 44 text

Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future

Slide 45

Slide 45 text

Everything is connected…

Slide 46

Slide 46 text

Evolution of Integration Microservices   Web  APIs   Real-­‐6me   Web     Oriented     Architecture   Enabling  Technologies   In-­‐Memory,  Mul6core   REST,  JSON   Demand  Drivers   Mobile,  Cloud   Fast  Data,  IoT   Services     Web  Services   Real-­‐6me   Service     Oriented   Architecture   Enabling  Technologies   XML,  SOAP,  WS-­‐*   Process  Modeling   Demand  Drivers   E-­‐Commerce   BPM   Interfaces   Adapters   Real-­‐6me   Enterprise     Applica6on   Integra6on     Enabling  Technologies   Client-­‐Server   Messaging  Middleware   Demand  Drivers   ERP   Analy6cs   Records   Batch  Jobs   Non-­‐real6me   Enabling  Technologies   Mainframe   ETL,  Databases   Data   Integra6on   Demand  Drivers   Data  Processing   MIS   Are  we   there  yet?   Level-­‐Up  by     u=lizing  the   lessons,   assets  and   prac=ces  of   the  previous   Level     Accelera=ng  Produc=vity  &    Agility    

Slide 47

Slide 47 text

Enterprise Integration Patterns (EIPs) … •  … are needed everywhere! •  Not just in an Integration Framework or Integration Product!

Slide 48

Slide 48 text

Connectivity Routing Transformation Complexity of Integration Integra6on   Products   Integra6on  Suite   Low High Integration Framework INTEGRATION Tooling Monitoring Support + API MANAGEMENT BIG DATA INTERNET OF THINGS STREAMING ANALYTICS „YOU NAME IT“ + When to use an Integration Suite?

Slide 49

Slide 49 text

EIPs for Streaming Analytics Stream Processing for Real Time Processing of Big Masses of Data à Fast Data

Slide 50

Slide 50 text

EIPs for Open API and API Management Your   Enterprise   Closed  APIs   EDI   Web   B2B  SOA   FTP   Pre-­‐defined  integra6on  points   Limited,  trusted  partners   Strictly  constrained  interac6ons   Enterprise  friendly   Open  APIs   Opportunis6c  access  points     Many  partners,  untrusted   Encourage  new  ideas   Developer  friendly   Your   Enterprise   Partners   Innovators   Known   Par6es   Consumers   Unknown   Par6es   Suppliers   Coope66on   Employees   API   API   API   API   API   API   API   API   API   API   API   API   API   API   API   API   API   API   API   API  

Slide 51

Slide 51 text

EIPs for API Management (e.g. Transformation or Routing) Composite Consumer Domain Target Service Domain Target Facade Consumer Target Provider Facade Target Target Endpoint Facade Operation Target Provider Facade Policies Target Policies Router Routing Policies Partner Policies Facade Endpoint Target Operation © Copyright 2000-2015 TIBCO Software Inc.

Slide 52

Slide 52 text

© Copyright 2000-2015 TIBCO Software Inc. EIPs for Log Analytics and Operational Intelligence Collect   Store   Index  &   Search   Parse  &   Normalize   Visually   Analyze   Format  &   Forward   © Copyright 2000-2014 TIBCO Software Inc.

Slide 53

Slide 53 text

EIPs for Business Logic E.g. with Microservices: •  Services implementing a limited set of business functions •  Services developed, deployed and scaled independently Integration Service Monolith application SOA Integration Service Business Service Business Service Business Service Micro Service API Gateway SaaS Business Service

Slide 54

Slide 54 text

EIPs for Integration of “Internet of Everything” à Real Time Integration with Things, APIs, Business Processes, Cloud, Social, etc. hep://blogs.cisco.com/ioe/how-­‐the-­‐internet-­‐of-­‐everything-­‐will-­‐change-­‐the-­‐worldfor-­‐the-­‐beeer-­‐infographic  

Slide 55

Slide 55 text

55 Integration Suite © Copyright 2000-2015 TIBCO Software Inc. Do  I  have  to  use  one   Integra*on  Suite  for   all  these  use  cases?  

Slide 56

Slide 56 text

One or more frameworks / vendors? Frameworks or Products from different Vendors •  No license costs (for open source frameworks) •  A lot of glue code •  Testing / Bugfixing •  No (combined) support Frameworks or Products from a single Vendor •  All integrated within one framework or product? à Vision! •  Not integrated? à Often reality! •  Loosely coupled, highly integrated frameworks or products? à Best choice!

Slide 57

Slide 57 text

Agenda Ø  Application Integration Ø  Enterprise Integration Patterns Ø  Modeling Ø  Frameworks and Tools Ø  Big Data, Internet of Things, Microservices Ø  Future

Slide 58

Slide 58 text

Future of EIPs: The BAD News •  New Patterns (since release of the book)? ß NOT YET –  Conversations –  Complex Event Processing –  Error Handling •  More Details: „EIP Visions“ ß TO BE DETERMINED –  EIP I: Messaging à done –  EIP II: Conversations à 50 page paper –  EIP III: Process à tbd –  EIP IV: Event Processing à tbd http://www.eaipatterns.com/ramblings/72_eipvolumes.html

Slide 59

Slide 59 text

Future of EIPs: The GOOD News •  New Patterns not necessary! ß OFTEN –  EIPs were based on Messaging already, which is a good thing (usually) –  Conversations == State == Anti Pattern (often) –  Integration scenarios can be solved with existing patterns (usually) •  Better Tooling ß YES –  More frameworks and products (e.g. Stream Processing for Big Data / Fast Data, Internet of Things) –  Stable, mature tools –  New standards (MQTT, OMG's DDS, Eclipse Open IoT Stack, etc.) –  Support for new hardware (iBeacons, OSIsoft Pi, Google Glass, etc.) –  Ease of use (no coding) –  Better scalability

Slide 60

Slide 60 text

Future of Integration and EIPs Buy  this  book     if  you  do  not  own  it  already!   Integration will get even more important in the future than it is today! The number of different data sources and technologies increases even more than in the past: •  CRM, ERP, Mainframe, B2B (EDIFACT), etc. will not disappear •  DWH, Hadoop cluster, event / streaming server, In-Memory DB – all of them have to communicate •  Cloud, Mobile, APIs, Big Data, Internet of Things are no option, but our future! EVERYTHING HAS TO BE INTEGRATED!

Slide 61

Slide 61 text

Did you get the Key Take-Aways?

Slide 62

Slide 62 text

Integration is the key factor for success of business in the future! Enterprise Integration Patterns are used everywhere! Real time integration is the game changer in a Big Data and IoT world! Recap: Key Take-Aways

Slide 63

Slide 63 text

Questions? Kai Wähner [email protected] @KaiWaehner www.kai-waehner.de LinkedIn / Xing à Please connect!