Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)

Navigating SAP’s Integration Options (Mastering SAP Technologies 2013)

This presentation from Mastering SAP Technologies 2013 provides an overview of popular integration approaches, maps them to SAP's integration tools and concludes with some lessons learnt in their application.

A5604f2015f157ee20ceb1ea6e609bc9?s=128

Sascha Wenninger

July 29, 2013
Tweet

Transcript

  1. None
  2. Naviga&ng  SAP’s  Integra&on  Op&ons   Lessons  Learnt  Integra&ng  SAP  into

     a  Complex  Landscape Sascha  Wenninger @sufw    
  3. About Me Technical Architect Focus: SAP Integration Opinionated Co-founder of

    Blue T SAP Mentor Wannabe Performance Engineer Enterprise IT !== Boring 3
  4. 4 https://secure.flickr.com/photos/good_day/159927879/

  5. There were files. 5

  6. Then came RFC 6

  7. 7

  8. So What Do You Choose? 8

  9. https://secure.flickr.com/photos/manannan_alias_fanch/315630347/ 9

  10. 10 h1p://www.flickr.com/photos/freddiebrown/6296214984/

  11. 11 h1ps://secure.flickr.com/photos/92583675@N00/2322009245/

  12. 12 h1ps://secure.flickr.com/photos/pascalbovet/4375204691/

  13. ?

  14. TINLOTR

  15. None
  16. One Integration Approach to Rule them All

  17. One Integration Approach to Rule them All

  18. One ____________ to Rule them All

  19. One ____________ to Rule them All

  20. Use the Right Tool for the Job! “Right  Tool  for

     the  Job?”,  by  Bruce  Murray
  21. 21 h1p://www.flickr.com/photos/rideyourbike/2275993007/

  22. 22 Back to First Principles

  23. Push or Pull? That is the Question. 23

  24. 24 Data  Flow A B Source DesVnaVon

  25. 25 A B Actual  IntegraVon? A  sends  to  B A

    B B  fetches  from  A or
  26. 26 A knows B Low latency easy A is active

    party A has control A may not know B Low latency possible B is active party B has control A B A B
  27. 27 or h1p://www.flickr.com/photos/gcwest/2686928531/

  28. Synchronous: Source is responsible to make itself understood 28 h1p://www.flickr.com/photos/48975388@N07/7163918352/

  29. Asynchronous: Target is responsible to ensure it understands 29 h1p://www.dumpaday.com/wp-­‐content/uploads/2013/02/funny-­‐pictures-­‐dumpaday-­‐28.jpg

  30. They’re the only binary choices you have to make! 30

  31. <sidebar> 31

  32. Synchronous == Best Effort, “Kiddie Stuff” Asynchronous == Reliable, Guaranteed

    Delivery, “Enterprisey” 32
  33. Synchronous == Best Effort, “Kiddie Stuff” Asynchronous == Reliable, Guaranteed

    Delivery, “Enterprisey” 33
  34. Remember the OSI Model? 34 7 Application 6 Presentation 5

    Session 4 Transport 3 Network 2 Data Link 1 Physical More or less leaky abstractions Inherently Synchronous
  35. Everything in Computing is Ultimately Synchronous! 35 “Turtle  Tower”,  by

     Andreas  Al
  36. 36 (It’s all about error handling!)

  37. </sidebar> 37

  38. For Example: Getting data into an Enterprise Data Warehouse (using

    SAP BW as an example) 38
  39. SAP  BW 39 Inbound Data Layer Transformation Layer Reporting Data

    Layer Reporting Tool Visualisation Tool Source System, e.g. SAP ERP Extractor Process Chain Process Chain Queries Browser access
  40. SAP  BW 40 Source System A Transformation Layer Reporting Data

    Layer Reporting Tool Visualisation Tool Source System B Source System C Source System D Source System E Inbound Data Layer
  41. 41 A B C D BW E “Blue  Marble  Next

     GeneraVon,  Raw  Bathymetry”,  by  NASA  Visible  Earth
  42. 42

  43. 43 A B C D BW “Blue  Marble  Next  GeneraVon,

     Raw  Bathymetry”,  by  NASA  Visible  Earth E
  44. 44 Best Bets for Push vs. Pull Decisions n Sources,

    1 Target 1 Source, n Targets “dumb” Source No/few intermediaries Many intermediaries Low latency needed ✔ ✔ ✔ ✔ ✔ ✔
  45. 45 Shared Database ETL (Extract, Transform, Load) File Transfer Message-Based

    SOA (Service Oriented Architecture) REST h1p://www.flickr.com/photos/marktee/7545627352
  46. 46 Shared Database

  47. 47 Shared Database

  48. ETL: Extract, Transform, Load 48

  49. File Transfer 49

  50. Message-Based 50

  51. SOA: Service Oriented Architecture 51

  52. REST: Representational State Transfer, aka “Web APIs” 52

  53. When to Use Which Approach? 53 Image  credit:  Jenny  Shirey

  54. 54 Criteria: Data Volume 1 kB 10 kB 100 kB

    1 MB 10 MB 100 MB 1 GB ETL File Transfer Message-Based SOA REST
  55. 55 Criteria: Frequency 1/week 1/day 1/hour 1/min 1/sec 10/sec ETL

    File Transfer Message-Based SOA REST
  56. 56 Criteria: Application Capabilities None Rudimentary Intermediate Sophisticated ETL File

    Transfer Message-Based SOA REST
  57. Criteria: Synchronicity 57 Fire  &  Forget! Asynchronous@ Synchronous$ ETL ✔

    ✔ File Transfer ✔ Message-Based ✔ SOA ✔ ✔ REST ✔ ✔ !      No  technical  acknowledgment  received  by  Sender @  Technical  (delivery)  acknowledgment  received  by  Sender $    Request/Response  communicaVon
  58. Criteria: Data or Functionality? 58 Data-­‐Centric Func&onality-­‐Centric ETL ✔ File

    Transfer ✔ Message-Based ✔ SOA ✔ REST ✔
  59. 59 Our Tools h1p://www.flickr.com/photos/tunnelarmr/2435107204/

  60. 60

  61. 61

  62. Best for Caution with • SAP’s Middleware product. –Origins in

    EAI, Message-based integration –XML-centric, many protocol adapters –Design-time SOA features, file transfer capabilities 62 PI: Process Integration •Push-oriented integration •XML data formats •Stateless processing •e.g. Message Router, Channel or Translator patterns •High-volume synchronous scenarios •Large (~200MB+) messages •Scenarios requiring keeping state •e.g. collect, distributed transactions, etc. •B2B integration
  63. Best for Caution with • SAP’s “middleware stack”: PI 7.3

    + BPM + BRM –Java-only installation with much improved (10x!) runtime performance –Next-gen NetWeaver BPM runtime for stateful processing 63 PO: Process Orchestration •Push-oriented integration •XML data formats •Stateless processing •e.g. Message Router, Channel or Translator patterns •High-volume synchronous scenarios •Large (~200MB+) messages •Short timeframe projects (skills availability) •B2B integration (although investment is increasing)
  64. Best for Caution with • SAP’s ETL Tooling: Extract >

    Transform > Cleanse > Load –Bulk data transfers at the database level –Useful for replicating content of data warehouses 64 BODS: BusinessObjects Data Services •Data-centric integration •Pull-oriented integration •Large volume of data •Low frequency, high latency •Data quality enforcement •Bulk loads into HANA •Application-to-Application integration •Lower latency requirements •Infrequently-changing data •Granular information
  65. ALE - specifically IDocs 65 • SAP-proprietary message-based integration •

    Well integrated into SAP applications, some heavily rely on it. –Slowly being supplanted by web services, but not yet. • Mature and feature-rich Best for Caution with •Integrating standard functionality of different SAP applications with each other •Forward Error Handling •Processing messages in bulk, or in sequence •Integrating applications not built by SAP. •Can enhance “Fortress SAP” perceptions •Enhancing/extending standard IDocs
  66. 66

  67. ABAP Batch Jobs • "The 80's called, they want their

    integration back" • Nevertheless still useful in some cases –But use XML. Tab-delimited files should have gone extinct in the 80s. 67 Best for Caution with •Fire & Forget asynchronous transfer •Long-running "message" creation •Often lowest-common denominator •Outbound from SAP ;-) •SAP on the inbound side; error handling is generally bespoke. •High-volume or high-frequency interactions
  68. Best for Caution with • SAP’s Web Service Layer –translates

    XML to ABAP, and back 68 ABAP Proxies •Proper outside-in web service design •Logging •Idempotency •Forward Error Handling •WS-* support •Relying only on ESR modeling •Some industry-standard XML Schemas NW ABAP (e.g. ECC, CRM) BAPI ABAP Classes ABAP Proxy Layer SOAP Client
  69. Best for Caution with • SAP’s REST-inspired OData API Layer

    for: –Business Suite –HANA –NW BPM, Business Workflow –etc. 69 NetWeaver Gateway •Functionality-centric integration •Pull-oriented integration •Client/server architectures •Multi-request interactions •Exposing public APIs directly. •Formats other than OData or JSON NW ABAP (e.g. ECC, CRM) BAPI ABAP Classes Workflow ECC/CRM etc. NW BPM NW Gateway JSON/ OData Client
  70. Best for Caution with • SAP’s generic HTTP Server Layer

    –direct access to HTTP requests –hand-craft responses to include any content. 70 ABAP HTTP Handlers •Complex, functionally-rich REST APIs •Intimate control over content: •from Plain-text to Binary •Interesting 'hacks' •Large numbers of 'cookie-cutter' interfaces •requires hand-crafting •Learning curve NW ABAP (e.g. ECC, CRM) BAPI ABAP Classes ABAP HTTP Handler HTTP Client
  71. 71 Assessing Fit h1p://www.flickr.com/photos/90369723@N00/5580038871/

  72. 72 PI PO BO  DS ALE/IDocs ETL ✔ File Transfer

    ✔ ✔ ✔ Message-Based ✔ ✔ ✔ SOA ✔ ✔ REST
  73. 73 Batch  Jobs ABAP  Proxies Gateway HTTP  Handler ETL ✔

    File Transfer ✔ Message-Based ✔ SOA ✔ ✔ ✔ REST ✔ ✔
  74. 74

  75. Use the right tool for the job. 75 h1p://cheezburger.com/3535420672

  76. Think XML Schema 76

  77. Modeling XML using ESR Data Type objects is too restrictive:

    Time-consuming Cannot extend elements No abstract types No xs:choice, xs:all, xs:any, etc. Poor support for industry-standard XML Schemas Lots of work-arounds... 77
  78. 78 0%! 20%! 40%! 60%! 80%! 100%! ABAP Proxy! ESR!

    Supported! Partially Supported! Not Supported!
  79. 㾺 Plan for reuse at the right level! 79

  80. 80 㿃✔ 㿃✔ 㿃?

  81. Don’t aim for reuse. Let it emerge. 81 h1p://www.flickr.com/photos/yukonblizzard/2926897104/

  82. Reusable or 82 Future Legacy? Cumbersome? One Size Fits None?

    Regression Testing Nightmare?
  83. YAGNI + Refactor 83 h1p://www.flickr.com/photos/mcgraths/3248483447/

  84. You Ain't Gonna Need It + Refactor 84 h1p://www.flickr.com/photos/mcgraths/3248483447/

  85. Don’t try to predict the future. You will be wrong.

    If in doubt, leave it out. “As simple as possible” Plan to refactor to improve! 85
  86. Build tests. 86 h1p://www.flickr.com/photos/kwl/4809326028/

  87. 87 h1p://agilemanifesto.org

  88. 88

  89. 89 Business value over technical strategy Evolutionary refinement over pursuit

    of initial perfection
  90. Main Message 90 http://tomfishburne.com/2012/07/one-size-fits-none.html

  91. Key Points to Take Home You will need more than

    one tool. Get the interaction right. Aim for simplicity. This helps implementation and support. 91
  92. None
  93. Sascha Wenninger @sufw sascha@BlueT.com.au +61 403 933 472

  94. Further Reading Enterprise Integration Patterns (Hohpe & Woolf). The Book

    and the Site. Positioning Process Integration and Data Services, by SAP The Practical Science of Batch Size, by Don Reinertsen. Video and slides. Comments on the SOA Manifesto, by co-author Stefan Tilkov 94