WebSphere Message Broker in Shared Runtime Environments

This session is a "lessons learned" about WebSphere Message Broker in shared runtime environments at various companies and the typical environment configurations and common set-ups with regards to high availability and workload balancing. What kind of solutions do we see implemented on top of message broker what are the demands for these solutions in terms of availability and isolation? How do we cater for these needs in a shared runtime environment? While there's going to be technical solutions highlighted in this session, we will also to a large extent take a look at the organization developing solutions targeting a shared runtime environment and how different organizations pose different requirements and challenges. This session will only cover WebSphere Message Broker on distributed platforms.

Transcript

1. © 2008 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM. TMM20: WebSphere Message Broker in Shared Runtime Environments Mårten Gustafson, Zystems marten.gustafson@zystems.se
2. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 •- The integration infrastructure evolution •- As integration becomes it’s own discipline the need for organizational models around it becomes apparent •- Enter the Integration Competency Center (ICC)
3. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Point to Point Spaghetti •- The integration infrastructure evolution •- As integration becomes it’s own discipline the need for organizational models around it becomes apparent •- Enter the Integration Competency Center (ICC)
4. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Point to Point Spaghetti Middleware Adoption •- The integration infrastructure evolution •- As integration becomes it’s own discipline the need for organizational models around it becomes apparent •- Enter the Integration Competency Center (ICC)
5. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Point to Point Spaghetti Middleware Adoption EAI Focusing on Application Integration •- The integration infrastructure evolution •- As integration becomes it’s own discipline the need for organizational models around it becomes apparent •- Enter the Integration Competency Center (ICC)
6. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Point to Point Spaghetti Middleware Adoption EAI Focusing on Application Integration ESB Focusing on Reusable Services •- The integration infrastructure evolution •- As integration becomes it’s own discipline the need for organizational models around it becomes apparent •- Enter the Integration Competency Center (ICC)
7. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Point to Point Spaghetti Middleware Adoption EAI Focusing on Application Integration ESB Focusing on Reusable Services BPM Business Process Management •- The integration infrastructure evolution •- As integration becomes it’s own discipline the need for organizational models around it becomes apparent •- Enter the Integration Competency Center (ICC)
8. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Parts of an ICC (according to Zystems) •- Communication •- Internal sales, marketing/PR •- Governance •- Architectural guidelines, patterns harvesting, best practices •- Delivery •- Development, QA •- Operations •- Runtime management, maintenance, patching, sizing, etc
9. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Parts of an ICC (according to Zystems) Communication •- Communication •- Internal sales, marketing/PR •- Governance •- Architectural guidelines, patterns harvesting, best practices •- Delivery •- Development, QA •- Operations •- Runtime management, maintenance, patching, sizing, etc
10. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Parts of an ICC (according to Zystems) Communication Governance •- Communication •- Internal sales, marketing/PR •- Governance •- Architectural guidelines, patterns harvesting, best practices •- Delivery •- Development, QA •- Operations •- Runtime management, maintenance, patching, sizing, etc
11. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Parts of an ICC (according to Zystems) Communication Delivery Governance •- Communication •- Internal sales, marketing/PR •- Governance •- Architectural guidelines, patterns harvesting, best practices •- Delivery •- Development, QA •- Operations •- Runtime management, maintenance, patching, sizing, etc
12. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Parts of an ICC (according to Zystems) Communication Delivery Governance Operations •- Communication •- Internal sales, marketing/PR •- Governance •- Architectural guidelines, patterns harvesting, best practices •- Delivery •- Development, QA •- Operations •- Runtime management, maintenance, patching, sizing, etc
13. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Parts of an ICC (according to Zystems) Communication Delivery Governance Operations ICC •- Communication •- Internal sales, marketing/PR •- Governance •- Architectural guidelines, patterns harvesting, best practices •- Delivery •- Development, QA •- Operations •- Runtime management, maintenance, patching, sizing, etc
14. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Parts of an ICC (according to Zystems) Communication Delivery Governance Operations ICC •- Communication •- Internal sales, marketing/PR •- Governance •- Architectural guidelines, patterns harvesting, best practices •- Delivery •- Development, QA •- Operations •- Runtime management, maintenance, patching, sizing, etc
15. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Agenda •- Projects (delivery) poses requirements on runtime environments (operations) •- Runtime environments dictate usage patterns/restricitions/possiblities (explicit or implicit) that must be documented (governance) •- Projects (delivery) must conform to the runtime environment constraints (governance)
16. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Agenda Delivery Governance Operations •- Projects (delivery) poses requirements on runtime environments (operations) •- Runtime environments dictate usage patterns/restricitions/possiblities (explicit or implicit) that must be documented (governance) •- Projects (delivery) must conform to the runtime environment constraints (governance)
17. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Development Organizations and their requirements on a shared runtime
18. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 ICC Organizational Models as defined by Schmidt & Lyle in “Integration Competency Center, An Implementation Methodology” © 2005, Informatica •- Project silos: No sharing, no methodology •- Best-practices: Standard procedures for operations, development, patterns - No support or development •- Standard services: Standardizes hardware and software, naming conventions and metadata specifications •- Shared-services: Provides standardized runtime and manages it (HA, D/R etc), supports testing, development, deployment, repository •- Central-services: Central development, operations and governance organization, Budget and charge-back models etc, Dependency tracking and asset management •- Self-service: Automated processes (“BPM your ICC”), self-service release, change and configuration management tools •- Shared and central is the most common from our experience Recommended reading: Schmidt, John Lyle, David Integration Competency Center An Implementation Methodology © 2005 Informatica Corporation
19. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 ICC Organizational Models as defined by Schmidt & Lyle in “Integration Competency Center, An Implementation Methodology” © 2005, Informatica Project silos •- Project silos: No sharing, no methodology •- Best-practices: Standard procedures for operations, development, patterns - No support or development •- Standard services: Standardizes hardware and software, naming conventions and metadata specifications •- Shared-services: Provides standardized runtime and manages it (HA, D/R etc), supports testing, development, deployment, repository •- Central-services: Central development, operations and governance organization, Budget and charge-back models etc, Dependency tracking and asset management •- Self-service: Automated processes (“BPM your ICC”), self-service release, change and configuration management tools •- Shared and central is the most common from our experience Recommended reading: Schmidt, John Lyle, David Integration Competency Center An Implementation Methodology © 2005 Informatica Corporation
20. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 ICC Organizational Models as defined by Schmidt & Lyle in “Integration Competency Center, An Implementation Methodology” © 2005, Informatica Project silos Best practices •- Project silos: No sharing, no methodology •- Best-practices: Standard procedures for operations, development, patterns - No support or development •- Standard services: Standardizes hardware and software, naming conventions and metadata specifications •- Shared-services: Provides standardized runtime and manages it (HA, D/R etc), supports testing, development, deployment, repository •- Central-services: Central development, operations and governance organization, Budget and charge-back models etc, Dependency tracking and asset management •- Self-service: Automated processes (“BPM your ICC”), self-service release, change and configuration management tools •- Shared and central is the most common from our experience Recommended reading: Schmidt, John Lyle, David Integration Competency Center An Implementation Methodology © 2005 Informatica Corporation
21. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 ICC Organizational Models as defined by Schmidt & Lyle in “Integration Competency Center, An Implementation Methodology” © 2005, Informatica Project silos Best practices Standard services •- Project silos: No sharing, no methodology •- Best-practices: Standard procedures for operations, development, patterns - No support or development •- Standard services: Standardizes hardware and software, naming conventions and metadata specifications •- Shared-services: Provides standardized runtime and manages it (HA, D/R etc), supports testing, development, deployment, repository •- Central-services: Central development, operations and governance organization, Budget and charge-back models etc, Dependency tracking and asset management •- Self-service: Automated processes (“BPM your ICC”), self-service release, change and configuration management tools •- Shared and central is the most common from our experience Recommended reading: Schmidt, John Lyle, David Integration Competency Center An Implementation Methodology © 2005 Informatica Corporation
22. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 ICC Organizational Models as defined by Schmidt & Lyle in “Integration Competency Center, An Implementation Methodology” © 2005, Informatica Project silos Best practices Standard services Shared services •- Project silos: No sharing, no methodology •- Best-practices: Standard procedures for operations, development, patterns - No support or development •- Standard services: Standardizes hardware and software, naming conventions and metadata specifications •- Shared-services: Provides standardized runtime and manages it (HA, D/R etc), supports testing, development, deployment, repository •- Central-services: Central development, operations and governance organization, Budget and charge-back models etc, Dependency tracking and asset management •- Self-service: Automated processes (“BPM your ICC”), self-service release, change and configuration management tools •- Shared and central is the most common from our experience Recommended reading: Schmidt, John Lyle, David Integration Competency Center An Implementation Methodology © 2005 Informatica Corporation
23. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 ICC Organizational Models as defined by Schmidt & Lyle in “Integration Competency Center, An Implementation Methodology” © 2005, Informatica Project silos Best practices Standard services Shared services Central services •- Project silos: No sharing, no methodology •- Best-practices: Standard procedures for operations, development, patterns - No support or development •- Standard services: Standardizes hardware and software, naming conventions and metadata specifications •- Shared-services: Provides standardized runtime and manages it (HA, D/R etc), supports testing, development, deployment, repository •- Central-services: Central development, operations and governance organization, Budget and charge-back models etc, Dependency tracking and asset management •- Self-service: Automated processes (“BPM your ICC”), self-service release, change and configuration management tools •- Shared and central is the most common from our experience Recommended reading: Schmidt, John Lyle, David Integration Competency Center An Implementation Methodology © 2005 Informatica Corporation
24. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 ICC Organizational Models as defined by Schmidt & Lyle in “Integration Competency Center, An Implementation Methodology” © 2005, Informatica Project silos Best practices Standard services Shared services Central services Self-service •- Project silos: No sharing, no methodology •- Best-practices: Standard procedures for operations, development, patterns - No support or development •- Standard services: Standardizes hardware and software, naming conventions and metadata specifications •- Shared-services: Provides standardized runtime and manages it (HA, D/R etc), supports testing, development, deployment, repository •- Central-services: Central development, operations and governance organization, Budget and charge-back models etc, Dependency tracking and asset management •- Self-service: Automated processes (“BPM your ICC”), self-service release, change and configuration management tools •- Shared and central is the most common from our experience Recommended reading: Schmidt, John Lyle, David Integration Competency Center An Implementation Methodology © 2005 Informatica Corporation
25. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 ICC Organizational Models as defined by Schmidt & Lyle in “Integration Competency Center, An Implementation Methodology” © 2005, Informatica Project silos Best practices Standard services Shared services Central services Self-service •- Project silos: No sharing, no methodology •- Best-practices: Standard procedures for operations, development, patterns - No support or development •- Standard services: Standardizes hardware and software, naming conventions and metadata specifications •- Shared-services: Provides standardized runtime and manages it (HA, D/R etc), supports testing, development, deployment, repository •- Central-services: Central development, operations and governance organization, Budget and charge-back models etc, Dependency tracking and asset management •- Self-service: Automated processes (“BPM your ICC”), self-service release, change and configuration management tools •- Shared and central is the most common from our experience Recommended reading: Schmidt, John Lyle, David Integration Competency Center An Implementation Methodology © 2005 Informatica Corporation
26. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Typical shared services ICC BU BU BU BU Central “governing” ICC •- Standard services provide a central, possibly shared runtime environment •- Should participate in projects to understand needs, solutions and reasons for workarounds •- Good in theory, hard to enforce •- Must provide incentive towards the business (internal sales and marketing)
27. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Typical shared services ICC BU Project / dev team BU Project / dev team BU Project / dev team BU Project / dev team Central “governing” ICC •- Standard services provide a central, possibly shared runtime environment •- Should participate in projects to understand needs, solutions and reasons for workarounds •- Good in theory, hard to enforce •- Must provide incentive towards the business (internal sales and marketing)
28. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Typical shared services ICC BU Project / dev team BU Project / dev team BU Project / dev team BU Project / dev team ICC Central “governing” ICC •- Standard services provide a central, possibly shared runtime environment •- Should participate in projects to understand needs, solutions and reasons for workarounds •- Good in theory, hard to enforce •- Must provide incentive towards the business (internal sales and marketing)
29. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Typical shared services ICC BU Project / dev team BU Project / dev team BU Project / dev team BU Project / dev team ICC Governance Central “governing” ICC •- Standard services provide a central, possibly shared runtime environment •- Should participate in projects to understand needs, solutions and reasons for workarounds •- Good in theory, hard to enforce •- Must provide incentive towards the business (internal sales and marketing)
30. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Typical shared services ICC BU Project / dev team BU Project / dev team BU Project / dev team BU Project / dev team ICC Governance Operations Central “governing” ICC •- Standard services provide a central, possibly shared runtime environment •- Should participate in projects to understand needs, solutions and reasons for workarounds •- Good in theory, hard to enforce •- Must provide incentive towards the business (internal sales and marketing)
31. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Shared services runtime • Characteristics Central operations Used by project teams from disparate business units • Key things Isolation Auditing/Monitoring
32. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 ICC Example - Customer Case 1 • Shared Services ICC ~10 headcount  Architects and managers Operations as a separate entity  ~4 headcount ~4 projects on their way into the runtime •- Aims to provide assets and core ESB functionality •- Lack of presence in the projects themselves which cripples the establishment of patterns, practices etc
33. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Typical central services ICC BU BU BU BU ICC Delivery (project / dev team) Operations Governance •- While small (<10 people) •- Easy re-use and tuning of both technology and processes (agility) •- Large (>10 people) •- The ICC starts requiring internal processes to function •- Re-use •- Becomes harder •- Introduces risk of ripple effect when changing a component •-“Easier” and more “safe” to DIY •- Risk of “integration spaghetti” within the ESB
34. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Central services runtime • Used only by the ICC Central control within a closed team • Characteristics Central operations Used only by the ICC Central control within a closed team • Key things Message tracing/tracking Development guidelines/re-use/patterns •- When something goes wrong everybody points to the middle(ware) •- Track and trace to quickly be able to tell where a message is
35. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 ICC Example - Customer Case 2 • Central Services ICC ~50 headcount  Developers, architects, process modelers, operations staff ~330 flows  File transfer 48,5%  Transformation 35,9% Re-use Some, easy and obvious (e.g routing) Other than that, Re-use at “development time” (e.g “copy/paste”)
36. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 •- For the vast majority of organizations one, central, logical environment will suffice, however one physical will not…
37. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 •- For the vast majority of organizations one, central, logical environment will suffice, however one physical will not…
38. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Shared Runtime Environments
39. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Typical environment configurations Single node Active/Passive (Cold Standby) •Same IP address, etc •Failover delay incurred by cluster or virtualization technique Active/Active : >1 nodes •Multiple IP addresses •Participating nodes must be proxied by MQ cluster, HTTP load balancers or such
40. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Typical environment configurations Broker A Single node Active/Passive (Cold Standby) •Same IP address, etc •Failover delay incurred by cluster or virtualization technique Active/Active : >1 nodes •Multiple IP addresses •Participating nodes must be proxied by MQ cluster, HTTP load balancers or such
41. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Typical environment configurations Broker A Cluster or virtualization technique Single node Active/Passive (Cold Standby) •Same IP address, etc •Failover delay incurred by cluster or virtualization technique Active/Active : >1 nodes •Multiple IP addresses •Participating nodes must be proxied by MQ cluster, HTTP load balancers or such
42. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Broker B Typical environment configurations Broker A WMQ cluster / HTTP load balancer Single node Active/Passive (Cold Standby) •Same IP address, etc •Failover delay incurred by cluster or virtualization technique Active/Active : >1 nodes •Multiple IP addresses •Participating nodes must be proxied by MQ cluster, HTTP load balancers or such
43. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Broker B Typical environment configurations Broker A WMQ cluster / HTTP load balancer Single node Active/Passive (Cold Standby) •Same IP address, etc •Failover delay incurred by cluster or virtualization technique Active/Active : >1 nodes •Multiple IP addresses •Participating nodes must be proxied by MQ cluster, HTTP load balancers or such
44. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Broker B Typical environment configurations Broker A WMQ cluster / HTTP load balancer Single node Active/Passive (Cold Standby) •Same IP address, etc •Failover delay incurred by cluster or virtualization technique Active/Active : >1 nodes •Multiple IP addresses •Participating nodes must be proxied by MQ cluster, HTTP load balancers or such
45. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 QoS – Performance and Availability
46. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 QoS – Performance and Availability Cluster or virtualization technique WMQ cluster + HTTP load balancer High performance “zone” - Active/Active - Workload balancing - Continuous availability Broker A Broker B
47. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 QoS – Performance and Availability Cluster or virtualization technique WMQ cluster + HTTP load balancer High performance “zone” - Active/Active - Workload balancing - Continuous availability Broker A Low performance “zone” - One node - Failover delay Broker C Broker B
48. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Broker Isolation / Partitioning Broker EG •Permissions •Separate address spaces / processes •Simplifies maintenance and migration •Easier to log/audit •Different OS’s provide different facilities for enforcing limits on resource utilization for processes
49. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Broker Isolation / Partitioning Broker EG EG EG EG •Permissions •Separate address spaces / processes •Simplifies maintenance and migration •Easier to log/audit •Different OS’s provide different facilities for enforcing limits on resource utilization for processes
50. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Broker Isolation / Partitioning Broker EG EG EG EG •Permissions •Separate address spaces / processes •Simplifies maintenance and migration •Easier to log/audit •Different OS’s provide different facilities for enforcing limits on resource utilization for processes
51. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Broker Isolation / Partitioning Broker EG EG EG EG •Permissions •Separate address spaces / processes •Simplifies maintenance and migration •Easier to log/audit •Different OS’s provide different facilities for enforcing limits on resource utilization for processes
52. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Broker Isolation / Partitioning Broker EG EG EG EG •Permissions •Separate address spaces / processes •Simplifies maintenance and migration •Easier to log/audit •Different OS’s provide different facilities for enforcing limits on resource utilization for processes
53. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Broker Isolation / Partitioning Broker EG EG EG EG •Permissions •Separate address spaces / processes •Simplifies maintenance and migration •Easier to log/audit •Different OS’s provide different facilities for enforcing limits on resource utilization for processes
54. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Examples of real world environments
55. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Customer example 1 MQ cluster Solaris zones + Veritas cluster Broker A Broker B GW QM HTTP Load Balancer Broker C Dedicated, per project Shared between projects (preferred) Broker C Broker … Broker … No stats available 
56. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Customer example 2 MQ cluster A MQ cluster B Broker B2 Broker B1 Broker A1 Broker A2 GW QM1 GW QM2 Extranet DMZ Intranet HTTP load balancer HTTP load balancer - ~300 flows - ~25 000 messages / hour - Active/Active 2 nodes - Windows
57. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Customer example 3 LPAR LPAR AIX HACMP Broker A Broker A - ~130 flows - ~5000 messages / hour - Single node / Cold standby - AIX cluster failover to other LPAR
58. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Customer example 4 MQ cluster Microsoft Cluster Services Broker A Broker B GW QM - ~250 flows - ~60 000 messages / hour - Active/Active 2 nodes - Windows
59. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Active/Active Runtime Environments and Implications on Development
60. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 State •State on queue •Fragmented collections/aggregations •Due to QM failure (part of collection/aggregation marooned, other part on different QM) •Due to MQ workload balancing (collection/aggregation split between different QMs) •Long vs Short term state should be considered •A collection/aggregation that isn’t completed within a few seconds might be obsolete anyway, ie req/rep and client timeout etc
61. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Concurrency •File nodes •Rely on OS level file locking •Synchronized within the same execution group •Unlocked files will be processed, might cause corrupt/incomplete data if not finished •FTP retrieval not synchronized •Watch out for concurrency •Workarounds •Adapters and other file transfer solution
62. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 HTTP Broker A Broker B ? biphttplistener biphttplistener •HTTP Input •Externalized listener •Servlet in an embedded Apache Tomcat •Moveable with SupportPac IE01 •See also •http://www.ibm.com/developerworks/websphere/library/techarticles/0608_braithwaite/0608_braithwaite.html
63. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 HTTP Broker A HTTP load balancer Broker B ! biphttplistener biphttplistener •HTTP Input •Externalized listener •Servlet in an embedded Apache Tomcat •Moveable with SupportPac IE01 •See also •http://www.ibm.com/developerworks/websphere/library/techarticles/0608_braithwaite/0608_braithwaite.html
64. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 HTTP Broker A HTTP load balancer Broker B ! biphttplistener biphttplistener SupportPac IE01 •HTTP Input •Externalized listener •Servlet in an embedded Apache Tomcat •Moveable with SupportPac IE01 •See also •http://www.ibm.com/developerworks/websphere/library/techarticles/0608_braithwaite/0608_braithwaite.html
65. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 SOAP Broker B Broker A WMQ cluster / HTTP load balancer EG-embedded listener EG-embedded listener •SOAP Input •Internalized listener •Apache Axis •Not moveable with IE01 (at least as of now) •Ports per execution group
66. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Heads up •- Consult your EIS admnistrator/vendor/consultand to ensure that the nodes work as competing consumers
67. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Timers / Schedules •Can you handle dual deployment of timed flows?
68. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 TCP/IP input •IP-sprayer/load balancer
69. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Wrap up
70. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 General lack of data and best pracices • Why is there so little data, patterns and practices available out in the “wild”? In our experience  Because the products “just work” Both good and bad  Good: products proven as very stable for mission critical operation  Bad: if you break new ground or think “outside the box” there’s not much experience, best practices available, assets or patterns For being such a mature product WMB lacks public, and agreed upon, development guidelines, patterns and assets
71. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Examples of mission critical deployments • Manufacturing industry • Banking/Trading • Pension funds management • Construction If the integration platform stop, the business stop Some of our customers industries
72. Introducing DataPower TMM20: WebSphere Message Broker in Shared Runtime Environments WebSphere Technical Conference and Transaction & Messaging Technical Conference Barcelona, Spain, 2008 Shared runtime - Key takeaways • Isolate  Execution Groups as the unit of isolation  Examine your OS ability to limit resources for processes and/or users • Automate  Broker and EG creation  Permissions  File system structures  Deployment  Consider self-service deployment (at least for test/QA environments) • Govern  Development guidelines / harvest patterns / document key concepts  Implementation patterns adapted to the runtime environment  Req/Rep, Pub/Sub, Fan in/out, Collection, FTP, File transfer etc  Make sure the people responsible for governance participate in projects themselves