Tweet
Tweet

Slide 1

Slide 1 text

#redhat #rhsummit UNIFYING ANALYTICS UNIFYING ANALYTICS ACROSS DATA SOURCES WITH ACROSS DATA SOURCES WITH RED HAT RED HAT JBOSS DATA JBOSS DATA VIRTUALIZATION VIRTUALIZATION Naveen Malik - Principal Software Engineer Burak Serdar - Principal Software Engineer Ian Firman - Business Intelligence Architect June 29, 2016 1 1

Slide 2

Slide 2 text

GET THIS DECK NOW... GET THIS DECK NOW... #redhat #rhsummit 2 2

Slide 3

Slide 3 text

RED HAT RED HAT IT JOURNEY IT JOURNEY #redhat #rhsummit Opportunity Research Test Plan Execute Maintain 3 3

Slide 4

Slide 4 text

OUR JOURNEY TODAY OUR JOURNEY TODAY #redhat #rhsummit Opportunity Research Test Plan Execute Maintain 4 4 Ian Firman Burak Serdar Naveen Malik

Slide 5

Slide 5 text

WHAT IS JBOSS DATA WHAT IS JBOSS DATA VIRTUALIZATION? VIRTUALIZATION? #redhat #rhsummit JBoss Data Virtualization is a data abstraction solution that sits in front of multiple data sources, allows them to be treated as a single source and accessed by various data consumers and/or applications Heterogeneous Sources Heterogeneous Clients 5 5

Slide 6

Slide 6 text

OPPORTUNITY OPPORTUNITY BUSINESS CASE BUSINESS CASE #redhat #rhsummit Opportunity 6 6

Slide 7

Slide 7 text

Data is coming in fast #redhat #rhsummit OPPORTUNITY OPPORTUNITY 7 7

Slide 8

Slide 8 text

Data is coming in fast Diverse data consumers #redhat #rhsummit OPPORTUNITY OPPORTUNITY 8 8

Slide 9

Slide 9 text

Diverse data sources Data is coming in fast Diverse data consumers #redhat #rhsummit OPPORTUNITY OPPORTUNITY 9 9

Slide 10

Slide 10 text

OPPORTUNITY OPPORTUNITY Diverse data sources Data is coming in fast Diverse data consumers #redhat #rhsummit Combining real time and historical data 10 10

Slide 11

Slide 11 text

RED HAT RED HAT USE CASES USE CASES TRANSACTIONAL TRANSACTIONAL #redhat #rhsummit LOW COMPLEXITY (ONE END POINT) LOW COMPLEXITY (ONE END POINT) Report from Bugzilla Simplify queries Filter sensitive data Extract to Warehouse 11 11

Slide 12

Slide 12 text

RED HAT RED HAT USE CASES USE CASES OPERATIONAL OPERATIONAL #redhat #rhsummit MEDIUM COMPLEXITY (MANY END POINTS) MEDIUM COMPLEXITY (MANY END POINTS) "Where's my message?" Choice of reporting tools 12 12

Slide 13

Slide 13 text

RED HAT RED HAT USE CASES USE CASES ANALYTICAL ANALYTICAL #redhat #rhsummit HIGH COMPLEXITY (MANY END POINTS, DYNAMIC) HIGH COMPLEXITY (MANY END POINTS, DYNAMIC) Data Science - Discovery, Mining, Advanced Analytics Real-time and Historical 13 13

Slide 14

Slide 14 text

"All problems in computer science can be solved by another level of abstraction" paraphrased from David Wheeler #redhat #rhsummit 14 14

Slide 15

Slide 15 text

LOGICAL ARCHITECTURE LOGICAL ARCHITECTURE #redhat #rhsummit Data Sources Data Warehouse(s) Files Data Consumers BI Reports & Analytics Mobile ESB & ETL 15 15

Slide 16

Slide 16 text

LOGICAL ARCHITECTURE LOGICAL ARCHITECTURE #redhat #rhsummit Data Sources Data Warehouse(s) Files 16 16 Data Consumers BI Reports & Analytics Mobile ESB & ETL

Slide 17

Slide 17 text

LOGICAL ARCHITECTURE LOGICAL ARCHITECTURE #redhat #rhsummit JBOSS Data Virtualization Integrated and abstracted sources Data Sources 17 17 Data Warehouse(s) Files Data Consumers BI Reports & Analytics Mobile ESB & ETL Multiple protocol access

Slide 18

Slide 18 text

LOGICAL ARCHITECTURE LOGICAL ARCHITECTURE #redhat #rhsummit Business Logic and data formatting Central Security JBOSS Data Virtualization Integrated and abstracted sources 18 18 Data Sources Data Warehouse(s) Files Data Consumers BI Reports & Analytics Mobile ESB & ETL Multiple protocol access

Slide 19

Slide 19 text

LOGICAL ARCHITECTURE LOGICAL ARCHITECTURE #redhat #rhsummit Business Logic and data formatting Central Security JBOSS Data Virtualization Integrated and abstracted sources 19 19 Data Sources Data Warehouse(s) Files Data Consumers BI Reports & Analytics Mobile ESB & ETL Multiple protocol access

Slide 20

Slide 20 text

VIRTUAL DATABASES VIRTUAL DATABASES DEFINITION DEFINITION #redhat #rhsummit A virtual database (or VDB) is a container for components used to integrate data from multiple data sources, so that they can be accessed in an integrated manner through a single, uniform API. 20 20

Slide 21

Slide 21 text

VDB STRATEGY VDB STRATEGY SOURCES SOURCES #redhat #rhsummit 21 21

Slide 22

Slide 22 text

VDB STRATEGY VDB STRATEGY BASE VDB BASE VDB Abstracts the physical source #redhat #rhsummit 22 22

Slide 23

Slide 23 text

VDB STRATEGY VDB STRATEGY VIRTUAL DATA MART VIRTUAL DATA MART #redhat #rhsummit Combine Base VDBs for analysis/reporting Business logic/formatting applied Data security applied at this layer Abstracts the physical source 23 23

Slide 24

Slide 24 text

#redhat #rhsummit DEVELOP DEVELOP VDB LIFECYCLE VDB LIFECYCLE Virtual Database 24 24

Slide 25

Slide 25 text

#redhat #rhsummit DEPLOY DEPLOY VDB LIFECYCLE VDB LIFECYCLE Virtual Database 25 25

Slide 26

Slide 26 text

#redhat #rhsummit USE! USE! VDB LIFECYCLE VDB LIFECYCLE Virtual Database 26 26

Slide 27

Slide 27 text

PLAN PLAN ARCHITECTURE ARCHITECTURE #redhat #rhsummit Opportunity Plan 27 27

Slide 28

Slide 28 text

THINGS TO CONSIDER THINGS TO CONSIDER DESIGN OPPORTUNITIES DESIGN OPPORTUNITIES #redhat #rhsummit 28 28

Slide 29

Slide 29 text

THINGS TO CONSIDER THINGS TO CONSIDER FLEXIBILITY FLEXIBILITY #redhat #rhsummit WHERE TO DEPLOY? WHERE TO DEPLOY? Cloud and on premise Real-time data vs. materialized views Locality to clients 29 29

Slide 30

Slide 30 text

OPEN HYBRID CLOUD OPEN HYBRID CLOUD Heterogeneous Sources Heterogeneous Clients #redhat #rhsummit 30 30

Slide 31

Slide 31 text

THINGS TO CONSIDER THINGS TO CONSIDER SCALABILITY SCALABILITY #redhat #rhsummit HOW EASY TO SCALE HOW EASY TO SCALE Initial starting point Easily scale by adding and removing nodes Clients isolated from infrastructure changes 31 31

Slide 32

Slide 32 text

SIZING SIZING JDV SIZING TOOL JDV SIZING TOOL INPUT: REQUIREMENTS INPUT: REQUIREMENTS OUTPUT: RECOMMENDATION OUTPUT: RECOMMENDATION How much data? How is data being accessed? CPU Storage Memory JVM Architecture #redhat #rhsummit 32 32

Slide 33

Slide 33 text

THINGS TO CONSIDER THINGS TO CONSIDER SECURITY SECURITY #redhat #rhsummit IN TRANSIT, AT REST, AUTH? IN TRANSIT, AT REST, AUTH? Transport Layer Security (TLS) Disk encryption Authentication Authorization 33 33

Slide 34

Slide 34 text

JBOSS EAP ROLE MANAGEMENT JBOSS EAP ROLE MANAGEMENT SAML LDAP Basic Auth Custom more.. SECURITY SECURITY ROLE BASED ACCESS CONTROL ROLE BASED ACCESS CONTROL #redhat #rhsummit 34 34

Slide 35

Slide 35 text

Object #redhat #rhsummit SECURITY SECURITY DEFINED AT... DEFINED AT... 35 35

Slide 36

Slide 36 text

Object Row SECURITY SECURITY DEFINED AT... DEFINED AT... #redhat #rhsummit 36 36

Slide 37

Slide 37 text

Object Row Field #redhat #rhsummit SECURITY SECURITY DEFINED AT... DEFINED AT... 37 37

Slide 38

Slide 38 text

PHYSICAL ARCHITECTURE PHYSICAL ARCHITECTURE Amazon Web Services and AWS are trademarks of Amazon.com, Inc. or its affiliates in the United States and/or other countries. #redhat #rhsummit 38 38

Slide 39

Slide 39 text

EXECUTE EXECUTE IMPLEMENTATION IMPLEMENTATION Opportunity Plan Execute #redhat #rhsummit 39 39

Slide 40

Slide 40 text

#redhat #rhsummit Virtual Database MAKING IT HAPPEN! MAKING IT HAPPEN! TOOLS TOOLS 40 40

Slide 41

Slide 41 text

#redhat #rhsummit Virtual Database MAKING IT HAPPEN! MAKING IT HAPPEN! TOOLS TOOLS jcliff 41 41

Slide 42

Slide 42 text

PUPPET & JCLIFF PUPPET & JCLIFF #redhat #rhsummit jcliff . . . Configuration Snippets 42 42

Slide 43

Slide 43 text

PUPPET & JCLIFF PUPPET & JCLIFF #redhat #rhsummit jcliff . . . Differences Configuration Snippets 43 43

Slide 44

Slide 44 text

PUPPET & JCLIFF PUPPET & JCLIFF #redhat #rhsummit Differences jcliff . . . Configuration Snippets 44 44

Slide 45

Slide 45 text

DATA SOURCES DATA SOURCES REDSHIFT RESOURCE REDSHIFT RESOURCE #redhat #rhsummit jcliff::datasource { 'redshift_ds': } jndi_name => 'java:/redshift', url => hiera('jdvbi::redshift::url'), driver_name => 'RedshiftJDBC4-1.1.1.0001.jar', username => hiera('jdvbi::redshift::username'), 45 45

Slide 46

Slide 46 text

DATA SOURCES DATA SOURCES REDSHIFT RESOURCE REDSHIFT RESOURCE #redhat #rhsummit { "datasource" => { "redshift_ds" => { "jndi-name" => "java:/redshift", "driver-name" => "RedshiftJDBC4-1.1.1.0001.jar", "enabled" => "true", " " " " jcliff::datasource { 'redshift_ds': } jndi_name => 'java:/redshift', url => hiera('jdvbi::redshift::url'), driver_name => 'RedshiftJDBC4-1.1.1.0001.jar', username => hiera('jdvbi::redshift::username'), 46 46

Slide 47

Slide 47 text

#redhat #rhsummit jcliff::teiid_salesforce_ra { 'sfcom': jndi_name =>'java:/sf_ds', url => hiera('jdvbi::salesforce::url'), username => hiera('jdvbi:salesforce:username'), SALESFORCE SALESFORCE DEFINE RESOURCE ADAPTER DEFINE RESOURCE ADAPTER 47 47

Slide 48

Slide 48 text

#redhat #rhsummit { "resource-adapter" => { "sf_ra" => { "module" => "org.jboss.teiid.resource-adapter.salesforce:main", "transaction-support" => "NoTransaction", "connection-definitions" => { "sf_ra" => { "enabled" => true, "jndi-name" => "java:/sf_ds", "config-properties" => { jcliff::teiid_salesforce_ra { 'sfcom': jndi_name =>'java:/sf_ds', url => hiera('jdvbi::salesforce::url'), username => hiera('jdvbi:salesforce:username'), SALESFORCE SALESFORCE DEFINE RESOURCE ADAPTER DEFINE RESOURCE ADAPTER 48 48

Slide 49

Slide 49 text

CONNECTIONS CONNECTIONS SECURE DATABASE SECURE DATABASE # Add a teiid JDBC transport with TLS jcliff::configfile { 'ssl-jdbc.conf': content > template('jbossdvbi/ssl jdbc conf erb') #redhat #rhsummit 49 49

Slide 50

Slide 50 text

CONNECTIONS CONNECTIONS SECURE DATABASE SECURE DATABASE {"teiid" => { "transport" => { "jdbc" => { "keystore-key-alias" => "<%=@keystore_alias%>", "keystore-key-password" => "<%=@keystore_password%>", "keystore-password" => "<%=@keystore_password%>", "keystore-type" => "JKS", "socket-binding" => "teiid-jdbc", "ssl-authentication-mode" => "1-way", # Add a teiid JDBC transport with TLS jcliff::configfile { 'ssl-jdbc.conf': content > template('jbossdvbi/ssl jdbc conf erb') #redhat #rhsummit 50 50

Slide 51

Slide 51 text

CONNECTIONS CONNECTIONS SECURE CLIENTS SECURE CLIENTS # Add a socket binding for TLS JDBC jcliff::socket_binding { 'teiid-jdbc': #redhat #rhsummit 51 51

Slide 52

Slide 52 text

CONNECTIONS CONNECTIONS SECURE CLIENTS SECURE CLIENTS { "standard-sockets" => { "socket-binding" => { " " "t iid jdb " # Add a socket binding for TLS JDBC jcliff::socket_binding { 'teiid-jdbc': #redhat #rhsummit 52 52

Slide 53

Slide 53 text

VIRTUAL DATABASES VIRTUAL DATABASES DEPLOY VDB DEPLOY VDB #redhat #rhsummit # Retrieve VDB from staging area exec { 'get-vdb': command => "wget ..." } # deploy VDB 53 53

Slide 54

Slide 54 text

VIRTUAL DATABASES VIRTUAL DATABASES DEPLOY VDB DEPLOY VDB #redhat #rhsummit { "deployments" => { "my.vdb" => { # Retrieve VDB from staging area exec { 'get-vdb': command => "wget ..." } # deploy VDB 54 54

Slide 55

Slide 55 text

CONCLUSION CONCLUSION #redhat #rhsummit 55 55

Slide 56

Slide 56 text

VALUE VALUE IS IT WORKING OUT? IS IT WORKING OUT? #redhat #rhsummit CURRENT DEPLOYMENTS CURRENT DEPLOYMENTS MORE IN THE FUTURE! MORE IN THE FUTURE! Marketing VDB Data Scientists VDB 56 56

Slide 57

Slide 57 text

USEFUL NEW FEATURES USEFUL NEW FEATURES NEW THINGS FROM THE PRODUCT! NEW THINGS FROM THE PRODUCT! #redhat #rhsummit NEW AND EXCITING NEW AND EXCITING Dynamic VDB Unified RBAC Redshift Translator 57 57

Slide 58

Slide 58 text

CONCLUSION CONCLUSION PARTING THOUGHTS PARTING THOUGHTS Opportunity Plan Execute #redhat #rhsummit 58 58

Slide 59

Slide 59 text

CONCLUSION CONCLUSION PARTING THOUGHTS PARTING THOUGHTS Fast integration. Heterogeneous sources. Decoupling. #redhat #rhsummit Plan Execute 59 59

Slide 60

Slide 60 text

CONCLUSION CONCLUSION PARTING THOUGHTS PARTING THOUGHTS Centralize business logic. Centralize security. Flexible architecture. Fast integration. Heterogeneous sources. Decoupling. #redhat #rhsummit Execute 60 60

Slide 61

Slide 61 text

Repeatable. Automated. Supported. CONCLUSION CONCLUSION PARTING THOUGHTS PARTING THOUGHTS Centralize business logic. Centralize security. Flexible architecture. Fast integration. Heterogeneous sources. Decoupling. #redhat #rhsummit 61 61

Slide 62

Slide 62 text

#redhat #rhsummit QUESTIONS? QUESTIONS? 62 62

Slide 63

Slide 63 text

LEARN. NETWORK. LEARN. NETWORK. EXPERIENCE OPEN SOURCE. EXPERIENCE OPEN SOURCE. #redhat #rhsummit 63 63

Slide 64

Slide 64 text

APPENDIX APPENDIX ADDITIONAL MATERIAL ADDITIONAL MATERIAL #redhat #rhsummit 64 64

Slide 65

Slide 65 text

RESOURCES RESOURCES PRODUCTS PRODUCTS #redhat #rhsummit Red Hat JBoss Data Virtualization https://www.redhat.com/en/technologies/jboss-middleware/data-virtualization Red Rat CloudForms https://www.redhat.com/en/technologies/cloud-computing/cloudforms Ansible https://www.ansible.com/ 65 65

Slide 66

Slide 66 text

RESOURCES RESOURCES TOOLS TOOLS #redhat #rhsummit JBoss Data Virtualization Sizing Architecture Tool https://access.redhat.com/labs/jbossdvsat/ jcliff https://github.com/bserdar/jcliff puppet-jcliff https://github.com/bserdar/puppet-jcliff 66 66