Slide 1

Slide 1 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactional Systems Properties and Mechanisms Jorge Simão, Ph.D. [email protected] DB TX APP SingSUG – Singapore Spring Users Group 8 Jan. 2015

Slide 2

Slide 2 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

Slide 3

Slide 3 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Server Scenarios $ $ $

Slide 4

Slide 4 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactional Properties (ACID) ● Atomicity – Transaction as indivisible unit-of-work ● Consistency – Integrity Constraints on Data ● Isolation – Concurrent access should not lead to inconsistency ● Durability – Operation effects should persist

Slide 5

Slide 5 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Atomicity APP RDBMS DB read(A)=500 read(B)=200 write(A)=500-100=400 write(B)=200+100=300 A = 500 B = 200 ∑=700 ∑=700 ∑'=600 $ $ $ commit ID Balance A 500 B 700 T_ACCOUNT Consistency In the Presence of Failures Tx A B 100$

Slide 6

Slide 6 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Consistency ID Balance 1 500 2 700 T_ACCOUNT NOT NULL >=0 ID CITY COUNTRY AC_ID 100 Lisbon Portugal 1 T_ADDRESS FOREGIN KEY Impose Constraints on Data Model

Slide 7

Slide 7 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Isolation: Race Conditions APP RDBMS DB read(A)=500 read(A)=500 write(A)=500+100=600 write(A)=500+200=700 A = 500 commit commit Concurrency Control Tx 1 Tx 2 A 200$ A 100$ ∑=700 ∑'=600 $ TX 1 TX 2 $ $ Serializibility

Slide 8

Slide 8 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Durability APP RDBMS DB $ commit Tx Table Op Row Value Log/Journal Data Persistence Log

Slide 9

Slide 9 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Weak ACID Properties ● Benefits of ACID: – Guaranteed consistency – Simple programming model and error semantics ● Problem with ACID: – Expensive to implement with distributed resources ● How to Scale-Out Data Management? – Limited Scope of Transaction ● Atomicity only for resources in same Server – Eventual Consistency ● Delay Propagation of Updates Dropping Global Consistency

Slide 10

Slide 10 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples

Slide 11

Slide 11 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactional Resources ● Database Resources – RDBMS & ORM, NoSql DBs ● Messaging Resources – JMS API, AMQP, STOMP ● Enterprise Information System – Java Connector ● Other Data Resources – Middleware/App Level – LDAP, TX-FS, TX-VMem DB NoSql

Slide 12

Slide 12 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactions in Messaging P C C Broker queue: q1 TXp 1 TXc 2 DB send= write recv = read + [update] + [del]

Slide 13

Slide 13 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Failures and Message Duplicates Msg1 commit() process TxBroker.commit (crash) C $ Broker Msg1 TxBroker.rollback DB C

Slide 14

Slide 14 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Multi-Resource Transactions APP RDBMS DB Broker DB TXDB TXBroker

Slide 15

Slide 15 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Multi-Commits Partial-Failure and Duplicate Handling APP RDBMS DB Broker DB TXBroker $ TXDB Connection dbConnection = dataSource.getConnection(); Session session = brokerConnection.createSession(); //receive message(s) Message msg = ... //check for duplicates: YES → Ignore; NO → Process if (!isDuplicate(msg)) { //process messages //write to DB } dbConnection.commit(); session.commit();

Slide 16

Slide 16 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Slave 2 Master Replicated Data APP Server DB Slave 1 DB DB Write* Read* Read* .us .eu .apj APP Server .us .eu APP Server .apj connect(“master”, [“slave1”, “slave2”]) Synchronize Write* Write* Read* Master-Server Replication

Slide 17

Slide 17 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Master 3 Master 1 Clustering APP Server DB Master 2 APP Server APP Server connect([“master1”]) Master-Master Replication LAN DB DB Cluster connect([“master1”, “master2”, “master3”]) Current Primary (for Resource X)

Slide 18

Slide 18 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Replicated Message Brokers Broker: Master DB Broker: Slave 1 DB Broker: Slave 2 DB P C C P Federation vs. Clustering

Slide 19

Slide 19 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples

Slide 20

Slide 20 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Atomicity with Transactional Log write(A)= 400 write(B)= 300 A = 500 B = 200 commit ID Balance A 500 B 700 T_ACCOUNT Tx Op Row Value 9 W A 500 9 W C 900 9 Commit 10 Rollback 11 W A 400 11 W B 300 11 Commit 12 W C 400 Log/Journal Logging Transaction Operations Tx 11 DB RDBMS APP head tail ID Balance A 400 B 300 Flush Log Truncate Log

Slide 21

Slide 21 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Isolation: Locks APP RDBMS DB read(A)=500 read(A)=600 write(A)=500+100=600 write(A)=600+200=800 A = 500 commit commit Pessimistic Concurrency Control Tx1 Tx2 A 200$ A 100$ ∑=800 Lock(A;W) Lock(A;W) ACK ACK ACK Lock(A;W)

Slide 22

Slide 22 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Isolation: Versioning APP RDBMS DB A = 500 Optimistic Concurrency Control Tx1 Tx2 A 200$ A 100$ ACK read(A)=500;v=1 read(A)=500;v=1 write(A)=500+100=600;v=1 write(A)=500+200=700;v=1 commit rollback ∑=700 v=1 v=2 $ Rollback ID Balance Version A 700 2 B 300 1 T_ACCOUNT

Slide 23

Slide 23 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Read-Only Transactions read(A)=500 read(A)=600 A = 500 commit commit Tx1 Tx2 Lock(A;R) Lock(A;R) ACK A = 500 ● Report Generator ● Event Notifier ● Business Intelligence Report ● Page Profile ● Document Listing

Slide 24

Slide 24 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Isolation Levels read(A)=500 read(A)=500 write(A)=800 A = 500 commit commit Read-Only Concurrent with Write Transactions TxR 1 TxW 2 Lock(A;R) Lock(A;W) ACK ACK Repeatable-Read Read Committed read(A)=500 read(A)=800 write(A)=800 A = 500 commit commit TxR 1 TxW 2 Lock(A;R) Lock(A;W) ACK ACK read(A)=500 read(A)=800 write(A)=800 A = 500 commit rollback TxR 1 TxW 2 Lock(A;R) Lock(A;W) ACK Read Uncommitted A = 800 A = 800 A = 500 Dirty Read read(A)=500

Slide 25

Slide 25 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware DB DB Two-Phase Commit Protocol (XA) TxMgr Resource Mgr Coordinator Prepare x N Pre-commit 1 Pre-commit N Commit x N phase 1 phase 2 Pre-commit (keep logs & locks)

Slide 26

Slide 26 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Server 3 Server 1 Primary (Master) Auto-Failover DB Server 2 Heartbeat & Elections DB DB Previous Primary (for Resource X) $ ♥ ♥ ♥ $ $ Election New Primary (for Resource X) $ $

Slide 27

Slide 27 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Server 3 Server 1 Partition Detection & Handling DB Server 2 Quorum-Based Consensus DB DB ♥ $ Election Election Partition 1 Partition 2

Slide 28

Slide 28 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples

Slide 29

Slide 29 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JDBC Transaction Management public void transfer(Long debitId, Long creditId, BigDecimal amount) { String sql = “update T_ACCOUNT set BALANCE=? WHERE ID=?”; try { Connection connection = dataSource.getConnection(); connection.setAutoCommit(false); PreparedStatement st = connection.prepareStatement(sql); st.setParameter(1, balance1 + amount); st.setParameter(2, debitId); st.execute(); st.setParameter(1, balance2 + amount); st.setParameter(2, creditId); st.execute(); connection.commit(); } catch (SQLException e) { connection.rollback(); throw new DataAccessException(e); } finally { connection.close(); } Debit Operation Credit Operation TX

Slide 30

Slide 30 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Modular Application APP Driver RDBMS DB SQL results Infrastructure Layer Data-Access Layer Service layer Multi-Tier Architecture (2,3+ Layers) Presentation layers public interface AccountRepository { Account getAccount(Long id); void updateBalance(Account account); ... }

Slide 31

Slide 31 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transaction Management ● Transaction scope may include: – Multiple calls to the same DAO – Operations in different DAOs – Operations in different Resources (JTA) ● Data-Access Technology Independence: – Service-Layer Implementation – DAO API ● Connection Must be Shared – Thread/Request Contextualized Service Layer Data-Access Layer Issues

Slide 32

Slide 32 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactional Interceptor ServiceImpl DAO Transactional Interceptor TransactionManager Proxy begin() commit() Client Object DB AOP – with Proxy Power Service Thread-Safe

Slide 33

Slide 33 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transaction Management Annotation-Driven Declarative Demarcation @Singleton public class TransferServiceImpl implements TransferService { @Inject private AccountRepository accountRepository; @Inject private OperationsRepository operationsRepository; @Transactional public Transfer transfer(Long creditId, Long debitId, BigDecimal amount) { // Do transfer with AccountRepository ... // Write confirmation with OperationsRepository ... ... } }

Slide 34

Slide 34 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transaction Management Local Transaction Propagation @Singleton public class TransferServiceImpl implements TransferService { ... @Transactional public Transfer transfer(...) { // do transfer // write confirmation with OperationsRepository ... … emailService.send(msg); } } @Singleton public class EmailServiceImpl implements EmailService { @Transactional public void send(Message msg) { ... } } method1 method2 TX 1 TX 1 TX 2

Slide 35

Slide 35 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transaction Management Remote Transaction Propagation method1 method2 TX 1 TX 1 TX 2 public class TransferServiceImpl ... { ... @Transactional public Transfer transfer(...) { … remoteEmailService.send(msg); } } @Singleton public class RemoteEmailService ... { @Transactional public void send(Message msg) { ... } } Host1 Host2

Slide 36

Slide 36 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

Slide 37

Slide 37 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactions in Spring Uniform Approach @Service public class TransferServiceImpl implements TransferService { @Autowired private AccountRepository accountRepository; @Autowired private OperationsRepository operationsRepository; @Transactional public Transfer transfer(Long creditId, Long debitId, BigDecimal amount) { Account debitAccount = accountRepository.getAccount(creditId); Account creditAccount = accountRepository.getAccount(debitId); debitAccount.debit(amount); creditAccount.credit(amount); accountRepository.update(debitAccount); accountRepository.update(creditAccount); Transfer transfer = new Transfer(debitAccount, creditAccount, amount); operationsRepository.add(transfer); return transfer; } } import org.springframework.stereotype.Service; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.transaction.annotation.Transactional;

Slide 38

Slide 38 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transaction Configuration Isolation, Propagation, Rollback Rules @Service public class TransferServiceImpl implements TransferService { @Transactional(isolation=Isolation.SERIALIZABLE, noRollbackFor=EmailException.class) public Transfer transfer(Long creditId, Long debitId, BigDecimal amount) { … emailService.sendConfirmation(Transfer.buildMessage(transfer)); return transfer; } } @Service public class EmailServiceImpl implements EmailService { @Transactional(propagation=Propagation.REQUIRES_NEW) public void send(Message message) { … } }

Slide 39

Slide 39 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JDBC Repositories @Repository public class JdbcAccountRepository implements AccountRepository { @Autowired private DataSource dataSource; private JdbcTemplate jdbcTemplate; @PostConstruct public void init() { jdbcTemplate = new JdbcTemplate(dataSource); } public Account getAccount(Long id) { String sql = “select * from T_ACCOUNT where ID=?”; return jdbcTemplate.query(sql, id); } public void updateBalance(Account account) { String sql = “update T_ACCOUNT set BALANCE=? where ID=?”; jdbcTemplate.update(sql, account.getBalance(), account.getId()); } } import org.springframework.stereotype.Repository; import org.springframework.beans.factory.annotation.Autowired; import java.lang.annotation.PostConstruct;

Slide 40

Slide 40 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactions in Spring Configuration @Configuration @EnableTransactionManagament @Import({JdbcConfig.class, JpaConfig.class, DBConfig.class}); @ComponentScan(“myapp.services”) @PropertySource(“app.properties”) public class AppConfig { … } @Configuration @Profile(“jdbc”) public class JdbcConfig { @Bean PlatformTransactionManager txMgr(DataSource dataSource) { return new DataSourceTransactionManager(dataSource); } } @Configuration @Profile(“jpa”) public class JpaConfig { … } import org.springframework.context.annotation.*;

Slide 41

Slide 41 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactions in Spring Conditional Beans import org.springframework.context.annotation.*; @Configuration @ImportResource(“db-config.xml”) public class DBConfig { @Autowired private Environment env; @Bean @Reachable DataSource master() { .. } @Bean @Reachable DataSource slave1() { .. } @Bean @Reachable DataSource slave2() { .. } } public class ReachableCondition implements Condition { public boolean matches(ConditionContext context, AnnotatedTypeMetadata metadata) { … } } @Conditional(ReachableCondition.class) @Target({ElementType.TYPE, ElementType.METHOD}) @Retention({RetentionPolicy.RUNTIME}) public @interface Reachable { } import java.lang.annotation.*;

Slide 42

Slide 42 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

Slide 43

Slide 43 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware ORM Architecture Automating Object to/from RDB Mapping Cache Annotations XML Meta-Data Persistence API Database Abstraction Layer Client DAO DB SQL ✔ SQL Query Generation ✔ Mapping ✔ Column Extraction ✔ Parameter Extraction ✔ Association Managm. ✔ DB Abstraction ✔ Declarative Config ✔ Proxy Creation ✔ Schema Creation Config DataSource Properties Criteria Query API + OQL Queries Criteria Query

Slide 44

Slide 44 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JPA Architecture EntityManager & EntityManagerFactory public class JpaAccountRepository { EntityManager entityManager; ... } EntityManager Data-Access Object Entity Instance Entity Instance Persistence Context (Cache) Persistence Unit DB EntityManagerFactory Connection DataSource

Slide 45

Slide 45 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JPA Entities Annotation-Driven Meta-Data @Entity @Table(“T_ACCOUNT”) public class Account { @Id private Long id; private String name; @Column(name=“DOB”) @Temporal(TemporalType.DATE) private Date birthday; @OneToMany @JoinColumn(name=“ACCOUNT_ID”) private Set
addresses; public Account() { } } Physical Mapping @Table @Column @JoinColumn Annotations Logical/Model @Entity @Id @OneToMany 1:N Account Address

Slide 46

Slide 46 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JPA Transactions public class JpaAccountDao implements AccountDao { @Inject private EntityManagerFactory emf; public void importAccounts(List accounts) { EntityManager em = emf.createEntityManager(); Transaction tx = em.getTransaction(); try { tx.begin(); for (Accout account: accounts) { entityManager.persist(account); } tx.commit(); } catch (Exception e) { tx.rollback(); } } ... } Programmatic Demarcation

Slide 47

Slide 47 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JPA DAOs public class JpaAccountDao implements AccountDao { @PersistenceContext private EntityManager entityManager; public JpaAccountDao() {} public void add(Account account) { entityManager.persist(account); } ... } EntityManagerProxy EntityManager Proxy @Singleton @Repository import javax.persistence.PersistenceContext; EJB&JSR330 Spring @ApplicationScoped CDI

Slide 48

Slide 48 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JPA Configuration @Configuration @Profile(“jpa”) public class JpaConfig { @Bean PlatformTransactionManager txMgr(EntityManagerFactory emf) { return new JpaTransactionManager(emf); } @Bean LocalContainerEntityManagerFactoryBean createEntityManagerFactory(DataSource dataSource) { LocalContainerEntityManagerFactoryBean emfb = new LocalContainerEntityManagerFactoryBean(); emfb.setPersistenceUnitName("onlineshop"); emfb.setPackagesToScan(“com.onlineshop.domain”); emfb.setDataSource(dataSource); return new emfb; } }

Slide 49

Slide 49 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JPA Configuration import org.einnovator.jpa.provider.PersistenceUnitInfoBuilder; import org.einnovator.jpa.Jpa; @Singleton @Jpa public class JpaTestConfig { @Factory public EntityManagerFactory createEntityManagerFactory(DataSource dataSource) { return new PersistenceUnitInfoBuilder() .name("onlineshop") .packagesToScan(EntitySupport.class) .dataSource(dataSource) .property("hibernate.dialect", "org.hibernate.dialect.HSQLDialect") .property("hibernate.hbm2ddl.auto", "create") .property("hibernate.show_sql", true) .buildEntityManagerFactory(); } }

Slide 50

Slide 50 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

Slide 51

Slide 51 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware MySQL Replication ● Asynchronous Master-Slave ● Semi-Synchronous Master-Slave ● Clustering ● Partitioning Replication Architectures

Slide 52

Slide 52 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware MySQL Replication mysql> CHANGE MASTER TO -> MASTER_HOST='master_host_name', -> MASTER_USER='replication_user_name', -> MASTER_PASSWORD='replication_password', -> MASTER_LOG_FILE='recorded_log_file_name', -> MASTER_LOG_POS=recorded_log_position; [mysqld] log-bin=mysql-bin server-id=1 [mysqld] server-id=2 Master Slave Master-Slave Configuration

Slide 53

Slide 53 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JAVA-MySQL Connector JDBC Connection to Master+Servers @Configuration @ImportResource(“db-config.xml”) public class DBConfig { @Autowired private Environment env; @Bean @Profile(“mysqlWAN”) DataSource masterserver() { return new DriverManagerDataSource( "jdbc:mysql:replication://master,slave1,slave2,slave3/myapp-db"); } ... } import org.springframework.jdbc.datasource.DriverManagerDataSource;

Slide 54

Slide 54 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

Slide 55

Slide 55 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware MongoDB ● Structured Document Data-Model ● BJSON Optimized Serialization ● Document Collection Abstraction ● Queries on Document Collection ● Projection Retrieval & Partial Updates Data-Model { stocks : [ {sym:”PIV”, value:120.13, org: }, {sym:”LNX”, value:88.99, org: }, {sym:”SGT”, value:23.12, org: } ], date : '06-01-2015', exchange: 'nasdaq' } { _id: , sym:”PIV”, name:”Pivotal, Inc.”, industry: “IT”, revenue: [ { year: 2014, qrts: [“8.5M”,“10.5M”, “14.0M”, “22.0M” ]}], workers: 2100 } company stocks

Slide 56

Slide 56 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware MongoDB ● Limited Transactions Scope – Operations in Single Document – No Atomicity Across Documents&Servers – Distributed ACID also available (but expensive) ● Read on Secondary Servers might not be latest – Not Propagated yet from Primary Transactions WRITE_CONCERN

Slide 57

Slide 57 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware MongoDB Replication ● Replication with Resplica-Sets – Recent Versions: Primary-Secondary+Auto-Failover – Early Versions: Master-Slaves Configuration > mongod --replSet "rs0" {--config $HOME/.mongodb/config} > mongo mongo> rs.initiate() mongo> rs.conf() mongo> rs.add(“mongo2.mydomain”) mongo> rs.add(“mongo3.mydomain”) > mongod --port 27017 --dbpath /d/mongodb/rs0-0 --replSet rs > mongod --port 27018 --dbpath /d/mongodb/rs0-1 --replSet rs > mongod --port 27019 --dbpath /d/mongodb/rs0-2 --replSet rs @Profile(“dev”) @Profile(“prod”)

Slide 58

Slide 58 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware MongoDB Partitioning ● Partitioning (Sharding) of Documents in Collection Partitioning (Sharding) > mongod > mongo mongo>

Slide 59

Slide 59 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

Slide 60

Slide 60 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware GemFire ● Distributed Object-Store – Key-Value Store++ with Object Queries ● Custom Optimized Binary Serialization ● Region = Collections of Objects ● In-Memory Data-Grid - Use cases: – Distributed Cache – High-Performance DataBase ● Taps into underlying storage for persistence – RDBMS, NoSql, HDFS (Hadoop) Overview

Slide 61

Slide 61 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware GemFire Data-Model Cache Data Control TransactionListener Region Region Region Region Region Control CacheListener CacheLoader CacheWriter Region: “books” Data Entry Key Value Meta-Data Entry Key Value Meta-Data DB Region java.util.Map

Slide 62

Slide 62 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware GemFire Region Types a.acme.com “Users” bucket[1] Key hash() bucket[1] bucket[2] bucket[3] ... bucket[4] Partitioned Replicated Distributed Local b.acme.com “Users” bucket[2] bucket[7] c.acme.com “Users” bucket[5] bucket[9] a.acme.com “Stocks” b.acme.com “Stocks” c.acme.com “Stocks” K1 V1 K2 V2 K3 V3 K1 V1 K2 V2 K3 V3 K1 V1 K2 V2 K3 V3 a.acme.com “Docs” b.acme.com “Docs” c.acme.com “Docs” K1 V1 K2 V2 K1 V1 K3 V3 K2 V2 K4 V4 K7 V7 a.acme.com “F(X)” K1 V1 K2 V2 K7 V7

Slide 63

Slide 63 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware GemFire Transactions Propagatation and Concurrency Control Replicated&Distributed a.acme.com “Stocks” b.acme.com “Stocks” c.acme.com “Stocks” Scope Propagation Wait Concurrency Control Local No NO None Distributed YES NO None Distributed+ACK YES YES Versions

Slide 64

Slide 64 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware GemFire Transactions API CacheTransactionManager txmgr = c.getCacheTransactionManager(); try { txmgr.begin(); Integer balance = cash.get(customer.getId()); Integer newBalance = Integer.valueOf(balance - price); cash.put(customer.getId(), newBalance); … txmgr.commit(); } catch (CommitConflictException e) { txmgr.rollback(); } Cache c = CacheFactory.create(DistributedSystem.connect(null)); AttributesFactory af = new AttributesFactory(); af.setDataPolicy(DataPolicy.REPLICATE); Region cash = c.createRegion("cash", af.create());

Slide 65

Slide 65 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

Slide 66

Slide 66 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JMS API ConnectionFactory Connection Message Producer Message Consumer singleton TCP/IP broker connection messaging context Message createConnection() createProducer() createConsumer() Session createSession() create*Message() MessageListener App MessageListener setMessageListener() Message onMessage()

Slide 67

Slide 67 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JMS - Transactions Session session = connection.createSession(true, 0); Producer producer = session.createProducer(destination); try { for (int i=0; i<3; i++) { Message msg = session.createTextMessage(); msg.setText(“Hello JMS – Part 1”); producer.send(msg); } session.commit(); } catch (Exception e) { session.rollback(); } ConnectionFactory factory = new ABCProviderConnectionFactory(); Destination queue = new ABCQueueDestination(“queue-q1”); Connection connection = factory.createConnection(); connection.activate(); Sending

Slide 68

Slide 68 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JMS - Transactions Session session = connection.createSession(true, 0); Consumer consumer = session.createConsumer(destination); try { for (int i=0; i<3; i++) { TextMessage msg = (TextMessage) consumer.receive(); System.out.println(msg.getText()); } session.commit(); } catch (Exception e) { session.rollback(); } Receiving

Slide 69

Slide 69 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Spring JMS Config @Configuration public class JmsConfig { @Bean JmsTemplate jmsTemplate(ConnectionFactory factory) { return new JmsTemplate(factory); } @Bean ConnectionFactory connectionFactory() { return new ConnectionFactory(); } @Bean Queue stocksQueue() { return new ABCQueue(); } @Bean MessageListenerContainer messageListenerContainer() { return new SimpleMessageListenerContainer(); } } @Autowired JmsTemplate template; @Autowired Destination stocksQueue;

Slide 70

Slide 70 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Spring JMS final Stock[] stocks = … jmsTemplate.exectute(stocksQueue, (Session s, MessageProducer p) → { for (Stock stock: stocks) { Message msg = converter.convertToMessage(s, stock); p.send(msg); } } ); Sending ProduceCallback jmsTemplate.setSessionTransacted(true); MessageConverter converter = jmsTemplate.getMessageConverter(); jmsTemplate.convertAndSend(stocksQueue, stock); TX

Slide 71

Slide 71 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Spring JMS final List stocks = new ArrayList<>(); jmsTemplate.exectute(stocksQueue, (Session s, MessageConsumer p) → { do { Message msg = c.receive(); stocks.add((Stock)converter.convertFromMessage(msg)); } while (/*more*/); } ); Receiving - Synchronous ConsumerCallback jmsTemplate.setSessionTransacted(true); MessageConverter converter = jmsTemplate.getMessageConverter(); Stock stock = (Stock) jmsTemplate.receiveAndConvert(); TX

Slide 72

Slide 72 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Spring JMS Receiving - Asynchronous TX @Component public class StockProcessor { @JmsListener(destination=“stocksQueue”) public void processStock(Stock stock) { //process stock... } } @Configuration @ComponentScan(“myapp.jms”) @EnableJms public class JmsConfig { ... @Bean MessageListenerContainer messageListenerContainer() { DefaultMessageListenerContainer container = new DefaultMessageListenerContainer(); container.setTransacter(true); return container; } }

Slide 73

Slide 73 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

Slide 74

Slide 74 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware AMQP and RabbitMQ P C C Broker queue: q1 X queue: q2 C

Slide 75

Slide 75 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware RabbitMQ – Java Client API ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); Connection connection = factory.newConnection(); Channel channel = connection.createChannel(); channel.close(); connection.close(); import com.rabbitmq.client.Channel; //...use channel to send&recv messages...

Slide 76

Slide 76 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware RabbitMQ – Sending Messages try { String[] stocks = {"SYM RabbitMQ = 99.9", "SYM LinuxLTD = 10.0", "SYM XMQ = 50.0"}; channel.txSelect(); for (String stock: stocks) { channel.basicPublish(“stocks-x”, “IT”, null, stock.getBytes()); } channel.txCommit(); } catch (Exception e) { channel.txRollback(); }

Slide 77

Slide 77 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware RabbitMQ – Receiving Messages try { String[] stocks = {"SYM RabbitMQ = 99.9", "SYM LinuxLTD = 10.0", "SYM XMQ = 50.0"}; channel.txSelect(); for (int i=0; i<3; i++) { GetResponse msg = channel.basicGet(“trading-q”, true); System.out.println(new String(msg.getBytes())); } channel.txCommit(); } catch (Exception e) { channel.txRollback(); }

Slide 78

Slide 78 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware STOMP CONNECT accept-version:1.2 host:stomp.github.org ^@ Simple (or Streaming) Text Orientated Messaging Protocol CONNECTED version:1.2 ^@ SEND destination:/queue/stocks-q content-type:text/plain SYM PIV; 123.75 ^@ BEGIN transaction:tx1 ^@ COMMIT transaction:tx1 ^@ http://stomp.github.io/

Slide 79

Slide 79 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

Slide 80

Slide 80 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JTA Java Transactions Architecture JDBC Resource Drivers JMS Connection JDBC XA XA Resource (adaptors) JMS XA DB Broker Msg Broker DB JTA TxMgr APP Global TX ● Application Server ● Standalone App JEE

Slide 81

Slide 81 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JTA - API @Stateless @TransactionManagement(BEAN) public class StockServiceImpl implements StockService { @Resource private UserTransaction tx; @Resource private DataSource dataSource; @Resource private ConnectionFactory connectionFactory; public void foo() { tx.begin(); // Do work on dataSource // Do work on connectionFactory tx.commit(); } } EJB Model

Slide 82

Slide 82 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

Slide 83

Slide 83 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware LDAP ● Naming and Directory Service ● Structure Names Data-Model ● Keeps many types of Information: – Organizational – Users – Security Policy – Custom ● No native support for ACID Transactions – Only Durability and Consistency Rules

Slide 84

Slide 84 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Spring LDAP ● Simplified API to connect and interact with LDAP server ● Emulates Transactions using Compensatory Undo Operations – Atomicity Only – Programatic API – Integrated with Declarative @Transactional ● No concurrency control (isolation) – Except if conflict detected by app and rollback

Slide 85

Slide 85 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Conclusions ● Transactions are very often essential – For Reliability and Consistency ● Multiple Kinds of Transactional Resources ● Distribution of Transactions is Expensive – Trade-off: Scalability vs. Consistency vs. Availability ● Basic Mechanisms Well-Understood – Many optimizations possible ● AOP Simplifies Transaction Management Thank You

Slide 86

Slide 86 text

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Learning More ● Training – Core Spring – Enterprise Integration with Spring – RabbitMQ – Java 8, JEE7 ● Resources – Spring.io – Spring Downloads & Docs – JPalace.org – RefCards, Tutorials, Articles