JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactional Systems Properties and Mechanisms Jorge Simão, Ph.D. jorge.simao@einnovator.org DB TX APP SingSUG – Singapore Spring Users Group 8 Jan. 2015

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Server Scenarios $ $ $

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactional Properties (ACID) ● Atomicity – Transaction as indivisible unit-of-work ● Consistency – Integrity Constraints on Data ● Isolation – Concurrent access should not lead to inconsistency ● Durability – Operation effects should persist

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Atomicity APP RDBMS DB read(A)=500 read(B)=200 write(A)=500-100=400 write(B)=200+100=300 A = 500 B = 200 ∑=700 ∑=700 ∑'=600 $ $ $ commit ID Balance A 500 B 700 T_ACCOUNT Consistency In the Presence of Failures Tx A B 100$

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Consistency ID Balance 1 500 2 700 T_ACCOUNT NOT NULL >=0 ID CITY COUNTRY AC_ID 100 Lisbon Portugal 1 T_ADDRESS FOREGIN KEY Impose Constraints on Data Model

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Isolation: Race Conditions APP RDBMS DB read(A)=500 read(A)=500 write(A)=500+100=600 write(A)=500+200=700 A = 500 commit commit Concurrency Control Tx 1 Tx 2 A 200$ A 100$ ∑=700 ∑'=600 $ TX 1 TX 2 $ $ Serializibility

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Durability APP RDBMS DB $ commit Tx Table Op Row Value Log/Journal Data Persistence Log

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Weak ACID Properties ● Benefits of ACID: – Guaranteed consistency – Simple programming model and error semantics ● Problem with ACID: – Expensive to implement with distributed resources ● How to Scale-Out Data Management? – Limited Scope of Transaction ● Atomicity only for resources in same Server – Eventual Consistency ● Delay Propagation of Updates Dropping Global Consistency

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactional Resources ● Database Resources – RDBMS & ORM, NoSql DBs ● Messaging Resources – JMS API, AMQP, STOMP ● Enterprise Information System – Java Connector ● Other Data Resources – Middleware/App Level – LDAP, TX-FS, TX-VMem DB NoSql

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactions in Messaging P C C Broker queue: q1 TXp 1 TXc 2 DB send= write recv = read + [update] + [del]

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Failures and Message Duplicates Msg1 commit() process TxBroker.commit (crash) C $ Broker Msg1 TxBroker.rollback DB C

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Multi-Resource Transactions APP RDBMS DB Broker DB TXDB TXBroker

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Multi-Commits Partial-Failure and Duplicate Handling APP RDBMS DB Broker DB TXBroker $ TXDB Connection dbConnection = dataSource.getConnection(); Session session = brokerConnection.createSession(); //receive message(s) Message msg = ... //check for duplicates: YES → Ignore; NO → Process if (!isDuplicate(msg)) { //process messages //write to DB } dbConnection.commit(); session.commit();

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Slave 2 Master Replicated Data APP Server DB Slave 1 DB DB Write* Read* Read* .us .eu .apj APP Server .us .eu APP Server .apj connect(“master”, [“slave1”, “slave2”]) Synchronize Write* Write* Read* Master-Server Replication

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Master 3 Master 1 Clustering APP Server DB Master 2 APP Server APP Server connect([“master1”]) Master-Master Replication LAN DB DB Cluster connect([“master1”, “master2”, “master3”]) Current Primary (for Resource X)

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Replicated Message Brokers Broker: Master DB Broker: Slave 1 DB Broker: Slave 2 DB P C C P Federation vs. Clustering

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Atomicity with Transactional Log write(A)= 400 write(B)= 300 A = 500 B = 200 commit ID Balance A 500 B 700 T_ACCOUNT Tx Op Row Value 9 W A 500 9 W C 900 9 Commit 10 Rollback 11 W A 400 11 W B 300 11 Commit 12 W C 400 Log/Journal Logging Transaction Operations Tx 11 DB RDBMS APP head tail ID Balance A 400 B 300 Flush Log Truncate Log

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Isolation: Locks APP RDBMS DB read(A)=500 read(A)=600 write(A)=500+100=600 write(A)=600+200=800 A = 500 commit commit Pessimistic Concurrency Control Tx1 Tx2 A 200$ A 100$ ∑=800 Lock(A;W) Lock(A;W) ACK ACK ACK Lock(A;W)

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Isolation: Versioning APP RDBMS DB A = 500 Optimistic Concurrency Control Tx1 Tx2 A 200$ A 100$ ACK read(A)=500;v=1 read(A)=500;v=1 write(A)=500+100=600;v=1 write(A)=500+200=700;v=1 commit rollback ∑=700 v=1 v=2 $ Rollback ID Balance Version A 700 2 B 300 1 T_ACCOUNT

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Read-Only Transactions read(A)=500 read(A)=600 A = 500 commit commit Tx1 Tx2 Lock(A;R) Lock(A;R) ACK A = 500 ● Report Generator ● Event Notifier ● Business Intelligence Report ● Page Profile ● Document Listing

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Isolation Levels read(A)=500 read(A)=500 write(A)=800 A = 500 commit commit Read-Only Concurrent with Write Transactions TxR 1 TxW 2 Lock(A;R) Lock(A;W) ACK ACK Repeatable-Read Read Committed read(A)=500 read(A)=800 write(A)=800 A = 500 commit commit TxR 1 TxW 2 Lock(A;R) Lock(A;W) ACK ACK read(A)=500 read(A)=800 write(A)=800 A = 500 commit rollback TxR 1 TxW 2 Lock(A;R) Lock(A;W) ACK Read Uncommitted A = 800 A = 800 A = 500 Dirty Read read(A)=500

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware DB DB Two-Phase Commit Protocol (XA) TxMgr Resource Mgr Coordinator Prepare x N Pre-commit 1 Pre-commit N Commit x N phase 1 phase 2 Pre-commit (keep logs & locks)

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Server 3 Server 1 Primary (Master) Auto-Failover DB Server 2 Heartbeat & Elections DB DB Previous Primary (for Resource X) $ ♥ ♥ ♥ $ $ Election New Primary (for Resource X) $ $

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Server 3 Server 1 Partition Detection & Handling DB Server 2 Quorum-Based Consensus DB DB ♥ $ Election Election Partition 1 Partition 2

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JDBC Transaction Management public void transfer(Long debitId, Long creditId, BigDecimal amount) { String sql = “update T_ACCOUNT set BALANCE=? WHERE ID=?”; try { Connection connection = dataSource.getConnection(); connection.setAutoCommit(false); PreparedStatement st = connection.prepareStatement(sql); st.setParameter(1, balance1 + amount); st.setParameter(2, debitId); st.execute(); st.setParameter(1, balance2 + amount); st.setParameter(2, creditId); st.execute(); connection.commit(); } catch (SQLException e) { connection.rollback(); throw new DataAccessException(e); } finally { connection.close(); } Debit Operation Credit Operation TX

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Modular Application APP Driver RDBMS DB SQL results Infrastructure Layer Data-Access Layer Service layer Multi-Tier Architecture (2,3+ Layers) Presentation layers public interface AccountRepository { Account getAccount(Long id); void updateBalance(Account account); ... }

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transaction Management ● Transaction scope may include: – Multiple calls to the same DAO – Operations in different DAOs – Operations in different Resources (JTA) ● Data-Access Technology Independence: – Service-Layer Implementation – DAO API ● Connection Must be Shared – Thread/Request Contextualized Service Layer Data-Access Layer Issues

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactional Interceptor ServiceImpl DAO Transactional Interceptor TransactionManager Proxy begin() commit() Client Object DB AOP – with Proxy Power Service Thread-Safe

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transaction Management Annotation-Driven Declarative Demarcation @Singleton public class TransferServiceImpl implements TransferService { @Inject private AccountRepository accountRepository; @Inject private OperationsRepository operationsRepository; @Transactional public Transfer transfer(Long creditId, Long debitId, BigDecimal amount) { // Do transfer with AccountRepository ... // Write confirmation with OperationsRepository ... ... } }

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transaction Management Local Transaction Propagation @Singleton public class TransferServiceImpl implements TransferService { ... @Transactional public Transfer transfer(...) { // do transfer // write confirmation with OperationsRepository ... … emailService.send(msg); } } @Singleton public class EmailServiceImpl implements EmailService { @Transactional public void send(Message msg) { ... } } method1 method2 TX 1 TX 1 TX 2

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transaction Management Remote Transaction Propagation method1 method2 TX 1 TX 1 TX 2 public class TransferServiceImpl ... { ... @Transactional public Transfer transfer(...) { … remoteEmailService.send(msg); } } @Singleton public class RemoteEmailService ... { @Transactional public void send(Message msg) { ... } } Host1 Host2

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactions in Spring Uniform Approach @Service public class TransferServiceImpl implements TransferService { @Autowired private AccountRepository accountRepository; @Autowired private OperationsRepository operationsRepository; @Transactional public Transfer transfer(Long creditId, Long debitId, BigDecimal amount) { Account debitAccount = accountRepository.getAccount(creditId); Account creditAccount = accountRepository.getAccount(debitId); debitAccount.debit(amount); creditAccount.credit(amount); accountRepository.update(debitAccount); accountRepository.update(creditAccount); Transfer transfer = new Transfer(debitAccount, creditAccount, amount); operationsRepository.add(transfer); return transfer; } } import org.springframework.stereotype.Service; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.transaction.annotation.Transactional;

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transaction Configuration Isolation, Propagation, Rollback Rules @Service public class TransferServiceImpl implements TransferService { @Transactional(isolation=Isolation.SERIALIZABLE, noRollbackFor=EmailException.class) public Transfer transfer(Long creditId, Long debitId, BigDecimal amount) { … emailService.sendConfirmation(Transfer.buildMessage(transfer)); return transfer; } } @Service public class EmailServiceImpl implements EmailService { @Transactional(propagation=Propagation.REQUIRES_NEW) public void send(Message message) { … } }

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JDBC Repositories @Repository public class JdbcAccountRepository implements AccountRepository { @Autowired private DataSource dataSource; private JdbcTemplate jdbcTemplate; @PostConstruct public void init() { jdbcTemplate = new JdbcTemplate(dataSource); } public Account getAccount(Long id) { String sql = “select * from T_ACCOUNT where ID=?”; return jdbcTemplate.query(sql, id); } public void updateBalance(Account account) { String sql = “update T_ACCOUNT set BALANCE=? where ID=?”; jdbcTemplate.update(sql, account.getBalance(), account.getId()); } } import org.springframework.stereotype.Repository; import org.springframework.beans.factory.annotation.Autowired; import java.lang.annotation.PostConstruct;

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactions in Spring Configuration @Configuration @EnableTransactionManagament @Import({JdbcConfig.class, JpaConfig.class, DBConfig.class}); @ComponentScan(“myapp.services”) @PropertySource(“app.properties”) public class AppConfig { … } @Configuration @Profile(“jdbc”) public class JdbcConfig { @Bean PlatformTransactionManager txMgr(DataSource dataSource) { return new DataSourceTransactionManager(dataSource); } } @Configuration @Profile(“jpa”) public class JpaConfig { … } import org.springframework.context.annotation.*;

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Transactions in Spring Conditional Beans import org.springframework.context.annotation.*; @Configuration @ImportResource(“db-config.xml”) public class DBConfig { @Autowired private Environment env; @Bean @Reachable DataSource master() { .. } @Bean @Reachable DataSource slave1() { .. } @Bean @Reachable DataSource slave2() { .. } } public class ReachableCondition implements Condition { public boolean matches(ConditionContext context, AnnotatedTypeMetadata metadata) { … } } @Conditional(ReachableCondition.class) @Target({ElementType.TYPE, ElementType.METHOD}) @Retention({RetentionPolicy.RUNTIME}) public @interface Reachable { } import java.lang.annotation.*;

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware ORM Architecture Automating Object to/from RDB Mapping Cache Annotations XML Meta-Data Persistence API Database Abstraction Layer Client DAO DB SQL ✔ SQL Query Generation ✔ Mapping ✔ Column Extraction ✔ Parameter Extraction ✔ Association Managm. ✔ DB Abstraction ✔ Declarative Config ✔ Proxy Creation ✔ Schema Creation Config DataSource Properties Criteria Query API + OQL Queries Criteria Query

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JPA Architecture EntityManager & EntityManagerFactory public class JpaAccountRepository { EntityManager entityManager; ... } EntityManager Data-Access Object Entity Instance Entity Instance Persistence Context (Cache) Persistence Unit DB EntityManagerFactory Connection DataSource

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JPA Entities Annotation-Driven Meta-Data @Entity @Table(“T_ACCOUNT”) public class Account { @Id private Long id; private String name; @Column(name=“DOB”) @Temporal(TemporalType.DATE) private Date birthday; @OneToMany @JoinColumn(name=“ACCOUNT_ID”) private Set

addresses; public Account() { } } Physical Mapping @Table @Column @JoinColumn Annotations Logical/Model @Entity @Id @OneToMany 1:N Account Address

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JPA Transactions public class JpaAccountDao implements AccountDao { @Inject private EntityManagerFactory emf; public void importAccounts(List accounts) { EntityManager em = emf.createEntityManager(); Transaction tx = em.getTransaction(); try { tx.begin(); for (Accout account: accounts) { entityManager.persist(account); } tx.commit(); } catch (Exception e) { tx.rollback(); } } ... } Programmatic Demarcation

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JPA DAOs public class JpaAccountDao implements AccountDao { @PersistenceContext private EntityManager entityManager; public JpaAccountDao() {} public void add(Account account) { entityManager.persist(account); } ... } EntityManagerProxy EntityManager Proxy @Singleton @Repository import javax.persistence.PersistenceContext; EJB&JSR330 Spring @ApplicationScoped CDI

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JPA Configuration @Configuration @Profile(“jpa”) public class JpaConfig { @Bean PlatformTransactionManager txMgr(EntityManagerFactory emf) { return new JpaTransactionManager(emf); } @Bean LocalContainerEntityManagerFactoryBean createEntityManagerFactory(DataSource dataSource) { LocalContainerEntityManagerFactoryBean emfb = new LocalContainerEntityManagerFactoryBean(); emfb.setPersistenceUnitName("onlineshop"); emfb.setPackagesToScan(“com.onlineshop.domain”); emfb.setDataSource(dataSource); return new emfb; } }

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JPA Configuration import org.einnovator.jpa.provider.PersistenceUnitInfoBuilder; import org.einnovator.jpa.Jpa; @Singleton @Jpa public class JpaTestConfig { @Factory public EntityManagerFactory createEntityManagerFactory(DataSource dataSource) { return new PersistenceUnitInfoBuilder() .name("onlineshop") .packagesToScan(EntitySupport.class) .dataSource(dataSource) .property("hibernate.dialect", "org.hibernate.dialect.HSQLDialect") .property("hibernate.hbm2ddl.auto", "create") .property("hibernate.show_sql", true) .buildEntityManagerFactory(); } }

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware MySQL Replication ● Asynchronous Master-Slave ● Semi-Synchronous Master-Slave ● Clustering ● Partitioning Replication Architectures

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware MySQL Replication mysql> CHANGE MASTER TO -> MASTER_HOST='master_host_name', -> MASTER_USER='replication_user_name', -> MASTER_PASSWORD='replication_password', -> MASTER_LOG_FILE='recorded_log_file_name', -> MASTER_LOG_POS=recorded_log_position; [mysqld] log-bin=mysql-bin server-id=1 [mysqld] server-id=2 Master Slave Master-Slave Configuration

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JAVA-MySQL Connector JDBC Connection to Master+Servers @Configuration @ImportResource(“db-config.xml”) public class DBConfig { @Autowired private Environment env; @Bean @Profile(“mysqlWAN”) DataSource masterserver() { return new DriverManagerDataSource( "jdbc:mysql:replication://master,slave1,slave2,slave3/myapp-db"); } ... } import org.springframework.jdbc.datasource.DriverManagerDataSource;

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware MongoDB ● Structured Document Data-Model ● BJSON Optimized Serialization ● Document Collection Abstraction ● Queries on Document Collection ● Projection Retrieval & Partial Updates Data-Model { stocks : [ {sym:”PIV”, value:120.13, org: }, {sym:”LNX”, value:88.99, org: }, {sym:”SGT”, value:23.12, org: } ], date : '06-01-2015', exchange: 'nasdaq' } { _id: , sym:”PIV”, name:”Pivotal, Inc.”, industry: “IT”, revenue: [ { year: 2014, qrts: [“8.5M”,“10.5M”, “14.0M”, “22.0M” ]}], workers: 2100 } company stocks

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware MongoDB ● Limited Transactions Scope – Operations in Single Document – No Atomicity Across Documents&Servers – Distributed ACID also available (but expensive) ● Read on Secondary Servers might not be latest – Not Propagated yet from Primary Transactions WRITE_CONCERN

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware MongoDB Replication ● Replication with Resplica-Sets – Recent Versions: Primary-Secondary+Auto-Failover – Early Versions: Master-Slaves Configuration > mongod --replSet "rs0" {--config $HOME/.mongodb/config} > mongo mongo> rs.initiate() mongo> rs.conf() mongo> rs.add(“mongo2.mydomain”) mongo> rs.add(“mongo3.mydomain”) > mongod --port 27017 --dbpath /d/mongodb/rs0-0 --replSet rs > mongod --port 27018 --dbpath /d/mongodb/rs0-1 --replSet rs > mongod --port 27019 --dbpath /d/mongodb/rs0-2 --replSet rs @Profile(“dev”) @Profile(“prod”)

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware MongoDB Partitioning ● Partitioning (Sharding) of Documents in Collection Partitioning (Sharding) > mongod > mongo mongo>

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware GemFire ● Distributed Object-Store – Key-Value Store++ with Object Queries ● Custom Optimized Binary Serialization ● Region = Collections of Objects ● In-Memory Data-Grid - Use cases: – Distributed Cache – High-Performance DataBase ● Taps into underlying storage for persistence – RDBMS, NoSql, HDFS (Hadoop) Overview

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware GemFire Data-Model Cache Data Control TransactionListener Region Region Region Region Region Control CacheListener CacheLoader CacheWriter Region: “books” Data Entry Key Value Meta-Data Entry Key Value Meta-Data DB Region java.util.Map

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware GemFire Region Types a.acme.com “Users” bucket[1] Key hash() bucket[1] bucket[2] bucket[3] ... bucket[4] Partitioned Replicated Distributed Local b.acme.com “Users” bucket[2] bucket[7] c.acme.com “Users” bucket[5] bucket[9] a.acme.com “Stocks” b.acme.com “Stocks” c.acme.com “Stocks” K1 V1 K2 V2 K3 V3 K1 V1 K2 V2 K3 V3 K1 V1 K2 V2 K3 V3 a.acme.com “Docs” b.acme.com “Docs” c.acme.com “Docs” K1 V1 K2 V2 K1 V1 K3 V3 K2 V2 K4 V4 K7 V7 a.acme.com “F(X)” K1 V1 K2 V2 K7 V7

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware GemFire Transactions Propagatation and Concurrency Control Replicated&Distributed a.acme.com “Stocks” b.acme.com “Stocks” c.acme.com “Stocks” Scope Propagation Wait Concurrency Control Local No NO None Distributed YES NO None Distributed+ACK YES YES Versions

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware GemFire Transactions API CacheTransactionManager txmgr = c.getCacheTransactionManager(); try { txmgr.begin(); Integer balance = cash.get(customer.getId()); Integer newBalance = Integer.valueOf(balance - price); cash.put(customer.getId(), newBalance); … txmgr.commit(); } catch (CommitConflictException e) { txmgr.rollback(); } Cache c = CacheFactory.create(DistributedSystem.connect(null)); AttributesFactory af = new AttributesFactory(); af.setDataPolicy(DataPolicy.REPLICATE); Region cash = c.createRegion("cash", af.create());

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JMS API ConnectionFactory Connection Message Producer Message Consumer singleton TCP/IP broker connection messaging context Message createConnection() createProducer() createConsumer() Session createSession() create*Message() MessageListener App MessageListener setMessageListener() Message onMessage()

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JMS - Transactions Session session = connection.createSession(true, 0); Producer producer = session.createProducer(destination); try { for (int i=0; i<3; i++) { Message msg = session.createTextMessage(); msg.setText(“Hello JMS – Part 1”); producer.send(msg); } session.commit(); } catch (Exception e) { session.rollback(); } ConnectionFactory factory = new ABCProviderConnectionFactory(); Destination queue = new ABCQueueDestination(“queue-q1”); Connection connection = factory.createConnection(); connection.activate(); Sending

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JMS - Transactions Session session = connection.createSession(true, 0); Consumer consumer = session.createConsumer(destination); try { for (int i=0; i<3; i++) { TextMessage msg = (TextMessage) consumer.receive(); System.out.println(msg.getText()); } session.commit(); } catch (Exception e) { session.rollback(); } Receiving

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Spring JMS Config @Configuration public class JmsConfig { @Bean JmsTemplate jmsTemplate(ConnectionFactory factory) { return new JmsTemplate(factory); } @Bean ConnectionFactory connectionFactory() { return new ConnectionFactory(); } @Bean Queue stocksQueue() { return new ABCQueue(); } @Bean MessageListenerContainer messageListenerContainer() { return new SimpleMessageListenerContainer(); } } @Autowired JmsTemplate template; @Autowired Destination stocksQueue;

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Spring JMS final Stock[] stocks = … jmsTemplate.exectute(stocksQueue, (Session s, MessageProducer p) → { for (Stock stock: stocks) { Message msg = converter.convertToMessage(s, stock); p.send(msg); } } ); Sending ProduceCallback jmsTemplate.setSessionTransacted(true); MessageConverter converter = jmsTemplate.getMessageConverter(); jmsTemplate.convertAndSend(stocksQueue, stock); TX

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Spring JMS final List stocks = new ArrayList<>(); jmsTemplate.exectute(stocksQueue, (Session s, MessageConsumer p) → { do { Message msg = c.receive(); stocks.add((Stock)converter.convertFromMessage(msg)); } while (/*more*/); } ); Receiving - Synchronous ConsumerCallback jmsTemplate.setSessionTransacted(true); MessageConverter converter = jmsTemplate.getMessageConverter(); Stock stock = (Stock) jmsTemplate.receiveAndConvert(); TX

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Spring JMS Receiving - Asynchronous TX @Component public class StockProcessor { @JmsListener(destination=“stocksQueue”) public void processStock(Stock stock) { //process stock... } } @Configuration @ComponentScan(“myapp.jms”) @EnableJms public class JmsConfig { ... @Bean MessageListenerContainer messageListenerContainer() { DefaultMessageListenerContainer container = new DefaultMessageListenerContainer(); container.setTransacter(true); return container; } }

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware AMQP and RabbitMQ P C C Broker queue: q1 X queue: q2 C

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware RabbitMQ – Java Client API ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); Connection connection = factory.newConnection(); Channel channel = connection.createChannel(); channel.close(); connection.close(); import com.rabbitmq.client.Channel; //...use channel to send&recv messages...

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware RabbitMQ – Sending Messages try { String[] stocks = {"SYM RabbitMQ = 99.9", "SYM LinuxLTD = 10.0", "SYM XMQ = 50.0"}; channel.txSelect(); for (String stock: stocks) { channel.basicPublish(“stocks-x”, “IT”, null, stock.getBytes()); } channel.txCommit(); } catch (Exception e) { channel.txRollback(); }

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware RabbitMQ – Receiving Messages try { String[] stocks = {"SYM RabbitMQ = 99.9", "SYM LinuxLTD = 10.0", "SYM XMQ = 50.0"}; channel.txSelect(); for (int i=0; i<3; i++) { GetResponse msg = channel.basicGet(“trading-q”, true); System.out.println(new String(msg.getBytes())); } channel.txCommit(); } catch (Exception e) { channel.txRollback(); }

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware STOMP CONNECT accept-version:1.2 host:stomp.github.org ^@ Simple (or Streaming) Text Orientated Messaging Protocol CONNECTED version:1.2 ^@ SEND destination:/queue/stocks-q content-type:text/plain SYM PIV; 123.75 ^@ BEGIN transaction:tx1 ^@ COMMIT transaction:tx1 ^@ http://stomp.github.io/

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JTA Java Transactions Architecture JDBC Resource Drivers JMS Connection JDBC XA XA Resource (adaptors) JMS XA DB Broker Msg Broker DB JTA TxMgr APP Global TX ● Application Server ● Standalone App JEE

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware JTA - API @Stateless @TransactionManagement(BEAN) public class StockServiceImpl implements StockService { @Resource private UserTransaction tx; @Resource private DataSource dataSource; @Resource private ConnectionFactory connectionFactory; public void foo() { tx.begin(); // Do work on dataSource // Do work on connectionFactory tx.commit(); } } EJB Model

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Outline ● Properties of Transactions ● Kinds of Transactional Systems ● Mechanisms for Transactions ● Transaction Managament ● Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware LDAP ● Naming and Directory Service ● Structure Names Data-Model ● Keeps many types of Information: – Organizational – Users – Security Policy – Custom ● No native support for ACID Transactions – Only Durability and Consistency Rules

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Spring LDAP ● Simplified API to connect and interact with LDAP server ● Emulates Transactions using Compensatory Undo Operations – Atomicity Only – Programatic API – Integrated with Declarative @Transactional ● No concurrency control (isolation) – Except if conflict detected by app and rollback

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Conclusions ● Transactions are very often essential – For Reliability and Consistency ● Multiple Kinds of Transactional Resources ● Distribution of Transactions is Expensive – Trade-off: Scalability vs. Consistency vs. Availability ● Basic Mechanisms Well-Understood – Many optimizations possible ● AOP Simplifies Transaction Management Thank You

JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware Learning More ● Training – Core Spring – Enterprise Integration with Spring – RabbitMQ – Java 8, JEE7 ● Resources – Spring.io – Spring Downloads & Docs – JPalace.org – RefCards, Tutorials, Articles