Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Transactions with Spring by Jorge Simao

SingaSUG
January 08, 2015

Transactions with Spring by Jorge Simao

Presented at Singapore Spring User Group on January 8th

SingaSUG

January 08, 2015
Tweet

More Decks by SingaSUG

Other Decks in Technology

Transcript

  1. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Transactional Systems Properties and Mechanisms Jorge Simão, Ph.D. [email protected] DB TX APP SingSUG – Singapore Spring Users Group 8 Jan. 2015
  2. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Outline • Properties of Transactions • Kinds of Transactional Systems • Mechanisms for Transactions • Transaction Managament • Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP
  3. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Transactional Properties (ACID) • Atomicity – Transaction as indivisible unit-of-work • Consistency – Integrity Constraints on Data • Isolation – Concurrent access should not lead to inconsistency • Durability – Operation effects should persist
  4. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Atomicity APP RDBMS DB read(A)=500 read(B)=200 write(A)=500-100=400 write(B)=200+100=300 A = 500 B = 200 ∑=700 ∑=700 ∑'=600  $ $ $ commit ID Balance A 500 B 700 T_ACCOUNT Consistency In the Presence of Failures Tx A B 100$
  5. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Consistency ID Balance 1 500 2 700 T_ACCOUNT NOT NULL >=0 ID CITY COUNTRY AC_ID 100 Lisbon Portugal 1 T_ADDRESS FOREGIN KEY Impose Constraints on Data Model
  6. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Isolation: Race Conditions APP RDBMS DB read(A)=500 read(A)=500 write(A)=500+100=600 write(A)=500+200=700 A = 500 commit commit Concurrency Control Tx 1 Tx 2 A 200$ A 100$ ∑=700 ∑'=600  $ TX 1 TX 2  $  $ Serializibility
  7. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Durability APP RDBMS DB $ commit Tx Table Op Row Value Log/Journal  Data Persistence Log
  8. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Weak ACID Properties • Benefits of ACID: – Guaranteed consistency – Simple programming model and error semantics • Problem with ACID: – Expensive to implement with distributed resources • How to Scale-Out Data Management? – Limited Scope of Transaction • Atomicity only for resources in same Server – Eventual Consistency • Delay Propagation of Updates Dropping Global Consistency
  9. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Outline • Properties of Transactions • Kinds of Transactional Systems • Mechanisms for Transactions • Transaction Managament • Practical Examples
  10. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Transactional Resources • Database Resources – RDBMS & ORM, NoSql DBs • Messaging Resources – JMS API, AMQP, STOMP • Enterprise Information System – Java Connector • Other Data Resources – Middleware/App Level – LDAP, TX-FS, TX-VMem DB NoSql
  11. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Transactions in Messaging P C C Broker queue: q1 TXp 1 TXc 2 DB send= write recv = read + [update] + [del]
  12. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Failures and Message Duplicates Msg1 commit() process TxBroker.commit (crash) C $ Broker Msg1 TxBroker.rollback DB C
  13. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Multi-Resource Transactions APP RDBMS DB Broker DB TXDB TXBroker
  14. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Multi-Commits Partial-Failure and Duplicate Handling APP RDBMS DB Broker DB TXBroker $  TXDB Connection dbConnection = dataSource.getConnection(); Session session = brokerConnection.createSession(); //receive message(s) Message msg = ... //check for duplicates: YES → Ignore; NO → Process if (!isDuplicate(msg)) { //process messages //write to DB } dbConnection.commit(); session.commit();
  15. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Slave 2 Master Replicated Data APP Server DB Slave 1 DB DB Write* Read* Read* .us .eu .apj APP Server .us .eu APP Server .apj connect(“master”, [“slave1”, “slave2”]) Synchronize Write* Write* Read* Master-Server Replication
  16. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Master 3 Master 1 Clustering APP Server DB Master 2 APP Server APP Server connect([“master1”]) Master-Master Replication LAN DB DB Cluster connect([“master1”, “master2”, “master3”]) Current Primary (for Resource X)
  17. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Replicated Message Brokers Broker: Master DB Broker: Slave 1 DB Broker: Slave 2 DB P C C P Federation vs. Clustering
  18. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Outline • Properties of Transactions • Kinds of Transactional Systems • Mechanisms for Transactions • Transaction Managament • Practical Examples
  19. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Atomicity with Transactional Log write(A)= 400 write(B)= 300 A = 500 B = 200 commit ID Balance A 500 B 700 T_ACCOUNT Tx Op Row Value 9 W A 500 9 W C 900 9 Commit 10 Rollback 11 W A 400 11 W B 300 11 Commit 12 W C 400 Log/Journal Logging Transaction Operations Tx 11 DB RDBMS APP head tail ID Balance A 400 B 300 Flush Log Truncate Log
  20. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Isolation: Locks APP RDBMS DB read(A)=500 read(A)=600 write(A)=500+100=600 write(A)=600+200=800 A = 500 commit commit Pessimistic Concurrency Control Tx1 Tx2 A 200$ A 100$ ∑=800  Lock(A;W) Lock(A;W) ACK ACK ACK Lock(A;W)
  21. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Isolation: Versioning APP RDBMS DB A = 500 Optimistic Concurrency Control Tx1 Tx2 A 200$ A 100$ ACK read(A)=500;v=1 read(A)=500;v=1 write(A)=500+100=600;v=1 write(A)=500+200=700;v=1 commit rollback ∑=700  v=1 v=2  $ Rollback ID Balance Version A 700 2 B 300 1 T_ACCOUNT
  22. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Read-Only Transactions read(A)=500 read(A)=600 A = 500 commit commit Tx1 Tx2 Lock(A;R) Lock(A;R) ACK A = 500 • Report Generator • Event Notifier • Business Intelligence Report • Page Profile • Document Listing
  23. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Isolation Levels read(A)=500 read(A)=500 write(A)=800 A = 500 commit commit Read-Only Concurrent with Write Transactions TxR 1 TxW 2 Lock(A;R) Lock(A;W) ACK ACK Repeatable-Read Read Committed read(A)=500 read(A)=800 write(A)=800 A = 500 commit commit TxR 1 TxW 2 Lock(A;R) Lock(A;W) ACK ACK read(A)=500 read(A)=800 write(A)=800 A = 500 commit rollback TxR 1 TxW 2 Lock(A;R) Lock(A;W) ACK Read Uncommitted A = 800 A = 800 A = 500 Dirty Read read(A)=500
  24. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    DB DB Two-Phase Commit Protocol (XA) TxMgr Resource Mgr Coordinator Prepare x N Pre-commit 1 Pre-commit N Commit x N phase 1 phase 2 Pre-commit (keep logs & locks)
  25. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Server 3 Server 1 Primary (Master) Auto-Failover DB Server 2 Heartbeat & Elections DB DB Previous Primary (for Resource X) $ ♥ ♥ ♥ $ $ Election New Primary (for Resource X) $ $
  26. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Server 3 Server 1 Partition Detection & Handling DB Server 2 Quorum-Based Consensus DB DB ♥ $ Election Election  Partition 1 Partition 2
  27. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Outline • Properties of Transactions • Kinds of Transactional Systems • Mechanisms for Transactions • Transaction Managament • Practical Examples
  28. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JDBC Transaction Management public void transfer(Long debitId, Long creditId, BigDecimal amount) { String sql = “update T_ACCOUNT set BALANCE=? WHERE ID=?”; try { Connection connection = dataSource.getConnection(); connection.setAutoCommit(false); PreparedStatement st = connection.prepareStatement(sql); st.setParameter(1, balance1 + amount); st.setParameter(2, debitId); st.execute(); st.setParameter(1, balance2 + amount); st.setParameter(2, creditId); st.execute(); connection.commit(); } catch (SQLException e) { connection.rollback(); throw new DataAccessException(e); } finally { connection.close(); } Debit Operation Credit Operation TX
  29. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Modular Application APP Driver RDBMS DB SQL results Infrastructure Layer Data-Access Layer Service layer Multi-Tier Architecture (2,3+ Layers) Presentation layers public interface AccountRepository { Account getAccount(Long id); void updateBalance(Account account); ... }
  30. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Transaction Management • Transaction scope may include: – Multiple calls to the same DAO – Operations in different DAOs – Operations in different Resources (JTA) • Data-Access Technology Independence: – Service-Layer Implementation – DAO API • Connection Must be Shared – Thread/Request Contextualized Service Layer Data-Access Layer Issues
  31. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Transactional Interceptor ServiceImpl DAO Transactional Interceptor TransactionManager Proxy begin() commit() Client Object DB AOP – with Proxy Power Service Thread-Safe
  32. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Transaction Management Annotation-Driven Declarative Demarcation @Singleton public class TransferServiceImpl implements TransferService { @Inject private AccountRepository accountRepository; @Inject private OperationsRepository operationsRepository; @Transactional public Transfer transfer(Long creditId, Long debitId, BigDecimal amount) { // Do transfer with AccountRepository ... // Write confirmation with OperationsRepository ... ... } }
  33. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Transaction Management Local Transaction Propagation @Singleton public class TransferServiceImpl implements TransferService { ... @Transactional public Transfer transfer(...) { // do transfer // write confirmation with OperationsRepository ... … emailService.send(msg); } } @Singleton public class EmailServiceImpl implements EmailService { @Transactional public void send(Message msg) { ... } } method1 method2 TX 1 TX 1 TX 2
  34. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Transaction Management Remote Transaction Propagation method1 method2 TX 1 TX 1 TX 2 public class TransferServiceImpl ... { ... @Transactional public Transfer transfer(...) { … remoteEmailService.send(msg); } } @Singleton public class RemoteEmailService ... { @Transactional public void send(Message msg) { ... } } Host1 Host2
  35. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Outline • Properties of Transactions • Kinds of Transactional Systems • Mechanisms for Transactions • Transaction Managament • Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP
  36. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Transactions in Spring Uniform Approach @Service public class TransferServiceImpl implements TransferService { @Autowired private AccountRepository accountRepository; @Autowired private OperationsRepository operationsRepository; @Transactional public Transfer transfer(Long creditId, Long debitId, BigDecimal amount) { Account debitAccount = accountRepository.getAccount(creditId); Account creditAccount = accountRepository.getAccount(debitId); debitAccount.debit(amount); creditAccount.credit(amount); accountRepository.update(debitAccount); accountRepository.update(creditAccount); Transfer transfer = new Transfer(debitAccount, creditAccount, amount); operationsRepository.add(transfer); return transfer; } } import org.springframework.stereotype.Service; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.transaction.annotation.Transactional;
  37. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Transaction Configuration Isolation, Propagation, Rollback Rules @Service public class TransferServiceImpl implements TransferService { @Transactional(isolation=Isolation.SERIALIZABLE, noRollbackFor=EmailException.class) public Transfer transfer(Long creditId, Long debitId, BigDecimal amount) { … emailService.sendConfirmation(Transfer.buildMessage(transfer)); return transfer; } } @Service public class EmailServiceImpl implements EmailService { @Transactional(propagation=Propagation.REQUIRES_NEW) public void send(Message message) { … } }
  38. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JDBC Repositories @Repository public class JdbcAccountRepository implements AccountRepository { @Autowired private DataSource dataSource; private JdbcTemplate jdbcTemplate; @PostConstruct public void init() { jdbcTemplate = new JdbcTemplate(dataSource); } public Account getAccount(Long id) { String sql = “select * from T_ACCOUNT where ID=?”; return jdbcTemplate.query(sql, id); } public void updateBalance(Account account) { String sql = “update T_ACCOUNT set BALANCE=? where ID=?”; jdbcTemplate.update(sql, account.getBalance(), account.getId()); } } import org.springframework.stereotype.Repository; import org.springframework.beans.factory.annotation.Autowired; import java.lang.annotation.PostConstruct;
  39. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Transactions in Spring Configuration @Configuration @EnableTransactionManagament @Import({JdbcConfig.class, JpaConfig.class, DBConfig.class}); @ComponentScan(“myapp.services”) @PropertySource(“app.properties”) public class AppConfig { … } @Configuration @Profile(“jdbc”) public class JdbcConfig { @Bean PlatformTransactionManager txMgr(DataSource dataSource) { return new DataSourceTransactionManager(dataSource); } } @Configuration @Profile(“jpa”) public class JpaConfig { … } import org.springframework.context.annotation.*;
  40. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Transactions in Spring Conditional Beans import org.springframework.context.annotation.*; @Configuration @ImportResource(“db-config.xml”) public class DBConfig { @Autowired private Environment env; @Bean @Reachable DataSource master() { .. } @Bean @Reachable DataSource slave1() { .. } @Bean @Reachable DataSource slave2() { .. } } public class ReachableCondition implements Condition { public boolean matches(ConditionContext context, AnnotatedTypeMetadata metadata) { … } } @Conditional(ReachableCondition.class) @Target({ElementType.TYPE, ElementType.METHOD}) @Retention({RetentionPolicy.RUNTIME}) public @interface Reachable { } import java.lang.annotation.*;
  41. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Outline • Properties of Transactions • Kinds of Transactional Systems • Mechanisms for Transactions • Transaction Managament • Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP
  42. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    ORM Architecture Automating Object to/from RDB Mapping Cache Annotations XML Meta-Data Persistence API Database Abstraction Layer Client DAO DB SQL ✔ SQL Query Generation ✔ Mapping ✔ Column Extraction ✔ Parameter Extraction ✔ Association Managm. ✔ DB Abstraction ✔ Declarative Config ✔ Proxy Creation ✔ Schema Creation Config DataSource Properties Criteria Query API + OQL Queries Criteria Query
  43. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JPA Architecture EntityManager & EntityManagerFactory public class JpaAccountRepository { EntityManager entityManager; ... } EntityManager Data-Access Object Entity Instance Entity Instance Persistence Context (Cache) Persistence Unit DB EntityManagerFactory Connection DataSource
  44. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JPA Entities Annotation-Driven Meta-Data @Entity @Table(“T_ACCOUNT”) public class Account { @Id private Long id; private String name; @Column(name=“DOB”) @Temporal(TemporalType.DATE) private Date birthday; @OneToMany @JoinColumn(name=“ACCOUNT_ID”) private Set<Address> addresses; public Account() { } } Physical Mapping @Table @Column @JoinColumn Annotations Logical/Model @Entity @Id @OneToMany 1:N Account Address
  45. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JPA Transactions public class JpaAccountDao implements AccountDao { @Inject private EntityManagerFactory emf; public void importAccounts(List<Account> accounts) { EntityManager em = emf.createEntityManager(); Transaction tx = em.getTransaction(); try { tx.begin(); for (Accout account: accounts) { entityManager.persist(account); } tx.commit(); } catch (Exception e) { tx.rollback(); } } ... } Programmatic Demarcation
  46. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JPA DAOs public class JpaAccountDao implements AccountDao { @PersistenceContext private EntityManager entityManager; public JpaAccountDao() {} public void add(Account account) { entityManager.persist(account); } ... } EntityManagerProxy EntityManager Proxy @Singleton @Repository import javax.persistence.PersistenceContext; EJB&JSR330 Spring @ApplicationScoped CDI
  47. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JPA Configuration @Configuration @Profile(“jpa”) public class JpaConfig { @Bean PlatformTransactionManager txMgr(EntityManagerFactory emf) { return new JpaTransactionManager(emf); } @Bean LocalContainerEntityManagerFactoryBean createEntityManagerFactory(DataSource dataSource) { LocalContainerEntityManagerFactoryBean emfb = new LocalContainerEntityManagerFactoryBean(); emfb.setPersistenceUnitName("onlineshop"); emfb.setPackagesToScan(“com.onlineshop.domain”); emfb.setDataSource(dataSource); return new emfb; } }
  48. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JPA Configuration import org.einnovator.jpa.provider.PersistenceUnitInfoBuilder; import org.einnovator.jpa.Jpa; @Singleton @Jpa public class JpaTestConfig { @Factory public EntityManagerFactory createEntityManagerFactory(DataSource dataSource) { return new PersistenceUnitInfoBuilder() .name("onlineshop") .packagesToScan(EntitySupport.class) .dataSource(dataSource) .property("hibernate.dialect", "org.hibernate.dialect.HSQLDialect") .property("hibernate.hbm2ddl.auto", "create") .property("hibernate.show_sql", true) .buildEntityManagerFactory(); } }
  49. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Outline • Properties of Transactions • Kinds of Transactional Systems • Mechanisms for Transactions • Transaction Managament • Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP
  50. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    MySQL Replication • Asynchronous Master-Slave • Semi-Synchronous Master-Slave • Clustering • Partitioning Replication Architectures
  51. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    MySQL Replication mysql> CHANGE MASTER TO -> MASTER_HOST='master_host_name', -> MASTER_USER='replication_user_name', -> MASTER_PASSWORD='replication_password', -> MASTER_LOG_FILE='recorded_log_file_name', -> MASTER_LOG_POS=recorded_log_position; [mysqld] log-bin=mysql-bin server-id=1 [mysqld] server-id=2 Master Slave Master-Slave Configuration
  52. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JAVA-MySQL Connector JDBC Connection to Master+Servers @Configuration @ImportResource(“db-config.xml”) public class DBConfig { @Autowired private Environment env; @Bean @Profile(“mysqlWAN”) DataSource masterserver() { return new DriverManagerDataSource( "jdbc:mysql:replication://master,slave1,slave2,slave3/myapp-db"); } ... } import org.springframework.jdbc.datasource.DriverManagerDataSource;
  53. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Outline • Properties of Transactions • Kinds of Transactional Systems • Mechanisms for Transactions • Transaction Managament • Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP
  54. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    MongoDB • Structured Document Data-Model • BJSON Optimized Serialization • Document Collection Abstraction • Queries on Document Collection • Projection Retrieval & Partial Updates Data-Model { stocks : [ {sym:”PIV”, value:120.13, org: <ObjectId1>}, {sym:”LNX”, value:88.99, org: <ObjectId2>}, {sym:”SGT”, value:23.12, org: <ObjectId3>} ], date : '06-01-2015', exchange: 'nasdaq' } { _id: <ObjectId1>, sym:”PIV”, name:”Pivotal, Inc.”, industry: “IT”, revenue: [ { year: 2014, qrts: [“8.5M”,“10.5M”, “14.0M”, “22.0M” ]}], workers: 2100 } company stocks
  55. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    MongoDB • Limited Transactions Scope – Operations in Single Document – No Atomicity Across Documents&Servers – Distributed ACID also available (but expensive) • Read on Secondary Servers might not be latest – Not Propagated yet from Primary Transactions WRITE_CONCERN
  56. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    MongoDB Replication • Replication with Resplica-Sets – Recent Versions: Primary-Secondary+Auto-Failover – Early Versions: Master-Slaves Configuration > mongod --replSet "rs0" {--config $HOME/.mongodb/config} > mongo mongo> rs.initiate() mongo> rs.conf() mongo> rs.add(“mongo2.mydomain”) mongo> rs.add(“mongo3.mydomain”) > mongod --port 27017 --dbpath /d/mongodb/rs0-0 --replSet rs > mongod --port 27018 --dbpath /d/mongodb/rs0-1 --replSet rs > mongod --port 27019 --dbpath /d/mongodb/rs0-2 --replSet rs @Profile(“dev”) @Profile(“prod”)
  57. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    MongoDB Partitioning • Partitioning (Sharding) of Documents in Collection Partitioning (Sharding) > mongod > mongo mongo>
  58. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Outline • Properties of Transactions • Kinds of Transactional Systems • Mechanisms for Transactions • Transaction Managament • Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP
  59. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    GemFire • Distributed Object-Store – Key-Value Store++ with Object Queries • Custom Optimized Binary Serialization • Region = Collections of Objects • In-Memory Data-Grid - Use cases: – Distributed Cache – High-Performance DataBase • Taps into underlying storage for persistence – RDBMS, NoSql, HDFS (Hadoop) Overview
  60. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    GemFire Data-Model Cache Data Control TransactionListener Region Region Region Region Region Control CacheListener CacheLoader CacheWriter Region: “books” Data Entry Key Value Meta-Data Entry Key Value Meta-Data DB Region java.util.Map
  61. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    GemFire Region Types a.acme.com “Users” bucket[1] Key hash() bucket[1] bucket[2] bucket[3] ... bucket[4] Partitioned Replicated Distributed Local b.acme.com “Users” bucket[2] bucket[7] c.acme.com “Users” bucket[5] bucket[9] a.acme.com “Stocks” b.acme.com “Stocks” c.acme.com “Stocks” K1 V1 K2 V2 K3 V3 K1 V1 K2 V2 K3 V3 K1 V1 K2 V2 K3 V3 a.acme.com “Docs” b.acme.com “Docs” c.acme.com “Docs” K1 V1 K2 V2 K1 V1 K3 V3 K2 V2 K4 V4 K7 V7 a.acme.com “F(X)” K1 V1 K2 V2 K7 V7
  62. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    GemFire Transactions Propagatation and Concurrency Control Replicated&Distributed a.acme.com “Stocks” b.acme.com “Stocks” c.acme.com “Stocks” Scope Propagation Wait Concurrency Control Local No NO None Distributed YES NO None Distributed+ACK YES YES Versions
  63. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    GemFire Transactions API CacheTransactionManager txmgr = c.getCacheTransactionManager(); try { txmgr.begin(); Integer balance = cash.get(customer.getId()); Integer newBalance = Integer.valueOf(balance - price); cash.put(customer.getId(), newBalance); … txmgr.commit(); } catch (CommitConflictException e) { txmgr.rollback(); } Cache c = CacheFactory.create(DistributedSystem.connect(null)); AttributesFactory af = new AttributesFactory(); af.setDataPolicy(DataPolicy.REPLICATE); Region<String, Integer> cash = c.createRegion("cash", af.create());
  64. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Outline • Properties of Transactions • Kinds of Transactional Systems • Mechanisms for Transactions • Transaction Managament • Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP
  65. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JMS API ConnectionFactory Connection Message Producer Message Consumer singleton TCP/IP broker connection messaging context Message createConnection() createProducer() createConsumer() Session createSession() create*Message() MessageListener App MessageListener setMessageListener() Message onMessage()
  66. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JMS - Transactions Session session = connection.createSession(true, 0); Producer producer = session.createProducer(destination); try { for (int i=0; i<3; i++) { Message msg = session.createTextMessage(); msg.setText(“Hello JMS – Part 1”); producer.send(msg); } session.commit(); } catch (Exception e) { session.rollback(); } ConnectionFactory factory = new ABCProviderConnectionFactory(); Destination queue = new ABCQueueDestination(“queue-q1”); Connection connection = factory.createConnection(); connection.activate(); Sending
  67. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JMS - Transactions Session session = connection.createSession(true, 0); Consumer consumer = session.createConsumer(destination); try { for (int i=0; i<3; i++) { TextMessage msg = (TextMessage) consumer.receive(); System.out.println(msg.getText()); } session.commit(); } catch (Exception e) { session.rollback(); } Receiving
  68. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Spring JMS Config @Configuration public class JmsConfig { @Bean JmsTemplate jmsTemplate(ConnectionFactory factory) { return new JmsTemplate(factory); } @Bean ConnectionFactory connectionFactory() { return new ConnectionFactory(); } @Bean Queue stocksQueue() { return new ABCQueue(); } @Bean MessageListenerContainer messageListenerContainer() { return new SimpleMessageListenerContainer(); } } @Autowired JmsTemplate template; @Autowired Destination stocksQueue;
  69. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Spring JMS final Stock[] stocks = … jmsTemplate.exectute(stocksQueue, (Session s, MessageProducer<Void> p) → { for (Stock stock: stocks) { Message msg = converter.convertToMessage(s, stock); p.send(msg); } } ); Sending ProduceCallback<T> jmsTemplate.setSessionTransacted(true); MessageConverter converter = jmsTemplate.getMessageConverter(); jmsTemplate.convertAndSend(stocksQueue, stock); TX
  70. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Spring JMS final List<Stock> stocks = new ArrayList<>(); jmsTemplate.exectute(stocksQueue, (Session s, MessageConsumer<Void> p) → { do { Message msg = c.receive(); stocks.add((Stock)converter.convertFromMessage(msg)); } while (/*more*/); } ); Receiving - Synchronous ConsumerCallback<T> jmsTemplate.setSessionTransacted(true); MessageConverter converter = jmsTemplate.getMessageConverter(); Stock stock = (Stock) jmsTemplate.receiveAndConvert(); TX
  71. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Spring JMS Receiving - Asynchronous TX @Component public class StockProcessor { @JmsListener(destination=“stocksQueue”) public void processStock(Stock stock) { //process stock... } } @Configuration @ComponentScan(“myapp.jms”) @EnableJms public class JmsConfig { ... @Bean MessageListenerContainer messageListenerContainer() { DefaultMessageListenerContainer container = new DefaultMessageListenerContainer(); container.setTransacter(true); return container; } }
  72. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Outline • Properties of Transactions • Kinds of Transactional Systems • Mechanisms for Transactions • Transaction Managament • Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP
  73. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    RabbitMQ – Java Client API ConnectionFactory factory = new ConnectionFactory(); factory.setHost("localhost"); Connection connection = factory.newConnection(); Channel channel = connection.createChannel(); channel.close(); connection.close(); import com.rabbitmq.client.Channel; //...use channel to send&recv messages...
  74. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    RabbitMQ – Sending Messages try { String[] stocks = {"SYM RabbitMQ = 99.9", "SYM LinuxLTD = 10.0", "SYM XMQ = 50.0"}; channel.txSelect(); for (String stock: stocks) { channel.basicPublish(“stocks-x”, “IT”, null, stock.getBytes()); } channel.txCommit(); } catch (Exception e) { channel.txRollback(); }
  75. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    RabbitMQ – Receiving Messages try { String[] stocks = {"SYM RabbitMQ = 99.9", "SYM LinuxLTD = 10.0", "SYM XMQ = 50.0"}; channel.txSelect(); for (int i=0; i<3; i++) { GetResponse msg = channel.basicGet(“trading-q”, true); System.out.println(new String(msg.getBytes())); } channel.txCommit(); } catch (Exception e) { channel.txRollback(); }
  76. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    STOMP CONNECT accept-version:1.2 host:stomp.github.org ^@ Simple (or Streaming) Text Orientated Messaging Protocol CONNECTED version:1.2 ^@ SEND destination:/queue/stocks-q content-type:text/plain SYM PIV; 123.75 ^@ BEGIN transaction:tx1 ^@ COMMIT transaction:tx1 ^@ http://stomp.github.io/
  77. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Outline • Properties of Transactions • Kinds of Transactional Systems • Mechanisms for Transactions • Transaction Managament • Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP
  78. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JTA Java Transactions Architecture JDBC Resource Drivers JMS Connection JDBC XA XA Resource (adaptors) JMS XA DB Broker Msg Broker DB JTA TxMgr APP Global TX • Application Server • Standalone App JEE
  79. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    JTA - API @Stateless @TransactionManagement(BEAN) public class StockServiceImpl implements StockService { @Resource private UserTransaction tx; @Resource private DataSource dataSource; @Resource private ConnectionFactory connectionFactory; public void foo() { tx.begin(); // Do work on dataSource // Do work on connectionFactory tx.commit(); } } EJB Model
  80. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Outline • Properties of Transactions • Kinds of Transactional Systems • Mechanisms for Transactions • Transaction Managament • Practical Examples – RDBMS: Spring JDBC, JPA/ORM, MySql – NoSQL: MongoDB, GemFire – Messaging: JMS, AMQP+RabbitMQ – JTA – LDAP
  81. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    LDAP • Naming and Directory Service • Structure Names Data-Model • Keeps many types of Information: – Organizational – Users – Security Policy – Custom • No native support for ACID Transactions – Only Durability and Consistency Rules
  82. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Spring LDAP • Simplified API to connect and interact with LDAP server • Emulates Transactions using Compensatory Undo Operations – Atomicity Only – Programatic API – Integrated with Declarative @Transactional • No concurrency control (isolation) – Except if conflict detected by app and rollback
  83. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Conclusions • Transactions are very often essential – For Reliability and Consistency • Multiple Kinds of Transactional Resources • Distribution of Transactions is Expensive – Trade-off: Scalability vs. Consistency vs. Availability • Basic Mechanisms Well-Understood – Many optimizations possible • AOP Simplifies Transaction Management Thank You
  84. JPalace.org – Software Engineering School © 2012-2014 EInnovator, Enterprise Middleware

    Learning More • Training – Core Spring – Enterprise Integration with Spring – RabbitMQ – Java 8, JEE7 • Resources – Spring.io – Spring Downloads & Docs – JPalace.org – RefCards, Tutorials, Articles