ChTest Virtual cluster at your finger tips 

Purpose ChTest is a library for writing automatic tests for Oracle Coherence data grid. Challenge: To test a cluster you need a cluster. Solution: Simulate a cluster, manage “virtualized” java process with smooth API. Plus, a lot of neat stuff to manage Coherence nodes. 

Simple case @Rule public DisposableCohCloud cloud = new DisposableCohCloud(); @Test public void simple_cluster() { // Present for typical single node cluster cloud.all().presetFastLocalCluster(); cloud.node("storage.**").localStorage(true); cloud.node("client.**").localStorage(false); // Simulates DefaultCacheServer based process cloud.node("storage.**").autoStartServices(); // declaring specific nodes to be created CohNode storage = cloud.node("storage.1"); CohNode client1 = cloud.node("client.1"); CohNode client2 = cloud.node("client.2"); // now we have 3 specific nodes in cloud // all of then will be initialized in parallel cloud.all().ensureCluster(); final String cacheName = "distr-a"; client1.exec(new Runnable() { @Override public void run() { NamedCache cache = CacheFactory.getCache(cacheName); Assert.assertNull(cache.get("A")); cache.put("A", "aaa"); } }); client2.exec(new Runnable() { @Override public void run() { NamedCache cache = CacheFactory.getCache(cacheName); Assert.assertEquals("aaa", cache.get("A")); } }); } 

Simple case @Rule public DisposableCohCloud cloud = new DisposableCohCloud(); @Test public void simple_cluster() { // Present for typical single node cluster cloud.all().presetFastLocalCluster(); cloud object is managing all your virtual nodes @Rule will ensure that all nodes created in test would be terminated Common settings for “virtual” cluster: WKA, timeouts, etc Will apply to all nodes in this cluster 

Convenient method to set tangosol.coherence.distributed.localstorage system property Simple case cloud.node("storage.**").localStorage(true); cloud.node("client.**").localStorage(false); // Simulates DefaultCacheServer based process cloud.node("storage.**").autoStartServices(); Simulate DefaultCacheServer behavior for stoarge nodes Using wild card in node name, creates a rule which will apply all matching nodes 

Simple case // declaring specific nodes to be created CohNode storage = cloud.node("storage.1"); CohNode client1 = cloud.node("client.1"); CohNode client2 = cloud.node("client.2"); Our cluster will have one storage enabled and 2 storage disable nodes Mentioning a node name without a wild card is a declaration of specific node Nothing is created at this point, initialization is lazy 

Simple case // now we have 3 specific nodes in cloud // all of then will be initialized in parallel cloud.all().ensureCluster(); All nodes which have been declare will be initialized in parallel. Coherence loads about 5000 classes, utilizing more CPU cores provides noticeable speed up “Virtual” nodes are initialized once on first attemp to execute something inside of node Convenient method to call CacheFactory.ensureCluster() for “virtual” node 

Simple case … [client.1] 2013-03-14 00:12:15.205/27.817 Oracle Coherence GE 3.7.1.3 (thread=Cluster, member=2): Member(Id=3, Timestam… [storage.1] 2013-03-14 00:12:15.205/27.817 Oracle Coherence GE 3.7.1.3 (thread=Cluster, member=n/a): This Member(Id=3… [storage.1] 2013-03-14 00:12:15.406/28.018 Oracle Coherence GE 3.7.1.3 (thread=Cluster, member=n/a): Member(Id=2, Times… [storage.1] 2013-03-14 00:12:15.572/28.184 Oracle Coherence GE 3.7.1.3 (thread=Cluster, member=n/a): Member 1 joined Se… [client.2] 2013-03-14 00:12:15.640/28.252 Oracle Coherence GE 3.7.1.3 (thread=ISOLATE[client.2] Cluster, member=1): Mem… [storage.1] 2013-03-14 00:12:15.641/28.253 Oracle Coherence GE 3.7.1.3 (thread=ISOLATE[storage.1] Cluster, member=n/a):… … Coherence member detail contains “virtual” node name Console output from each node is prefixed for convenience while “virtual” cluster is started let’s take look at console … …erMemberSet( …isMember=Member(Id=3, Timestamp=2013-03-14 00:12:15.129, Address=127.0.0.1:51739, MachineId=3073, Location=site:,process:st… …destMember=Member(Id=1, Timestamp=2013-03-14 00:12:09.908, Address=127.0.0.1:51735, MachineId=3073, Location=site:,process:… …tualMemberSet=MemberSet(Size=3 …Member(Id=1, Timestamp=2013-03-14 00:12:09.908, Address=127.0.0.1:51735, MachineId=3073, Location=site:,process:client.2, R… …Member(Id=2, Timestamp=2013-03-14 00:12:14.76, Address=127.0.0.1:51737, MachineId=3073, Location=site:,process:client.1, Ro… …Member(Id=3, Timestamp=2013-03-14 00:12:15.129, Address=127.0.0.1:51739, MachineId=3073, Location=site:,process:storage.1, … … 

Simple case Threads created in “virtual” node context are prefixed with it’s name let’s lookup at threads Thread dump will also have “virtual” node names 

Simple case client1.exec(new Runnable() { @Override public void run() { NamedCache cache = CacheFactory.getCache(cacheName); Assert.assertNull(cache.get("A")); cache.put("A", "aaa"); } }); exec will execute Runnable or Callable synchronously in context of your virtual nodes This object will be “recreated” in different classloader and executed in different thread You can use JUnit’s Assert here. Exception will be forwarded to caller. You can use Coherence API in the usual way 

Simple case Wrapping up  Cluster of three nodes was created in JUnit test  Nodes have independent configuration  Nodes have independent system properties  You can execute arbitrary code in scope of any node  Parallel initialization  Can be run under debugger 

Bidirectional communications public interface RemotePut extends Remote { public void put(Object key, Object value); } RemotePut remoteService = client1.exec(new Callable() { @Override public RemotePut call() { final NamedCache cache = CacheFactory.getCache(cacheName); return new RemotePut() { @Override public void put(Object key, Object value) { cache.put(key, value); } }; } }); public interface RemotePut extends Remote { public void put(Object key, Object value); } @SuppressWarnings("unused") @Test public void bidirectional_remoting() { // Present for typical single node cluster cloud.all().presetFastLocalCluster(); cloud.node("storage.**").localStorage(true); cloud.node("client.**").localStorage(false); // Simulates DefaultCacheServer based process cloud.node("storage.**").autoStartServices(); // declaring specific nodes to be created CohNode storage = cloud.node("storage.1"); CohNode client1 = cloud.node("client.1"); CohNode client2 = cloud.node("client.2"); // now we have 3 specific nodes in cloud // all of then will be initialized in parallel cloud.all().ensureCluster(); final String cacheName = "distr-a"; RemotePut remoteService = client1.exec(new Callable() { @Override public RemotePut call() { final NamedCache cache = CacheFactory.getCache(cacheName); return new RemotePut() { @Override public void put(Object key, Object value) { cache.put(key, value); } }; } }); remoteService.put("A", "aaa"); client2.exec(new Runnable() { @Override public void run() { NamedCache cache = CacheFactory.getCache(cacheName); Assert.assertEquals("aaa", cache.get("A")); } }); } remoteService.put("A", "aaa"); 

Bidirectional communications public interface RemotePut extends Remote { public void put(Object key, Object value); } RemotePut remoteService = client1.exec(new Callable() { @Override public RemotePut call() { final NamedCache cache = CacheFactory.getCache(cacheName); return new RemotePut() { @Override public void put(Object key, Object value) { cache.put(key, value); } }; } }); remoteService.put("A", "aaa"); Extending java.rmi.Remotexec will mark interface for auto export Unlike Java RMI, there is no need to declare RemoteException for every method Result of callable will be serialized and transferred to caller Objects implementing remote interfaces are automatically replaced with remote stub during serialization Here we got a remote stub, not a real implementation of interface Call to a stub, will be converted to “remote” call to instance we have created in “virtualized” node few lines above 

Out of process nodes If class loader isolation is not enough … • node.outOfProcess(true) • separate slave JVM process will be spawned • same API will work for out-of-process nodes • JVM options (e.g. -Xmx) could be passed • slave’s console will be prefixed and show on master • if master process dies, slave will die too • even if master has been killed by debugger or kill -9 

Tweaking classpath You could tweak classpath for each node • include or exclude jars from classpath • works for both classloader isolation and out-of-process slaves • you can start cluster with different versions of coherence.jar • for your convenience node.useCoherenceVersion("3.7.1.8")  current version of cohernce.jar would be excluded  required version would be included from …  … eigther local maven cache …  … or from resources (should packaged in special way) 

Simulating node crash Default TCMP timeout is 10 minutes Too long for a test! For in-process node, shutdown will kill all threads thread in its scope TCP ring detects process liveliness, it will prevent death detection of “virtualized”node cloud.all().setTCMPTimeout(15000); cloud.all().enableTcpRing(false); cloud.node("server.0").shutdown(); System.out.println("It takes some time for Coherence" + " to detect node failure is \"virtualized\" node"); For out-of-process node, its process would be destroyed. 

Coherence MBean support Want to see Coherence Mbeans?  node.enaleJmx(true) Each node has separate domain All “virtualized” nodes are exposed All “virtualized” nodes are exposed 

Try it yourself … org.gridkit.coherence-tools chtest 0.1 test [email protected] https://code.google.com/p/gridkit/wiki/ChTest