The Measured Performance of Communication and Serialization Primitives Brian Blose, Phillip Burdette, and Gregory M. Kapfhammer Department of Computer Science Allegheny College http : //cs.allegheny.edu/˜gkapfham/ The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 1/15

Communication Primitives How does object encoding impact communication? Contribution: A benchmarking framework to compare the performance of sockets and XML-RPC The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 2/15

Serialization Primitives F P JVM Heap allocate serialize How does object encoding impact serialization? Contribution: A benchmarking framework to compare the performance of binary and XML serialization The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 3/15

Program Execution with a JVM Program Stack Fast? Interpreter? Machine Virtual JIT? Adaptive? Native Code Cache Heap methodA testOne Input Output Byte Code P JVM implementation and configuration impacts performance The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 4/15

Experiment Design Communication: sockets and XML-RPC Serialization: XStream, JBoss, Java Serialize and Externalize Select Java 1.5.0, GNU/Linux with kernel 2.6.12, 3 GHz P4, 1 GB main memory, 1 MB L1 Cache, CPU hyperthreading Use operating system and language-based timers to calculate response time and space overheads Execute ten trials and calculate arithmetic means, standard deviations, and confidence intervals Understand internal behavior of the Java virtual machine The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 5/15

Micro Benchmarks Experiment Sent by client Received by client SS Single primitive Single primitive SV Single primitive Vector VS Vector Single primitive VV Vector Vector Use benchmarks similar to those proposed by Allman et al. Implement the benchmarks in the Java language ExperimentCampaign framework uses Perl and Mathematica The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 6/15

Micro Benchmarks II Experiment Sent by client Received by client FIND (SS) Single primitive Single primitive FACT (SV) Single primitive Vector GCD (VS) Vector Single primitive REV (VV) Vector Vector Benchmarks use sockets and Apache XML-RPC Benchmarks perform a simple computation on the server Configure the client and server to execute on same node The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 7/15

Micro Benchmark I S SS X SS S SV X SV S VS X VS S VV X VV Benchmarks 0.02 0.04 0.06 0.08 0.1 0.12 0.14 Time seconds S SS X SS S SV X SV S VS X VS S VV X VV Micro Experiments Language Based Timer 0.0004 0.0859 0.0013 0.0897 0.0012 0.0908 0.0237 0.0897 XML-RPC shows greater response time with more dispersion The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 8/15

Micro Benchmark II S FIND X FIND S FACT X FACT S GCD X GCD S REV X REV Benchmarks 0.25 0.5 0.75 1 1.25 1.5 1.75 2 Time seconds S FIND X FIND S FACT X FACT S GCD X GCD S REV X REV Macro Experiments Language Based Timer 0.0017 0.0863 0.0023 0.093 0.0022 0.0857 0.0033 1.7605 X-REV exhibits high response time due to string parsing The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 9/15

Using Very Large Vectors size(V ) size(V ) (bytes) R(VV, S) (sec) R(VV, X) (sec) 5000 80,520 0.298 0.347 10000 161,000 0.598 0.523 50000 927,720 18.784 1.697 At smaller vector sizes sockets demonstrate slightly better response times XML-RPC shows better response time when size(V ) = 50000 : why? The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 10/15

Explanatory Power of GC size(V ) YGC (count) YGC (sec) FGC (count) FGC (sec) 5000 16 .008 0 0 10000 63 .023 4 .050 50000 1645 .697 663 10.375 size(V ) YGC (count) YGC (sec) FGC (count) FGC (sec) 5000 14 .016 0 0 10000 27 .022 1 .020 50000 123 .695 5 .143 Varying the heap size of socket JVM yields similar results The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 11/15

GC Allocation Rate S-VV allocates 710, 374, 184 bytes and X-VV only allocates 54, 101, 312 bytes At benchmark termination, S-VV has 4, 773, 224 bytes and X-VV has 7, 234, 520 bytes of live objects Sockets use char[] and XML-RPC uses java.nio.CharBuffer Can we use past GC behavior to predict future program performance? The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 12/15

Serialization Response Time 0 250 500 750 1000 1250 1500 1750 Time msec LL JS S JS M JS L JB S JB M JB L JE S JE M JE L XS S XS M XS L Primitive JS S JS M JS L JB S JB M JB L JE S JE M JE L XS S XS M XS L 0 500 1000 1500 2000 Time msec DLL JS S JS M JS L JB S JB M JB L JE S JE M JE L XS S XS M XS L JS S JS M JS L JB S JB M JB L JE S JE M JE L XS S XS M XS L Ser Deser Serialize and deserialize a LinkedList XS-L exhibits high response time due to parsing and validation JS and JE demonstrate a low response time The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 13/15

Serialization Trade-Offs (JS) 0 1000 2000 3000 4000 5000 Size of ADT elements JS LL 0 50 100 150 200 250 300 Time msec Deser Ser JS response time varies as ADT size increases (not for XS) The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 14/15

Conclusions A suite of benchmarks to measure the performance of communication and serialization primitives Experiments reveal a trade-off in the performance of the two primitives Extend the study to new primitives and JVMs Focus on remote communication, long running benchmarks, and the measurement of throughput Consider the use of new abstract data types What are your suggestions? The Measured Performance of Communication and Serialization Primitives, NITLE IPCC, June 19, 2007 – p. 15/15