Kapfhammer Department of Computer Science Allegheny College http : //cs.allegheny.edu/˜gkapfham/ Measuring the Performance of an XML-Based Communication Primitive, RICSS, September 22, 2006 – p. 1/15
the World?” Dear Sirs, I am. Sincerely yours, G. K. Chesterton Measuring the Performance of an XML-Based Communication Primitive, RICSS, September 22, 2006 – p. 2/15
Contribution: A benchmarking framework to compare the performance of sockets and XML-RPC Measuring the Performance of an XML-Based Communication Primitive, RICSS, September 22, 2006 – p. 3/15
Sun RPC or XML-RPC Does the communication primitive impact performance? How do we measure performance and/or correctness? Measuring the Performance of an XML-Based Communication Primitive, RICSS, September 22, 2006 – p. 4/15
Virtual JIT? Adaptive? Native Code Cache Heap methodA testOne Input Output Byte Code P JVM implementation and configuration impacts performance Measuring the Performance of an XML-Based Communication Primitive, RICSS, September 22, 2006 – p. 5/15
Single primitive Single primitive SV Single primitive Vector VS Vector Single primitive VV Vector Vector Use benchmarks similar to those proposed by Allman et al. Implement the benchmarks in the Java language ExperimentCampaign framework uses Perl and Mathematica Measuring the Performance of an XML-Based Communication Primitive, RICSS, September 22, 2006 – p. 6/15
FIND (SS) Single primitive Single primitive FACT (SV) Single primitive Vector GCD (VS) Vector Single primitive REV (VV) Vector Vector Benchmarks use sockets and Apache XML-RPC Benchmarks perform a simple computation on the server Configure the client and server to execute on same node Measuring the Performance of an XML-Based Communication Primitive, RICSS, September 22, 2006 – p. 7/15
GHz P4, 1 GB main memory, 1 MB L1 Cache, CPU hyperthreading Use operating system and language-based timers to calculate R(B, P), R∆ (B, P, P ), and R% ∆ (B, P, P ) Replace the socket communication primitive with XML-RPC Execute ten trials and calculate arithmetic means, standard deviations, and confidence intervals Formulate the null hypothesis as H0 : µR(B,P) = µR(B,P ) Use the Welch’s approximate t-test with α = .01 Measuring the Performance of an XML-Based Communication Primitive, RICSS, September 22, 2006 – p. 8/15
SV S VS X VS S VV X VV Benchmarks 0.02 0.04 0.06 0.08 0.1 0.12 0.14 Time seconds S SS X SS S SV X SV S VS X VS S VV X VV Micro Experiments Language Based Timer 0.0004 0.0859 0.0013 0.0897 0.0012 0.0908 0.0237 0.0897 XML-RPC shows greater response time with more dispersion Measuring the Performance of an XML-Based Communication Primitive, RICSS, September 22, 2006 – p. 9/15
FACT S GCD X GCD S REV X REV Benchmarks 0.25 0.5 0.75 1 1.25 1.5 1.75 2 Time seconds S FIND X FIND S FACT X FACT S GCD X GCD S REV X REV Macro Experiments Language Based Timer 0.0017 0.0863 0.0023 0.093 0.0022 0.0857 0.0033 1.7605 X-REV exhibits high response time due to string parsing Measuring the Performance of an XML-Based Communication Primitive, RICSS, September 22, 2006 – p. 10/15
X-VV only allocates 54, 101, 312 bytes At benchmark termination, S-VV has 4, 773, 224 bytes and X-VV has 7, 234, 520 bytes of live objects Sockets use char[] and XML-RPC uses java.nio.CharBuffer Can we use past GC behavior to predict future program performance? Measuring the Performance of an XML-Based Communication Primitive, RICSS, September 22, 2006 – p. 13/15
of communication primitives A comparison of sockets and XML-RPC that we can extend to other primitives Experiments reveal a trade-off in the performance of the two primitives Extend the study to new primitives and JVMs Focus on remote communication, long running benchmarks, and the measurement of throughput What are your suggestions? Measuring the Performance of an XML-Based Communication Primitive, RICSS, September 22, 2006 – p. 14/15
participation! http : //cs.allegheny.edu/˜gkapfham/research/ Measuring the Performance of an XML-Based Communication Primitive, RICSS, September 22, 2006 – p. 15/15