Kapfhammer, Daniel M. Fiedler Kristen Walcott, Thomas Richardson Department of Computer Science Allegheny College Ahmed Amer, Panos Chrysanthis Department of Computer Science University of Pittsburgh A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 1/20
while varying the number of clients Tuple space aging technique that automatically populates tuple space before benchmark execution Detailed empirical study that evaluates tuple space performance, time overhead associated with aging, and impact of aging on space performance A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 2/20
phases Client Cj starts up Tdelay ∈ [Tmin , Tmin + V ] ms after Cj−1 Client pauses for Tdelay ms between the write and take phases Measure response time, R(Si , Cj , O), and throughput, X(Si , O, q) Startup Aging Aging Cleanup Write Take Shutdown Pause A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 4/20
tuple space but take will execute faster than normal Age with either automatically generated or recorded/derived workloads {r, t, w}-frequency defines the fraction of the workload that will be associated with each space operation Define a frequency for each possible Entry type so that A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 5/20
MB main memory GNU/Linux kernel 2.4.18-14smp, Java 1.4.2 compiler, Java 1.4.2 VM in HotSpot client mode, Jini 1.2.1 LinuxThreads 0.10 was configured with a one-to-one mapping between Java threads and kernel processes Clients C1 , . . . , Cq executed on the same machine as JavaSpace Si Other configurations are possible and additional experiments are currently being conducted A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 7/20
q 250 275 300 325 350 375 400 Throughput Operations sec NullIO StringIO ArrayIO FileIO When space is not aged, throughput knees at 8 or 14 clients A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 9/20
of Clients q 320 330 340 350 360 370 380 390 Throughput operations sec 10. 20. 30. 40. 50. 60. 70. 80. Response Time ms Response Time Throughput When throughput knees at 8 clients, average response time continues to increase linearly A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 10/20
of Clients q 245 250 255 260 265 270 Throughput operations sec 20. 40. 60. 80. Response Time ms Response Time Throughput When throughput knees at 14 clients, average response time continues to increase linearly A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 11/20
W 25 50 75 100 125 150 175 200 Benchmark Execution Time sec 1000 3000 6000 12000 55.2 59.7 66.6 78. 97.6 105.3 115.7 137. 14 8 When |W| = 3000 there is a 30% increase in NullIO execution time A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 12/20
W 20 40 60 80 100 Aging’s % of Execution Time 1000 3000 6000 12000 8.65 14.99 22.29 30.87 5.1 9.1 14.2 20.3 14 8 Aging never consumes more than 31% of entire benchmark time A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 13/20
W 20 40 60 80 100 Cleaning’s % of Execution Time 1000 3000 6000 12000 5.6 9.5 14.1 18.3 3.2 5.6 8.6 11.3 14 8 Cleaning incurs less time overhead than aging due to snapshot A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 14/20
12000 Workload Size W 200 250 300 350 Throughput Operations sec NullIO StringIO ArrayIO FileIO Aging reduces tuple space throughput as workload size increases A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 15/20
10000 12000 Workload Size W 20 25 30 35 40 Response Time ms NullIO StringIO ArrayIO FileIO Aging increases tuple space response time as workload size increases A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 16/20
et al. evaluate tuple space performance in the context of bioinformatics Noble and Zlateva measure tuple space performance for astrophysics computations Hancke et al. and Neve et al. measure tuple space performance through the use of statistically guided experiments Smith and Seltzer introduced file system aging A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 17/20
(ii) remote client interactions, (iii) different tuple space implementations, (iv) new versions of Jini and JavaSpaces Workload studies for tuple space-based applications Additional micro, macro, and application-specific benchmarks Definition-use testing for tuple space-based applications: how do you know your application puts the right data into the space? A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 18/20
tuple space aging In current SETTLE configuration, JavaSpaces can support between eight and fourteen concurrent local clients without reducing average response time Tuple space aging can be performed with acceptable time overhead Aging does support the characterization of worst-case performance A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 19/20
Performance. In ACM SIGMETRICS Performance Evaluation Review. December, 2005. http : //cs.allegheny.edu/˜gkapfham/research/settle/ A Tuple Space Benchmarking and Performance Testing Framework, 9th JCM, October 19-20, 2005 – p. 20/20
Gregory Kapfhammer
gkapfham
reverentgeek
rmw
skipperchong
carmenhchung
keathley
productmarketing
clairelew
thoeni
jcasabona
