Share
Embed
Transcript
25 slides
-
General Idea How well are existing applications supported by PaaS
providers from a (non-)functional viewpoint? The focus is on migrating JEE applications to Google App Engine. Find methods and tools to test PaaS providers for • fast response times • elasticity and scalability. 2 -
Related Work • Strong focus on IaaS providers ◦ CPU
and memory tests ◦ EC2 vs. Google Compute Engine vs. … • Efficient VM provisioning • Comparison of various NoSQL data stores • Load tests for traditional software stacks ◦ LAMP / JEE web applications ◦ SQL databases in the background • Cloud services for scientific computing 3 -
Examined Areas of (Non-)Functional Testing • Portability of Applications ◦
Migrating JPA-based applications to App Engine • Load & Performance Tests ◦ Simulation of high load situations ◦ Spot bottlenecks in production environments ◦ Identify limits of the NoSQL Google Cloud Datastore • Scalability Tests ◦ How do Java applications scale on Google App Engine? 4 -
JPA: From JEE to App Engine • JPA as common
ground for relational data ◦ JPA is a persistence API for Java objects ◦ It spans a wide range of relational database features ◦ App Engine provides a DataNucleus plugin for the NoSQL Datastore service • Various aspects of JPA have been tested ◦ Entity mappings & table definitions ◦ Relationships between entities ◦ Collection mappings ◦ Transaction support & entity group management ◦ Queries 6 -
Datastore vs. Relational DB Entities Key Data 123123 1150 Max
Mustermann M 123,92 null null 1 937489 1220 Jennifer Johnson F 92,10 null T-Shirt 2 ... ... ... ... ... ... ... ... Relational Database Table & Entities NoSQL Datastore Entities 8 -
Datastore Transactions • Datastore’s scope of transaction is defined by
the entity group membership ◦ This membership is determined by an entity’s key ◦ RDBMS allow arbitrary entities to be part of a transaction ◦ “Owned relationships” enforce both sides to be member of the same entity group • Every insert / update / delete of an entity is always part of an entity group transaction • Only 25 entity groups can be part of a transaction 9 -
Using JPA on App Engine is painful and frustrating! The
Objectify framework turned out to be an opinionated alternative. -
Performance Case Study Finding evidence for good scalability and elasticity
of App Engine web applications -
Why a case study? • We can only find evidence
for scalability in real world environments, not in labs / simulations ◦ App Engine is a proprietary technology ◦ Impossible to derive any model from the implementation ◦ We don’t have any information for the global state of the cloud environment • Benchmarks are a valid tool for case studies ◦ App Engine has no advanced monitoring for customers ◦ Only reliable way is to measure response times • Goal: form a strong chain of evidence 12 -
Other empirical research methods • Simulations ◦ Only possible for
open source stacks ◦ Often used for proof of concept implementation, e.g. to show efficiency of load balancing algorithms • Surveys ◦ Not possible if the customer base is not disclosed ◦ Hard to find a representative group • Analysing existing artifacts ◦ Google does only provide a development server ◦ Development server is not representative 13 -
Research Questions Is JMeter an adequate load test tool for
cloud-based applications? How does App Engine react on incoming traffic? Any effects on the Datastore performance? What are the consequences of the 1 write per entity group limit in real world deployments? 14 -
Running a Load Test Step 2: JMeter HTTP-based test PHP
Webserver Step 4: Download and Merge Results 16 Local Machine Step 1: Shell Scripts as Controllers Step 3: Store results as a CSV file -
Key Results The following slides only show a selection of
the gained insights and collected results -
Throughput: Light vs. Heavy Entities • 7,5 minutes vs. 43
minutes test runtime • 12x higher error rate for heavyweight • 50 ms per write vs. 180 ms per write! 18 -
Creating large entity groups Group entities of the same entity
group into a single transaction during creates or updates. 19 -
Parallel Writes to single Entity Group: Datastore Contention Most updates
took only 1 attempt, but some are running extremely long. 20 -
Research Questions – Answers Is JMeter an adequate load test
tool for cloud-based applications? Yes, JMeter … • performed well in a distributed setup • was easy to maintain and execute • generated sufficient load to stress the application and the Datastore service 21 -
Research Questions – Answers How does App Engine react on
incoming traffic? Any effects on the Datastore performance? • Auto-scales the application, no upper limit for instance count found • During instant high load some minor performance issues have been found • Datastore read performance “unlimited”, write performance depends on instances 22 -
Research Questions – Answers What are the consequences of the
1 write per entity group limit in real world deployments? • Not strictly enforced by App Engine • 90 writes per second at the peak • Some writes stucked in multiple attempts • Conclusion: Datastore tries to minimize the effects of contention for most of the users 23 -
SUMMARY • JPA-based migrations are hard in practice ◦ Objectify
framework is a specialized alternative • Performance Case Study showed no bottlenecks ◦ Read performance is limited by the budget ◦ Write performance is not great, but avoidable by design • App Engine is a very specialized product ◦ Hard to migrate back to a on-premise stack ◦ Google’s Cloud Datastore is different to other NoSQL stores 24