Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Final Master Thesis Presentation

Final Master Thesis Presentation

Philipp Naderer

June 01, 2016

Transcript

25 slides
  1. General Idea How well are existing applications supported by PaaS

    providers from a (non-)functional viewpoint? The focus is on migrating JEE applications to Google App Engine. Find methods and tools to test PaaS providers for • fast response times • elasticity and scalability. 2
  2. Related Work • Strong focus on IaaS providers ◦ CPU

    and memory tests ◦ EC2 vs. Google Compute Engine vs. … • Efficient VM provisioning • Comparison of various NoSQL data stores • Load tests for traditional software stacks ◦ LAMP / JEE web applications ◦ SQL databases in the background • Cloud services for scientific computing 3
  3. Examined Areas of (Non-)Functional Testing • Portability of Applications ◦

    Migrating JPA-based applications to App Engine • Load & Performance Tests ◦ Simulation of high load situations ◦ Spot bottlenecks in production environments ◦ Identify limits of the NoSQL Google Cloud Datastore • Scalability Tests ◦ How do Java applications scale on Google App Engine? 4
  4. JPA: From JEE to App Engine • JPA as common

    ground for relational data ◦ JPA is a persistence API for Java objects ◦ It spans a wide range of relational database features ◦ App Engine provides a DataNucleus plugin for the NoSQL Datastore service • Various aspects of JPA have been tested ◦ Entity mappings & table definitions ◦ Relationships between entities ◦ Collection mappings ◦ Transaction support & entity group management ◦ Queries 6
  5. Datastore vs. Relational DB Entities Key Data 123123 1150 Max

    Mustermann M 123,92 null null 1 937489 1220 Jennifer Johnson F 92,10 null T-Shirt 2 ... ... ... ... ... ... ... ... Relational Database Table & Entities NoSQL Datastore Entities 8
  6. Datastore Transactions • Datastore’s scope of transaction is defined by

    the entity group membership ◦ This membership is determined by an entity’s key ◦ RDBMS allow arbitrary entities to be part of a transaction ◦ “Owned relationships” enforce both sides to be member of the same entity group • Every insert / update / delete of an entity is always part of an entity group transaction • Only 25 entity groups can be part of a transaction 9
  7. Using JPA on App Engine is painful and frustrating! The

    Objectify framework turned out to be an opinionated alternative.
  8. Why a case study? • We can only find evidence

    for scalability in real world environments, not in labs / simulations ◦ App Engine is a proprietary technology ◦ Impossible to derive any model from the implementation ◦ We don’t have any information for the global state of the cloud environment • Benchmarks are a valid tool for case studies ◦ App Engine has no advanced monitoring for customers ◦ Only reliable way is to measure response times • Goal: form a strong chain of evidence 12
  9. Other empirical research methods • Simulations ◦ Only possible for

    open source stacks ◦ Often used for proof of concept implementation, e.g. to show efficiency of load balancing algorithms • Surveys ◦ Not possible if the customer base is not disclosed ◦ Hard to find a representative group • Analysing existing artifacts ◦ Google does only provide a development server ◦ Development server is not representative 13
  10. Research Questions Is JMeter an adequate load test tool for

    cloud-based applications? How does App Engine react on incoming traffic? Any effects on the Datastore performance? What are the consequences of the 1 write per entity group limit in real world deployments? 14
  11. Running a Load Test Step 2: JMeter HTTP-based test PHP

    Webserver Step 4: Download and Merge Results 16 Local Machine Step 1: Shell Scripts as Controllers Step 3: Store results as a CSV file
  12. Key Results The following slides only show a selection of

    the gained insights and collected results
  13. Throughput: Light vs. Heavy Entities • 7,5 minutes vs. 43

    minutes test runtime • 12x higher error rate for heavyweight • 50 ms per write vs. 180 ms per write! 18
  14. Creating large entity groups Group entities of the same entity

    group into a single transaction during creates or updates. 19
  15. Parallel Writes to single Entity Group: Datastore Contention Most updates

    took only 1 attempt, but some are running extremely long. 20
  16. Research Questions – Answers Is JMeter an adequate load test

    tool for cloud-based applications? Yes, JMeter … • performed well in a distributed setup • was easy to maintain and execute • generated sufficient load to stress the application and the Datastore service 21
  17. Research Questions – Answers How does App Engine react on

    incoming traffic? Any effects on the Datastore performance? • Auto-scales the application, no upper limit for instance count found • During instant high load some minor performance issues have been found • Datastore read performance “unlimited”, write performance depends on instances 22
  18. Research Questions – Answers What are the consequences of the

    1 write per entity group limit in real world deployments? • Not strictly enforced by App Engine • 90 writes per second at the peak • Some writes stucked in multiple attempts • Conclusion: Datastore tries to minimize the effects of contention for most of the users 23
  19. SUMMARY • JPA-based migrations are hard in practice ◦ Objectify

    framework is a specialized alternative • Performance Case Study showed no bottlenecks ◦ Read performance is limited by the budget ◦ Write performance is not great, but avoidable by design • App Engine is a very specialized product ◦ Hard to migrate back to a on-premise stack ◦ Google’s Cloud Datastore is different to other NoSQL stores 24

More Decks by Philipp Naderer

Other Decks in Research