Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Locomotor: transparent migration of client-side database code

C1e4202cb329b1e3343a571dff94c68c?s=47 Michael Mior
September 01, 2017

Locomotor: transparent migration of client-side database code

Server-side execution is a well-known method for improving the performance of database applications. Running code on the database server eliminates round trips to the client application resulting in significantly reduced latency. However, the common approach of explicitly writing server-side code in stored procedures has significant drawbacks. Application developers must develop and maintain code in two separate languages and manually partition code between the client and server. Code shipping is a viable alternative but still requires an explicit specication of which code can be run on the server. We propose a hybrid shipping approach based on static analysis which automatically partitions client code and only ships code to the server which is likely to improve performance.
We demonstrate the viability of this approach in a prototype system which we call Locomotor. Locomotor operates on Python applications using the Redis key-value store. Through static analysis, it identifies fragments of code which can benet from being executed on the server and automatically performs translation to execute the fragments on the server. Unlike many previous systems, Locomotor is not pattern-based and is able to ship a wide variety of code. By shipping code to the server, Locomotor is able to achieve significant performance gains over client-side execution with no modifications to the application code.

C1e4202cb329b1e3343a571dff94c68c?s=128

Michael Mior

September 01, 2017
Tweet

Transcript

  1. Locomotor Transparent Migration of Client-Side Database Code Michael Mior –

    University of Waterloo
  2. Stored Procedures ▸ Complex database operations often require queries and

    updates to be mixed with application code ▸ Using stored procedures can reduce latency by cutting back on round trips
  3. Stored Procedures ▸ Server-side scripting languages differ from the language

    used to develop the application ▸ Independent maintenance is required ▸ Required data needs to be carefully transferred in both directions
  4. Locomotor ▸ Automatically convert annotated Python functions into server-side Lua

    scripts in Redis ▸ Patch the Python code at runtime to call the dynamically-generated script ▸ Updated code is automatically re-translated and deployed
  5. Shipping Example Redis Data Model Key-value store where values can

    be complex types (e.g. maps, lists) item:1 → { name: 'Foo', category: 'Bar' } category:Bar → ['item:1', …]
  6. Shipping Example Application Code hmset('item:' + str(key), {'name': 'Foo'}) lpush('category:Books',

    'item:1') ids = lrange('category:' + category, 0, -1) items = [] for id in ids: items.append(hget(id, 'name')) return items
  7. Shipping Example Application Code hmset('item:' + str(key), {'name': 'Foo'}) lpush('category:Books',

    'item:1') ids = lrange('category:' + category, 0, -1) items = [] for id in ids: items.append(hget(id, 'name')) return items Each of these calls requires a round trip to the server! And once per iteration hmset lpush lrange for id in ids: hget
  8. Shipping Example Server Script local category = 'category:' .. ARGV[1]

    local item_keys = redis.call('lrange', 'category:' .. category, 0, -1) local items = {} for _, key in ipairs(item_keys) do table.insert(items, redis.call('hget', key, 'name')) end return items
  9. Shipping Example Server Script local category = 'category:' .. ARGV[1]

    local item_keys = redis.call('lrange', 'category:' .. category, 0, -1) local items = {} for _, key in ipairs(item_keys) do table.insert(items, redis.call('hget', key, 'name')) end return items Only one round trip needed
  10. Auto Translate 1. Data type conversions 2. Differing language semantics

    3. Built-in functions 4. Loop constructs …
  11. Deploy Each time an annotated function is run, it is

    translated/shipped on the fly @redis_server def get_category(redis, cat): ...
  12. Evaluation Client

  13. Evaluation • Shipping code reduced round trips from 24 to

    8 • With an inefficient server-side implementation, we still achieve a 4× reduction in runtime
  14. Summary ▸ Translation of client code to server-side scripting languages

    is a viable approach to optimization ▸ This optimization can be automated with careful observation of language semantics
  15. Future Work ▸ Explore other DB/language combos such as MongoDB

    and JavaScript ▸ Automated selection of code fragments to translate and move ▸ Use of low-level interfaces (e.g. Redis modules)
  16. Questions?