Share
Embed
We All Make Distributed Systems - wroc_love.rb 2017
Presentation for my talk at wroc_love.rb 2017 conference.
mrzasa
March 19, 2017
Transcript
35 slides
-
WE ALL MAKE WE ALL MAKE WE ALL MAKE WE
ALL MAKE WE ALL MAKE WE ALL MAKE WE ALL MAKE WE ALL MAKE WE ALL MAKE WE ALL MAKE WE ALL MAKE WE ALL MAKE WE ALL MAKE WE ALL MAKE WE ALL MAKE WE ALL MAKE DISTRIBUTED SYSTEMS DISTRIBUTED SYSTEMS DISTRIBUTED SYSTEMS DISTRIBUTED SYSTEMS DISTRIBUTED SYSTEMS DISTRIBUTED SYSTEMS MACIEJ RZĄSA @mjrzasa -
ME Software Engineer @ TextMaster main interests writing software that
matters self-organising teams distributed systems (occasionally) knowledge sharing Rzeszów Ruby User Group ( ) Rzeszów University of Technology rrug.pl -
AGENDA Why? - reasons to care about this talk What?
- limitations of distributed system How? - case study -
A collection of independent computers that appears to its users
as a single coherent system Andrew Tannenbaum A distributed system is one where a machine I’ve never heard of can cause my program to fail. Leslie Lamport DISTRIBUTED SYSTEM -
A collection of independent computers that appears to its users
as a single coherent system Andrew Tannenbaum DISTRIBUTED SYSTEM -
single-copy consistency not the same as ACID- consistency example: write
on android, read on web CONSISTENCY -
every non-failing node responds meaningfully example: I can work on
any client (web/android/iOS/WP) AVAILABILITY -
system works even with some messages missing network failure: loss
of packets offline-mode PARTITION TOLERACE If all you have is a timeout, everything looks like a partition - @nkeywal -
you cannot have all three evidence: try to save offline
and read on a different client THREE GUARANTEES: CAP THEOREM -
consistency and availability no partition handling single host network is
unreliable CA: I FEEL LUCKY -
consistency and partition tolerance consistency guaranteed no offline-mode, app works
only with network connection limited features possible (read-only) after reconnection: fetching data (one-way sync) convenient for developers CP: ORDNUNG MUSS SEIN -
availability and partition tolerance app usable all the time offline
mode two-way sync required profitable for the client AP: WORK AROUND THE CLOCK -
CAP CA - I feel lucky CP - Ordnung muss
sein AP - Work around the clock The choice of availability over consistency is a business choice, not a technical one. - @coda -
CASE STUDY CASE STUDY CASE STUDY CASE STUDY CASE STUDY
CASE STUDY CASE STUDY CASE STUDY CASE STUDY CASE STUDY source: wikipedia.org -
APPLICATION OVERVIEW domain: recycling mobile app for field workers +
web panel for admins offline: payments, product catalog trade-offs: validation, consistency challenges: synchronization, conflict resolution -
SYNC SYNC SYNC SYNC SYNC SYNC SYNC SYNC SYNC SYNC
ALL THE DATA! ALL THE DATA! ALL THE DATA! ALL THE DATA! ALL THE DATA! ALL THE DATA! source: memegenerator.com -
SYNC — ONE WAY (CP) pricing, published ~1/week, bulk of
them at the same time (Monday) version 1.0: fetch all changes at once growing data size: timeouts, memory limitations on android obvious solution: pagination -
PAGINATION GET /items? page=1 GET /items? since=1234 to=3456 GET /items?
since=1234 page_size=3 -
SYNC — TWO WAY (AP) tickets (a kind of shopping
cart) payments offline (!) every client (android) creates its own tickets and adds ticket items tickets available on the server and set to other mobile clients -
CLIENT SERVER SYNC — TWO WAY (AP) def sync() timestamp
= get_last_sync_timestamp client_changes = choose_updated(timestamp) server_changes = send(client_changes, timestamp) # wait... store(server_changes) set_sync_timestamp(Time.now) end def sync(client_changes, timestamp) server_changes = choose_updated(timestamp) store(client_changes) send(server_changes) end -
TICKET EDIT cancelling tickets (purchase) on the server changed tickets
set to the client FAIL! both sides edits a ticket, changes lost solution: sync of status changes, not whole tickets -
ONE DOES NOT SIMPLY ONE DOES NOT SIMPLY ONE DOES
NOT SIMPLY ONE DOES NOT SIMPLY ONE DOES NOT SIMPLY ONE DOES NOT SIMPLY ONE DOES NOT SIMPLY ONE DOES NOT SIMPLY ONE DOES NOT SIMPLY ONE DOES NOT SIMPLY SYNC MUTABLE STATE SYNC MUTABLE STATE SYNC MUTABLE STATE SYNC MUTABLE STATE SYNC MUTABLE STATE SYNC MUTABLE STATE source: youtube.com -
RETRANSMISSION Client sends payment records Server saves records and updates
account data What if the client loses connection? -
RETRANSMISSION def sync(data) create_transaction(data) account.redeem(data.amount) end PUT /transactions { amount:
1000 } def sync(data) transaction = find_transaction(data.uuid) return if transaction create_transaction(data) account.redeem(data.amount) end PUT /transactions { uuid: "2db1ec4c-...", amount: 1000 } -
CLIENT-SIDE IDENTIFIERS AUTOINCREMENT is not very useful in distributed environment
;-) UUID (v4) really low risk of collision, may be generated on the client-side sync can be repeated multiple times helpful on failures (of the network, server, client) effect: idempotence e4043456-b29e-4d80-afaf-4d65246f1d36 -
SYNCHRONIZATION PATTERNS let client decide on data scope and size
exclude received changes in two-way sync sync immutable rather than mutable identify data on client to assure idempotence GET /items? since=1234 page_size=3 -
WEB AS DISTRIBUTED SYSTEM limitations of distributed systems are applicable
to web/mobile apps as well the network is unreliable CAP: consistency - availability - partition tolerance: you cannot have all three CP and AP approaches may be mixed in one app synchronize immutable values and apply them to mutable objects (events vs entities) idempotency matters source of the icons on my diarams: http:/ /www.flaticon.com -
REFERENCES : data safety test in various distributed systems :
really? ;-) source of the icons on my diagrams: CAP 12 years later Jepsen Netwok is reliable Starbucks Does Not Use Two-Phase Commit Latency: The New Web Performance Bottleneck http:/ /www.flaticon.com