Peer-to-Peer Replication on Android

Building synchronized peer-to-peer systems on Android with Network Service Discovery (NSD) and CouchbaseLite (1.X)

- Problem statement: building a POS on Android
- Architectural constraints: Android OS, multiple devices, no single point of failure, must work offline
- Peer-to-Peer technologies available in Android
- Overview of the technologies we are using: Network Service Discovery and CouchbaseLite

Transcript

1. Peer-to-Peer Replication on Android Jeremiah Sheehy | Yum.dev | February 2020
2. Android Point of Sale 2
3. Android Point of Sale Consumer grade hardware 3
4. Android Point of Sale Consumer grade hardware Easy to set up 4
5. Android Point of Sale Consumer grade hardware Easy to set up Support between 1 and 20 devices in a store 5
6. Android Point of Sale Consumer grade hardware Easy to set up Support between 1 and 20 devices in a store No single point of failure 6
7. Android Point of Sale Consumer grade hardware Easy to set up Support between 1 and 20 devices in a store No single point of failure Must work offline (no outbound internet) 7
8. Peer-to-Peer 8
9. Peer-to-Peer How do you discover other nodes? 9
10. Peer-to-Peer How do you discover other nodes? How do you transmit data to other nodes? 10
11. Peer-to-Peer How do you discover other nodes? How do you transmit data to other nodes? How do you structure your network? 11
12. Supported on Android WiFi-Direct Bluetooth Nearby Connections API Network Service Discovery 12
13. Supported on Android WiFi-Direct Bluetooth Nearby Connections API Network Service Discovery 13
14. Supported on Android WiFi-Direct Bluetooth Nearby Connections API Network Service Discovery 14
15. Supported on Android WiFi-Direct Bluetooth Nearby Connections API Network Service Discovery 15
16. Supported on Android WiFi-Direct Bluetooth Nearby Connections API Network Service Discovery 16
17. Network Service Discovery (NSD) mDNSResponder created by Apple (Bonjour) 17
18. Network Service Discovery (NSD) mDNSResponder created by Apple (Bonjour) 18 https://github.com/andriydruk/Bonjou
19. Network Service Discovery (NSD) mDNSResponder created by Apple (Bonjour) 19 https://github.com/andriydruk/Bonjou
20. Network Service Discovery (NSD) mDNSResponder created by Apple (Bonjour) 20 https://github.com/andriydruk/Bonjou
21. Network Service Discovery (NSD) mDNSResponder created by Apple (Bonjour) 21 https://github.com/andriydruk/Bonjou Added to Android in API 16 (4.1)
22. Network Service Discovery (NSD) mDNSResponder created by Apple (Bonjour) 22 https://github.com/andriydruk/Bonjou Added to Android in API 16 (4.1) Supported by any device with a network card
23. Network Service Discovery (NSD) mDNSResponder created by Apple (Bonjour) 23 https://github.com/andriydruk/Bonjou Added to Android in API 16 (4.1) Supported by any device with a network card Multicast DNS and DNS-SD
24. Network Service Discovery (NSD) 24 https://github.com/andriydruk/Bonjou
25. Network Service Discovery (NSD) 25 https://github.com/andriydruk/Bonjou
26. NSD for Peer-to-Peer 26
27. Define our Network Service serviceInfo = new NsdServiceInfo(); // _<protocol>._<transportlayer> serviceInfo.setServiceType("_p2pnsd._tcp."); // The name is subject to change based on conflicts // with other services advertised on the same network. serviceInfo.setServiceName("P2P_NSD_"+uuid); // Preferred** service port serviceInfo.setPort(3334); // Requires API Level 21+ serviceInfo.setAttribute("key1", "Some text value"); serviceInfo.setAttribute("key2", "Some other text value"); 27
28. Define our Network Service serviceInfo = new NsdServiceInfo(); // _<protocol>._<transportlayer> serviceInfo.setServiceType("_p2pnsd._tcp."); // The name is subject to change based on conflicts // with other services advertised on the same network. serviceInfo.setServiceName("P2P_NSD_"+uuid); // Preferred** service port serviceInfo.setPort(3334); // Requires API Level 21+ serviceInfo.setAttribute("key1", "Some text value"); serviceInfo.setAttribute("key2", "Some other text value"); 28
29. Define our Network Service serviceInfo = new NsdServiceInfo(); // _<protocol>._<transportlayer> serviceInfo.setServiceType("_p2pnsd._tcp."); // The name is subject to change based on conflicts // with other services advertised on the same network. serviceInfo.setServiceName("P2P_NSD_"+uuid); // Preferred** service port serviceInfo.setPort(3334); // Requires API Level 21+ serviceInfo.setAttribute("key1", "Some text value"); serviceInfo.setAttribute("key2", "Some other text value"); 29
30. Define our Network Service serviceInfo = new NsdServiceInfo(); // _<protocol>._<transportlayer> serviceInfo.setServiceType("_p2pnsd._tcp."); // The name is subject to change based on conflicts // with other services advertised on the same network. serviceInfo.setServiceName("P2P_NSD_"+uuid); // Preferred** service port serviceInfo.setPort(3334); // Requires API Level 21+ serviceInfo.setAttribute("key1", "Some text value"); serviceInfo.setAttribute("key2", "Some other text value"); 30
31. Register our Service registrationListener = new NsdManager.RegistrationListener() { @Override public void onServiceRegistered(NsdServiceInfo NsdServiceInfo) { // Save the service name. Android may have changed it in order to // resolve a conflict, so update the name you initially requested // with the name Android actually used. serviceName = NsdServiceInfo.getServiceName(); } @Override public void onRegistrationFailed(NsdServiceInfo serviceInfo, int errorCode) {} @Override public void onServiceUnregistered(NsdServiceInfo arg0) {} @Override public void onUnregistrationFailed(NsdServiceInfo serviceInfo, int errorCode) {} }; 31
32. Register our Service registrationListener = new NsdManager.RegistrationListener() { @Override public void onServiceRegistered(NsdServiceInfo NsdServiceInfo) { // Save the service name. Android may have changed it in order to // resolve a conflict, so update the name you initially requested // with the name Android actually used. serviceName = NsdServiceInfo.getServiceName(); } @Override public void onRegistrationFailed(NsdServiceInfo serviceInfo, int errorCode) {} @Override public void onServiceUnregistered(NsdServiceInfo arg0) {} @Override public void onUnregistrationFailed(NsdServiceInfo serviceInfo, int errorCode) {} }; 32 registrationListener = new NsdManager.RegistrationListener() { @Override public void onServiceRegistered(NsdServiceInfo NsdServiceInfo) { // Save the service name. Android may have changed it in order to // resolve a conflict, so update the name you initially requested // with the name Android actually used. serviceName = NsdServiceInfo.getServiceName(); } //... }; nsdManager = Context.getSystemService(Context.NSD_SERVICE); nsdManager.registerService(serviceInfo, NsdManager.PROTOCOL_DNS_SD, registrationListener);
33. Service Discovery 33
34. Discover Services discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 34
35. Discover Services discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 35 discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) { // A service was found! Do something with it. if (!service.getServiceType().equals("_p2pnsd._tcp.")) { // This is not our service type, ignore it. } else if (service.getServiceName().equals(serviceName)) { // This is the service we are broadcasting, ignore it } else if (service.getServiceName().contains("P2P_NSD")){ nsdManager.resolveService(service, resolveListener); } } //.. };
36. Why Resolve? 36
37. Why Resolve? 37 Service Registered
38. Why Resolve? 38 Service Registered Service Unregistered (correctly)
39. Discover Services discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 39 discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) { // A service was found! Do something with it. if (!service.getServiceType().equals("_p2pnsd._tcp.")) { // This is not our service type, ignore it. } else if (service.getServiceName().equals(serviceName)) { // This is the service we are broadcasting, ignore it } else if (service.getServiceName().contains("P2P_NSD")){ nsdManager.resolveService(service, resolveListener); } } //.. };
40. Resolve Service discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 40 resolveListener = new NsdManager.ResolveListener() { @Override public void onResolveFailed(NsdServiceInfo serviceInfo, int errorCode) { // handle error } @Override public void onServiceResolved(NsdServiceInfo serviceInfo) { // start replicating data to our peer String url = serviceInfo.getHost().getHostAddress()+":"+serviceInfo.getPort(); startReplications(url); } };
41. Couchbase 41
42. Setting it up discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 42 couchManager = new Manager(new AndroidContext(context), Manager.DEFAULT_OPTIONS); couchDb = couchManager.getDatabase(DB_NAME); // init peer to peer replication listener (This is the service we advertised earlier) LiteListener listener = new LiteListener(couchManager, 3334); Thread thread = new Thread(listener); thread.start();
43. Setting it up discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 43 couchManager = new Manager(new AndroidContext(context), Manager.DEFAULT_OPTIONS); couchDb = couchManager.getDatabase(DB_NAME); // init peer to peer replication listener (This is the service we advertised earlier) LiteListener listener = new LiteListener(couchManager, 3334); Thread thread = new Thread(listener); thread.start();
44. Setting it up discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 44 couchManager = new Manager(new AndroidContext(context), Manager.DEFAULT_OPTIONS); couchDb = couchManager.getDatabase(DB_NAME); // init peer to peer replication listener (This is the service we advertised earlier) LiteListener listener = new LiteListener(couchManager, 3334); Thread thread = new Thread(listener); thread.start();
45. Starting Replications 45
46. Starting Replications 46 resolveListener = new NsdManager.ResolveListener() { @Override public void onResolveFailed(NsdServiceInfo serviceInfo, int errorCode) { // handle error } @Override public void onServiceResolved(NsdServiceInfo serviceInfo) { // start replicating data to our peer String url = serviceInfo.getHost().getHostAddress()+":"+serviceInfo.getPort(); startReplications(url); } };
47. Starting Replications 47 private void startReplications(String syncUrl){ Replication push = couchDB.createPushReplication(syncUrl); push.setContinuous(true); push.start(); Replication pull = couchDB.createPullReplication(syncUrl); pull.setContinuous(true); pull.start(); }
48. Starting Replications 48 private void startReplications(String syncUrl){ Replication push = couchDB.createPushReplication(syncUrl); push.setContinuous(true); push.start(); Replication pull = couchDB.createPullReplication(syncUrl); pull.setContinuous(true); pull.start(); }
49. Writing Data discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 49 // In NoSQL everything is basically a key : value pair Map<String, Object> properties = new HashMap<>(); properties.put("TYPE", "Employee"); properties.put("name", "Jeremiah"); // Create a new document Document document = couchDb.createDocument(); // Save the document to the database document.putProperties(properties); { "TYPE": "Employee", "name": "Jeremiah" }
50. Writing Data discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 50 // In NoSQL everything is basically a key : value pair Map<String, Object> properties = new HashMap<>(); properties.put("TYPE", "Employee"); properties.put("name", "Jeremiah"); // Create a new document Document document = couchDb.createDocument(); // Save the document to the database document.putProperties(properties); { "TYPE": "Employee", "name": "Jeremiah" }
51. Writing Data discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 51 // In NoSQL everything is basically a key : value pair Map<String, Object> properties = new HashMap<>(); properties.put("TYPE", "Employee"); properties.put("name", "Jeremiah"); // Create a new document Document document = couchDb.createDocument(); // Save the document to the database document.putProperties(properties); { "TYPE": "Employee", "name": "Jeremiah" }
52. Writing Data discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 52 // In NoSQL everything is basically a key : value pair Map<String, Object> properties = new HashMap<>(); properties.put("TYPE", "Employee"); properties.put("name", "Jeremiah"); // Create a new document Document document = couchDb.createDocument(); // Save the document to the database document.putProperties(properties); { "TYPE": "Employee", "name": "Jeremiah" }
53. Writing Data discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 53 // In NoSQL everything is basically a key : value pair Map<String, Object> properties = new HashMap<>(); properties.put("TYPE", "Employee"); properties.put("name", "Jeremiah"); // Create a new document Document document = couchDb.createDocument(); // Save the document to the database document.putProperties(properties); { "TYPE": "Employee", "name": "Jeremiah" }
54. Listening for Changes discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 54 // Create a view emloyeeView = couchDb.getView("EMPLOYEE_VIEW"); emloyeeView.setMap((doc, emitter) -> { if (doc.get("TYPE").equals("Employee")) { emitter.emit(doc); } }); // Create live query Query query = emloyeeView.createQuery(); employeeLiveQuery = query.toLiveQuery(); employeeLiveQuery.addChangeListener(event -> { // TODO: Handle Changes }); employeeLiveQuery.start()
55. Listening for Changes discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 55 // Create a view emloyeeView = couchDb.getView("EMPLOYEE_VIEW"); emloyeeView.setMap((doc, emitter) -> { if (doc.get("TYPE").equals("Employee")) { emitter.emit(doc); } }); // Create live query Query query = emloyeeView.createQuery(); employeeLiveQuery = query.toLiveQuery(); employeeLiveQuery.addChangeListener(event -> { // TODO: Handle Changes }); employeeLiveQuery.start()
56. Listening for Changes discoveryListener = new NsdManager.DiscoveryListener() { @Override public void onDiscoveryStarted(String regType) {} @Override public void onServiceFound(NsdServiceInfo service) {} // @Override public void onServiceLost(NsdServiceInfo service) {} @Override public void onDiscoveryStopped(String serviceType) {} @Override public void onStartDiscoveryFailed(String serviceType, int errorCode) { } @Override public void onStopDiscoveryFailed(String serviceType, int errorCode) { } }; 56 // Create a view emloyeeView = couchDb.getView("EMPLOYEE_VIEW"); emloyeeView.setMap((doc, emitter) -> { if (doc.get("TYPE").equals("Employee")) { emitter.emit(doc); } }); // Create live query Query query = emloyeeView.createQuery(); employeeLiveQuery = query.toLiveQuery(); employeeLiveQuery.addChangeListener(event -> { // TODO: Handle Changes }); employeeLiveQuery.start()
57. Questions? 57
58. We are hiring! 58 email: jeremiah.sheehy@yum.com web: https://Yum.dev github: @jsh33hy