[Sascha Wolter] The Physical Web: Context is King!

Presentation from GDG DevFest Ukraine 2016.
Learn more at: https://devfest.gdg.org.ua

Transcript

1. The Physical Web™: Context is King! Internet of Uncanny Things Lecture Sascha Wolter, @saschawolter September 2016 Source: Mattel's Barbie Hello Dreamhouse, via https://youtu.be/lgax5y7hVpc
2. Speed kills! In 1830 people though, your brain stops working travelling more than 20 mph. And anyone going at more than 30 mph would have all air sucked out of them and would suffocate to death. Source:Tony Robinson's Weird World of Wonders! British
3. Internet of (Important) Things Image: https://en.wikipedia.org/wiki/File:RegVarneyATM.jpg, https://www.ucc.asn.au/services/drink.ucc 1967 1982
4. Image: Google Mark Weiser 1988: “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” Tim Höttges, Deutsche Telekom, 2014: “Everything that can be connected, will be connected.”
5. Context or exact Location Image: Google Barcode NFC/RFID: 0.2m Bluetooth: 0.3m – 50m GPS: 10m WiFi: 25m - 200m Cell ID: 50m - 3km/35km (city/rural) IP Location: Region Roaming: Country …
6. Barcode versus Beacon QR code • Explicit / Manual • Quick Response Code • User Experience • Camera / App • Environment / Brightness • Who scans what? Beacon • Implicit / Automatic Image: http://english.sina.com/business/p/2015/1212/873118.html / news.on Quick Response?
7. Extend the web to everyday physical objects. Walk up and use anything via URIs Image: Google, Videos: https://www.youtube.com/channel/UCr3EFqJTy1YjlWErFH4fpfw
8. Physical Web™ How-to 1. URL/Deeplink to a Thing • Busstop: http://mobile.bahn.de/bin/mobil/bhftafel. exe/dox?use_realtime_filter=1&si=653360 &bt=dep&p=1111111111&mx=20&start=y es 2. Shortener (e.g. goo.gl) • http://goo.gl/tzoAH2 3. Configure Beacon 4. Install Beacon 5. Use it… •
9. The Physical Web™ Experience iOS
10. Google’s IoT Bouquet (excerpt) • Brillo and Weave • Android to hardware for IoT with an embedded OS. (https://developers.google.com/brillo/) • Communications platform for IoT devices. (https://developers.google.com/weave/) • Nest • Real connected homes for real people. (https://nest.com/) • Revolv • The Revolv app and hub don’t work anymore. (http://revolv.com/) • Thread • Built for the Connected Home on open standards and IPv6/6LoWPAN protocols. (http://threadgroup.org/) • OnHuB • A router (i.e. supports Bluetooth® Smart Ready, 802.15.4 (ZigBee, Thread ready), and Weave (Weave ready). (https://on.google.com/hub/) Source: Google • Google Home • A voice-activated home to everyday tasks. • Fuchsia • Fuchsia (a new Operating System). (https://fuchsia.googlesource.com/) • Google Assistant (Google Now) • Google Now is an intelligent personal assistant. • Trust API (aka Abacus) • Intends to replace password via context (i.e. biometry). • Beacons • A platform for marking up the world to make your apps and devices work smarter by providing timely, contextual information. I.e. Eddystone, Proximity Beacon API, Nearby API, and Place Picker. (https://developers.google.com/beacons/) • … Physical Web
11. Technical Image: Google There is only one packet format for BLE which can be 80 to 376 bits in length including the Protocol Data Unit (PDU) for advertising or data packets.
12. Image: https://en.wikipedia.org/wiki/Eddystone_Lighthouse Landscape • AltBeacon • Apple‘s property: iBeacon™ • Estimote • Gimbal™ • PayPal™ Beacon • yoints • … • Bluetooth® SIG (Special Interest Group) • Bluetooth® Smart Beacon • Eddystone™ • https://github.com/google/eddystone/tree/master/branding
13. Use Cases Image: Estimote
14. Some Bluetooth® Facts • Bluetooth / BLE is a wireless protocol • Bluetooth uses UHF radio waves in the ISM band from 2.4 to 2.485 GHz divided into channels with frequency hopping • Signal strength is an indicator for proximity (RSSI, received signal strength indicator) • BLE has reduced power consumption • Roughly 90% of all mobile phones sold today are Bluetooth enabled, according to Bluetooth SIG. • Bluetooth SIG predicts more than 90% of Bluetooth-enabled smartphones will support the low energy standard by 2018. 90% Proximity
15. Some Bluetooth® Glitches Physics Signal (RSSI) strength will be effected by reflections and attenuation (e.g. fluids). Software “Unfortunately, Bluetooth Share has Stopped” is a common issue. It is solved in Android 4.4.4 (KitKat, API Level 19). http://www.cl.cam.ac.uk/research/dtg/www/publica tions/public/acnt2/ubicomp2005-bluetooth.pdf https://github.com/RadiusNetworks/bluetooth- crash-resolver Image: https://commons.wikimedia.org/wiki/File:%27Tom-Ton_-_Fat_Boy%27_Wellcome_L0040521.jpg
16. Eddystone™, what’s so special? Openness • It is an open Bluetooth 4.0 protocol • While iBeacon™ is officially supported by iOS devices only, Eddystone™ has official support for both iOS and Android Packet types / frames • Eddystone-UID (identifier) • Namespace as UUID • Instance (6 bytes), much like Major and Minor • Eddystone-URL • Eddystone-TLM (telemetry) • battery voltage • temperature • number of packets since last reboot • beacon uptime since last reboot • … Image: Estimote
17. Contextual Content Reader circle bound to location (Geo-fence) shown at http://www.exacteditions.com/
18. Proximity Marketing “shopBeacon” location-specific deals, discounts, recommendations, and rewards already livein at Macy’s Source: http://www.shopkick.com/shopbeacon
19. Coupons and Tickets ballparks with beacon sensors will shoot messages upcoming MLB season Source: http://recode.net/2014/02/14/major-league-baseball-completes-ibeacon-installation-at-first-two-ballparks/
20. Presence Beacon could be placed in shelves, near doors or just at display dummy Source: Starwood | YouTube /
21. Tracking Stairway usage of employees Source: http://www.toprun.e-7.com/
22. Securing Hold on to the important things made by Device Vendor Source: http://www.nokia.com/global/products/
23. Umbrella Never lose your umbrella again! Source: https://www.kickstarter.com/projects/896798832/davek-alert-umbrella-never-lose-your-umbrella-agai
24. Payment Payment and Authorization and Identity started by PayPal Source: http://https://www.paypal.com/us/webapps/mpp/beacon
25. Prototyping Node-RED | Node.js* BLE Hardware Rasperry Pi *BLE processing based on “noble” npm install eddystone-beacon-scanner
26. Vendor specific tools, e.g. Estimote, Inc. and Bluetooth Scanner nRF Master Control Panel (BLE) https://play.google.com/store/apps/details?id=no.nordicsemi.android.mcp&hl=en
27. Permissions
28. Coding a simple pure BLE Scanner public class MainActivity extends Activity { // Declare Bluetooth adapter private BluetoothManager bluetoothManager; private BluetoothAdapter bluetoothAdapter; private BluetoothLeScanner bluetoothLeScanner; // prior Marshmallow you had bluetoothAdapter.startLeScan(new BluetoothAdapter.LeScanCallback() {... // The Eddystone Service UUID, 0xFEAA. private static final ParcelUuid EDDYSTONE_SERVICE_UUID = ParcelUuid.fromString("0000FEAA-0000-1000-8000-00805F9B34FB"); // Eddystone frame types private static final byte TYPE_UID = 0x00; private static final byte TYPE_URL = 0x10; private static final byte TYPE_TLM = 0x20; // ... Declaration
29. Coding a simple pure BLE Scanner @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); // Initialize view // ... // Initialize Bluetooth adapter bluetoothManager = (BluetoothManager) getSystemService(Context.BLUETOOTH_SERVICE); bluetoothAdapter = bluetoothManager.getAdapter(); bluetoothLeScanner = bluetoothAdapter.getBluetoothLeScanner(); if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) { if (checkSelfPermission(Manifest.permission.ACCESS_COARSE_LOCATION) == PackageManager.PERMISSION_GRANTED) { startScan(); } else { requestPermissions(new String[]{Manifest.permission.ACCESS_COARSE_LOCATION}, 1); } } } Initialization
30. BLE in a nutshell: Scan record (UID) Fixed: 02 01 06 03 03 aa fe Length: 15 Fixed: 16 aa fe Frame Type: 00 TX Power: ed Namespace: ed d1 eb ea c0 4e 5d ef a0 17 Instance: c5 61 2a 8c c2 53 ... 04 09 ... 45 53 54 03 03 Bat. Service: 0f 18 0e ... 16 0a 18 Mac: 53 c2 8c 2a 61 c5 ... 00 00 00 00 00 00 00 00 00 00 00 00 00 00
31. Coding a simple pure BLE Scanner private void startScan() { ScanSettings settings = new ScanSettings.Builder() .setScanMode(ScanSettings.SCAN_MODE_LOW_LATENCY) .build(); bluetoothLeScanner.startScan(null, settings, new ScanCallback() { @Override public void onScanResult(int callbackType, ScanResult result) { super.onScanResult(callbackType, result); byte[] data = result.getScanRecord().getServiceData(EDDYSTONE_SERVICE_UUID); if (data == null) return; byte frameType = data[0]; if (frameType != TYPE_UID) return; // ... } }); } Scanning Identify
32. Coding a simple pure BLE Scanner @Override public void onScanResult(int callbackType, ScanResult result) { // ... String namespace = new BigInteger(1, Arrays.copyOfRange(data, 2, 12)).toString(16); String instance = new BigInteger(1, Arrays.copyOfRange(data, 12, 18)).toString(16); if (!(namespace + instance).equals("edd1ebeac04e5defa017" + "c5612a8cc253")) return; // Signal strength int rssi = result.getRssi(); // Signal strength emitted by the beacon in dBm at 0 meters distance. int txPower = data[1]; // pathLoss = (txPower at 0m - rssi); double distance = Math.pow(10, ((txPower - rssi) - 41) / 20.0); // because rssi is unstable, usually only proximity zones are used: // - immediate (very close to the beacon) // - near (about 1-3 m from the beacon) // - far (further away or the signal is fluctuating too much to make a better estimate) // - unknown Log.i("distance", String.format("%.2fm", distance)); // ... }); Proximity
33. Coding a simple BLE Scanner Stopwatch BLE Receiver BLE Sender
34. Coding Sensors and Telemetry List<ScanFilter> filters = new ArrayList<ScanFilter>(); filters.add( new ScanFilter.Builder() .setServiceUuid(EDDYSTONE_SERVICE_UUID) .setDeviceAddress("C5:61:2A:8C:C2:53") /* .setServiceData(EDDYSTONE_SERVICE_UUID, new byte[]{TYPE_TLM}) */ .build()); bluetoothLeScanner.startScan(filters, settings, new ScanCallback() { @Override public void onScanResult(int callbackType, ScanResult result) { super.onScanResult(callbackType, result); byte[] data = result.getScanRecord().getServiceData(EDDYSTONE_SERVICE_UUID); byte frameType = data[0]; if (frameType!= TYPE_TLM) return; // ... } }); Identify/Filter
35. BLE in a nutshell: Scan record (TLM) Fixed: 02 01 06 03 03 aa fe Length: 11 Fixed: 16 aa fe Frame Type: 20 Version: 00 Volt: 0c 06 Temp: 13 00 Adv. Count: 00 0f 70 77 Sec. Count: 00 14 4e 70 ... 04 09 ... 45 53 54 03 03 Bat. Service: 0f 18 0e ... 16 0a 18 Mac: 53 c2 8c 2a 61 c5 ... 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
36. Coding Sensors and Telemetry bluetoothLeScanner.startScan(filters, settings, new ScanCallback() { @Override public void onScanResult(int callbackType, ScanResult result) { super.onScanResult(callbackType, result); // ... // Version int version = data[1]; // Battery voltage, 1 mV/bit int vbatt = new BigInteger(Arrays.copyOfRange(data, 2, 4)).intValue(); // Beacon temperature double temp = new BigInteger(Arrays.copyOfRange(data, 4, 6)).intValue()/256.0; // Advertising PDU count BigInteger advCount = new BigInteger(Arrays.copyOfRange(data, 6, 10)); // Time since power-on or reboot, 0.1 second resolution BigInteger secCount = new BigInteger(Arrays.copyOfRange(data, 10, 14)); Log.i("temperature", String.format("%.2f°C", temp)); } }); Process TLM
37. Source: https://www.backyardbrains.com/ 100 billion connections will be generated and 2 million new sensors will be deployed every hour by 2025 (Huawei)
38. OECD M2M: from 5 billion connected devices to 50 billion in 2020
39. Pacif-i™ – The world’s smartest dummy with temperature sensor, proximity sensor, and buzzer alarm
40. Source: Vivid My friend Cayla Uncanny Valley of Assistance Internet of Uncanny Things
41. Source: The Simpsons, 2001 Anticipation and Empathy
42. Source: http://www.jre-water.com/Suica/index.html Voice and Facial Recognition Gender, Age and Mood
43. Security, Responsibility and Dogfooding Hippocratic Oath and Eating your own Dogfood. Teaching Responsibility and Enlightenment versus Believe in Politics and Regulation Image: Sascha Wolter, Hour of Code 2015, see also https://code.org/learn
44. Businesses are coming to realize that attracting developers is the fastest route to innovation. Moreover, attracting developers means attracting external investment, which finances innovation and expansion. Source: Benno Luthiger, Open-Source-Jahrbuch 2004 INNOVATION AND PROSUMERS Innovation Prosumer: Consumer and Producer Millions and millions of dollars are being spent to attract developers: The millions of dollars in developer marketing efforts serve one purpose: to persuade developers to use a specific platform, network, tool or API set to generate innovations and to reach more consumers. In short: Developers consume an Platform, API, etc. and produce something based on it. Source: Developer Economics 2012, www.developereconomics.com Some Consumers turn into Developers to solve their needs. Video: youtube/Microsoft
45. Internet of Everything Source: http://www.instructables.com/id/Uber-Home-Automation/step15/ Need driven Development
46. see also Project Malmo (http://research.microsoft.com/en-US/groups/mip-ai/default.aspx) Start doing a smattering of REST with SSE
47. Value Innovation Practiceand Exchange to find new opportunities. Source: Siematic, Discovery Channel, BSH, Microsoft, ZDF
48. LEGO-based Design Process similar to Design and Hybrid Thinking Source: http://www.adobe.com/macromedia/events/john_gay/ 1. Choose a problem: Build a LEGO ship. 2. Develop a vision: What sort of ship will it be? How big will it be? What will it carry? 3. Build: Build the framework of the ship. 4. Fill in the details: Design and build the details of the ship, ramps, doors, etc. 5. Test: Drive the cars around the ship and sail the ship while exploring the house. 6. Refine: Take parts of the ship apart and make them better. 7. Learn: Take what you learned from building this ship and use it to build a better one next time.
49. The Physical Web™: Context is King! Internet of Uncanny Things Lecture Sascha Wolter | @saschawolter September 2016 more at http://wolter.biz Image: Sascha Wolter, Prototypic Driving School 2007