Building interactive wall décor from junk phones

Why throw away your old phones when you can create a digital work of art? With just some basic arts and crafts skills, you can build a collage of phones with wires intertwining and running across your wall like a circuit board. It can display photo galleries, shopping lists, music visualizations, and make a wicked Zoom background.

In this talk, you'll learn how Node.js and a Raspberry Pi can manage multiple Android devices over USB. We'll explore how to build visualizations using browser APIs, and how to integrate with the wider Internet of Things. Let's give your old mobile phones a new life.

Transcript

1. Building interactive wall décor from junk phones Tiger Oakes - @Not_Woods 1
2. Hi! I’m Tiger. Software Engineer working on Microsoft Loop. • Worked on Edge, Chrome, and Firefox. • From Hawai’i, now based in Seattle. • Likes computers 2
3. Broken cell phones 👵 🧼 🌲 4
5. Hoarding old tech 6 Upcycling old tech
7. CellWall
10. 12 Cells ADB WebSockets Remote HTTP POST Server On / Off Smart home Home Assistant Architecture
11. 01 Build with physical parts. Shopping at the hardware store 13
12. Shopping list Base wood panel 1. Plywood panel 2. Sandpaper 3. White acrylic paint 4. Paint brush 5. Wood screws or command strips for mounting to wall
13. Shopping list Base wood panel 1. Plywood panel 2. Sandpaper 3. White acrylic paint 4. Paint brush 5. Wood screws or command strips for mounting to wall
14. Shopping list Mounts, connectors 6. Old cell phones and tablets running Android 7. Raspberry Pi or another computer for the server 8. USB cables 9. USB hub 10.Velcro strips 11.Wire clips
15. Shopping list Mounts, connectors 6. Old cell phones and tablets running Android 7. Raspberry Pi or another computer for the server 8. USB cables 9. USB hub 10.Velcro strips 11.Wire clips
16. Shopping list Mounts, connectors 6. Old cell phones and tablets running Android 7. Raspberry Pi or another computer for the server 8. USB cables 9. USB hub 10.Velcro strips 11.Wire clips
17. Shopping list = €74.70 Base wood panel 1. Plywood panel 2. Sandpaper 3. White acrylic paint 4. Paint brush 5. Wood screws or command strips for mounting to wall Mounts, connectors €26.60 6. Old cell phones and tablets running Android 7. Raspberry Pi or another computer for the server 8. Velcro strips 9. USB cables 10.Wire clips 11.USB hub €48.10
18. 20 Cells ADB WebSockets Remote HTTP POST On / Off Smart home Home Assistant Architecture Server
19. 02 Talk to your phone over USB. Using Android command line tools to control your USB cables 21
20. USB hub connecting to server 22
21. Android Debug Bridge Command-line tool to help with developing Android apps. 23
22. # List connected devices > adb devices List of devices attached 1e778e25 device emulator-5554 device 24
23. # Open a Unix shell on the Android device > adb –s <serial> shell 25
24. # Command to install an app > adb –s ABC12345 install ./app_name.apk # Command to simulate pressing power button > adb –s ABC12345 shell input keyevent 26 # Command to open website on phone > adb –s ABC12345 shell am start –a android.intent.action.VIEW –d https://example.com 26
25. Enable USB debugging Set up developer mode on each device https://aka.ms/debug-android 27
26. > adb devices * daemon not running; starting now at tcp:5037 * daemon started successfully List of devices attached ID1234 device device IDabcd device This phone has no ID! Making up a serial code and updating the phone 28
27. # List connected devices with long output > adb devices -l List of devices attached ID1234 device usb:11 transport_id:1 device usb:22 transport_id:2 IDabcd device usb:33 transport_id:3 # Use USB port ID > adb –s usb:11 shell input keyevent 26 # Use transport ID > adb –t 3 shell input keyevent 26 29
28. Rooting your Android phone Gain superuser access so you can change a string 30
29. This phone has no Wi-Fi! Using the USB cable like an Ethernet cable with reverse port forwarding 31
30. # Send TCP requests over a port to the server via USB, not WiFi > adb –s ABC12345 reverse tcp:<device-port> tcp:<system-port> # Reverse forward TCP request on port 3000 to server port 3000 > adb –s ABC12345 reverse tcp:3000 tcp:3000 32
31. This phone has no screen! Replacing the screen? 33
32. github.com/ Genymobile/ scrcpy pronounced "screen copy" Use your computer to see and control your phone’s screen 34
33. 35 Cells ADB WebSockets Remote HTTP POST On / Off Smart home Home Assistant Architecture Server
34. 03 Control phones from a server. Automating ADB commands 36
36. Finally, he’s talking about JavaScript! 38
37. https://npm.im/appium-adb import { ADB } from 'appium-adb'; const adb = await ADB.createADB(); const devices = await adb.getConnectedDevices(); 39
38. https://npm.im/appium-adb Command line # List connected devices > adb devices List of devices attached 2315332 device 1ee21d device E2dbeeDR device JavaScript import { ADB } from 'appium-adb'; const adb = await ADB.createADB(); const devices = await adb.getConnectedDevices(); console.log(devices); // [{ udid: '2315332' }, // { udid: '1ee21d' }, // { udid: 'E2dbeeDR' }] 40
39. Control devices with Node async function helloWorld() { const devices = await getConnectedDevices(); await Promise.all(devices.map(async (device) => { const KEYCODE_POWER = 26; await device.keyevent(KEYCODE_POWER); await device.startUri('https://example.com'); }); }); 41
40. Server Communication protocol speed 42 Client ADB
41. Server Communication protocol speed 43 Client ADB WebSockets TCP
42. Server Communication protocol speed 44 Client ADB Establish connection and turn on phones WebSockets Send display data quickly
43. Represent state with websites If you are presenting a website, an internet product or an app, you can place a screenshot of it here. 45 https://example.com
44. Latest Software for 1st Gen Kindle Fire Released 2013. Android 4.4 Firefox Nightly 102 Released yesterday. 46
45. 04 Coordinating across clients. Splitting up data on the server for the cell phones 47
46. const size = { width: window.innerWidth, height: window.innerHeight, }; 48
47. 49 0,0 1250,10 Loading…
51. Distribute lines of text async function showText(text) { const lines = text.split('\n'); // Roughly sort based on ascending x and y position const scorePosition = (pos) => (pos.x * 100) + (pos.y * 500); const devices = (await getConnectedDevices()) .sort((a, b) => scorePosition(a) – scorePosition(b)); for (let i = 0; i < Math.min(lines.length, devices.length); i++) { await devices[i].sendUrl(`/page/text?text=${lines[i]}`); } } 53
52. 54 Cells ADB WebSockets Remote HTTP POST On / Off Smart home Home Assistant Architecture Server
53. 05 Integrate with other services. Remotes, smart homes, and more 55
54. import express from 'express'; const app = express(); app.post('/api/power', async function (req, res) { const devices = await getConnectedDevices(); await Promise.all(devices.map(async (device) => { const KEYCODE_POWER = 26; await device.keyevent(KEYCODE_POWER); }); res.send('Toggled power'); }); 56
55. 57 <form action="/api/power" method="post" > <button type="submit"> On / Off </button> </form> On / Off
56. Server Communication protocol 58 Cell ADB WebSockets On / Off Remote HTTP POST
57. Other services 59 Cell ADB WebSockets Services HTTP POST Server & Webhooks Webhook call fired by service
58. 60 Web server Reminder Notified! Smart home software
59. Home Assistant https://home-assistant.io Open source smart home software to coordinate devices 61
60. Speaker integration 62
61. Power of the Web! 63 Services Server Internet of Things Smart home
62. 64 Cells ADB WebSockets Remote HTTP POST Server On / Off Smart home Home Assistant Architecture
63. home- assistant.io Install on Raspberry Pi, Docker Container, old computer, and more! 65
64. tigeroakes.com @Not_Woods Thank you! T H A N K Y O U 66
65. Credits. Presentation Template: SlidesMania Sample Images: Unsplash Fonts used in this presentation: Cascadia Code 67