Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building interactive wall décor from junk phones

Building interactive wall décor from junk phones

Why throw away your old phones when you can create a digital work of art? With just some basic arts and crafts skills, you can build a collage of phones with wires intertwining and running across your wall like a circuit board. It can display photo galleries, shopping lists, music visualizations, and make a wicked Zoom background.

In this talk, you'll learn how Node.js and a Raspberry Pi can manage multiple Android devices over USB. We'll explore how to build visualizations using browser APIs, and how to integrate with the wider Internet of Things. Let's give your old mobile phones a new life.

Tiger Oakes

May 28, 2022
Tweet

Other Decks in Programming

Transcript

  1. Building interactive wall décor from junk phones Tiger Oakes -

    @Not_Woods 1
  2. Hi! I’m Tiger. Software Engineer working on Microsoft Loop. •

    Worked on Edge, Chrome, and Firefox. • From Hawai’i, now based in Seattle. • Likes computers 2
  3. Broken cell phones 👵 🧼 🌲 4

  4. 5

  5. Hoarding old tech 6 Upcycling old tech

  6. 7

  7. CellWall

  8. 9

  9. 10

  10. 12 Cells ADB WebSockets Remote HTTP POST Server On /

    Off Smart home Home Assistant Architecture
  11. 01 Build with physical parts. Shopping at the hardware store

    13
  12. Shopping list Base wood panel 1. Plywood panel 2. Sandpaper

    3. White acrylic paint 4. Paint brush 5. Wood screws or command strips for mounting to wall
  13. Shopping list Base wood panel 1. Plywood panel 2. Sandpaper

    3. White acrylic paint 4. Paint brush 5. Wood screws or command strips for mounting to wall
  14. Shopping list Mounts, connectors 6. Old cell phones and tablets

    running Android 7. Raspberry Pi or another computer for the server 8. USB cables 9. USB hub 10.Velcro strips 11.Wire clips
  15. Shopping list Mounts, connectors 6. Old cell phones and tablets

    running Android 7. Raspberry Pi or another computer for the server 8. USB cables 9. USB hub 10.Velcro strips 11.Wire clips
  16. Shopping list Mounts, connectors 6. Old cell phones and tablets

    running Android 7. Raspberry Pi or another computer for the server 8. USB cables 9. USB hub 10.Velcro strips 11.Wire clips
  17. Shopping list = €74.70 Base wood panel 1. Plywood panel

    2. Sandpaper 3. White acrylic paint 4. Paint brush 5. Wood screws or command strips for mounting to wall Mounts, connectors €26.60 6. Old cell phones and tablets running Android 7. Raspberry Pi or another computer for the server 8. Velcro strips 9. USB cables 10.Wire clips 11.USB hub €48.10
  18. 20 Cells ADB WebSockets Remote HTTP POST On / Off

    Smart home Home Assistant Architecture Server
  19. 02 Talk to your phone over USB. Using Android command

    line tools to control your USB cables 21
  20. USB hub connecting to server 22

  21. Android Debug Bridge Command-line tool to help with developing Android

    apps. 23
  22. # List connected devices > adb devices List of devices

    attached 1e778e25 device emulator-5554 device 24
  23. # Open a Unix shell on the Android device >

    adb –s <serial> shell 25
  24. # Command to install an app > adb –s ABC12345

    install ./app_name.apk # Command to simulate pressing power button > adb –s ABC12345 shell input keyevent 26 # Command to open website on phone > adb –s ABC12345 shell am start –a android.intent.action.VIEW –d https://example.com 26
  25. Enable USB debugging Set up developer mode on each device

    https://aka.ms/debug-android 27
  26. > adb devices * daemon not running; starting now at

    tcp:5037 * daemon started successfully List of devices attached ID1234 device device IDabcd device This phone has no ID! Making up a serial code and updating the phone 28
  27. # List connected devices with long output > adb devices

    -l List of devices attached ID1234 device usb:11 transport_id:1 device usb:22 transport_id:2 IDabcd device usb:33 transport_id:3 # Use USB port ID > adb –s usb:11 shell input keyevent 26 # Use transport ID > adb –t 3 shell input keyevent 26 29
  28. Rooting your Android phone Gain superuser access so you can

    change a string 30
  29. This phone has no Wi-Fi! Using the USB cable like

    an Ethernet cable with reverse port forwarding 31
  30. # Send TCP requests over a port to the server

    via USB, not WiFi > adb –s ABC12345 reverse tcp:<device-port> tcp:<system-port> # Reverse forward TCP request on port 3000 to server port 3000 > adb –s ABC12345 reverse tcp:3000 tcp:3000 32
  31. This phone has no screen! Replacing the screen? 33

  32. github.com/ Genymobile/ scrcpy pronounced "screen copy" Use your computer to

    see and control your phone’s screen 34
  33. 35 Cells ADB WebSockets Remote HTTP POST On / Off

    Smart home Home Assistant Architecture Server
  34. 03 Control phones from a server. Automating ADB commands 36

  35. 37

  36. Finally, he’s talking about JavaScript! 38

  37. https://npm.im/appium-adb import { ADB } from 'appium-adb'; const adb =

    await ADB.createADB(); const devices = await adb.getConnectedDevices(); 39
  38. https://npm.im/appium-adb Command line # List connected devices > adb devices

    List of devices attached 2315332 device 1ee21d device E2dbeeDR device JavaScript import { ADB } from 'appium-adb'; const adb = await ADB.createADB(); const devices = await adb.getConnectedDevices(); console.log(devices); // [{ udid: '2315332' }, // { udid: '1ee21d' }, // { udid: 'E2dbeeDR' }] 40
  39. Control devices with Node async function helloWorld() { const devices

    = await getConnectedDevices(); await Promise.all(devices.map(async (device) => { const KEYCODE_POWER = 26; await device.keyevent(KEYCODE_POWER); await device.startUri('https://example.com'); }); }); 41
  40. Server Communication protocol speed 42 Client ADB

  41. Server Communication protocol speed 43 Client ADB WebSockets TCP

  42. Server Communication protocol speed 44 Client ADB Establish connection and

    turn on phones WebSockets Send display data quickly
  43. Represent state with websites If you are presenting a website,

    an internet product or an app, you can place a screenshot of it here. 45 https://example.com
  44. Latest Software for 1st Gen Kindle Fire Released 2013. Android

    4.4 Firefox Nightly 102 Released yesterday. 46
  45. 04 Coordinating across clients. Splitting up data on the server

    for the cell phones 47
  46. const size = { width: window.innerWidth, height: window.innerHeight, }; 48

  47. 49 0,0 1250,10 Loading…

  48. 50

  49. 51

  50. 52

  51. Distribute lines of text async function showText(text) { const lines

    = text.split('\n'); // Roughly sort based on ascending x and y position const scorePosition = (pos) => (pos.x * 100) + (pos.y * 500); const devices = (await getConnectedDevices()) .sort((a, b) => scorePosition(a) – scorePosition(b)); for (let i = 0; i < Math.min(lines.length, devices.length); i++) { await devices[i].sendUrl(`/page/text?text=${lines[i]}`); } } 53
  52. 54 Cells ADB WebSockets Remote HTTP POST On / Off

    Smart home Home Assistant Architecture Server
  53. 05 Integrate with other services. Remotes, smart homes, and more

    55
  54. import express from 'express'; const app = express(); app.post('/api/power', async

    function (req, res) { const devices = await getConnectedDevices(); await Promise.all(devices.map(async (device) => { const KEYCODE_POWER = 26; await device.keyevent(KEYCODE_POWER); }); res.send('Toggled power'); }); 56
  55. 57 <form action="/api/power" method="post" > <button type="submit"> On / Off

    </button> </form> On / Off
  56. Server Communication protocol 58 Cell ADB WebSockets On / Off

    Remote HTTP POST
  57. Other services 59 Cell ADB WebSockets Services HTTP POST Server

    & Webhooks Webhook call fired by service
  58. 60 Web server Reminder Notified! Smart home software

  59. Home Assistant https://home-assistant.io Open source smart home software to coordinate

    devices 61
  60. Speaker integration 62

  61. Power of the Web! 63 Services Server Internet of Things

    Smart home
  62. 64 Cells ADB WebSockets Remote HTTP POST Server On /

    Off Smart home Home Assistant Architecture
  63. home- assistant.io Install on Raspberry Pi, Docker Container, old computer,

    and more! 65
  64. tigeroakes.com @Not_Woods Thank you! T H A N K Y

    O U 66
  65. Credits. Presentation Template: SlidesMania Sample Images: Unsplash Fonts used in

    this presentation: Cascadia Code 67