Slide 1
Slide 1 text
Building
interactive
wall décor from
junk phones
Tiger Oakes - @Not_Woods
1
Slide 2
Slide 2 text
Hi! I’m Tiger.
Software Engineer working on
Microsoft Loop.
● Worked on Edge, Chrome, and Firefox.
● From Hawai’i, now based in Seattle.
● Likes computers
2
Slide 3
Slide 3 text
Broken cell phones
👵 🧼 🌲
4
Slide 4
Slide 4 text
5
Slide 5
Slide 5 text
Hoarding
old tech
6
Upcycling
old tech
Slide 6
Slide 6 text
7
Slide 7
Slide 7 text
CellWall
Slide 8
Slide 8 text
9
Slide 9
Slide 9 text
10
Slide 10
Slide 10 text
12
Cells
ADB
WebSockets
Remote
HTTP POST
Server
On / Off
Smart home
Home Assistant
Architecture
Slide 11
Slide 11 text
01 Build with
physical
parts.
Shopping at the hardware store
13
Slide 12
Slide 12 text
Shopping list
Base wood panel
1. Plywood panel
2. Sandpaper
3. White acrylic paint
4. Paint brush
5. Wood screws or command strips
for mounting to wall
Slide 13
Slide 13 text
Shopping list
Base wood panel
1. Plywood panel
2. Sandpaper
3. White acrylic paint
4. Paint brush
5. Wood screws or command strips
for mounting to wall
Slide 14
Slide 14 text
Shopping list
Mounts, connectors
6. Old cell phones and tablets
running Android
7. Raspberry Pi or another
computer for the server
8. USB cables
9. USB hub
10.Velcro strips
11.Wire clips
Slide 15
Slide 15 text
Shopping list
Mounts, connectors
6. Old cell phones and tablets
running Android
7. Raspberry Pi or another
computer for the server
8. USB cables
9. USB hub
10.Velcro strips
11.Wire clips
Slide 16
Slide 16 text
Shopping list
Mounts, connectors
6. Old cell phones and tablets
running Android
7. Raspberry Pi or another
computer for the server
8. USB cables
9. USB hub
10.Velcro strips
11.Wire clips
Slide 17
Slide 17 text
Shopping list = €74.70
Base wood panel
1. Plywood panel
2. Sandpaper
3. White acrylic paint
4. Paint brush
5. Wood screws or command strips
for mounting to wall
Mounts, connectors
€26.60
6. Old cell phones and tablets
running Android
7. Raspberry Pi or another
computer for the server
8. Velcro strips
9. USB cables
10.Wire clips
11.USB hub
€48.10
Slide 18
Slide 18 text
20
Cells
ADB
WebSockets
Remote
HTTP POST
On / Off
Smart home
Home Assistant
Architecture
Server
Slide 19
Slide 19 text
02 Talk to your
phone over
USB.
Using Android command line tools to control
your USB cables
21
Slide 20
Slide 20 text
USB hub connecting to server
22
Slide 21
Slide 21 text
Android
Debug
Bridge
Command-line tool to help with
developing Android apps.
23
Slide 22
Slide 22 text
# List connected devices
> adb devices
List of devices attached
1e778e25 device
emulator-5554 device
24
Slide 23
Slide 23 text
# Open a Unix shell on the Android device
> adb –s shell
25
Slide 24
Slide 24 text
# Command to install an app
> adb –s ABC12345 install ./app_name.apk
# Command to simulate pressing power button
> adb –s ABC12345 shell input keyevent 26
# Command to open website on phone
> adb –s ABC12345 shell am start
–a android.intent.action.VIEW
–d https://example.com
26
Slide 25
Slide 25 text
Enable USB
debugging
Set up developer mode on
each device
https://aka.ms/debug-android
27
Slide 26
Slide 26 text
> adb devices
* daemon not running; starting now at tcp:5037
* daemon started successfully
List of devices attached
ID1234 device
device
IDabcd device
This phone has
no ID!
Making up a serial code and
updating the phone
28
Slide 27
Slide 27 text
# List connected devices with long output
> adb devices -l
List of devices attached
ID1234 device usb:11 transport_id:1
device usb:22 transport_id:2
IDabcd device usb:33 transport_id:3
# Use USB port ID
> adb –s usb:11 shell input keyevent 26
# Use transport ID
> adb –t 3 shell input keyevent 26
29
Slide 28
Slide 28 text
Rooting your
Android phone
Gain superuser access so you can
change a string
30
Slide 29
Slide 29 text
This phone has
no Wi-Fi!
Using the USB cable like an
Ethernet cable with reverse port
forwarding
31
Slide 30
Slide 30 text
# Send TCP requests over a port to the server via USB, not WiFi
> adb –s ABC12345 reverse tcp: tcp:
# Reverse forward TCP request on port 3000 to server port 3000
> adb –s ABC12345 reverse tcp:3000 tcp:3000
32
Slide 31
Slide 31 text
This phone has
no screen!
Replacing the screen?
33
Slide 32
Slide 32 text
github.com/
Genymobile/
scrcpy
pronounced "screen copy"
Use your computer to see and
control your phone’s screen
34
Slide 33
Slide 33 text
35
Cells
ADB
WebSockets
Remote
HTTP POST
On / Off
Smart home
Home Assistant
Architecture
Server
Slide 34
Slide 34 text
03 Control
phones from a
server.
Automating ADB commands
36
Slide 35
Slide 35 text
37
Slide 36
Slide 36 text
Finally, he’s talking about JavaScript! 38
Slide 37
Slide 37 text
https://npm.im/appium-adb
import { ADB } from 'appium-adb';
const adb = await ADB.createADB();
const devices = await adb.getConnectedDevices();
39
Slide 38
Slide 38 text
https://npm.im/appium-adb
Command line
# List connected devices
> adb devices
List of devices attached
2315332 device
1ee21d device
E2dbeeDR device
JavaScript
import { ADB } from 'appium-adb';
const adb = await ADB.createADB();
const devices =
await adb.getConnectedDevices();
console.log(devices);
// [{ udid: '2315332' },
// { udid: '1ee21d' },
// { udid: 'E2dbeeDR' }]
40
Slide 39
Slide 39 text
Control devices with Node
async function helloWorld() {
const devices = await getConnectedDevices();
await Promise.all(devices.map(async (device) => {
const KEYCODE_POWER = 26;
await device.keyevent(KEYCODE_POWER);
await device.startUri('https://example.com');
});
});
41
Slide 40
Slide 40 text
Server
Communication protocol speed
42
Client
ADB
Slide 41
Slide 41 text
Server
Communication protocol speed
43
Client
ADB
WebSockets
TCP
Slide 42
Slide 42 text
Server
Communication protocol speed
44
Client
ADB
Establish connection
and turn on phones
WebSockets
Send display
data quickly
Slide 43
Slide 43 text
Represent state
with websites
If you are presenting a website, an
internet product or an app, you can
place a screenshot of it here.
45
https://example.com
Slide 44
Slide 44 text
Latest Software for 1st Gen Kindle Fire
Released
2013.
Android
4.4
Firefox
Nightly
102
Released
yesterday.
46
Slide 45
Slide 45 text
04 Coordinating
across
clients.
Splitting up data on the server for the
cell phones
47
Slide 46
Slide 46 text
const size = {
width: window.innerWidth,
height: window.innerHeight,
};
48
Slide 47
Slide 47 text
49
0,0
1250,10
Loading…
Slide 48
Slide 48 text
50
Slide 49
Slide 49 text
51
Slide 50
Slide 50 text
52
Slide 51
Slide 51 text
Distribute lines of text
async function showText(text) {
const lines = text.split('\n');
// Roughly sort based on ascending x and y position
const scorePosition = (pos) => (pos.x * 100) + (pos.y * 500);
const devices = (await getConnectedDevices())
.sort((a, b) => scorePosition(a) – scorePosition(b));
for (let i = 0; i < Math.min(lines.length, devices.length); i++) {
await devices[i].sendUrl(`/page/text?text=${lines[i]}`);
}
}
53
Slide 52
Slide 52 text
54
Cells
ADB
WebSockets
Remote
HTTP POST
On / Off
Smart home
Home Assistant
Architecture
Server
Slide 53
Slide 53 text
05 Integrate
with other
services.
Remotes, smart homes, and more
55
Slide 54
Slide 54 text
import express from 'express';
const app = express();
app.post('/api/power', async function (req, res) {
const devices = await getConnectedDevices();
await Promise.all(devices.map(async (device) => {
const KEYCODE_POWER = 26;
await device.keyevent(KEYCODE_POWER);
});
res.send('Toggled power');
});
56
Slide 55
Slide 55 text
57
On / Off
On / Off
Slide 56
Slide 56 text
Server
Communication protocol
58
Cell
ADB
WebSockets
On / Off
Remote
HTTP POST
Slide 57
Slide 57 text
Other services
59
Cell
ADB
WebSockets
Services
HTTP POST
Server
& Webhooks
Webhook call
fired by
service
Slide 58
Slide 58 text
60
Web
server
Reminder
Notified!
Smart home
software
Slide 59
Slide 59 text
Home Assistant
https://home-assistant.io
Open source smart home
software to coordinate
devices
61
Slide 60
Slide 60 text
Speaker
integration
62
Slide 61
Slide 61 text
Power of the Web!
63
Services
Server
Internet
of Things
Smart home
Slide 62
Slide 62 text
64
Cells
ADB
WebSockets
Remote
HTTP POST
Server
On / Off
Smart home
Home Assistant
Architecture
Slide 63
Slide 63 text
home-
assistant.io
Install on Raspberry Pi,
Docker Container,
old computer, and more!
65
Slide 64
Slide 64 text
tigeroakes.com
@Not_Woods
Thank you! T
H
A
N
K
Y
O
U
66
Slide 65
Slide 65 text
Credits.
Presentation Template: SlidesMania
Sample Images: Unsplash
Fonts used in this presentation: Cascadia Code
67