Better with Friends: Android+Arduino+NFC

Thinking beyond multi-touch gestures, Android's got a few tricks up its sleeve to take user interactions beyond the screen and into the physical world.

With the Android Open Accessory or Bluetooth APIs, devices can interface with external hardware such as Arduino microcontrollers to sense your environment and act upon real world objects. And with NFC support, Android devices can make some of these physical interactions as easy as a handshake.

In this session, we'll explore some commercial and experimental projects that utilize these interesting features and the APIs behind them.

Presented at FITC Spotlight: Android (June 2012) and SCREENS (Sept 2012): http://www.fitc.ca/events/presentations/presentation.cfm?event=131&presentation_id=2011

Transcript

1. klab.ca/spotlightandroid Google+: klab.ca/+ Twitter: @PearlChen Open Accessory (Or "Android: The Loveable Mobile Platform That Plays Well with External Devices") NFC Bluetooth Better with Friends Pearl Chen pearl@karma-laboratory.com
2. klab.ca/spotlightandroid The future is already here — it's just not very evenly distributed. William Gibson
3. About me ( Illustration created using modified output from visualize.me and my LinkedIn profile. ) Some Comp Sci classes back there too
4. Some of my web projects
5. Some of non-web my projects
6. What inspires me A-Nerve by CuteCircuit
7. The Third Industrial Revolution From The Economist, April 2012 A number of remarkable technologies are converging: clever software, novel materials, more dexterous robots, new processes (notably 3D printing) and a whole range of web-based services. The factory of the past was based on cranking out zillions of identical products. The factory of the future will focus on mass customisation.
8. Microcontrollers 1975: Microchip PIC MaKey MaKey Arduino Basic Stamp
9. Hardware startups becoming viable [The founder] also shared a photo of the first Pebble prototype using an Arduino and some similarly cobbled-together hardware. That was four years ago. Pebble Kickstarter campaign as of June 2012 via Slashgear
10. Looking at the hardware trends • Small, yet mighty • Multi-screen • Always on • $$$ ⇾ $ Steve Mann - world’s first cyborg Project Glass by Google
11. ...and the software trends Location, location, location. Context, context, context.
12. And how is this related to Android? This talk will cover: • External hardware integration with Android Open Accessory • NFC for low-friction interactions (not related to mobile payments) • Combining with Bluetooth
13. klab.ca/spotlightandroid Professional Android Sensor Programming Disclaimer: I authored chapter 11 (NFC) & co-authored 10 (AOA) (more info here)
14. klab.ca/spotlightandroid Android Open Accessory
15. klab.ca/spotlightandroid Android Open Accessory - Demo!
16. klab.ca/spotlightandroid Android Development Kit (ADK) Arduino Mega ADK
17. klab.ca/spotlightandroid More Android Development Kits! SparkFun IOIO ODroid ADK Microchip PIC24F ADK
18. Basic firmware/sketch on Arduino ADK #include <Max3421e.h> #include <Usb.h> #include <AndroidAccessory.h> AndroidAccessory acc("Manufacturer", "Model", "Description", "1.0", "http://www.android.com", "0000000012345678"); Google-supplied C++ library USB and USB host libraries Create a new instance of the AndroidAccessory void setup() { acc.powerOn(); } void loop() { byte msg[0]; if (acc.isConnected()) { //send something to the Android app acc.write(msg, 1); //or read something int len = acc.read(msg, sizeof(msg), 1); } } Note: You can get a fully working but barebones Arduino ADK sketch here: iheartrobotics.com/2011/07/arduino-mega-adk-setup-notes.html Convenience method that simply calls the powerOn() method in the Max3421e library. Continually check for connection to Android app Create data that the Android app can read Read data from Android app into msg variable
19. Accessory Filter Resource Needs to match the Arduino sketch exactly In xml/accessory_filter.xml: <resources> <usb-accessory manufacturer="Manufacturer" model="Model" version="1.0" /> </resources>
20. Android Manifest <manifest ...> <uses-sdk android:minSdkVersion="12" /> <uses-feature android:name="android.hardware.usb.accessory" /> <application ...> <activity ...> <meta-data android:name="android.hardware.usb.action.USB_ACCESSORY_ATTACHED" android:resource="@xml/accessory_filter" /> <intent-filter> <action android:name="android.hardware.usb.action.USB_ACCESSORY_ATTACHED"/> </intent-filter> </activity> </application> </manifest> In AndroidManifest.xml (3.1+): From previous slide Use intent filter to launch this activity when ADK is plugged in
21. Activity import com.android.hardware.usb.UsbAccessory; import com.android.hardware.usb.UsbManager; //... UsbManager manager = (UsbManager) getSystemService(Context.USB_SERVICE); UsbAccessory accessory = (UsbAccessory) intent.getParcelableExtra(UsbManager.EXTRA_ACCESSORY); ParcelFileDescriptor fileDescriptor = manager.openAccessory(accessory); FileDescriptor fd = fileDescriptor.getFileDescriptor(); FileInputStream inputStream = new FileInputStream(fd); byte[] buffer = new byte[16384]; int ret = inputStream.read(buffer); FileOutputStream outputStream = new FileOutputStream(fd); mOutputStream.write(buffer); In the .java file for your Activity (3.1+): Read data from the Arduino via a FileInputStream Write data to the Arduino via a FileOutputStream
22. Project ideas So what can you make with the Open Accessory APIs and an Arduino? 1 + 1 = 3 Things already available on your phone Electronics components you can add Super awesome stuff!
23. klab.ca/spotlightandroid Things already available on your phone touch screen camera wifi/3G accelerometer GPS mass storage proximity/light sensors temperature sensor microphone NFC
24. klab.ca/spotlightandroid Electronics components you can add Electronics inputs and outputs using LittleBits as an example of various components
25. klab.ca/spotlightandroid Super awesome stuff! Music Beta: “Now Playing” by Chris Juergen Aerogarden Monitor by Sam Steele Space Spheres by NASA Floating Sensor Project by UC Berkeley
26. klab.ca/spotlightandroid Why Open Accessories are “open” • iOS developer fees: $99/yr. • Approval required to be in MFi program; must sign NDAs. • Additional Apple licensing fees and 3rd party certification. • Final product needs to be approved by Apple • One supplier (Avent) produces the connectors. • Fill out zero forms to start developing. • Pay nothing to Google -- now or ever. • No approval process from Google to go to market. • USB connectors and drivers are readily available.
27. Speaking of connectors...
28. Are there wireless ADKs? Bluetooth Shields Coming soon? Wifi Shield w/ integrated chip IOIO w/ USB Bluetooth dongle Coming soon? ADK2012
29. klab.ca/spotlightandroid NFC (near field communications)
30. What is NFC? Radiowaves NFC Scanner NFC Sticker: antenna and flash storage
31. How is it different than RFID? Low frequency radio waves can travel far distances. High frequency waves cannot. NFC works within 10cm only. 13.56MHz = NFC
32. How is it different than RFID? Storage with RFID is also limited to simple IDs versus the complex data NFC tags can hold. NFC Forum Type Popular Products of This Type Operations Specifications Rewrite Capabilities Available Memory Communication Speed Price Range (price per unit) 1 Broadcom Topaz ISO 14443A User rewritable; can be marked as read-only by user 96 bytes, expandable to 2KB 106kbit/s Low (~$1-2 USD) 2 MIFARE UltraLight ISO 14443A User rewritable; can be marked as read-only by user 48 bytes, 144 bytes is common, expandable to 2KB 106kbit/s Low (~$1-2 USD) 3 Sony FeliCa JIS X 6319-4 Manufacture pre- configured to be read- only or re-writable. variable, theoretical 1MB 212kbit/s or 424kbit/s High (~$8-10 USD or higher) 4 NXP DESFire, NXP SmartFX ISO 14443A, ISO 14443B Manufacture pre- configured to be read- only or rewritable. 4KBfor DESFire, up to 32KBfor SmartFX Up to 424kbit/s Medium-High (~$3-4 USD) MIFARE Type Operations Specifications Rewrite Capabilities Available Memory Communicatio n Speed Price Range (price per unit) Classic 1K ISO 14443A compatible, but NDEF is formatted using a proprietary protocol User rewritable; only manufacturer can mark as read-only 752 bytes 106kbit/s Low (~$1 USD) Classic 4K ISO 14443A compatible, but NDEF is formatted using a proprietary protocol User rewritable; only manufacturer can mark as read-only 3440 bytes 106kbit/s Low-Medium (~$2 USD)
33. klab.ca/spotlightandroid NFC - Demo!
34. Android Manifest <manifest ...> <uses-sdk android:minSdkVersion="10" /> <uses-feature android:name="android.hardware.nfc" android:required="true" /> <uses-permission android:name="android.permission.NFC" /> <application ...> <activity ...> <intent-filter> <action android:name="android.nfc.action.NDEF_DISCOVERED"/> <category android:name="android.intent.category.DEFAULT" /> <data android:mimeType="text/plain" /> </intent-filter> </activity> </application> </manifest> In AndroidManifest.xml (4.0+): Could be custom MIME Type like: application/root.gast.playground.nfc
35. Activity - Reacting to NFC scans import android.nfc.NfcAdapter; import android.nfc.NfcAdapter.CreateNdefMessageCallback; import android.nfc.NfcEvent; //... NfcAdapter mNfcAdapter = NfcAdapter.getDefaultAdapter(this); Boolean nfcEnabled = mNfcAdapter.isEnabled(); mNfcAdapter.enableForegroundDispatch(this, mNfcPendingIntent, mReadTagFilters, null); mNfcAdapter.setNdefPushMessageCallback(this,this); In the .java file for your Activity: Give priority to the foreground activity to receive NFC scan intents. Enable Android Beam for peer-to-peer
36. Activity - Formatting to NDEF Format import android.nfc.NdefMessage; import android.nfc.NdefRecord; import android.nfc.tech.Ndef; import android.nfc.tech.NdefFormatable; //... // get the values from the form's text fields: Editable nameField = mName.getText(); JSONObject computerSpecs = new JSONObject(); computerSpecs.put("name", nameField); String data = computerSpecs.toString(); // create a new NDEF record w/ NDEF message using the app's custom MIME type: String mimeType = "application/root.gast.playground.nfc"; byte[] mimeBytes = mimeType.getBytes(Charset.forName("UTF-8")); byte[] dataBytes = data.getBytes(Charset.forName("UTF-8")); byte[] id = new byte[0]; NdefRecord record = new NdefRecord(NdefRecord.TNF_MIME_MEDIA, mimeBytes, id, dataBytes); NdefMessage m = new NdefMessage(new NdefRecord[] { record }); In the .java file for your Activity: JSON for readability, not performance Unique to your app Not hard at all!
37. Activity - Reading NDEF Format import android.nfc.Tag; //... Parcelable[] rawMsgs = intent.getParcelableArrayExtra(NfcAdapter.EXTRA_NDEF_MESSAGES); NdefMessage[] msgs = new NdefMessage[rawMsgs.length]; for (int i = 0; i < rawMsgs.length; i++) { msgs[i] = (NdefMessage) rawMsgs[i]; } NdefRecord record = msgs[0].getRecords()[0]; byte[] payload = record.getPayload(); In the .java file for your Activity: The info you wanted. (e.g. string or json)
38. klab.ca/spotlightandroid A case for NFC • Create low friction interactions • Augment physical objects (even offline) • Make your software app more tangible • Engage others to share and connect • Increase the viralness of your app • Possibility to be platform agnostic
39. klab.ca/spotlightandroid Mobile OS support for NFC Chart updated for Sept 27, 2012
40. klab.ca/spotlightandroid Don’t worry, this isn’t Apple bashing... By touching phones? Hey! What did you just do there? Oh, I just sent him a playlist.
41. klab.ca/spotlightandroid Thank you! Oh, and check out some upcoming Arduino workshops! Sign up for mailing list on karma-laboratory.com. Google+: klab.ca/+ Twitter: @PearlChen Pearl Chen pearl@karma-laboratory.com