Adapting ARM Platform to support Android Accessory

Describes how to create your own Android Dev Kit from the example sources and a USB-Host capable ARM device. Code example:https://github.com/jdannenbring/android-arm-accessory

Transcript

1. 16 August 2012 Adapting Your ARM Platform to Support the Android Accessory Protocol (AD-2002) By Jesse Dannenbring Embedded Software Engineer jdannenbring@adeneo-embedded.com Technical White Paper
2. Date Contents Introduction 2 Android ADK2011 / 2012 limitations 3 Porting the Android Accessory Software 3 Libusb Software Implementation 5 Accessory Software 7 Android Application Software 9 Summary 10 References 10 Introduction The Android Open Accessory Development Kit (ADK) gives product developers the hardware and software tools for creating a product that can interact with Android devices via a Bluetooth or USB connection. The interactions can take the form of control, sensor information transfer, and raw data transfer. It could be the Android device that is transmitting its sensor data, or it could be the accessory. The hardware and software are both open source, and give a great deal of flexibility to the accessory designer. Though the software and hardware designs are delivered as a single ADK, the software is only loosely coupled with the hardware designs. In fact, the changes between the ADK2011 and the ADK2012 are quite striking. For instance, the ADK2011 was based on the ATmega2560 microcontroller and the ADK2012 is based on the ARM 32-bit Cortex M3 microprocessor. These changes are possible because the communication protocol that defines the connection between the Android device and a compatible accessory relies on standard USB hardware. We will introduce the Android Open Accessory Protocol (AOA) and show how to easily port the software to any ARM-based device with USB host capabilities. For our example the iMX6Q Cortex A9 microprocessor was chosen as the basis for the Accessory device. USB host communication from an application is best handled from a supporting library that is widely available. Libusb is an open source C library that gives access to USB devices on a wide number of operating systems. It is a popular library for Linux-based devices,
3. though our example utilizes it in Android 4.0 ICS for the iMX6Q Cortex A9. The stock Nexus 7 tablet running Android 4.1 Jellybean was chosen as the Android device to connect to the Accessory. This device was chosen for its out-of-the-box support of Jellybean which includes AOA 2.0 features such as audio streaming. Our software example implementation is an Android Accessory device that functions as an Audio dock for streaming music from the Nexus 7 tablet. It also functions as a picture slideshow controlled from the Nexus 7 tablet. Android ADK2011 / 2012 limitations Currently the Android Accessory Software can be viewed as two separate software pieces. The software running on the Accessory Hardware and the software that runs on the Android device. The software that runs on the accessory can be logically separated further into that which handles communication with the Android device, and that which is controlling the accessory hardware. If the accessory supports it the software may also handle user input and display. The software running on the Android device is an application that handles communication with the accessory, accepts input from the user, and displays information to the user. The software running on the Android device is completely portable and changes for each accessory that it attaches to. The software on the accessory that handles AOA Protocol communication is very easy to port, as it relies on standard USB communication constructs. The software that handles control of the hardware is not portable, but any custom hardware will already have a Board Support Package (BSP), a set of drivers, and possibly an OS that provides this control and functionality. The only task is to integrate the portable AOA protocol communication with the custom boards BSP. While the ADK 2011 / 2012 Hardware design provides a reference design for creating an Android Accessory it does limit the hardware to use particular microcontroller and firmware. A variety of board manufacturers do provide reference boards that are based on the Android ADK design, but this still limits the developer to use a single manufacturers firmware and software tools. Porting the Android Accessory Software Sources for the ported Android Accessory SW are available at: https://github.com/jdannenbring/android- arm-accessory https://github.com/jdannenbring/android- arm-accessory-app Android Open Accessory The Android Open Accessory (AOA) 1.0 protocol allows an Android device to interact with an Android USB Accessory in a special accessory mode. There are four steps: 1. Wait for and detect connected devices 2. Determine the device’s accessory mode support
4. 3. Attempt to start the device in accessory mode 4. Establish communications The manner in which these steps are handled is explained in further detail in the section Libusb Software Implementation. The AOA 2.0 protocol further extends AOA to provide an optional USB audio interface, HID device registration, and Bluetooth support. The USB audio interface in AOA 2.0 utilizes an Isochronous USB endpoint connection for streaming the audio data. The audio data is limited to 2-Channel, 16-bit PCM @ 44100 KHz sample rate. Libusb Portability The key benefit of using libusb as the basis for the ADK software on the Accessory is its portability. Libusb is specifically architected to decouple the OS specific components from the rest of the library. Libusb is also readily available for Linux and porting the library to Android is trivial since Android also relies on a Linux kernel to run. Another reason libusb was chosen is because it supports Isochronous USB transfers (a requirement for the AOA2.0 audio streaming feature). Using non-ADK Hardware There are many benefits to using non-ADK hardware. A developer may wish to differentiate their Android Accessory product and thus want to design custom hardware. Other times a developer may already have an existing embedded design developed for another purpose, and wish to add ADK features and support to that design. This might be a car infotainment unit that could make use of the audio streaming capabilities of Android Open Accessory 2.0, or a home environment unit that could leverage the graphical capabilities of an Android device to display sensor information and control. Adding support for Android Open Accessory to an existing device that already has USB host capabilities is a cheap alternative that likely only requires a software upgrade. Another scenario is a developer wants to design an Accessory but the hardware specifications of the standard ADK do not have the processing capabilities that a latest generation ARM-based SOC might provide, such as image processing, video decoding, or networking. In this case porting the ADK software to this SOC is a requirement. It may even be the case that a developer doesn’t want to wait to get their hands on the ADK hardware and wants to start experimenting with the software right away. Utilizing Android API’s Android ICS was chosen as the OS example for the iMX6Q Accessory because of the libraries that are available for USB connection detection and the AudioTrack API. Android allows a developer to easily create a filter for various USB events, such as connection/disconnection of hardware. This is shown in the section Accessory Software. The AudioTrack API is just one of the audio API available on Android. The other Android Audio API’s (MediaPlayer and SoundPool) are suitable for file media playback or low latency short sound samples. However, the AudioTrack API is suitable for streaming and allows directly
5. setting the number of channels, sample rate, and number of bits per sample. Creating an AudioTrack is shown in the section Accessory Software. For our example we are utilizing libusb which requires native calls via jni. The isochronous transfer callback occurs in the context of this native code, thus it is necessary to call the AudioTrack’s write() method from native code as well. This is necessary because the Android API’s do not support Isochronous USB communication, and thus libusb is a requirement to implement the audio streaming on the Accessory hardware. Libusb Software Implementation Though Android ICS has USB Host capabilities, it does not support isochronous transfers. Since the audio stream is sent by the Android Jellybean operating system via an isochronous endpoint the Android API’s for USB host on the iMX6Q were not viable. Libusb is utilized instead as it provides this isochronous support. Additionally, libusb is a popular C library among many OS including Linux and is purposefully written in a manner that allows easy porting to new operating systems. This makes it the perfect candidate for implementing the Android Open Accessory Protocol and services running on new hardware. During development of this project demonstration the software implementing the AOA protocol was tested on the host Linux development machine before being ported to the Android ICS on ARM. Porting libusb-1.0 to Android requires first downloading the libusb-1.0 sources and then creating an Android.mk file. The Android.mk file specifies the local source and includes files in this way: - Android.mk - LOCAL_SRC_FILES:= \ libusb/core.c \ libusb/descriptor.c \ libusb/io.c \ libusb/sync.c \ libusb/os/linux_usbfs.c LOCAL_C_INCLUDES += $(LOCAL_PATH)/android \ $(LOCAL_PATH)/libusb \ $(LOCAL_PATH)/libusb/os Then the target module name is specified (libusb) and building a shared library is indicated: - Android.mk - LOCAL_MODULE:= libusb LOCAL_MODULE_TAGS := optional LOCAL_PRELINK_MODULE := false include $(BUILD_SHARED_LIBRARY) Finally the source files for libusb already include a config.h file that is normally generated when compiling libusb for Linux. Instead of using automake tools though, we are using the Android build system so this file is added manually and simply contains: - android/config.h - #define OS_LINUX #define ENABLE_LOGGING #define API_EXPORTED #define TIMESPEC_TO_TIMEVAL(tv, ts) { \ (tv)->tv_sec = (ts)->tv_sec; \ (tv)->tv_usec =
6. (ts)->tv_nsec / 1000; \ } This modified libusb folder (named libusb-android) is placed in the Android sources at /external/libusb- android and the libusb module target is added to the list of PRODUCT_PACKAGES in the device.mk for your board: PRODUCT_PACKAGES += \ libusb \ lsusb To build libusb for your android device then simply select your board via the command: $ lunch combo Then rebuild the sources and ramdisk and flash the board. To just rebuild libusb run: $ mmm external/libusb-android The libusb.so library will be compiled and installed to: /out/target/product/mydeviceboard/ system/lib/libusb.so Pushing the library to the running device with adb is simple: $ adb remount /system $ adb push libusb.so /system/lib For development purposes it can also be useful to port lsusb which is an application that displays a lot of useful information about connected USB devices and descriptors. Lsusb can be ported quite easily by simply creating an Android.mk file and declaring the target module as a BUILD_EXECUTABLE. Referencing the libusb library is also necessary since that is what the lsusb program relies on. - Android.mk - LOCAL_PATH:= $(call my-dir) include $(CLEAR_VARS) LOCAL_SRC_FILES:= lsusb.c names.c devtree.c usbmisc.c lsusb-t.c LOCAL_MODULE := lsusb LOCAL_C_INCLUDES += external/libusb-android/libusb/ LOCAL_SHARED_LIBRARIES := libc libusb LOCAL_MODULE_TAGS := optional include $(BUILD_EXECUTABLE) Building lsusb is done in a similar manner to libusb above (as it has also been added to the PRODUCT_PACKAGES list): $ mmm external/lsusb The binary is installed to: /out/target/product/mydeviceboard/system /bin/lsusb Pushing the library to a running device with adb: $ adb remount /system $ adb push lsusb /system/bin It may be necessary to allow access to the /dev/bus/usb nodes from within an Android application context. To do this modify the ueventd.rc file for the device to give full permissions to /dev/bus/usb Once a working libusb has been ported software for the AOA protocol can be written for an Android application on the accessory.
7. Accessory Software The full source code for the ported Accessory software is available at: https://github.com/jdannenbring/andr oid-arm-accessory The Android application for the accessory has a control flow that involves the following sequence of events: A) Detect a USB attached event B) Determine the level of AOA support that the newly attached USB device has. C) Setup a separate Android Service for standard AOA1.0 communication and AOA2.0 audio streaming. The control flow is shown in the Threads of Execution figure below. For step A the Android Activity utilizes the UsbManager to register for an USB_DEVICE_ATTACHED intent. This is done by using an intent-filter in the AndroidManifest.xml. The relevant steps are described in detail on the Android Developer website here. Upon receiving the intent the activity parses out the USB Vendor / Product Id’s and proceeds to step B. For step B a few JNI calls for handling the native libusb library communication are required. The native calls that handle AOA device discovery and initial setup are: connectToAccessory() checkAccessorySupport() Finally once certain level of AOA support is established the Activity spawns a new Service and this service proceeds to step C.
8. For step C the native call that handles the standard AOA communication is: monitorCommands() For the audio streaming AOA 2.0 communication the ADKUSBService creates a new Audio Track class: atrack = new AudioTrack( AudioManager.STREAM_MUSIC, 44100, AudioFormat.CHANNEL_OUT_STERE O, AudioFormat.ENCODING_PCM_16BI T, 70560, AudioTrack.MODE_STREAM); atrack.play(); Next a new Thread is created for handling the actual writing of audio data to the AudioTrack. This is necessary to properly transfer the data from the Isochronous USB transfer complete callback thread to the AudioTrack class. (new Thread(new Runnable() { @Override public void run() { write(); } })).start(); Write() is a native AudioWrite() call that transfers data from a circular buffer to the AudioTrack object. The Isochronous USB transfer is setup with the native call: setupISO() The Audio Data Flow from the Isochronous USB endpoint to the audio hardware is shown in the figure below. An issue that was encountered when attempting to stream the audio over USB to the audio hardware. Consistent underruns were occurring due to the speed of the USB
9. stream. Audio underruns are caused when the audio hardware does not have enough data samples in its hardware buffer to keep up with the intended playback speed. The underruns will result in a ‘hiccup’ or no sound being played for a short time that indicates a lack of data in the buffer. After running some basic calculations and tests it was found that the audio stream data sent from the Android Jellybean device via the USB isochronous transfer is far less than the needed 44.1K samples per second. The results are shown here: usb_audio_callback() time = 49.400999 total transfer = 8638972 AVERAGE Bytes / Sec transferred = 174874.438191 time = 49.401043 total transfer = 8643560 AVERAGE Bytes / Sec transferred = 174967.157121 time = 49.400993 total transfer = 8643384 AVERAGE Bytes / Sec transferred = 174963.769236 time = 49.401007 total transfer = 8643208 AVERAGE Bytes / Sec transferred = 174960.158424 The stream is a 2-channel, 16-bit stream thus each sample is 4 bytes. From the test results it can be seen that the average speed of the stream is 43.735 K samples per second. This means that the USB audio stream is behind the desired rate by roughly 365 samples every second. The ADK2012 hardware skews its sampling rate by small steps in order to accommodate this difference. However, for our example only common discrete sampling rates are supported by the audio hardware. To help reduce this audible issue, our solution was to create a circular buffer to minimize underruns. The usb audio transfer callback is allowed to fill a portion of the circular buffer before the AudioWrite starts transferring data to the audio hardware. This solution reduces the issue to a 2 second ‘hiccup’ every few minutes. Obviously utilizing an additional buffer does increase the latency of the audio stream. Android Application Software The Android Application runs on the Android phone or tablet and handles connection and communication with the Accessory. The application that is started on the Android Device must: - Discover the accessory when it is attached - Communicate with the accessory Discovering the Accessory is done using an intent-filter. Creating the intent-filter is described here. Once the intent is captured, the application must setup communication with the Accessory. This is done with the UsbAccessory class: if(getIntent().getAction().equals(" android.hardware.usb.action.USB_ACC ESSORY_ATTACHED")){ UsbAccessory accessory = (UsbAccessory)getIntent().getParcel ableExtra(UsbManager.EXTRA_ACCESSOR Y); UsbManager manager = ((UsbManager) getSystemService(Context.USB_SERVIC E));
10. FileDescriptor fd = manager.openAccessory(accesso ry).getFileDescriptor(); mOut = new FileOutputStream(fd); mIn = new FileInputStream(fd); } At this point the file streams can be utilized for sending / receiving developer-defined commands and data. Summary Porting the ADK software to a new hardware platform allows for greater flexibility to developers. Using non- ADK hardware benefits developers by leveraging existing hardware platforms, and allowing the use of more powerful processing of the latest ARM hardware SOC’s. This flexibility can help developers create more diverse, impressive Accessory devices or Accessory Kits for use as development platforms. Utilizing the libusb library allows the ADK software to now be easily ported to many different platforms including popular embedded OS such as Linux or Android. The example leverages existing Android API’s for enabling AOA 2.0 features like audio streaming. The example also uses Android’s UsbManager to detect USB connections. Android offers a wide number of media features that could allow an accessory to display a rich variety of content such as video playback, and image processing. References Di Cerbo, Manuel. “Turn your Linux computer into a huge Android USB Accessory” Using Android in Industrial Automation. 5-13-2011. http://android.serverbox.ch/?p=262 “libusb”. http://www.libusb.org/. 1 June 2012. “Android Developers”. http://developer.android.com/index.html. 1 June 2012. “monaka / libusb-android / overview”. https://bitbucket.org/monaka/libusb- android/. 1 June 2012.