Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Android Indoor Localization

Android Indoor Localization

Description of a high accuracy indoor localization system using Android devices and combine WiFi, Beacons and Magnetic Field.

Panagiotis Vasileiou

January 25, 2017
Tweet

More Decks by Panagiotis Vasileiou

Other Decks in Programming

Transcript

  1. Android Indoor Localization Panagiotis Vasileiou, Lead Software Engineer @ SchoolBusNotes

    S.A. panosvas 5th Athens Android Dev Meetup The Cube, 25 January, 2017
  2. Let's Start!  Can Android find me?  Sure! You

    cannot hide :)  Should I use Android?  Why not :)  Come on...  OK! Android contains everything you need to compose an indoor localization system  Multiple Sensors, Multiple Wireless Interfaces  And yes, you should use it!  There is huge adoption of Android devices  Android day to day evolves (some time backwards, but we can deal with it)
  3. Available Tools  Unfortunately (or fortunately), there is no predefined

    way to achieve this  Android gives you the tools, and we have to use them  OK then, what are these tools?  WiFi Networks: SSID, RSSI (Received Signal Strength Indicator)  Bluetooth: Beacons  Magnetic Field: x, y, z coefficients (what??)  Multiple Sensors: Gyroscope, Accelerometer, …
  4. Break it Down  What is the best?  Don't

    be hurry. As in many questions, it depends. Let me first describe some options  WiFi  Triangulation: the classic method that GPS is using. We need coverage at each point from at least 3 Access Points (actually, it is 4 as we need one extra for time) AP 1 AP 2 AP 3 ME
  5. Break it Down  WiFi  Fingerprinting: 2 phases 

    Phase 1 (offline or training): Create a Radio Map  Phase 2 (online): Find me based on the radio map and real measurements  Beacons  Triangulation  Fingerprinting  Nearbys: Room level identification. Low level transmit power results in room identification for each beacon reception
  6. Break it Down  Magnetic Field  Each exact location

    inside a building has a unique magnetic field signature!
  7. Pros - Cons  WiFi or Beacons  Triangulation 

    Pros  No training phase needed  Cons  Simultaneously coverage of at least 4 APs for each point  Bad results (more times worse than 5 meters off, based on the chosen propagation model  Fingerprinting  Pros  No need of multiple receptions at point  Good accuracy  Cons  Need a training phase (the more extensive the better)
  8. Pros - Cons  Magnetic Field  Pros  Good

    accuracy  Low battery consumption  Coverage everywhere (earth on duty!)  No infrastructure  Cons  Need a training phase (the more extensive the better)  There are serious differences of what each device measures
  9. Still... I cannot choose one  No worries! You can

    have all of them, or most of them :)  There are plenty of options and day to day research on that, but 2 well established ways to do this, are:  Extended Kalman Filters (adds memory to the system)  This is not new, as Android uses Kalman Filter before it serves each application with sensor data for example. The main use in Android is to smooth the sensor values  The Extended Kalman Filter is a common approach on sensor fusion  Particle Filters  A promising technique newly applied to combining multiple data
  10. Architecture for High Accuracy  Combination of  WiFi Fingerprinting

     Beacons Fingerprinting  Magnetic Field  Modified/Extended Kalman Filters + Pattern Search Algorithm  There is the need of a robust measurements campaign  Multiple different devices  Different orientations at same point  Multiple measurements at same point  No initial user interference
  11. Hardware  Custom Portable WiFi/BT APs  Beagleboards with Linux

     Estimote Beacons  Custom advertising interval and RSSI  Step Motor for measurements campaign
  12. Come back to Android!  What is the task? 

    Retrieve WiFi, Beacons and Magnetic Fields measurements every period of 'X'  I can read the developer.android.com for that...  Sure! But there are a couple of issues I faced with each measurement which is good to know...
  13. WiFi Measurements  We need 'X' measurements for each point

     Cannot get measurements as frequent as you want  Actually, you can, but you will get the same results  You are always based on the chipset and the Android kernel  Examples of minimum period for different Android versions:  Android 4: 5 seconds  Android 4.4.2: less than 2 seconds  Android 5/6: less than 1 second  What to use? 1, 2 or 5?  Let the system decide...
  14. WiFi Measurements  Get the WiFi Manager WifiManager wManager =

    (WifiManager) getSystemService(Context.WIFI_SERVICE);  Start scan and register receiver wManager.startScan(); registerReceiver(wifiReceiver, new IntentFilter(WifiManager.SCAN_RESULTS_AVAILABLE_ACTION));  On retrieved measurement do your job: System.out.println("WiFi Measurement: " + wifiCount + "..."); WifiManager wManager = (WifiManager) c.getSystemService(Context.WIFI_SERVICE); List<ScanResult> wifiList = wManager.getScanResults(); for (int i = 0; i < wifiList.size(); i++) { DeviceWifiMeasurements deviceWifiMeasurement = new DeviceWifiMeasurements(); ScanResult wifi = wManager.getScanResults().get(i); deviceWifiMeasurement.setSsid(wifi.SSID); deviceWifiMeasurement.setRssi(String.valueOf(wifi.level)); deviceWifiMeasurements.add(deviceWifiMeasurement); String outputInfo = "SSID: " + wifi.SSID + " " + "Level: " + wifi.level; System.out.println(outputInfo); }
  15. WiFi Measurements  Call startScan() again to wait for new

    measurement  In that way you get new results as soon as the kernel has new ones  After your desired number of measurements continue with next job: wifiCount++; if (wifiCount > wifiTotalCount) { System.out.println("WiFi Measurement Finished!"); … }
  16. Estimote Beacon Measurements  Limitations  Beacons needs an initial

    calibration for broadcasting intervals  Default 1 sec and minimum 0.2 sec  There are different ways to capture Estimote Beacons  Use the RangingListener from BeaconManager and do your job inside: public void onBeaconsDiscovered(Region region, final List<Beacon> beacons) { …... if (beaconCount < beaconTotalCount) { for (Beacon beacon : beacons) { DeviceBeaconMeasurements deviceBeaconMeasurement = new DeviceBeaconMeasurements(); deviceBeaconMeasurement.setMinor(String.valueOf(beacon.getMinor())); deviceBeaconMeasurement.setRssi(String.valueOf(beacon.getRssi())); deviceBeaconMeasurements.add(deviceBeaconMeasurement); Log.i("INFO", "Beacon Minor: " + beacon.getMinor() + " RSSI: " + beacon.getRssi()); } beaconCount++; } else { // Finished beacons measurements, so, go on... } }
  17. Magnetic Field Measurements  In WiFi and Beacons we want

    faster measurements but we cannot  In Magnetic Field (MF) Measurements we want slower measurements  The MF measurements arrive extremely fast from the continuous magnetometer of the smartphone  We need a better channel sampling for each point, so, we want to slow down our values  There are multiple ways to achieve that  Get all the values and filter out only those with specific intervals  Register and unregister to sensor in specific intervals  ...
  18. Magnetic Field Measurements  On very small intervals you can

    get continuous values and filter out, otherwise you can do something like: CountDownTimer magneticCountDownTimer = new CountDownTimer(3000, 1000) { @Override public void onTick(long millisUntilFinished) { sensorManager.registerListener(magneticListener, sensorManager.getDefaultSensor(Sensor.TYPE_MAGNETIC_FIELD),SensorManager.SENSOR_DELAY_NORMAL); } @Override public void onFinish() { retrieveBLEStats(); } };  And then in onSensorChanged do your job as usual @Override public void onSensorChanged(SensorEvent event) { if (event.sensor.getType() == Sensor.TYPE_MAGNETIC_FIELD) { DeviceMagneticMeasurements deviceMagneticMeasurement = new DeviceMagneticMeasurements(); deviceMagneticMeasurement.setxValue(String.valueOf(event.values[0])); deviceMagneticMeasurement.setyValue(String.valueOf(event.values[1])); deviceMagneticMeasurement.setzValue(String.valueOf(event.values[2])); deviceMagneticMeasurements.add(deviceMagneticMeasurement); sensorManager.unregisterListener(magneticListener); } }
  19. WiFi Measurements  At the end, we want a radio

    map for each one (WiFi, Beacons, Magnetic field)  Example of fingerprinting for single AP
  20. Measurements Campaign Hints  Export Distribution for each measurement. To

    help you:  Log-Normal for RSSI WiFi and Beacons  Normal for Magnetic Field (in general the orientation will also follow the Normal distribution)  In that way you will be able to:  Export an estimated error for each measurement estimation  Assist the algorithm to apply weights where we are more confident  If there is high error in WiFi estimation we will apply small weight
  21. Measurements Campaign Guidelines  If possible perform it in real

    conditions  Make as much measurements per point  Use different orientations per point  Use different devices  Create a density grid  Preferably remotely control the application, e.g.:  Proof of Concept: Socket  Production: GCM  Production: MQTT Paho at Android side and RabbitMQ + MQTT Adapter at Server side  ...
  22. Remote Start of Measurement  UDP Socket communication public class

    UDPClient extends Application implements Runnable { private final static int LISTENING_PORT = 37766; @Override public void run() { try { //Opening listening socket Log.e("UDP Receiver", "Opening listening socket on port " + LISTENING_PORT + "..."); DatagramSocket socket = new DatagramSocket(LISTENING_PORT); socket.setBroadcast(true); socket.setReuseAddress(true); Log.e("UDP Receiver", "Listening..."); while (true) { //Listening on socket byte[] buf = new byte[1024]; DatagramPacket packet = new DatagramPacket(buf, buf.length); socket.receive(packet); String message = new String(packet.getData()).trim(); Log.e("UDP Receiver", "UDP Packet Received:" + message); message.replaceAll("[\\~\\`\\!\\@\\#\\$\\%\\^\\&\\*\\(\\)\\_\\-\\+\\{\\}\\[\\]\\;\"\\|\\\\,\\.\\/\\<\\>\\?]", ""); if (message.contains("measure")) { ….
  23. Measurements Results WiFi Only Beacons Only Magnetic Field Only Combination

    (4 APs) Combination (6 APs) Error (m) 1.3 1.6 1.5 1.1 0.5  Very good results from WiFi  Beacon is lost packets due to latency and predefined intervals. For that reason, this year Estimote released dedicated beacons for localization  Magnetic field is really promising but the device needs an 8-figure calibration first  The combination alongside with a modified Extended Kalman filter with system memory may increase the accuracy of the system
  24. Online Phase Thoughts  The procedure can take place on

    Server or even on the mobile if the radio map is in an appropriate light format and the algorithm is efficient as well  For research reasons, in order to try different algorithms, we connected Server with Matlab via socket, so, this was the final architecture of the online phase
  25. Last but not Least... Orientation!  What we do if

    we want to orientate relative to the space?  In ship orientation example:
  26. Reference App Orientation in Android  Register for the sensor:

    sensorManager = (SensorManager) getSystemService(SENSOR_SERVICE); sensor = sensorManager.getDefaultSensor(Sensor.TYPE_ORIENTATION); sensorManager.registerListener(mySensorEventListener, sensor, SensorManager.SENSOR_DELAY_NORMAL);  In onSensorChanged get and update the azimuthal value: azimuth = event.values[0]; float degree = Math.round(event.values[0]);  Continuously Broadcast the up to date azimuthal in the WiFi network's broadcast address (most of times you can get it from DHCP): ... socket.setBroadcast(true); socket.setReuseAddress(true); String messageStr = "azimuth=" + String.valueOf(azimuth); InetAddress local = InetAddress.getByName(broadcastAddress); int msg_length=messageStr.length(); byte[] message = messageStr.getBytes(); DatagramPacket p = new DatagramPacket(message, msg_length, local, SERVER_PORT); socket.send(p); ...
  27. Reference App Orientation in Android  Follow the same procedure

    to listen for sensor updates  Setup a UDP Socket Server to retrieve the reference azimuthal  Continuously Perform the combination: float degree = azimuthLocal - azimuthReference;  Update any graphical staff such as compass
  28. Is There Any Other Option?  There are always options

     The users may be passive or active. Till now we mostly discussed the passive way  Some other solutions may contain:  Barcode/QR scanner: the user scan tags in walls for example to identify his/her exact location  RFID tags: the user keeps a tag (such as wristlet) and passes it in various entry points, for room level localization  And many other solutions...
  29. THANK YOU!  Resources  Android Measurements: https://github.com/panosvas/Measurements  Android

    Indoor Localization: https://github.com/panosvas/IndoorPosition  Android Orientation Reference: https://github.com/panosvas/OrientationReference  Android Orientation User: https://github.com/panosvas/OrientationPassenger  Server: https://github.com/panosvas/IndoorPositioningServer 5th Athens Android Dev Meetup The Cube, January 25, 2017