And then my phone became smarter - A journey into the Awareness API

And then my phone became smarter - A journey into the Awareness API

Let’s take a glance at the Awareness API and apply it to music recommendations.
The geeks have won ! We are all carrying tiny computers with lots of sensors.
With them, we can do more than display content, we can also make these devices aware of the world around us, making them smarter.
One way to do this is to leverage the Google Awareness API in order to have rich features with minimal input from the user.
It can trigger events when some conditions are put together : when you are close to a certain area, when you plug in headphones, when it is cold or sunny, etc
It can also be used to directly poll the sensors and get some information about the world around you.
Using all these capacities to facilitate your life, your device will know what to do and when to do it, with or without your input (as configured by the user). In this talk we will show you how to improve your music experience and take it to the next stage.

Here is the repo of the app presented : https://github.com/LostInContext/LostContext-App

4d2070e800356da57315d427aa2343f4?s=128

François Blavoet

August 15, 2016
Tweet

Transcript

  1. @francoisblavoet @sarrouna23 And then my phone became smarter

  2. @francoisblavoet @sarrouna23

  3. Smartphones are very personal devices People have different ways to

    use them Your phone has a wealth of sensors Use them to make it smarter
  4. 50 The number of times people check their phone everyday

    Source: The Consumer Barometer Survey 2014/15
  5. 50 The number of times people check their phone everyday

    31% of people use their phone 
 to listen to music every day Source: The Consumer Barometer Survey 2014/15
  6. Enter : The Awareness API

  7. • Interacting with the sensors • Coordinate the detection together

    • Use as little battery as possible Fence : An abstraction for something you want to detect Lots of code to do it manually but comes for free with awareness Enter : The Awareness API
  8. Several types of fences : • Location • Detected Activity

    • Beacon • Time • Headphone Fence API
  9. Let’s dive in the code !

  10. <meta-data
 android:name="com.google.android.awareness.API_KEY"
 android:value="AIzaSyBiEKKSbprkD5GHlSeR7Dd1mv6VsgRyIN0" />
 
 <meta-data
 android:name="com.google.android.geo.API_KEY"
 android:value="AIzaSyBiEKKSbprkD5GHlSeR7Dd1mv6VsgRyIN0" /> Awareness

    API
  11. @Override protected void onStart() {
 super.onStart();
 
 googleApiClient = new

    GoogleApiClient.Builder(this)
 .addApi(Awareness.API)
 .build();
 
 googleApiClient.registerConnectionCallbacks(connectionCallbacks);
 googleApiClient.registerConnectionFailedListener(connectionFailedListener);
 googleApiClient.connect();
 
 } @Override protected void onStop() {
 googleApiClient.disconnect();
 super.onStop();
 } Awareness API
  12. @Override protected void onStart() {
 super.onStart();
 
 googleApiClient = new

    GoogleApiClient.Builder(this)
 .addApi(Awareness.API)
 .build();
 
 googleApiClient.registerConnectionCallbacks(connectionCallbacks);
 googleApiClient.registerConnectionFailedListener(connectionFailedListener);
 googleApiClient.connect();
 
 } @Override protected void onStop() {
 googleApiClient.disconnect();
 super.onStop();
 } Awareness API
  13. @Override protected void onStart() {
 super.onStart();
 
 googleApiClient = new

    GoogleApiClient.Builder(this)
 .addApi(Awareness.API)
 .build();
 
 googleApiClient.registerConnectionCallbacks(connectionCallbacks);
 googleApiClient.registerConnectionFailedListener(connectionFailedListener);
 googleApiClient.connect();
 
 } @Override protected void onStop() {
 googleApiClient.disconnect();
 super.onStop();
 } Awareness API
  14. Awareness.FenceApi.updateFences(googleApiClient,
 fenceUpdateRequest); Fence API

  15. FenceUpdateRequest.Builder builder = new FenceUpdateRequest.Builder();
 builder.addFence("fencekey",
 awarenessFence,
 pendingIntent); builder.build(); Fence

    API
  16. FenceUpdateRequest.Builder builder = new FenceUpdateRequest.Builder();
 builder.addFence("fencekey",
 awarenessFence,
 pendingIntent); builder.build(); Fence

    API
  17. FenceUpdateRequest.Builder builder = new FenceUpdateRequest.Builder();
 builder.addFence("fencekey",
 awarenessFence,
 pendingIntent); builder.build(); Fence

    API
  18. FenceUpdateRequest.Builder builder = new FenceUpdateRequest.Builder();
 builder.addFence("fencekey",
 awarenessFence,
 pendingIntent); builder.build(); Fence

    API
  19. LocationFence.in(latitude, longitude, radius, dwellTimeMillis); LocationFence.entering(latitude, longitude, radius);
 LocationFence.exiting(latitude, longitude, radius);

    DetectedActivityFence.during(DetectedActivityFence.WALKING); DetectedActivityFence.starting(DetectedActivityFence.WALKING); DetectedActivityFence.stopping(DetectedActivityFence.WALKING);
 Fence API - How to build ?
  20. LocationFence.in(latitude, longitude, radius, dwellTimeMillis); LocationFence.entering(latitude, longitude, radius);
 LocationFence.exiting(latitude, longitude, radius);

    DetectedActivityFence.during(DetectedActivityFence.WALKING); DetectedActivityFence.starting(DetectedActivityFence.WALKING); DetectedActivityFence.stopping(DetectedActivityFence.WALKING);
 Fence API - How to build ?
  21. LocationFence.in(latitude, longitude, radius, dwellTimeMillis); LocationFence.entering(latitude, longitude, radius);
 LocationFence.exiting(latitude, longitude, radius);

    DetectedActivityFence.during(DetectedActivityFence.WALKING); DetectedActivityFence.starting(DetectedActivityFence.WALKING); DetectedActivityFence.stopping(DetectedActivityFence.WALKING);
 DetectedActivityFence.during(DetectedActivityFence.WALKING, DetectedActivityFence.ON_BICYCLE); Fence API - How to build ?
  22. Fence API - Let’s put everything together FenceUpdateRequest.Builder builder =

    new FenceUpdateRequest.Builder();
 
 builder.addFence("walking", awarenessFence,
 pendingIntent);
 
 FenceUpdateRequest updateRequest = builder.build();
 

  23. Fence API - Let’s put everything together FenceUpdateRequest.Builder builder =

    new FenceUpdateRequest.Builder();
 
 builder.addFence("walking", DetectedActivityFence.during(DetectedActivityFence.WALKING),
 pendingIntent);
 
 FenceUpdateRequest updateRequest = builder.build();
 

  24. FenceUpdateRequest.Builder builder = new FenceUpdateRequest.Builder();
 
 builder.addFence("walking",
 DetectedActivityFence.during(DetectedActivityFence.WALKING),
 pendingIntent);
 


    FenceUpdateRequest updateRequest = builder.build();
 
 
 Awareness.FenceApi.updateFences(googleApiClient, updateRequest); Fence API - Let’s put everything together
  25. @Override protected void onHandleIntent(Intent intent) {
 FenceState fenceState = FenceState.extract(intent);


    
 if (TextUtils.equals(fenceState.getFenceKey(), "myFence")) {
 switch (fenceState.getCurrentState()) {
 case FenceState.TRUE:
 Log.i(TAG, "the fence conditions are verified");
 break;
 case FenceState.FALSE:
 Log.i(TAG, "the fence conditions are NOT verified");
 break;
 
 case FenceState.UNKNOWN:
 Log.i(TAG, "I have no idea ¯\_(ϑ)_/¯ ”);
 }
 }
 } Fence API - React to the fence
  26. @Override protected void onHandleIntent(Intent intent) {
 FenceState fenceState = FenceState.extract(intent);


    
 if (TextUtils.equals(fenceState.getFenceKey(), "myFence")) {
 switch (fenceState.getCurrentState()) {
 case FenceState.TRUE:
 Log.i(TAG, "the fence conditions are verified");
 break;
 case FenceState.FALSE:
 Log.i(TAG, "the fence conditions are NOT verified");
 break;
 
 case FenceState.UNKNOWN:
 Log.i(TAG, "I have no idea ¯\_(ϑ)_/¯ ”);
 }
 }
 } Fence API - React to the fence
  27. @Override protected void onHandleIntent(Intent intent) {
 FenceState fenceState = FenceState.extract(intent);


    
 if (TextUtils.equals(fenceState.getFenceKey(), "myFence")) {
 switch (fenceState.getCurrentState()) {
 case FenceState.TRUE:
 Log.i(TAG, "the fence conditions are verified");
 break;
 case FenceState.FALSE:
 Log.i(TAG, "the fence conditions are NOT verified");
 break;
 
 case FenceState.UNKNOWN:
 Log.i(TAG, "I have no idea ¯\_(ϑ)_/¯ ”);
 }
 }
 } Fence API - React to the fence
  28. @Override protected void onHandleIntent(Intent intent) {
 FenceState fenceState = FenceState.extract(intent);


    
 if (TextUtils.equals(fenceState.getFenceKey(), "myFence")) {
 switch (fenceState.getCurrentState()) {
 case FenceState.TRUE:
 Log.i(TAG, "the fence conditions are verified");
 break;
 case FenceState.FALSE:
 Log.i(TAG, "the fence conditions are NOT verified");
 break;
 
 case FenceState.UNKNOWN:
 Log.i(TAG, "I have no idea ¯\_(ϑ)_/¯ ”);
 }
 }
 } Fence API - React to the fence
  29. Fence API - Wait, there is more ! FENCE AND

    OR FENCE FENCE NOT
  30. AwarenessFence pluggingInWhileOnFoot =
 AwarenessFence.and(DetectedActivityFence.during(ON_FOOT),
 HeadphoneFence.pluggingIn()); 
 
 AwarenessFence inThePark =

    LocationFence.in(latitude, longitude, radius, dwellTimeMillis);
 
 AwarenessFence complex = AwarenessFence.or(pluggingInWhileMoving, inThePark);
 
 AwarenessFence notInThePark = AwarenessFence.not(inThePark);
 Fence API - Wait, there is more !
  31. AwarenessFence pluggingInWhileOnFoot =
 AwarenessFence.and(DetectedActivityFence.during(ON_FOOT),
 HeadphoneFence.pluggingIn()); 
 
 AwarenessFence inThePark =

    LocationFence.in(latitude, longitude, radius, dwellTimeMillis);
 
 AwarenessFence complex = AwarenessFence.or(pluggingInWhileMoving, inThePark);
 
 AwarenessFence notInThePark = AwarenessFence.not(inThePark);
 Fence API - Wait, there is more !
  32. Fence API - Wait, there is more ! AwarenessFence pluggingInWhileOnFoot

    =
 AwarenessFence.and(DetectedActivityFence.during(ON_FOOT),
 HeadphoneFence.pluggingIn()); 
 
 AwarenessFence inThePark = LocationFence.in(latitude, longitude, radius, dwellTimeMillis);
 
 AwarenessFence complex = AwarenessFence.or(pluggingInWhileMoving, inThePark);
 
 AwarenessFence notInThePark = AwarenessFence.not(inThePark);

  33. AwarenessFence pluggingInWhileOnFoot =
 AwarenessFence.and(DetectedActivityFence.during(ON_FOOT),
 HeadphoneFence.pluggingIn()); 
 
 AwarenessFence inThePark =

    LocationFence.in(latitude, longitude, radius, dwellTimeMillis);
 
 AwarenessFence complex = AwarenessFence.or(pluggingInWhileMoving, inThePark);
 
 AwarenessFence notInThePark = AwarenessFence.not(inThePark);
 Fence API - Wait, there is more !
  34. Fence API - the final bits You can remove fences

    : 
 FenceUpdateRequest.Builder builder = new FenceUpdateRequest.Builder();
 builder.removeFence(“fenceKey”); And also query them directly : FenceQueryRequest queryAll = FenceQueryRequest.all();
 FenceQueryRequest querySome = FenceQueryRequest.forFences("headphone", "walking");
 
 Awareness.FenceApi.queryFences(googleApiClient, queryAll)
 

  35. Snapshot API

  36. Several types of options : • Location • Detected Activity

    • Beacon • Time • Headphone Snapshot API : The possibilities
  37. Several types of options : • Location • Detected Activity

    • Beacon • Time • Headphone • PlaceLikeliHood • Weather Snapshot API : The possibilities
  38. 
 Awareness.SnapshotApi.getWeather(mGoogleApiClient)
 .setResultCallback(new ResultCallback<WeatherResult>() {
 @Override
 public void onResult(@NonNull WeatherResult

    weatherResult) {
 if (!weatherResult.getStatus().isSuccess()) {
 Log.e(TAG, "Could not get weather.");
 return;
 }
 Weather weather = weatherResult.getWeather();
 Log.i(TAG, "Weather: " + weather);
 }
 }); Snapshot API : Sample code
  39. 
 Awareness.SnapshotApi.getWeather(mGoogleApiClient)
 .setResultCallback(new ResultCallback<WeatherResult>() {
 @Override
 public void onResult(@NonNull WeatherResult

    weatherResult) {
 if (!weatherResult.getStatus().isSuccess()) {
 Log.e(TAG, "Could not get weather.");
 return;
 }
 Weather weather = weatherResult.getWeather();
 Log.i(TAG, "Weather: " + weather);
 }
 }); Snapshot API : Sample code
  40. 
 Awareness.SnapshotApi.getWeather(mGoogleApiClient)
 .setResultCallback(new ResultCallback<WeatherResult>() {
 @Override
 public void onResult(@NonNull WeatherResult

    weatherResult) {
 if (!weatherResult.getStatus().isSuccess()) {
 Log.e(TAG, "Could not get weather.");
 return;
 }
 Weather weather = weatherResult.getWeather();
 Log.i(TAG, "Weather: " + weather);
 }
 }); Snapshot API : Sample code
  41. Awareness API : music streaming • Intuitive and easy to

    use • No restriction on the choice of the music application • Use Awareness API and available public APIs of music streaming services
  42. Magic Context

  43. Magic Context

  44. Magic Context

  45. Magic Context - next iterations • More music services •

    More fences types • More actions : autoplay, fence modification or deletion, …
  46. Magic Context - show me the code https://github.com/LostInContext/LostContext-App

  47. Awareness Api : Privacy • The awareness api gives you

    access to very private information. • Make clear to the user what data you’re using and how you’re using it. • If you’re storing data to the device make it clear to the user and make it deletable.
  48. None
  49. Questions ? fb@deezer.com twitter.com/francoisblavoet strabelsi@deezer.com twitter.com/sarrouna23

  50. Thanks Bhavik Singh Albert Reynaud Raj Ajrawat Maurice Chu Axelle

    Migniot Nicolas Le Breton