Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PuReWidgets toolkit

PuReWidgets toolkit

A presentation about the initial version of the PuReWidgets toolkit for interactive public display applications.


Jorge C. S. Cardoso

May 30, 2012

More Decks by Jorge C. S. Cardoso

Other Decks in Research


  1. The  PuReWidgets  toolkit  for   interac6ve  public  display   applica6ons

      Jorge  C.  S.  Cardoso   jorgecardoso@ieee.org     Ubicomp@UMinho     May  30,  2012   1  
  2. Content   •  Mo6va6on   •  PuReWidgets  features   • 

    Architecture   •  Applica6on  models   •  Programming  library  overview   •  HelloWorld   2  
  3. Mo6va6on   Complexity  of  incorpora6ng  interac6ve  features  in  public  display

     applica6ons.   Too  many  things  to  answer  when  designing  the  interac6on  for  a  public  display   applica6on:   •  What  controls  can  we  use  to  offer  interac6ve  features  to  users?   –  No  standard  interac6ve  controls  for  public  displays   •  What  interac6on  mechanisms  can  be  used  to  provide  those  features?   –  Various  possible  interac6on  mechanisms  (SMS,  BT  naming,  OBEX,  Email,  IM,  mobile  app,   desktop,  QR  codes,  touch,  etc.)   –  Different  loca6ons  may  have  different  interac6on  resources   •  How  will  those  features  work?   –  Desktop  interac6on  paradigm  does  not  generally  apply   –  Applica6ons  may  not  always  be  visible   •  How  can  the  applica6on  react  to  different  users?   –  The  interac6on  environment  is  mul6-­‐user   –  We  need  to  iden6fy/differen6ate  users   •  How  can  users  tell  what  they  can  do  with  the  applica6on?   –  Where  are  the  features  presented?   3  
  4. Mo6va6on   We  can  solve  all  those  ques6ons  for  ad

     hoc  applica6ons,  but   •  This  means  wasted  effort  in  developing  applica6ons   –  Programmers  must  deal  with  issues  extraneous  to  the  main   applica6on  logic   •  Will  certainly  lead  to  inconsistencies  in  the  interac6on   model   –  Each  solu6on  will  behave  slightly  differently   –  Will  confuse  users   We  need  a  programming  toolkit  for  interac6ve  public  display   applica6ons!   4  
  5. Toolkit   •  The  toolkit  should  provide  a  consistent  model

      for  user  interac6on  and  applica6on   development   – But  s6ll  allow  some  diversity  in  solu6ons   5  
  6. PuReWidgets   •  PuReWidgets  tries  to  solve  these  problems.  

    •  To  make  sure  it  covers  a  wide  range  of   applica6ons,  we  looked  at  various  interac6ve   public  display  systems  to  learn   – General  requirements   – Types  of  controls   – Types  of  input  mechanisms   6  
  7. PuReWidgets   Widget-­‐based  toolkit.     •  A  widget  represents

     an  interac6ve  feature.   •  Is  represented  by  class  in  an  object-­‐oriented  programming  model.   •  Applica6ons  instan6ate  widgets,  define  a  callback,  and  receive   interac6on  event  through  the  callback     •  Main  features   –  Mul6ple,  extensible,  controls   –  Independence  of  input  mechanisms  and  modali6es   –  Automa6cally  generated  graphical  interfaces   –  Asynchronous  interac6on   –  Concurrent  interac6on         –  Graphical  affordances   –  Server  &  client  libraries   7  
  8. Features   Mul6ple,  extensible,  controls.       •  Allows

     developers  to  create  rich  interac6ve  applica6ons.  Controls  can  be  extended  with  custom   func6onality.   •  We  studied  various  interac6ve  display  systems  to  define  a  set  of  basic  interac6ve  controls,  common  to  the   majority  of  applica6ons:   •  Ac6on  bugon   –  Trigger  an  ac6on  in  the  applica6on,  e.g.,  play  a  video   •  Op6on  selec6on   –  Selects  among  a  set  of  op6ons,  e.g.,  to  vote   •  Text  entry   –  Sends  text  to  the  applica6on,  e.g.,  a  comment,  tag,  search  keyword   •  Download   –  Receives  a  media  file  from  the  applica6on,  e.g.,  download  poster   •  Upload   –  Uploads  a  media  file  to  the  applica6on,  e.g.,  a  photo  to  be  displayed   •  Check-­‐in   –  Says  “I’m  here”   8  
  9. Features   Independence  of  input  mechanisms  and  modali6es.    

    •  Allows  developers  to  focus  on  the  core   applica6on  func6onality  and  not  on  the  low-­‐level   interac6on  details.   •  Allows  applica6ons  to  work  in  heterogeneous   loca6ons,  with  different  mechanism     •  Interac6on  can  be  accomplished  with  various   mechanisms  (SMS,  Bluetooth  naming,  QR  codes,   and  automa6cally  generated  graphical   interfaces).   9  
  10. Features   Automa6cally  generated  graphical  interfaces.     •  Provide

     a  richer  interac6on  for  desktop  and   mobile  devices.   •  PuReWidgets  generates  web-­‐based  interfaces   for  each  applica6on.     •  It  also  generates  QR  codes  for  each   applica6on's  feature     10  
  11. Features   Asynchronous  interac6on.     •  Allows  applica6ons  to

     receive  input  events   that  were  generated  when  the  applica6on   was  not  listening.   •  PuReWidgets  provides  a  persistent  input   queue  for  each  applica6on   •  Applica6ons  can  request  past  input  at  any   6me   11  
  12. Features   Concurrent  interac6on.     •  Allows  various  users

     to  interact  at  the  same   6me,  while  providing  applica6ons  with   iden6ty  informa6on  that  allows  them  to   differen6ate  users.   •  Every  input  event  carries  a  user  id  (if  available)   – Where  possible,  anonymous  interac6ons  are  s6ll   “iden6fied”  (more  on  this  later)   12  
  13. Features   Graphical  affordances.     •  Allows  users  to

     immediately  recognize  interac6ve   features  and  input  feedback.   •  Controls  have  an  op6onal  graphical   representa6on  that  can  be  used  on  the  public   display  interface.   –  Applica6ons  may  not  show  the  features  on  the  public   display,  but  they  will  s6ll  be  available     •  Input  feedback  is  also  (op6onally)  displayed  on   the  public  display.   13  
  14. Features   Server  &  client  libraries.     •  Programmers

     can  choose  to  use  just  the   server,  just  the  client,  or  both  libraries   together.   •  Allows  developers  to  choose  the  best   applica6on  model.   14  
  15. PuReWidgets  Architecture   15  

  16. Interac6on  services   •  PuReWidgets  service   –  Keeps  informa6on

     about  every  widget   –  Generates  graphical  interfaces  for  desktop,  mobile,  touch   plaporms     •  These  allow  anonymous  interac6ons,  but  they  generate  persistent   cookie-­‐based  random  ids.   –  Generates  QR  codes  for  every  widget   –  Generates  unique  textural  reference  codes  for  data-­‐based   interac6ons  (SMS,  BT  naming,  OBEX,  etc.)   •  IO  infrastructure   –  (from  Instant  places)   –  Provides  low-­‐level  data-­‐based  interac6on   16  
  17. PuReWidgets  library   •  Provides  an  object-­‐oriented  library  of  widgets

      •  Hides  the  communica6on  details  with  the   PuReWidgets  service   •  Provides  other  features,  unrelated  to  interac6on,   such  as   –  Applica6on  “place-­‐aware”  storage   •  PuReWidgets  provides  a  name-­‐value  server  datastore  for   easy  storage  of  place-­‐specific  seqngs/state   –  Skeleton  admin  interface  for  place-­‐owners   •  To  configure  place-­‐specific  seqngs   17  
  18. Local  applica6on  model   18  

  19. Local  applica6on  model   •  Applica6on  logic  on  the  public

     display  (player)   – Subject  to  the  schedule  6mings   •  May  not  be  able  to  react  (or  update  data   structures)  immediately  when  user  interacts   – But  will  eventually  (when  it  is  displayed  again)   •  When  displayed,  the  toolkit  periodically  asks   the  service  for  input  events   – And  triggers  the  appropriate  widget  event   19  
  20. Remote  applica6on  model   20  

  21. Remote  applica6on  model   •  Applica6on  logic  on  the  server

      •  Can  react  (or  update  data  structures)   immediately  to  user  input   – Even  if  not  on  the  public  display,  the  reac6on  can   be  visible  on  other  channels   •  The  toolkit  periodically  asks  the  service  for   input  events   – And  triggers  the  appropriate  widget  event   21  
  22. PuReWidgets  Implementa6on   •  Google  Appengine  (server)   •  Google

     Web  Toolkit  –  GWT   (client)   •  Overview  of  the  library   22  
  23. Using  PuReWidgets   •  Hello  World  applica6on  (local  model)  

    •  Video   23  
  24. Ini6al  Evalua6on/Development   process   •  Two  applica6ons  developed  

    so  far  (ignore  the  “graphic  design”,  please)   –  Youtube  player,  media   intensive,  complex  graphical   components  (several  panels   that  change  over  6me),  local   applica6on  model   –  Everybody  votes,  based  on   display  owner  created  content,   simpler  graphical  component,   combines  remote  and  local   applica6on  models   24  
  25. Ini6al  Evalua6on/Development   process   •  Con6nuous  refinement   – Develop

     interac6ve  public  display  applica6ons   – Gain  more  insight  about  the  difficul6es       – Refine  the  toolkit  to  address  the  iden6fied   problems   – Refactor  the  applica6ons  to  include  the  toolkit   changes   25  
  26. Summary   •  PuReWidgets  facilitates  applica6on   development   – Provides

     ready-­‐to-­‐use  interac6ve  controls   – Integrates  various  input  mechanisms  via  an  I/O   infrastructure   – Generates  applica6on  interfaces  for  desktop,   mobile,  and  QR  codes   – Provides  client  and  server  applica6on   development  models   26  
  27. The  end   27