$30 off During Our Annual Pro Sale. View Details »

Data in Your Browsers?!

Data in Your Browsers?!

Building better web apps with client-side data.
Video: http://www.youtube.com/watch?v=al_xsxF6Tdw

Irene Ros

March 18, 2014
Tweet

More Decks by Irene Ros

Other Decks in Technology

Transcript

  1. Data In Your
    Browser?!
    Data In Your
    Browser?!
    Tuesday, October 9, 12

    View Slide

  2. All rights reserved to
    Pete Karl
    Irene Ros
    @ireneros, [email protected]
    http://bocoup.com | http://ireneros.com | http://github.com/iros
    Irene Ros
    @ireneros, [email protected]
    http://bocoup.com | http://ireneros.com | http://github.com/iros
    Tuesday, October 9, 12

    View Slide

  3. Bocoup | http://bocoup.com
    What do you do with your data?
    Create
    Read
    Update
    Delete
    or “why would you be manipulating your data to begin with”
    Tuesday, October 9, 12

    View Slide

  4. Bocoup | http://bocoup.com
    What do you do with your data?
    Create
    Read
    Update
    Delete
    Filtering
    Computation
    or “why would you be manipulating your data to begin with”
    Tuesday, October 9, 12

    View Slide

  5. Bocoup | http://bocoup.com
    What do you do with your data?
    Create
    Read
    Update
    Delete
    Filtering
    Computation
    or “why would you be manipulating your data to begin with”
    AWESOME ZONE
    Tuesday, October 9, 12

    View Slide

  6. Why Client-Side?!
    Why Client-Side?!
    Yey, Browsers!
    They can actually
    Handle it!
    Less Load on the Server:
    More free resources not taken
    up by computation and data
    crunching
    Less Caching? Less storage
    When computations are user-
    specific
    Tuesday, October 9, 12

    View Slide

  7. How do we treat data?
    Rows of records
    Good for iteration
    (doing something with
    each record)
    Bad for figuring out
    something about a
    specific property
    Tuesday, October 9, 12

    View Slide

  8. var data = [
    {"name":"Abe Sapien","type":"hero","age":57},
    {"name":"Abin Sur","type":"hero","age":58},
    {"name":"Adam Monroe","type":"hero","age":59},
    {"name":"Adam Strange","type":"hero","age":60},
    {"name":"Agent 13","type":"hero","age":61}
    ];
    Tuesday, October 9, 12

    View Slide

  9. How do we treat data?
    Column-wise (table-ish)
    Good for figuring out
    something about a specific
    property
    Bad for iteration (doing
    something with each
    record)
    Tuesday, October 9, 12

    View Slide

  10. var data = [
    [ "Abe Sapien", "Abin Sur", "Adam Monroe", "Adam Strange", "Agent 13" ],
    [ "hero", "hero", "hero", "hero", "hero" ],
    [ 57, 58, 59, 60, 61 ]
    ];
    Tuesday, October 9, 12

    View Slide

  11. Follow Along!
    Follow Along!
    https://github.com/iros/clientsidedata
    *There are more libraries explored there
    than this presentation covers.
    Party on.
    *
    *
    Tuesday, October 9, 12

    View Slide

  12. Follow Along!
    Follow Along!
    https://github.com/iros/clientsidedata
    *There are more libraries explored there
    than this presentation covers.
    Party on.
    *
    *
    Tuesday, October 9, 12

    View Slide

  13. {
    "name": "Batman",
    "type": "hero",
    "id": 1339970999,
    "intelligence": 1.0,
    "strength": 0.28,
    "speed": 0.27,
    "durability": 0.42,
    "power": 0.37,
    "combat": 1.0,
    "Alternate_Identities": "Batman",
    "Aliases": "Insider, Matches Malone",
    "Nicknames": "The Dark Knight, The Caped Crusader, The Worlds Greatest ...",
    "Place_of_birth": "Crest Hill, Bristol Township; Gotham County",
    "First_appearance": "Detective Comics #27 (May, 1939)",
    "Alignment": "good",
    "Gender": "male",
    "Race": "",
    "Height_cm": "188",
    "Height_ft": "6'2",
    "Weight_lb": "210",
    "Weight_km": "95",
    "Eye_color": "blue",
    "Hair_color": "black",
    "Occupation": "Businessman",
    "Base": "Batcave, Stately Wayne Manor, Gotham City; Hall of Justice, Justice League Watchtower",
    "Relatives": "Dr. Thomas Wayne (father, deceased), Martha Wayne (mother, deceased), Philip Wayne (uncle,
    foster father, deceased), Alfred Pennyworth (butler, foster father), Dick Grayson
    (Nightwing - adopted son), Jason Todd (Robin II - adopted son, deceased), Tim Drake (Robin
    III - protégé)",
    "Group_Affiliation": "Batman Family, Batman Incorporated, Justice League, Outsiders, Wayne Enterprises,
    Club of Heroes",
    "Friends": "",
    "Enemies": "",
    "superpower_Ability_Shift": 0,
    "superpower_Absorption": 0,
    "superpower_Accuracy": 1,
    "superpower_Adaptation": 0,
    "superpower_Aerokinesis": 0,
    ...
    Tuesday, October 9, 12

    View Slide

  14. Heroes
    Villains
    data/just_heroes.json
    data/just_villains.json
    Tuesday, October 9, 12

    View Slide

  15. Heroes
    Villains
    data/just_heroes.json
    data/just_villains.json
    data/all.json
    Tuesday, October 9, 12

    View Slide

  16. Rows of Data!
    Tuesday, October 9, 12

    View Slide

  17. Bocoup | http://bocoup.com
    TaffyDB
    ‣ Database-like (records are json objs)
    ‣ Rich selection language
    ‣ Supports saving to LocalStorage
    ‣ Extensible
    ‣ No events (for updates, new rows etc.)
    ‣ “templating” support:
    var row = products({ item : 2 })
    .supplant("" +
    "{name}" +
    "{price}" +
    "");
    http://www.taffydb.com
    Tuesday, October 9, 12

    View Slide

  18. Bocoup | http://bocoup.com
    // create a database of heroes
    heroes = TAFFY(heroes);
    // find all the male & female heroes with a height
    var male_heroes = heroes({
    Gender : "male",
    type : "hero",
    Height_cm : { "!is" : "NA" }
    }),
    female_heroes = heroes({
    Gender : "female",
    type : "hero",
    Height_cm : { "!is" : "NA" }
    }),
    // find all heroes that are not male and are not female but have a height
    unknown_gender_heroes = heroes({
    type : "hero",
    Height_cm : { "!is" : "NA" }
    }, [{ Gender : { "!is" : "male" }},
    { Gender : { "!is" : "female" }}]
    );
    Tuesday, October 9, 12

    View Slide

  19. Are Abilities Related?!
    intelligence
    power
    durability
    speed
    combat
    http://jsbin.com/avovem/2/
    clientsidedata/code/taffy-db/competencies.html
    strength
    Tuesday, October 9, 12

    View Slide

  20. Columns
    of Data!
    Tuesday, October 9, 12

    View Slide

  21. Bocoup | http://bocoup.com
    Crossfilter
    ‣ Extremely fast
    ‣ Uses Typed arrays
    ‣ Doesn’t do so well with incomplete or
    messy data...
    http://square.github.com/crossfilter/
    Tuesday, October 9, 12

    View Slide

  22. Bocoup | http://bocoup.com
    // pass the data through the crossfilter
    var heroes_crossfilter = crossfilter(heroes);
    // dimension based on intelligence
    var intelligence_dim = heroes_crossfilter.dimension(function(hero) {
    return hero.intelligence;
    });
    // find all records that are in the upper quadrant
    var lower_quad_intelligence_filter = intelligence_dim.filter([0, 0.25]),
    lower_quad_intelligence = lower_quad_intelligence_filter.top(Infinity);
    // clear filter
    intelligence_dim.filterAll();
    Tuesday, October 9, 12

    View Slide

  23. Bocoup | http://bocoup.com
    Typed Arrays - why?
    ‣ Some operations are faster
    ‣ Quick manipulation of raw binary data
    ‣ Initialization to 0 by default
    ‣ You can share buffers
    ‣ Different views on the same data
    ‣ Smaller memory footprint (if you know
    the dimensions of your data!)
    ‣ Browser support? http://caniuse.com/#search=typed arrays
    ‣ Perf tests: http://jsperf.com/typed-arrays-vs-arrays/6
    http://jsperf.com/typed-array-iteration
    Tuesday, October 9, 12

    View Slide

  24. Bocoup | http://bocoup.com
    var buffer = new ArrayBuffer(5),
    totalUint8Array = new Uint8Array(buffer); //8-bit unsigned integer
    totalUint8Array[0] = NaN;
    totalUint8Array[1] = undefined;
    totalUint8Array[2] = null;
    totalUint8Array[3] = “cat”;
    totalUint8Array[4] = 258;
    console.log(totalUint8Array); >> [0, 0, 0, 0, 2]
    Typed Arrays - why not?
    ‣ NaN, Undefined, Null, strings turn to 0.
    ‣ Data Surprises
    http://jsfiddle.net/EXNtv/2/
    Tuesday, October 9, 12

    View Slide

  25. Bocoup | http://bocoup.com
    intelligence
    power
    durability
    speed
    combat
    How are competencies
    Distributed?
    http://jsbin.com/izefeh/1
    clientsidedata/code/crossfilter/visual.html
    Tuesday, October 9, 12

    View Slide

  26. Bocoup | http://bocoup.com
    The Miso Project - Dataset
    This library was written jointly by Bocoup & The Guardian - I am biased. Ping
    myself or Alex Graul anytime for help.
    Tuesday, October 9, 12

    View Slide

  27. Bocoup | http://bocoup.com
    Fetch Parse
    Compute
    Filter
    Derive
    Tuesday, October 9, 12

    View Slide

  28. Bocoup | http://bocoup.com
    Miso Dataset
    ‣ Node.js + Client-side
    ‣ Built-in importers and parsers for
    common data sources & structures
    ‣ Evented Views
    ‣ Common math functions
    ‣ Common derived dataset functions
    http://misoproject.com/dataset
    @misoprojec on github
    add
    remove
    update
    Tuesday, October 9, 12

    View Slide

  29. Bocoup | http://bocoup.com
    Tuesday, October 9, 12

    View Slide

  30. Bocoup | http://bocoup.com
    // Instantiate a new Miso Dataset object and point it
    // at our trusty hero database.
    var heroes = new Miso.Dataset({
    url: 'http://localhost:8000/data/just_heroes.json'
    // or data : yourJSONObjectArray....
    });
    // fetch data
    heroes.fetch().then(function() {
    console.log("Hero #:" + heroes.length);
    }) ;
    Tuesday, October 9, 12

    View Slide

  31. Bocoup | http://bocoup.com
    // Instantiate a new Miso Dataset object and point it
    // at our trusty hero database.
    var heroes = new Miso.Dataset({
    url: 'http://localhost:8000/data/just_heroes.json'
    // or data : yourJSONObjectArray....
    });
    // fetch data
    heroes.fetch().then(function() {
    console.log("Hero #:" + heroes.length);
    }) ;
    >> Hero #: 570
    Tuesday, October 9, 12

    View Slide

  32. Bocoup | http://bocoup.com
    // Instantiate a new Miso Dataset object and point it
    // at our trusty hero database.
    var heroes = new Miso.Dataset({
    url: 'http://localhost:8000/data/just_heroes.json'
    // or data : yourJSONObjectArray....
    });
    // fetch data
    heroes.fetch().then(function() {
    console.log("Hero #:" + heroes.length);
    }) ;
    >> Hero #: 570
    // Get all the hair color counts
    // and then sort them by the actual resulting count.
    var heroes_by_haircolor = heroes
    .countBy("Hair_color")
    .sort(function(row1, row2) {
    if (row1.count > row2.count) { return -1; }
    if (row1.count < row2.count) { return 1; }
    return 0;
    });
    heroes_by_haircolor.each(function(row) {
    print(row);
    });
    clientsidedata/code/miso.dataset/basic.html
    Tuesday, October 9, 12

    View Slide

  33. Bocoup | http://bocoup.com
    // Instantiate a new Miso Dataset object and point it
    // at our trusty hero database.
    var heroes = new Miso.Dataset({
    url: 'http://localhost:8000/data/just_heroes.json'
    // or data : yourJSONObjectArray....
    });
    // fetch data
    heroes.fetch().then(function() {
    console.log("Hero #:" + heroes.length);
    }) ;
    >> Hero #: 570
    Top hair colors for all
    heroes and villains
    •(156)
    •Black(112)
    •Brown(63)
    •Blond(57)
    •Red(42)
    •None(22)
    •Bald(16)
    •White(15)
    •Blonde(11)
    •Green(6)
    // Get all the hair color counts
    // and then sort them by the actual resulting count.
    var heroes_by_haircolor = heroes
    .countBy("Hair_color")
    .sort(function(row1, row2) {
    if (row1.count > row2.count) { return -1; }
    if (row1.count < row2.count) { return 1; }
    return 0;
    });
    heroes_by_haircolor.each(function(row) {
    print(row);
    });
    clientsidedata/code/miso.dataset/basic.html
    Tuesday, October 9, 12

    View Slide

  34. Bocoup | http://bocoup.com
    ALL THE SUPER POWERS!
    clientsidedata/code/miso.dataset/superpowers.html
    Tuesday, October 9, 12

    View Slide

  35. Bocoup | http://bocoup.com
    Conclusions!
    ‣ Rows or Columns?
    ‣ Mix and Match
    ‣ Beware of iterating too much
    ‣ Beware of premature optimization
    ‣ Going to search a lot? Cache it.
    ‣ Beware of over-caching - keys take space too
    ‣ Only using a subset of your data? Filter it, then
    act on it.
    ‣ Clean up your events
    ‣ Not everything can be client-side
    Tuesday, October 9, 12

    View Slide

  36. Bocoup | http://bocoup.com
    Irene Ros
    [email protected]
    @ireneros
    github.com/iros
    github.com/misoproject
    Questions? Things!
    Write with us:
    Gaming + JavaScript?
    http://buildnewgames.com/
    Interested in app control flow
    management?
    Contact me for early access to
    scene.js/storyboard.js (name
    pending)
    Tuesday, October 9, 12

    View Slide

  37. Same array of
    records, altered
    Object
    1. a: 10
    2. b: 20
    3. sum: 30
    4. __proto__: Object
    Object
    1. a: 50
    2. b: 60
    3. sum: 110
    4. __proto__: Object
    var data = [{ a: 10, b :20 }, { a : 50, b: 60 }];
    _.each(data, function(row) {
    row.sum = row.a + row.b;
    });
    >>
    New array
    [10, 40, 5340, 60]
    _.map([1,4,534,6], function(value) {
    return value * 10;
    });
    >>
    1
    _.min([1,4,3,5])
    Single Value
    >>
    _.inject(
    [1,2,3,4],
    function(memo, value) {
    return memo + value;
    },
    0
    );
    >> 10
    Tuesday, October 9, 12

    View Slide

  38. Who’s Taller?
    Heroes or Villains?
    http://jsbin.com/afupoh/11/
    clientsidedata/code/underscore/visual.html
    Tuesday, October 9, 12

    View Slide

  39. underscore.js vs lodash.js
    ‣ Drop in replacement.
    ‣ But different behavior!
    ‣ Features:
    ‣ AMD Support
    ‣ Lazy Binding
    ‣ _.forOwn - iterating over object’s own properties (no
    more fetching _.keys first.)
    ‣ _.clone - has deep cloning!
    ‣ _.merge - deep _.extend
    ‣ _.countBy
    ‣ Custom builds (mobile, content security policy, etc.)
    ‣ * Faster http://lodash.com/benchmarks
    Tuesday, October 9, 12

    View Slide