Upgrade to Pro — share decks privately, control downloads, hide ads and more …

JMaghreb 2014: Make sense of your (BIG) data!

JMaghreb 2014: Make sense of your (BIG) data!

Talk given at Casablanca, Morocco

Elasticsearch: you know, for search! But you can use it as well to compute information on live data.

During this session, you will discover how elasticsearch actually works behind the scene and why this great open source software is really more than "search".

We will inject marketing data into elasticsearch and build live a dashboard using Kibana, generic and powerful visualisation tool under Apache2 license.

In minutes, you will know how to build YOUR own dashboard and make sense of YOUR data!

Elasticsearch Inc

November 06, 2014
Tweet

More Decks by Elasticsearch Inc

Other Decks in Technology

Transcript

  1. #elasticsearch MAKE SENSE OF YOUR (BIG) DATA! David Pilato Technical

    advocate elasticsearch. @dadoonet
  2. None
  3. StartUp #elasticsearch #elasticsearch data ?

  4. StartUp #elasticsearch #elasticsearch

  5. StartUp #elasticsearch #elasticsearch

  6. StartUp #elasticsearch #elasticsearch

  7. StartUp #elasticsearch #elasticsearch

  8. StartUp #elasticsearch #elasticsearch

  9. StartUp #elasticsearch #elasticsearch

  10. StartUp #elasticsearch #elasticsearch

  11. StartUp #elasticsearch #elasticsearch

  12. StartUp #elasticsearch #elasticsearch

  13. None
  14. StartUp #elasticsearch #elasticsearch BIG data ?

  15. StartUp #elasticsearch #elasticsearch Source: http://www.csc.com/insights/flxwd/78931-big_data_just_beginning_to_explode

  16. StartUp #elasticsearch #elasticsearch Source: http://www.csc.com/insights/flxwd/78931-big_data_just_beginning_to_explode 35.000.000.000.000.000 mb

  17. StartUp #elasticsearch #elasticsearch Source: http://www.domo.com/learn/data-never-sleeps-2

  18. None
  19. StartUp #elasticsearch #elasticsearch search = like % ?

  20. StartUp #elasticsearch #elasticsearch search = like % ? SELECT doc.*,

    country.* FROM doc, country WHERE doc.country_code = country.code AND doc.date_doc > to_date('2011-12', 'yyyy-mm') AND doc.date_doc < to_date('2012-01', 'yyyy-mm') AND lower(country.name) = 'france' AND lower(doc.comment) LIKE '%product%' AND lower(doc.comment) LIKE '%david%';
  21. StartUp #elasticsearch #elasticsearch Search engine ?

  22. StartUp #elasticsearch #elasticsearch Search engine ?

  23. StartUp #elasticsearch #elasticsearch elasticsearch ?

  24. StartUp #elasticsearch #elasticsearch elasticsearch ? Lucene

  25. StartUp #elasticsearch #elasticsearch elasticsearch ? REST/JSON Lucene

  26. StartUp #elasticsearch #elasticsearch elasticsearch ? REST/JSON scalable Lucene

  27. StartUp #elasticsearch #elasticsearch elasticsearch ? plug & play REST/JSON scalable

    Lucene
  28. StartUp #elasticsearch #elasticsearch plug & play REST/JSON scalable Apache 2

    license Lucene elasticsearch
  29. #elasticsearch #elasticsearch Start… $ wget https://download.elasticsearch.org/elasticsearch/ elasticsearch/elasticsearch-1.4.0.tar.gz $ tar -xf

    elasticsearch-1.4.0.tar.gz $ ./elasticsearch-1.4.0/bin/elasticsearch [INFO ][node ][Ghost Maker] {1.4.0}[5645]: initializing
  30. #elasticsearch #elasticsearch … and play! $ curl -XPUT localhost:9200/sessions/session/1 -d

    '{ "title" : "Elasticsearch", "subtitle" : "Make sense of your (BIG) data !", "date" : "2014-11-06T10:30:00", "tags" : [ "elasticsearch", "jmaghreb", "bigdata" ], "speakers" : [{ "first_name" : "David", "last_name" : "Pilato" }] }'
  31. #elasticsearch #elasticsearch Search! $ curl http://localhost:9200/sessions/session/_search -d' { "query": {

    "multi_match": { "query": "elasticsearch jmaghreb david", "fields": [ "title^3", "tags^2", "speakers.first_name" ] } }, "post_filter": { "range": { "date": { "from": "2014-11-01", "to": "2014-12-01" } } } }'
  32. StartUp #elasticsearch #elasticsearch Compute?

  33. #elasticsearch #elasticsearch $ curl http://localhost:9200/sessions/session/_search -d' { "query": { ...

    }, "aggs": { "by_date": { "date_histogram": { "field": "date", "interval": "day", "format" : "dd/MM/yyyy" } } } }' Compute!
  34. #elasticsearch #elasticsearch $ curl http://localhost:9200/sessions/session/_search -d' { "query": { ...

    }, "aggs": { "by_date": { "date_histogram": { "field": "date", "interval": "day", "format" : "dd/MM/yyyy" } } } }' "by_date": [ { "key_as_string": "03/04/2014", "doc_count": 1 }, { "key_as_string": "12/04/2014", "doc_count": 2 }, { "key_as_string": "16/04/2014", "doc_count": 3 } ] Compute!
  35. #elasticsearch #elasticsearch $ curl http://localhost:9200/sessions/session/_search -d' { "query": { ...

    }, "aggs": { "by_date": { "date_histogram": { "field": "date", "interval": "day", "format" : "dd/MM/yyyy" } } } }' "by_date": [ { "key_as_string": "03/04/2014", "doc_count": 1 }, { "key_as_string": "12/04/2014", "doc_count": 2 }, { "key_as_string": "16/04/2014", "doc_count": 3 } ] Compute!
  36. None
  37. #mstechdays #elasticsearch StartUp #elasticsearch #elasticsearch • logs • twitter •

    github • marketing data • ... • your data • your big data Let’s make sense of …
  38. #mstechdays #elasticsearch StartUp #elasticsearch #elasticsearch • logs • twitter •

    github • marketing data • ... • your data • your big data Let’s make sense of … { "name":"Pilato David", "dateOfBirth":"1971-12-26", "gender":"male", "children":3, "marketing":{ "fashion":334, "music":3363, "hifi":2351 }, "address":{ "country":"France", "city":"Paris", "location": [2.332395, 48.861871] } }
  39. démo #mstechdays #elasticsearch StartUp #elasticsearch #elasticsearch MAKE SENSE OF YOUR

    (BIG) DATA! let’s inject some marketing documents…
  40. #elasticsearch ELASTICSEARCH

  41. elasticsearch.

  42. elasticsearch. elasticsearch kibana logstash Marvel

  43. elasticsearch.

  44. elasticsearch. Training (public and on-site)

  45. elasticsearch. Training (public and on-site) Development support Production support

  46. elasticsearch. Training (public and on-site) Development support Production support Marvel

  47. None
  48. None
  49. @dadoonet اركش we are hiring!