Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Unlocking Interplanetary Datasets with Real-Time Search

Unlocking Interplanetary Datasets with Real-Time Search

This talk was presented at the inaugural Elastic{ON} conference, http://elasticon.com

Session Abstract:
Hear how the Jet Propulsion Laboratory is using Elasticsearch to power flight telemetry and system dashboards from the Curiosity Mars rover and SMAP, as well as its internal search engine including a powerful video and people search.

Dd9d954997353b37b4c2684f478192d3?s=128

Elastic Co

March 11, 2015
Tweet

Transcript

  1. { } 29 Unlocking Interplanetary Datasets with Realtime Search Ricky

    Ma and Dan Isla
  2. { } 30 Ricky Ma Chief of Application Architecture Dan

    Isla Systems Engineer and Data Scientist Who We Are
  3. About JPL

  4. { } 32 We are one of the ten NASA

    field centers
  5. { } 33 we have about 5000 employees

  6. { } 34 located in Pasadena California...

  7. { } 35 and we make stuff

  8. { } 36 Spacecraft (some of the many…) 1st US

    Satellite 89 - Study Jupiter and its moons 77 - Most distant human-made object
  9. { } 37 First US Satellite

  10. { } 38 Furthest Man Made Object

  11. 39

  12. Spacecraft Designs People / Collaboration Documents and Data =

  13. 41

  14. Custom Admin panels

  15. Adaptive Learning

  16. None
  17. None
  18. http://space.io9.com/this-is-every-place-in-the-solar-system-ever-occupied-e-1645507693

  19. http://www.nasa.gov/sites/default/files/14-198.png

  20. None
  21. Legacy SQL Ground System Excel/Matlab PowerPoint E-Mail

  22. None
  23. None
  24. Improved Min/Max Stats metric loses exact time of min and

    max value. Top Hits nested aggregation, using _source to extract time.
  25. { } 53 Improved Min/Max Stats metric loses exact time

    of min and max value. Top Hits nested aggregation, using _source to extract time.
  26. { } 54 Data Pipeline and Indexing ETL

  27. { } 55

  28. None
  29. { } 57 Earth Science Earth Science: SMAP • 30x

    more telemetry than Mars mission. • 80 million telemetry points per day. • 3.8 billion docs generated during test and launch campaign. • 6 node i2.xlarge cluster, 1.5+ TB ingested. • Sub-second query time
  30. • Mesos to orchestrate multi cluster deployments of Elasticsearch •

    Docker containers for Elasticsearch Nodes • Hybrid Cloud infrastructure with AWS GovCloud and Nebula private cloud. Deployment
  31. _search?q=life:universe

  32. { } 60 Thanks for listening! Ricky Ma and Dan

    Isla