Slide 1
Slide 1 text
{ } 29
Unlocking Interplanetary Datasets
with Realtime Search
Ricky Ma and Dan Isla
Slide 2
Slide 2 text
{ } 30
Ricky Ma
Chief of Application
Architecture
Dan Isla
Systems Engineer and
Data Scientist
Who We Are
Slide 3
Slide 3 text
About JPL
Slide 4
Slide 4 text
{ } 32
We are one of the ten NASA field centers
Slide 5
Slide 5 text
{ } 33
we have about 5000 employees
Slide 6
Slide 6 text
{ } 34
located in Pasadena California...
Slide 7
Slide 7 text
{ } 35
and we make stuff
Slide 8
Slide 8 text
{ } 36
Spacecraft (some of the many…)
1st US Satellite 89 - Study Jupiter and its moons
77 - Most distant human-made object
Slide 9
Slide 9 text
{ } 37
First US Satellite
Slide 10
Slide 10 text
{ } 38
Furthest Man Made
Object
Slide 11
Slide 11 text
39
Slide 12
Slide 12 text
Spacecraft Designs
People / Collaboration
Documents and Data
=
Slide 13
Slide 13 text
41
Slide 14
Slide 14 text
Custom Admin panels
Slide 15
Slide 15 text
Adaptive Learning
Slide 16
Slide 16 text
No content
Slide 17
Slide 17 text
No content
Slide 18
Slide 18 text
http://space.io9.com/this-is-every-place-in-the-solar-system-ever-occupied-e-1645507693
Slide 19
Slide 19 text
http://www.nasa.gov/sites/default/files/14-198.png
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
Legacy SQL Ground System
Excel/Matlab
PowerPoint
E-Mail
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
No content
Slide 24
Slide 24 text
Improved Min/Max
Stats metric loses exact time of min and max value. Top Hits nested aggregation, using
_source to extract time.
Slide 25
Slide 25 text
{ } 53
Improved Min/Max
Stats metric loses exact time of min and max value. Top Hits nested aggregation, using
_source to extract time.
Slide 26
Slide 26 text
{ } 54
Data Pipeline and Indexing
ETL
Slide 27
Slide 27 text
{ } 55
Slide 28
Slide 28 text
No content
Slide 29
Slide 29 text
{ } 57
Earth Science
Earth Science: SMAP
● 30x more telemetry than
Mars mission.
● 80 million telemetry points
per day.
● 3.8 billion docs generated
during test and launch
campaign.
● 6 node i2.xlarge cluster,
1.5+ TB ingested.
● Sub-second query time
Slide 30
Slide 30 text
● Mesos to orchestrate multi
cluster deployments of
Elasticsearch
● Docker containers for
Elasticsearch Nodes
● Hybrid Cloud infrastructure
with AWS GovCloud and
Nebula private cloud.
Deployment
Slide 31
Slide 31 text
_search?q=life:universe
Slide 32
Slide 32 text
{ } 60
Thanks for listening!
Ricky Ma and Dan Isla