NVIDIA's User Experience Streaming Analytics: Data Intelligence with the Elastic Stack at Scale

NVIDIA collects metrics for every frame streamed, every second spent on GeForce NOW, its streaming video game platform, using the Elastic Stack – enabling business, engineering, operations, and quality assurance teams to assess quality and debug issues in real time.

Satish Varma Dandu l Software Manager, Data Science & Engineering l NVIDIA
Niranjan Nataraja l Lead Data Scientist l NVIDIA

Transcript

1. Satish Varma Dandu Software Manager Data Engineering & Data Science Niranjan Nataraja Lead Data Scientist Elastic{ON} - 2017
2. 2 AGENDA Ø Kratos Data Platform as–a-Service Ø Multi-Tenancy Ø GeForce Now Cloud Gaming Ø Use Cases Ø Quality of Experience Ø Operations Ø Business Ø Kratos Visual Logging
3. 3 Ø Data Platform security has been implemented with VPCs in AWS Ø Dashboard access using NVIDIA LDAP Security Ø Offers HA with no data-loss Ø Always On Architecture Ø Data replication High Availability Ø Handles 1M events/sec Ø Auto-scales the cluster automatically Scale Ø Log-to-Analytics Ø Kibana, JDBC access Ø Accessing data using BI tools Self Service Kratos Data Platform as-a-Service
4. 4 ARCHITECTURE 10K FEET API Gateway S3 Kafka Cluster ELK Cluster Spark + Presto Cluster BI Tools Real Time Events Batch logs
5. 5 Kratos – Data Platform Stats
6. 6 GEFORCE NOW
7. 7 GeForce Now
8. 8 GEFORCE NOW
9. 9 Kratos Visual Logging *Images created from quickmeme.com
10. 10 Kratos Visual Logging Ø Visual Logging is an extension & backward compatible to existing logging frameworks like log4j, log4net, pylog Ø Provides default meta-data out-of-the-box
11. 11 Use case #1: Dev-to-Search
12. 12 Use case #2: Zero-Engineering Visualizations/Charts
13. 13 Use case #3: Zero-Engineering Dashboards Ø Devs can log in a particular format and we can build some automated dashboards for yield, duration, error codes etc.
14. 14 Use case #4: Zero-Engineering BI Ø Kratos Visual logging offers JDBC support. So BI tools like Tableau, AWS Quick Sight can be used to query data directly Ø Engineers & Analysts can write sql queries to make data decisions from the log data directly
15. 15 NVIDIA Kratos Team Ø Jarod Maupin Ø Nixs Nataraja Ø Rohan Somvanshi Ø Ed Clune Ø Rohit Kulkarni Ø Satish Dandu Ø Contact : ElasticOn2017@Nvidia.com, sdandu@nvidia.com
16. 16 Thanks Q&A