Powering Real-Time Search at Microsoft

This talk was presented at the inaugural Elastic{ON} conference, http://elasticon.com

Session Abstract:
Pablo Castro, Director of Engineering at Microsoft, shares how Microsoft is leveraging Elasticsearch to power search and analytics across various products, including MSN, Microsoft Social Listening, and Azure Search.

Transcript

1. { } 18 Pablo Castro Director of Engineering
2. { } 19
3. { } 20
4. { } 21 Top-30 site of the Web 24,000 requests per second peaks 18 billion page views/month Content backend powered by Elasticsearch
5. { } Cache 24,000 requests per second (RPS) peaks “local” items not cached 800-1000 RPS filters + graph queries 30-50 ms latency 54 nodes/cluster 42 data 3 master 9 clients x4 regions near-real time content updates News Sports stats Movies … Elasticsearch Cluster Content
6. { } 23 How do you measure the effectiveness of an online marketing campaign in Microsoft Dynamics CRM? Elasticsearch used for indexing and analyzing all social signals Microsoft Social Listening
7. { } 24 Twitter Facebook Tumblr Wordpress News Filter topics Elasticsearch cluster Front-end Dynamics CRM 2 million items/day and growing 300 million items 30 data nodes, 3 masters Ubuntu Linux on Azure VMs analytics queries aggregations End user experience
8. { } 25 An offering for customers looking for a PaaS search-as-a- service solution in Azure Elasticsearch allowed us to bring a product to market faster, no wheels reinvented Search
9. { } 26 Azure Portal 
 or API Managed experience Online topology changes 8 regions globally > 600 clusters > 1000 nodes Growing fast Custom extensions: . Shard allocation . Master election . DSL queries . Language analyzers Elasticsearch cluster
10. { } 27
11. { } 28 Thanks for listening! pablo.castro@microsoft.com @pmc