Slide 1

Slide 1 text

Workday February 28, 2018 https://github.com/Workday Lessons Learned from A Search Application POC to Production Journey Angela Juang, Tech Lead Search Madhura Dudhgaonkar, Head of Search/ML

Slide 2

Slide 2 text

This presentation may contain forward-looking statements for which there are risks, uncertainties, and assumptions. If the risks materialize or assumptions prove incorrect, Workday’s business results and directions could differ materially from results implied by the forward-looking statements. Forward-looking statements include any statements regarding strategies or plans for future operations; any statements concerning new features, enhancements or upgrades to our existing applications or plans for future applications; and any statements of belief. Further information on risks that could affect Workday’s results is included in our filings with the Securities and Exchange Commission which are available on the Workday investor relations webpage: www.workday.com/company/investor_relations.php Workday assumes no obligation for and does not intend to update any forward-looking statements. Any unreleased services, features, functionality or enhancements referenced in any Workday document, roadmap, blog, our website, press release or public statement that are not currently available are subject to change at Workday’s discretion and may not be delivered as planned or at all. Customers who purchase Workday, Inc. services should make their purchase decisions upon services, features, and functions that are currently available. Safe Harbor Statement

Slide 3

Slide 3 text

Introductions Workday Confidential Angela Juang Tech Lead Search, Workday

Slide 4

Slide 4 text

Workday Confidential Elasticon 2017!

Slide 5

Slide 5 text

Creating things that delight people Workday Confidential

Slide 6

Slide 6 text

Creating things that delight people Workday Confidential

Slide 7

Slide 7 text

Creating things that delight people Workday Confidential

Slide 8

Slide 8 text

Workday Search Creating things that delight people Workday Confidential

Slide 9

Slide 9 text

Introductions Workday Confidential Madhura Dudhgaonkar Head of Search/ML, Workday

Slide 10

Slide 10 text

Mar 2009 - March 2006

Slide 11

Slide 11 text

Mar 2009 - March 2009 Mt. Denali | 20,320 ft | 6,194 m

Slide 12

Slide 12 text

Mar 2009 - March 2009 The Crazy Ones!

Slide 13

Slide 13 text

March 2009 - June 2010

Slide 14

Slide 14 text

July 2010

Slide 15

Slide 15 text

Mar 2009 - Audacious Goals Accelerated Learning and Mastery!

Slide 16

Slide 16 text

Workday Confidential Happy Madhura!

Slide 17

Slide 17 text

The Enterprise Cloud for Finance and HR Workday Confidential

Slide 18

Slide 18 text

Search and Machine Learning Teams San Francisco Dublin, Ireland Pleasanton Workday Confidential

Slide 19

Slide 19 text

Search @ Workday Recruiting Recruiting Search Relevance Workday Confidential 2007-2011 2012-2014 2015-2018

Slide 20

Slide 20 text

Mar 2009 - 2014 - Recruiting search needs upgrade Workday Confidential

Slide 21

Slide 21 text

Workday Confidential Options..Options..

Slide 22

Slide 22 text

Choice Workday Confidential

Slide 23

Slide 23 text

Search Relevance Search Scale

Slide 24

Slide 24 text

Search Relevance

Slide 25

Slide 25 text

scala & search & senior developer & (San Francisco or San Mateo or Dublin, Ireland) & enterprise software & file systems & distributed systems & query latency tuning Workday Confidential What Customers Told

Slide 26

Slide 26 text

Workday Confidential User Research

Slide 27

Slide 27 text

Angela Juang java Madhura Dudhgaonkar senior developer & scala Megan Gates Workday Confidential

Slide 28

Slide 28 text

Name Job Title Skill Workday Confidential They Cared About

Slide 29

Slide 29 text

Workday Confidential More User Research

Slide 30

Slide 30 text

Angela Gates 10 of 10000 Angela Juang Srini Nithyanandan Angela Gates . . Workday Confidential

Slide 31

Slide 31 text

Angela Gates & scala 10 of 5000 Angela Juang Jas Singh Angela Gates . . Workday Confidential

Slide 32

Slide 32 text

Angela Gates & scala & senior developer 10 of 2000 Angela Juang Jas Singh John Washington Angela Gates . Workday Confidential

Slide 33

Slide 33 text

Workday Confidential Eureka!

Slide 34

Slide 34 text

Workday Confidential Deep Search Find oysters (no rocks/fish) - Precision

Slide 35

Slide 35 text

Workday Confidential Shallow Search Find a pearl oyster in top 3-5 - Ranking

Slide 36

Slide 36 text

Solve the Right Search Problem

Slide 37

Slide 37 text

Workday Confidential Tune Precision and Victory

Slide 38

Slide 38 text

Workday Confidential

Slide 39

Slide 39 text

Angela Gates & scala & senior developer 10 of 500 Angela Juang Jas Singh John Washington Angela Gates . Workday Confidential

Slide 40

Slide 40 text

Workday Confidential What Is Going On?

Slide 41

Slide 41 text

Workday Confidential Spelunking Began

Slide 42

Slide 42 text

Job Application Form: Name * Skills Title Attachments * Workday Confidential

Slide 43

Slide 43 text

Cover Letters Resumes Portfolio docs Workday Confidential Nothing in Structured Fields!

Slide 44

Slide 44 text

John Washington 10 of 2000 John Singh (Ref: John Washington) John Washington Angela Gates (John Hopkins University) . . Workday Confidential

Slide 45

Slide 45 text

Workday Confidential Doc Parsing to Rescue

Slide 46

Slide 46 text

Angela Gates & scala & senior developer 10 of 20 Angela Gates Angela Juang Jas Singh John Washington . Workday Confidential

Slide 47

Slide 47 text

Check Data Quality Early! Garbage in, Garbage out. Workday Confidential

Slide 48

Slide 48 text

Total Results Workday Confidential Total Results

Slide 49

Slide 49 text

Precision @50 Workday Confidential Precision @50

Slide 50

Slide 50 text

Ndcg for Ranking Workday Confidential Ndcg for Ranking

Slide 51

Slide 51 text

Search Scale

Slide 52

Slide 52 text

Tenant Index Workday Confidential

Slide 53

Slide 53 text

Workday Confidential

Slide 54

Slide 54 text

Workday Confidential Unmanageable Indices

Slide 55

Slide 55 text

Workday Confidential

Slide 56

Slide 56 text

Workday Confidential

Slide 57

Slide 57 text

Understand your data and growth patterns! Workday Confidential

Slide 58

Slide 58 text

Cluster State Size: Before & After Workday Confidential Cluster State Size: Before & After

Slide 59

Slide 59 text

Workday Confidential Security

Slide 60

Slide 60 text

22 TB JVM heap <= 30GB per data node 734 data nodes (minimum!) Workday Confidential

Slide 61

Slide 61 text

Workday Confidential Outage Scenario

Slide 62

Slide 62 text

Workday Confidential

Slide 63

Slide 63 text

https://github.com/Workday/elasticrypt Workday Confidential

Slide 64

Slide 64 text

Plan for the unknown! Workday Confidential

Slide 65

Slide 65 text

But don’t reinvent the wheel. Workday Confidential

Slide 66

Slide 66 text

If you do invent, share it! :) Workday Confidential

Slide 67

Slide 67 text

Workday Confidential

Slide 68

Slide 68 text

Workday Confidential

Slide 69

Slide 69 text

● Solve the right search problem ● Check data quality early ● Understand the characteristics of your data and growth patterns ● Plan for the unknown Workday Confidential In Summary

Slide 70

Slide 70 text

We Are Hiring! Distributed systems engineer Elasticsearch engineer Search engineer Data scientist Machine Learning engineer Deep learning engineer Workday Confidential

Slide 71

Slide 71 text

TM Thank You