Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Elastic{ON} 2018 - Lessons Learned from Workday's Search Application Journey from POC to Production

Elastic Co
March 01, 2018

Elastic{ON} 2018 - Lessons Learned from Workday's Search Application Journey from POC to Production

Elastic Co

March 01, 2018
Tweet

More Decks by Elastic Co

Other Decks in Technology

Transcript

  1. Workday
    February 28, 2018
    https://github.com/Workday
    Lessons Learned from A Search
    Application POC to Production Journey
    Angela Juang, Tech Lead Search
    Madhura Dudhgaonkar, Head of Search/ML

    View Slide

  2. This presentation may contain forward-looking statements for which there are risks, uncertainties, and
    assumptions. If the risks materialize or assumptions prove incorrect, Workday’s business results and directions
    could differ materially from results implied by the forward-looking statements. Forward-looking statements
    include any statements regarding strategies or plans for future operations; any statements concerning new
    features, enhancements or upgrades to our existing applications or plans for future applications; and any
    statements of belief. Further information on risks that could affect Workday’s results is included in our filings with
    the Securities and Exchange Commission which are available on the Workday investor relations webpage:
    www.workday.com/company/investor_relations.php
    Workday assumes no obligation for and does not intend to update any forward-looking statements. Any
    unreleased services, features, functionality or enhancements referenced in any Workday document, roadmap,
    blog, our website, press release or public statement that are not currently available are subject to change at
    Workday’s discretion and may not be delivered as planned or at all.
    Customers who purchase Workday, Inc. services should make their purchase decisions upon services, features,
    and functions that are currently available.
    Safe Harbor Statement

    View Slide

  3. Introductions
    Workday Confidential
    Angela Juang
    Tech Lead Search,
    Workday

    View Slide

  4. Workday Confidential
    Elasticon 2017!

    View Slide

  5. Creating things
    that delight
    people
    Workday Confidential

    View Slide

  6. Creating things
    that delight
    people
    Workday Confidential

    View Slide

  7. Creating things
    that delight
    people
    Workday Confidential

    View Slide

  8. Workday
    Search
    Creating things
    that delight
    people
    Workday Confidential

    View Slide

  9. Introductions
    Workday Confidential
    Madhura Dudhgaonkar
    Head of Search/ML,
    Workday

    View Slide

  10. Mar 2009 - March 2006

    View Slide

  11. Mar 2009 - March 2009
    Mt. Denali | 20,320 ft | 6,194 m

    View Slide

  12. Mar 2009 - March 2009
    The Crazy Ones!

    View Slide

  13. March 2009 - June 2010

    View Slide

  14. July 2010

    View Slide

  15. Mar 2009 - Audacious Goals
    Accelerated Learning and Mastery!

    View Slide

  16. Workday Confidential
    Happy Madhura!

    View Slide

  17. The Enterprise Cloud for Finance and HR
    Workday Confidential

    View Slide

  18. Search and Machine Learning Teams
    San Francisco
    Dublin, Ireland
    Pleasanton
    Workday Confidential

    View Slide

  19. Search @ Workday
    Recruiting
    Recruiting Search Relevance
    Workday Confidential
    2007-2011 2012-2014 2015-2018

    View Slide

  20. Mar 2009 -
    2014 - Recruiting search needs upgrade
    Workday Confidential

    View Slide

  21. Workday Confidential
    Options..Options..

    View Slide

  22. Choice
    Workday Confidential

    View Slide

  23. Search Relevance
    Search Scale

    View Slide

  24. Search Relevance

    View Slide

  25. scala & search & senior developer & (San
    Francisco or San Mateo or Dublin, Ireland) &
    enterprise software & file systems & distributed
    systems & query latency tuning
    Workday Confidential
    What Customers Told

    View Slide

  26. Workday Confidential
    User Research

    View Slide

  27. Angela Juang
    java
    Madhura Dudhgaonkar
    senior developer & scala
    Megan Gates
    Workday Confidential

    View Slide

  28. Name
    Job Title
    Skill
    Workday Confidential
    They Cared About

    View Slide

  29. Workday Confidential
    More User Research

    View Slide

  30. Angela Gates
    10 of 10000
    Angela Juang
    Srini Nithyanandan
    Angela Gates
    .
    .
    Workday Confidential

    View Slide

  31. Angela Gates & scala
    10 of 5000
    Angela Juang
    Jas Singh
    Angela Gates
    .
    .
    Workday Confidential

    View Slide

  32. Angela Gates & scala & senior developer
    10 of 2000
    Angela Juang
    Jas Singh
    John Washington
    Angela Gates
    .
    Workday Confidential

    View Slide

  33. Workday Confidential
    Eureka!

    View Slide

  34. Workday Confidential
    Deep Search
    Find oysters (no rocks/fish)
    - Precision

    View Slide

  35. Workday Confidential
    Shallow Search
    Find a pearl oyster in top 3-5
    - Ranking

    View Slide

  36. Solve the Right Search Problem

    View Slide

  37. Workday Confidential
    Tune Precision and Victory

    View Slide

  38. Workday Confidential

    View Slide

  39. Angela Gates & scala & senior developer
    10 of 500
    Angela Juang
    Jas Singh
    John Washington
    Angela Gates
    .
    Workday Confidential

    View Slide

  40. Workday Confidential
    What Is Going On?

    View Slide

  41. Workday Confidential
    Spelunking Began

    View Slide

  42. Job Application Form:
    Name *
    Skills
    Title
    Attachments *
    Workday Confidential

    View Slide

  43. Cover Letters
    Resumes
    Portfolio docs
    Workday Confidential
    Nothing in Structured Fields!

    View Slide

  44. John Washington
    10 of 2000
    John Singh (Ref: John Washington)
    John Washington
    Angela Gates (John Hopkins University)
    .
    .
    Workday Confidential

    View Slide

  45. Workday Confidential
    Doc Parsing to Rescue

    View Slide

  46. Angela Gates & scala & senior developer
    10 of 20
    Angela Gates
    Angela Juang
    Jas Singh
    John Washington
    .
    Workday Confidential

    View Slide

  47. Check Data Quality Early!
    Garbage in, Garbage out.
    Workday Confidential

    View Slide

  48. Total Results
    Workday Confidential
    Total Results

    View Slide

  49. Precision @50
    Workday Confidential
    Precision @50

    View Slide

  50. Ndcg for Ranking
    Workday Confidential
    Ndcg for Ranking

    View Slide

  51. Search Scale

    View Slide

  52. Tenant Index
    Workday Confidential

    View Slide

  53. Workday Confidential

    View Slide

  54. Workday Confidential
    Unmanageable Indices

    View Slide

  55. Workday Confidential

    View Slide

  56. Workday Confidential

    View Slide

  57. Understand your data and growth patterns!
    Workday Confidential

    View Slide

  58. Cluster State Size: Before & After
    Workday Confidential
    Cluster State Size: Before & After

    View Slide

  59. Workday Confidential
    Security

    View Slide

  60. 22 TB
    JVM heap <= 30GB per data node
    734 data nodes (minimum!)
    Workday Confidential

    View Slide

  61. Workday Confidential
    Outage Scenario

    View Slide

  62. Workday Confidential

    View Slide

  63. https://github.com/Workday/elasticrypt
    Workday Confidential

    View Slide

  64. Plan for the unknown!
    Workday Confidential

    View Slide

  65. But don’t reinvent the wheel.
    Workday Confidential

    View Slide

  66. If you do invent, share it! :)
    Workday Confidential

    View Slide

  67. Workday Confidential

    View Slide

  68. Workday Confidential

    View Slide

  69. ● Solve the right search problem
    ● Check data quality early
    ● Understand the characteristics of your data and
    growth patterns
    ● Plan for the unknown
    Workday Confidential
    In Summary

    View Slide

  70. We Are Hiring!
    Distributed systems engineer
    Elasticsearch engineer
    Search engineer
    Data scientist
    Machine Learning engineer
    Deep learning engineer
    Workday Confidential

    View Slide

  71. TM
    Thank You

    View Slide