Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machine Learning from a Product Perspective

Inga Chen
September 08, 2017

Machine Learning from a Product Perspective

MACHINE LEARNING FROM A PRODUCT PERSPECTIVE

The trends of big data and cost-effective computing power have given way to an unprecedented focus on developing machine learning algorithms and open source tools, making machine learning more accessible and powerful today than ever. But building machine learning products is about much more than picking the right algorithm or library. It's a product manager's job to use machine learning for the right problems, and to make sure the end user experience solves those problems and fosters user trust. This talk will illustrate machine learning concepts through applications in products, provide a framework for determining problems that are good for machine learning to solve, and talk about how the product development cycle for machine learning-enabled products might differ from traditional software.

Agenda:
- What is ML?
- What problems are good for ML?
- What should product managers bring to the table?
- ML product development cycle
- Why machine learning now?

Inga Chen

September 08, 2017
Tweet

Transcript

  1. MACHINE LEARNING FROM A PRODUCT PERSPECTIVE I N G A

    C H E N
  2. @ I N G A C H E N S

    I L I C O N VA L L E Y N AT I V E D A U G H T E R O F S O F T WA R E + H A R D WA R E E N G I N E E R S I N V E S T M E N T B A N K E R T E C H M & A J E F F E R I E S S I L I C O N VA L L E Y E A R LY B U S I N E S S H I R E I o T S TA R T U P F O R C A R S A U T O M AT I C L A B S S A N F R A N C I S C O P R O D U C T M A N A G E R D ATA I N S I G H T S & F L E E T A U T O M AT I C L A B S S A N F R A N C I S C O P R O D U C T M A N A G E R A N A LY T I C S S Q U A R E S PA C E N E W YO R K C I T Y
  3. Millions of miles of driving Millions of websites Slice of

    the internet Drivers Fuel efficiency Car Maintenance SMBs Online presence and business
  4. Millions of miles of driving Millions of websites Slice of

    the internet Create an optimized experience for each user MACHINE LEARNING
  5. B E YO N D T H E A L

    G O R I T H M : B U I L D I N G M L P R O D U C T S W H AT S H O U L D P R O D U C T P E O P L E B R I N G T O T H E TA B L E ? B A C KG R O U N D : W H AT I S M L? W H AT P R O B L E M S A R E G O O D F O R M L? W H Y N O W MACHINE LEARNING FROM A PRODUCT PERSPECTIVE
  6. B E YO N D T H E A L

    G O R I T H M : B U I L D I N G M L P R O D U C T S W H AT P R O B L E M S A R E G O O D F O R M L? W H Y N O W MACHINE LEARNING FROM A PRODUCT PERSPECTIVE B A C KG R O U N D : W H AT I S M L? W H AT S H O U L D P R O D U C T P E O P L E B R I N G T O T H E TA B L E ?
  7. Artificial Intelligence Human Intelligence Exhibited by Machines

  8. GOALS OF AI: Reasoning Knowledge Planning Learning Natural Language Processing

    Perception Move & manipulate objects Artificial Intelligence Human Intelligence Exhibited by Machines
  9. “[Machine Learning is the] field of study that gives computers

    the ability to learn without being explicitly programmed Arthur Samuel, 1959
  10. “[Machine Learning is the] field of study that gives computers

    the ability to learn without being explicitly programmed (often more data than is humanly possible to go through) and makes a determination or prediction about something in the world. by observing data
  11. H O W D O E S M A C

    H I N E L E A R N I N G W O R K ?
  12. INPUT OUTPUT ML MODEL EMAIL SPAM NOT SPAM

  13. DATA EMAILS

  14. “CORRECT ANSWERS” IDEALLY…IT’S LABELED EMAILS LABELED AS SPAM / NOT

    SPAM
  15. 70% Training Set Validation Set 30% TEACHER GIVES EXAMPLES TO

    ILLUSTRATE A CONCEPT TEST
  16. B E YO N D T H E A L

    G O R I T H M : B U I L D I N G M L P R O D U C T S B A C KG R O U N D : W H AT I S M L? W H AT P R O B L E M S A R E G O O D F O R M L? W H Y N O W MACHINE LEARNING FROM A PRODUCT PERSPECTIVE W H AT S H O U L D P R O D U C T P E O P L E B R I N G T O T H E TA B L E ?
  17. D E C I S I O N S W

    I T H M A N Y I N P U T S and it’s unclear how to combine them W H AT P R O B L E M S A R E G O O D F O R M A C H I N E L E A R N I N G ?
  18. W H O W I L L W I N

    T H E E L E C T I O N ? Hillary vs Trump Education Income level Race Gender Geographic location Religion What car you drive
  19. None
  20. None
  21. D E C I S I O N S W

    I T H M A N Y O U T P U T S and it’s infeasible to show all outputs W H AT P R O B L E M S A R E G O O D F O R M A C H I N E L E A R N I N G ?
  22. S L A C K H I G H L

    I G H T S M A N A G E T H E D E L U G E O F W O R K M E S S A G E S
  23. M A K I N G D E C I

    S I O N S AT S C A L E for millions/billions of items or users W H AT P R O B L E M S A R E G O O D F O R M A C H I N E L E A R N I N G ?
  24. ML MODEL OUTPUT PREDICTED LABELS CONFIDENCE % CAT CAT NOT

    NOT NOT SUPERVISED LEARNING CAT NOT CAT CAT NOT TRAINING SET UNSEEN DATA INPUT VALIDATION SET I M A G E S E A R C H “ C AT ”
  25. D E C I S I O N S W

    I T H M A N Y I N P U T S D E C I S I O N S W I T H M A N Y O U T P U T S M A K I N G D E C I S I O N S AT S C A L E W H AT P R O B L E M S A R E G O O D F O R M A C H I N E L E A R N I N G ?
  26. PREDICT DISCOVER OPTIMIZE DETECT RANK AUTOMATE stock prices housing prices

    sales health outcomes pricing user experience (personalization) recommendations bank fraud fake news fractures road hazards search results newsfeed recommendations patterns trends customer segmentation human tasks not easily delineated by rules W H AT P R O B L E M S A R E G O O D F O R M A C H I N E L E A R N I N G ?
  27. U N S U P E R V I S

    E D L E A R N I N G D I S C O V E R PAT T E R N S
  28. P R O B L E M How should I

    improve my fuel efficiency based on my driving behavior?
  29. None
  30. 8 clusters

  31. What distinguishes those 8 clusters?

  32. What are the biggest factors affecting fuel efficiency of our

    drivers? Aggression on highway Impatience on local roads Weather, terrain, traffic
  33. NY Cabbie Grocery Go-Getter Highway Hustler Racecar Driver Suburban Scout

    Foxy Fighter Pilot Peaceful Sage Commuter
  34. None
  35. A U T O M AT E C O M

    P L E X TA S K S D E E P L E A R N I N G
  36. None
  37. S E L F - D R I V I

    N G C A R S
  38. P R O B L E M A slice of

    the internet: How do we build products for such a diverse user base?
  39. None
  40. None
  41. None
  42. None
  43. S O L U T I O N VISUAL SEARCH

    ENGINE FOR WEBSITES P R O B L E M How do we build products for such a diverse user base?
  44. None
  45. Deep Learning: Under the hood (SIMPLIFIED) Convolutional variational auto-encoder Recreate

    the image Find similarities Learns how to interpret the structure of an image hidden layers DEEP
  46. None
  47. None
  48. B E YO N D T H E A L

    G O R I T H M : B U I L D I N G M L P R O D U C T S B A C KG R O U N D : W H AT I S M L? W H AT P R O B L E M S A R E G O O D F O R M L? W H Y N O W MACHINE LEARNING FROM A PRODUCT PERSPECTIVE W H AT S H O U L D P R O D U C T P E O P L E B R I N G T O T H E TA B L E ?
  49. 1. IDENTIFY THE RIGHT PROBLEMS TO USE ML TO SOLVE

  50. 2. FIND OUT: DO YOU HAVE THE DATA AND IS

    IT GOOD?
  51. 3. CONSIDER THE NON-ML ALTERNATIVE Is that good enough?

  52. 4. KNOW HOW INPUTS + OUTPUTS ARE RELATED

  53. 5. DESIGN FOR WHEN THE MODEL FAILS Because it will

  54. USER TRUST Easy to lose Hard to win back

  55. W H AT S H O U L D P

    R O D U C T P E O P L E B R I N G T O T H E TA B L E ? B A C KG R O U N D : W H AT I S M L? W H AT P R O B L E M S A R E G O O D F O R M L? W H Y N O W MACHINE LEARNING FROM A PRODUCT PERSPECTIVE B E YO N D T H E A L G O R I T H M : B U I L D I N G M L P R O D U C T S
  56. PRIORITIZE BUILD LAUNCH LEARN MEASURE FOR SOFTWARE PRODUCTS P R

    O D U C T D E V E L O P M E N T C YC L E
  57. CLASSIFY AS ML CONSIDER NON-ML GATHER & PREP DATA PRODUCT

    CHANGES? BUILD & EVALUATE MODELS THROW AWAY IDENTIFY MODEL WEAKNESSES COLLECT MORE DATA REBUILD & IMPROVE LAUNCH PRODUCTION MODEL ACCURACY % TRADEOFFS: ACCURACY VS COMPUTE POWER VS VISIBILITY FOR SOFTWARE PRODUCTS USING ML P R O D U C T D E V E L O P M E N T C YC L E PRIORITIZE BUILD LAUNCH LEARN MEASURE
  58. INPUT OUTPUT MANY ML MODELS P R O D U

    C T S I N P R A C T I C E
  59. B A C KG R O U N D :

    W H AT I S M L? B E YO N D T H E A L G O R I T H M : B U I L D I N G M L P R O D U C T S W H AT S H O U L D P R O D U C T P E O P L E B R I N G T O T H E TA B L E ? W H AT P R O B L E M S A R E G O O D F O R M L? W H Y N O W MACHINE LEARNING FROM A PRODUCT PERSPECTIVE L E A R N I N G F R O M D ATA W I T H O U T B E I N G E X P L I C I T LY P R O G R A M M E D M A N Y I N P U T S , M A N Y O U T P U T S M A K I N G D E C I S I O N S AT S C A L E D E S I G N F O R W H E N T H E M O D E L FA I L S L A U N C H , R E B U I L D & I M P R O V E . T R A D E O F F S
  60. B E YO N D T H E A L

    G O R I T H M : B U I L D I N G M L P R O D U C T S W H AT YO U N E E D F O R M L B A C KG R O U N D : W H AT I S M L? W H AT P R O B L E M S A R E G O O D F O R M L? MACHINE LEARNING FROM A PRODUCT PERSPECTIVE W H Y N O W
  61. MACHINE LEARNING PRODUCTS MATURING OF ML ALGORITHMS with open source

    tools and deep learning MASSIVE COMPUTE POWER Cloud computation GPU processing ACCESSIBILITY OF OPEN DATA CommonCrawl - petabytes ImageNet - 14M labeled C O N V E R G I N G T R E N D S … YOUR COMPETITORS ARE DOING IT NOT JUST FOR BIG COMPANIES EVEN INDIVIDUALS CAN DO IT
  62. BUILD PRODUCTS THAT KNOW YOU BETTER THAN YOU KNOW YOURSELF

    CONSUMER M A C H I N E L E A R N I N G E N A B L E S …
  63. YOUR DRIVING

  64. “I wish my significant other understood me as well as

    Discover Weekly.” YOUR MUSIC TASTES
  65. EVERYTHING YOU BUY

  66. BUILD PRODUCTS THAT HELP BUSINESSES BE SUCCESSFUL SMB/ENTERPRISE M A

    C H I N E L E A R N I N G E N A B L E S …
  67. YOUR WORKFLOW

  68. EVERYTHING WE KNOW ABOUT SUCCESSFUL WEBSITES EVERYTHING WE KNOW ABOUT

    YOUR BUSINESS GOALS BUILD A SUCCESSFUL BUSINESS ONLINE
  69. twitter: @ingachen THANK YOU!