Machine Learning from a Product Perspective

2b8d8955a6215697a49d8c443154cb01?s=47 Inga Chen
September 08, 2017

Machine Learning from a Product Perspective

MACHINE LEARNING FROM A PRODUCT PERSPECTIVE

The trends of big data and cost-effective computing power have given way to an unprecedented focus on developing machine learning algorithms and open source tools, making machine learning more accessible and powerful today than ever. But building machine learning products is about much more than picking the right algorithm or library. It's a product manager's job to use machine learning for the right problems, and to make sure the end user experience solves those problems and fosters user trust. This talk will illustrate machine learning concepts through applications in products, provide a framework for determining problems that are good for machine learning to solve, and talk about how the product development cycle for machine learning-enabled products might differ from traditional software.

Agenda:
- What is ML?
- What problems are good for ML?
- What should product managers bring to the table?
- ML product development cycle
- Why machine learning now?

2b8d8955a6215697a49d8c443154cb01?s=128

Inga Chen

September 08, 2017
Tweet

Transcript

  1. 2.

    @ I N G A C H E N S

    I L I C O N VA L L E Y N AT I V E D A U G H T E R O F S O F T WA R E + H A R D WA R E E N G I N E E R S I N V E S T M E N T B A N K E R T E C H M & A J E F F E R I E S S I L I C O N VA L L E Y E A R LY B U S I N E S S H I R E I o T S TA R T U P F O R C A R S A U T O M AT I C L A B S S A N F R A N C I S C O P R O D U C T M A N A G E R D ATA I N S I G H T S & F L E E T A U T O M AT I C L A B S S A N F R A N C I S C O P R O D U C T M A N A G E R A N A LY T I C S S Q U A R E S PA C E N E W YO R K C I T Y
  2. 3.

    Millions of miles of driving Millions of websites Slice of

    the internet Drivers Fuel efficiency Car Maintenance SMBs Online presence and business
  3. 4.

    Millions of miles of driving Millions of websites Slice of

    the internet Create an optimized experience for each user MACHINE LEARNING
  4. 5.

    B E YO N D T H E A L

    G O R I T H M : B U I L D I N G M L P R O D U C T S W H AT S H O U L D P R O D U C T P E O P L E B R I N G T O T H E TA B L E ? B A C KG R O U N D : W H AT I S M L? W H AT P R O B L E M S A R E G O O D F O R M L? W H Y N O W MACHINE LEARNING FROM A PRODUCT PERSPECTIVE
  5. 6.

    B E YO N D T H E A L

    G O R I T H M : B U I L D I N G M L P R O D U C T S W H AT P R O B L E M S A R E G O O D F O R M L? W H Y N O W MACHINE LEARNING FROM A PRODUCT PERSPECTIVE B A C KG R O U N D : W H AT I S M L? W H AT S H O U L D P R O D U C T P E O P L E B R I N G T O T H E TA B L E ?
  6. 8.

    GOALS OF AI: Reasoning Knowledge Planning Learning Natural Language Processing

    Perception Move & manipulate objects Artificial Intelligence Human Intelligence Exhibited by Machines
  7. 9.

    “[Machine Learning is the] field of study that gives computers

    the ability to learn without being explicitly programmed Arthur Samuel, 1959
  8. 10.

    “[Machine Learning is the] field of study that gives computers

    the ability to learn without being explicitly programmed (often more data than is humanly possible to go through) and makes a determination or prediction about something in the world. by observing data
  9. 11.

    H O W D O E S M A C

    H I N E L E A R N I N G W O R K ?
  10. 16.

    B E YO N D T H E A L

    G O R I T H M : B U I L D I N G M L P R O D U C T S B A C KG R O U N D : W H AT I S M L? W H AT P R O B L E M S A R E G O O D F O R M L? W H Y N O W MACHINE LEARNING FROM A PRODUCT PERSPECTIVE W H AT S H O U L D P R O D U C T P E O P L E B R I N G T O T H E TA B L E ?
  11. 17.

    D E C I S I O N S W

    I T H M A N Y I N P U T S and it’s unclear how to combine them W H AT P R O B L E M S A R E G O O D F O R M A C H I N E L E A R N I N G ?
  12. 18.

    W H O W I L L W I N

    T H E E L E C T I O N ? Hillary vs Trump Education Income level Race Gender Geographic location Religion What car you drive
  13. 19.
  14. 20.
  15. 21.

    D E C I S I O N S W

    I T H M A N Y O U T P U T S and it’s infeasible to show all outputs W H AT P R O B L E M S A R E G O O D F O R M A C H I N E L E A R N I N G ?
  16. 22.

    S L A C K H I G H L

    I G H T S M A N A G E T H E D E L U G E O F W O R K M E S S A G E S
  17. 23.

    M A K I N G D E C I

    S I O N S AT S C A L E for millions/billions of items or users W H AT P R O B L E M S A R E G O O D F O R M A C H I N E L E A R N I N G ?
  18. 24.

    ML MODEL OUTPUT PREDICTED LABELS CONFIDENCE % CAT CAT NOT

    NOT NOT SUPERVISED LEARNING CAT NOT CAT CAT NOT TRAINING SET UNSEEN DATA INPUT VALIDATION SET I M A G E S E A R C H “ C AT ”
  19. 25.

    D E C I S I O N S W

    I T H M A N Y I N P U T S D E C I S I O N S W I T H M A N Y O U T P U T S M A K I N G D E C I S I O N S AT S C A L E W H AT P R O B L E M S A R E G O O D F O R M A C H I N E L E A R N I N G ?
  20. 26.

    PREDICT DISCOVER OPTIMIZE DETECT RANK AUTOMATE stock prices housing prices

    sales health outcomes pricing user experience (personalization) recommendations bank fraud fake news fractures road hazards search results newsfeed recommendations patterns trends customer segmentation human tasks not easily delineated by rules W H AT P R O B L E M S A R E G O O D F O R M A C H I N E L E A R N I N G ?
  21. 27.

    U N S U P E R V I S

    E D L E A R N I N G D I S C O V E R PAT T E R N S
  22. 28.

    P R O B L E M How should I

    improve my fuel efficiency based on my driving behavior?
  23. 29.
  24. 32.

    What are the biggest factors affecting fuel efficiency of our

    drivers? Aggression on highway Impatience on local roads Weather, terrain, traffic
  25. 34.
  26. 35.

    A U T O M AT E C O M

    P L E X TA S K S D E E P L E A R N I N G
  27. 36.
  28. 37.

    S E L F - D R I V I

    N G C A R S
  29. 38.

    P R O B L E M A slice of

    the internet: How do we build products for such a diverse user base?
  30. 39.
  31. 40.
  32. 41.
  33. 42.
  34. 43.

    S O L U T I O N VISUAL SEARCH

    ENGINE FOR WEBSITES P R O B L E M How do we build products for such a diverse user base?
  35. 44.
  36. 45.

    Deep Learning: Under the hood (SIMPLIFIED) Convolutional variational auto-encoder Recreate

    the image Find similarities Learns how to interpret the structure of an image hidden layers DEEP
  37. 46.
  38. 47.
  39. 48.

    B E YO N D T H E A L

    G O R I T H M : B U I L D I N G M L P R O D U C T S B A C KG R O U N D : W H AT I S M L? W H AT P R O B L E M S A R E G O O D F O R M L? W H Y N O W MACHINE LEARNING FROM A PRODUCT PERSPECTIVE W H AT S H O U L D P R O D U C T P E O P L E B R I N G T O T H E TA B L E ?
  40. 55.

    W H AT S H O U L D P

    R O D U C T P E O P L E B R I N G T O T H E TA B L E ? B A C KG R O U N D : W H AT I S M L? W H AT P R O B L E M S A R E G O O D F O R M L? W H Y N O W MACHINE LEARNING FROM A PRODUCT PERSPECTIVE B E YO N D T H E A L G O R I T H M : B U I L D I N G M L P R O D U C T S
  41. 56.

    PRIORITIZE BUILD LAUNCH LEARN MEASURE FOR SOFTWARE PRODUCTS P R

    O D U C T D E V E L O P M E N T C YC L E
  42. 57.

    CLASSIFY AS ML CONSIDER NON-ML GATHER & PREP DATA PRODUCT

    CHANGES? BUILD & EVALUATE MODELS THROW AWAY IDENTIFY MODEL WEAKNESSES COLLECT MORE DATA REBUILD & IMPROVE LAUNCH PRODUCTION MODEL ACCURACY % TRADEOFFS: ACCURACY VS COMPUTE POWER VS VISIBILITY FOR SOFTWARE PRODUCTS USING ML P R O D U C T D E V E L O P M E N T C YC L E PRIORITIZE BUILD LAUNCH LEARN MEASURE
  43. 58.

    INPUT OUTPUT MANY ML MODELS P R O D U

    C T S I N P R A C T I C E
  44. 59.

    B A C KG R O U N D :

    W H AT I S M L? B E YO N D T H E A L G O R I T H M : B U I L D I N G M L P R O D U C T S W H AT S H O U L D P R O D U C T P E O P L E B R I N G T O T H E TA B L E ? W H AT P R O B L E M S A R E G O O D F O R M L? W H Y N O W MACHINE LEARNING FROM A PRODUCT PERSPECTIVE L E A R N I N G F R O M D ATA W I T H O U T B E I N G E X P L I C I T LY P R O G R A M M E D M A N Y I N P U T S , M A N Y O U T P U T S M A K I N G D E C I S I O N S AT S C A L E D E S I G N F O R W H E N T H E M O D E L FA I L S L A U N C H , R E B U I L D & I M P R O V E . T R A D E O F F S
  45. 60.

    B E YO N D T H E A L

    G O R I T H M : B U I L D I N G M L P R O D U C T S W H AT YO U N E E D F O R M L B A C KG R O U N D : W H AT I S M L? W H AT P R O B L E M S A R E G O O D F O R M L? MACHINE LEARNING FROM A PRODUCT PERSPECTIVE W H Y N O W
  46. 61.

    MACHINE LEARNING PRODUCTS MATURING OF ML ALGORITHMS with open source

    tools and deep learning MASSIVE COMPUTE POWER Cloud computation GPU processing ACCESSIBILITY OF OPEN DATA CommonCrawl - petabytes ImageNet - 14M labeled C O N V E R G I N G T R E N D S … YOUR COMPETITORS ARE DOING IT NOT JUST FOR BIG COMPANIES EVEN INDIVIDUALS CAN DO IT
  47. 62.

    BUILD PRODUCTS THAT KNOW YOU BETTER THAN YOU KNOW YOURSELF

    CONSUMER M A C H I N E L E A R N I N G E N A B L E S …
  48. 64.

    “I wish my significant other understood me as well as

    Discover Weekly.” YOUR MUSIC TASTES
  49. 66.

    BUILD PRODUCTS THAT HELP BUSINESSES BE SUCCESSFUL SMB/ENTERPRISE M A

    C H I N E L E A R N I N G E N A B L E S …
  50. 68.

    EVERYTHING WE KNOW ABOUT SUCCESSFUL WEBSITES EVERYTHING WE KNOW ABOUT

    YOUR BUSINESS GOALS BUILD A SUCCESSFUL BUSINESS ONLINE