What understood about that we've used LUIS through the year

What understood that we've used LUIS through the year.

Transcript

1. What understood about that
   we’ve used LUIS through the year
   Shinichi Tanabe
   January 12, 2018 Minami Aoyama Night #5
   

   View Slide

2. Speaker
   Shinichi Tanabe
   　 (田邊 晋一/たなべ しんいち)
   ● NAVITIME JAPAN Co., Ltd.
   ○ Joined in 2008
   ○ Cogbot project
   ○ Programmer
   

   View Slide

3. Products
   

   View Slide

4. View Slide

5. Encounter
   

   View Slide

6. September 15, 2016
   

   View Slide

7. View Slide

8. View Slide

9. First impression
   

   View Slide

10. Easy to use, runs fast and smart.
    

    View Slide

11. Easy to use
    

    View Slide

12. Let’s go to the portal site!
    https://www.luis.ai
    

    View Slide

13. Step1. Create new app
    

    View Slide

14. View Slide

15. Step2. Add intent
    

    View Slide

16. View Slide

17. Step3. Add utterances
    

    View Slide

18. View Slide

19. Step4. Add entities
    

    View Slide

20. View Slide

21. View Slide

22. Step5. Train
    

    View Slide

23. View Slide

24. Step6. Test
    

    View Slide

25. View Slide

26. View Slide

27. View Slide

28. View Slide

29. View Slide

30. Step7. Publish
    

    View Slide

31. View Slide

32. View Slide

33. That’s all!
    

    View Slide

34. Furthermore...
    

    View Slide

35. You can get a happy bonus.
    

    View Slide

36. Versioning
    

    View Slide

37. View Slide

38. View Slide

39. View Slide

40. View Slide

41. Runs fast and smart
    

    View Slide

42. Comparison between
    and
    LUIS Dialogflow
    

    View Slide

43. Test model
    

    View Slide

44. Test model
    Intent Places.FindPlace
    Utterances おいしいカレーが食べたいな
    どこか近くでおすすめのレストランを教えて
    Entities Cuisine カレー
    PlaceType レストラン
    

    View Slide

45. Training speed
    

    View Slide

46. LUIS 2 - 4 sec
    Dialogflow 4 - 8 sec
    The training speed result of test model
    

    View Slide

47. Precision and recall
    

    View Slide

48. Test utterance
    LUIS Dialogflow
    Intent Entity Intent Entity
    おいしいカレーが食べたいな 〇 〇 〇 〇
    どこか近くでおすすめのレストラ
    ンを教えて
    〇 〇 〇 〇
    Precision result of test model
    

    View Slide

49. Test utterance
    LUIS Dialogflow
    Intent Entity Intent Entity
    おすすめのバーを教えて 〇 〇 × ×
    おすすめのバー教えて 〇 × × ×
    おいしいうどんが食べたい 〇 〇 〇 ×
    おいしいうどん食べたい 〇 × 〇 ×
    Recall result of test model
    

    View Slide

50. Yes, he was perfect!
    

    View Slide

51. Getting started
    

    View Slide

52. But, we had some questions.
    

    View Slide

53. Questions
    1. How should we defines intents and entities?
    2. How do we know accuracy and precision?
    3. When will he go GA(General Availability)?
    

    View Slide

54. 1. How should we defines intents and entities?
    

    View Slide

55. Anti pattern
    Utterance : Intent ≒ 1 : 1
    

    View Slide

56. Use or copy pre-build model positively.
    

    View Slide

57. View Slide

58. View Slide

59. 2. How do we know accuracy and precision?
    

    View Slide

60. Comprehensive test on model
    

    View Slide

61. Batch testing
    

    View Slide

62. Test result details in a visualized view.
    

    View Slide

63. Error matrix
    

    View Slide

64. True positive
    True negative
    Green zone indicates correct prediction
    

    View Slide

65. False negative
    False positive
    Red zone indicates incorrect prediction
    

    View Slide

66. 3. When will he go GA?
    

    View Slide

67. LUIS is now GA!!
    

    View Slide

68. How to get along with LUIS
    

    View Slide

69. Points
    1. Start small model which has few intents.
    2. Use or copy pre-build model positively.
    3. Raise requests before do something about that yourself.
    

    View Slide

70. Thank you!
    

    View Slide