Topic Modelling on Customer Reviews

A look at how N Brown approached classifying their Customer Reviews, whilst also taking us the company's data science journey.

Transcript

1. Topic Modelling on Customer Reviews Chris Billingham – N Brown Group
2. Who we are 2 Data Scientist and Analytical Business Partner Joined N Brown in 2016 Previous worked at Openreach Had a non-traditional Data Science upbringing
3. Background – Customer Pulse 3 “We need a better way of understanding what our customers are saying” “Excellent experience shopping both online and on the phone.” – JD Williams Customer “Order arrived very promptly, items were as described and pictured. Would order again.” – Simply Be Customer “I used “Joke a mo” to buy a pair of jeans and a polo shirt…” – Jacamo Customer
4. Customer Pulse – The Word Climate 4 What are people talking about this week? Positive Negative Rank Word Frequency WOW 1 product 18 61% 2 great 21 50% 3 pay 14 25% 4 wait 20 19% 5 quickly 16 14% 6 service 44 12% 7 stock 36 12% 8 experience 20 10% Rank Word Frequency WOW 1 dress 58 110% 2 shoes 51 107% 3 date 21 77% 4 friends 16 47% 5 jacamo 32 47% 6 things 20 45% 7 class 17 44% 8 comfortable 21 42%
5. The Word Climate – A Code Retrospective 5
6. The Word Climate – A Code Retrospective 6
7. Customer Pulse – What we learned 7 It (mostly) worked It only worked on my computer I didn’t entirely understand what it was doing It needed some manual finessing each week Other people want to know R The business was hungry for more detail
8. Customer Pulse – What happened next 8 “This is all well and good but what kind of things are our customer’s talking about?” “This code is a mess, why can’t other people run it? Hey what’s this tidytext thing?”
9. When Bethan met Julia 9 X
10. “The Jane Austen Epidemic” 10 “But every time I turn R on bloody Jane Austen is there.” “This seems much easier, I think we can use this over the tm package.”
11. “The Jane Austen Epidemic” 11
12. The N Brown R User Group 12 Bi Monthly or so Great opportunity to share “Analysts Anonymous” sessions An opportunity to berate people into using projects Cross-business
13. LDA or Topic Modelling – A Primer 13
14. LDA or Topic Modelling – A Primer of the Primer 14
15. “The Progress Bar” 15
16. stm – The Structure Topic Model 16
17. stm – The Structural Topic Model 17 Most importantly…
18. stm – First iteration 18
19. stm – The need for version control 19
20. N Brown Data Science x git & bitbucket 20
21. stm – The second iteration 21
22. Topic Modelling in the Cloud 22
23. stm – Cloud Craziness 23
24. stm – Cloud Craziness 24
26. Reflection 26 “I should really understand everything the package can do”
27. The Eureka moment 27
28. Removing Low Frequency Tokens 28
29. Topic Modelling Success 29
30. Topic Modelling – Hand Review – The Good 30
31. Topic Modelling – Hand Review – The Bad 31
32. Our Network of Topics by Correlation 32
33. Topic “Coherence” 33 Uniqueness of terms in topic How well entire topic holds together
34. Our Topics over Time 34
35. Our Detailed Topics over Time 35
36. What we learned about Topic Models 36 You should always, always properly read the manual You need to spend the time tuning your available tokens to get a coherent model The stm package is a fantastic tool for generating and assessing good quality models, particularly when you have accompanying metadata Progress bars are good Some of your topics are just going to be wonky, its down to you to minimise those At some point you’re going to go manual and hand classify
37. …and our Data Science Team 37 We now have a bi-monthly (or so) R User Group hosted by the team R is the primary tool of choice (we have acquired some Pythonistas as well) Use Projects (mostly) Are using Version Control (mostly) Currently in the process of building out an on-prem Data Science server for those bigger workloads
38. 38 Thank You @yobrenoops chris-billingham