Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Do Angry People Have Poor Grammar? An Exploration of Language Processing and Statistics in Python

Do Angry People Have Poor Grammar? An Exploration of Language Processing and Statistics in Python

This talk is about two things: natural language processing (NLP) and statistical dependence. We will embark on a data science workflow using various python scientific computing tools to better understand the behavior of commenters on Reddit. To do this we'll go through an introduction to sentiment analysis in Python (mostly using NLTK) and a swift explanation of the statistics of variable dependence.

We'll couple these freshly learned methods with an excellent dataset for this domain: every public reddit comment. We'll talk a bit about handling and preprocessing data of this size and character. Then we'll compile scores for both sentiment and spelling/grammar. In the end we may just discover if angry comment are also grammatically poor comments. And the audience will walk away a few more tools in scientific computing toolbelt.

Deck as presented at PyData Amsterdam 2016

77cacd503a0b6641e3b7a2e26dbbbaa1?s=128

Ben Fields

March 12, 2016
Tweet

Transcript

  1. Do Angry People Have Poor Grammar? Ben Fields

  2. An Exploration of Language Processing and Statistics in Python Ben

    Fields
  3. intro and motivation

  4. @alsothings - Do Angry People Have Poor Grammar? Have you

    ever noticed on social media
  5. @alsothings - Do Angry People Have Poor Grammar? that the

    ‘loudest’
  6. @alsothings - Do Angry People Have Poor Grammar? seem to

    not construct the best sentences?
  7. @alsothings - Do Angry People Have Poor Grammar? me too.

  8. @alsothings - Do Angry People Have Poor Grammar? But I

    am a skeptic
  9. @alsothings - Do Angry People Have Poor Grammar? http://dx.doi.org/10.1037/1089-2680.2.2.175

  10. @alsothings - Do Angry People Have Poor Grammar? So lets

    do some analysis
  11. @alsothings - Do Angry People Have Poor Grammar? 1. Find

    pile of comments 2. Measure style and sentiment 3. ???? 4. Profit
  12. @alsothings - Do Angry People Have Poor Grammar? 1. Find

    pile of comments 2. Measure style and sentiment 3. Statistical dependance? 4. Profit
  13. @alsothings - Do Angry People Have Poor Grammar? 1. Find

    pile of comments 2. Measure style and sentiment 3. Statistical dependance? 4. more twitter followers
  14. 1. Find pile of comments

  15. @alsothings - Do Angry People Have Poor Grammar? all of

    the reddit comments!
  16. @alsothings - Do Angry People Have Poor Grammar? https://www.reddit.com/r/datasets/comments/3bxlg7/ i_have_every_publicly_available_reddit_comment

  17. @alsothings - Do Angry People Have Poor Grammar? 1.7 Trillion

    reddit comments!
  18. @alsothings - Do Angry People Have Poor Grammar? 59 Million

    reddit comments! https://mega.nz/#!ysBWXRqK!yPXLr25PgJi184pbJU3GtnqUY4wG7YvuPpxJjEmnb9A
  19. @alsothings - Do Angry People Have Poor Grammar? (actually ~1%

    sample of that: 390k comments)
  20. @alsothings - Do Angry People Have Poor Grammar? from nltk

    import tokenize … for comment in pile_of_comment: num_tokens = len(filter(lambda t:t not in punctuation, tokenize.word_tokenize(comment['body']))) if num_tokens < 5: continue
  21. @alsothings - Do Angry People Have Poor Grammar? 319K comments

    from January 2015
  22. 2. Measure style and sentiment

  23. @alsothings - Do Angry People Have Poor Grammar? grammar?

  24. @alsothings - Do Angry People Have Poor Grammar? Context-Free Grammar!

    http://www.nltk.org/book/ch08.html
  25. @alsothings - Do Angry People Have Poor Grammar? Great for

    analysing complex and ambiguous sentence structure http://www.nltk.org/book/ch08.html
  26. @alsothings - Do Angry People Have Poor Grammar? http://www.imdb.com/title/tt0020640/

  27. @alsothings - Do Angry People Have Poor Grammar? “One morning,

    I shot an elephant in my pyjamas. How he got in my pyjamas, I don't know.” http://www.nltk.org/book/ch08.html
  28. @alsothings - Do Angry People Have Poor Grammar? “I shot

    an elephant in my pyjamas.” http://www.nltk.org/book/ch08.html
  29. @alsothings - Do Angry People Have Poor Grammar? S ->

    NP VP PP -> P NP NP -> Det N | Det N PP | 'I' VP -> V NP | VP PP Det -> 'an' | 'my' N -> 'elephant' | 'pyjamas' V -> 'shot' P -> 'in' http://www.nltk.org/book/ch08.html
  30. @alsothings - Do Angry People Have Poor Grammar? groucho_grammar =

    nltk.CFG.fromstring(from_last_slide) sent = ['I', 'shot', 'an', 'elephant', 'in', 'my', 'pyjamas'] parser = nltk.ChartParser(groucho_grammar) for tree in parser.parse(sent): print(tree) http://www.nltk.org/book/ch08.html
  31. @alsothings - Do Angry People Have Poor Grammar? (S (NP

    I) (VP (VP (V shot) (NP (Det an) (N elephant))) (PP (P in) (NP (Det my) (N pyjamas))))) (S (NP I) (VP (V shot) (NP (Det an) (N elephant) (PP (P in) (NP (Det my) (N pajamas)))))) http://www.nltk.org/book/ch08.html
  32. @alsothings - Do Angry People Have Poor Grammar? http://www.nltk.org/book/ch08.html S

    S NP VP NP VP VP PP V NP V NP P NP Det N Det N Det N PP P NP Det N I shot an elephant in my pyjamas I shot an elephant in my pyjamas
  33. @alsothings - Do Angry People Have Poor Grammar? S ->

    NP VP PP -> P NP NP -> Det N | Det N PP | 'I' VP -> V NP | VP PP Det -> 'an' | 'my' N -> 'elephant' | 'pyjamas' V -> 'shot' P -> 'in' http://www.nltk.org/book/ch08.html
  34. @alsothings - Do Angry People Have Poor Grammar? Extended ad

    nauseum: a definition of english
  35. @alsothings - Do Angry People Have Poor Grammar? Extended ad

    nauseum: a static simplification of english
  36. @alsothings - Do Angry People Have Poor Grammar? pCFG

  37. @alsothings - Do Angry People Have Poor Grammar? probabilistic CFG

  38. @alsothings - Do Angry People Have Poor Grammar? “The main

    problem is that there is no common agreement on what are grammatically correct (English) sentences; nor has anyone yet been able to offer a grammar precise enough to propose as definitive.” http://dl.acm.org/citation.cfm?id=1882777
  39. @alsothings - Do Angry People Have Poor Grammar? Style checking!

  40. @alsothings - Do Angry People Have Poor Grammar? lint for

    prose
  41. @alsothings - Do Angry People Have Poor Grammar? proselint.com

  42. @alsothings - Do Angry People Have Poor Grammar? proselint.com/write

  43. @alsothings - Do Angry People Have Poor Grammar? https://github.com/amperser/proselint/ {

    "max_errors": 1000, "checks": { "butterick.symbols" : true, "carlin.filth" : true, "consistency.spacing" : true, "consistency.spelling" : true, "garner.airlinese" : true, … "inc.corporate_speak" : true, "leonard.exclamation" : true, "leonard.hell" : true, … "write_good.weasel_words" : true, "wsj.athletes" : true } }
  44. @alsothings - Do Angry People Have Poor Grammar? https://github.com/amperser/proselint/ (from

    checks/leonard/exclamation.py) @memoize def check_repeated_exclamations(text): """Check the text.""" err = "leonard.exclamation.multiple" msg = u"Stop yelling. Keep your exclamation points under control." regex = r"[^A-Z]\b((\s[A-Z]+){3,})" return existence_check( text, [regex], err, msg, require_padding=False, ignore_case=False,max_errors=1, dotall=True)
  45. @alsothings - Do Angry People Have Poor Grammar? mean std

    dev min 25% 50% 75% max lints 0.4298 1.0861 0 0 0 1 400 normed lints 0.0216 0.0497 0 0 0 0.0178 1.2
  46. sentiment analysis http://www.nltk.org/howto/sentiment.html

  47. @alsothings - Do Angry People Have Poor Grammar?

  48. @alsothings - Do Angry People Have Poor Grammar? lul or

    luuuuuuuuuuuulz?
  49. @alsothings - Do Angry People Have Poor Grammar? VADER (Valence

    Aware Dictionary for sEntiment Reasoning) http://comp.social.gatech.edu/papers/icwsm14.vader.hutto.pdf
  50. @alsothings - Do Angry People Have Poor Grammar? Test Condition

    Example Text Baseline Yay. Another good phone interview. Punctuation1 Yay! Another good phone interview! Punctuation1 + Degree Mod. Yay! Another extremely good phone interview! Punctuation2 Yay!! Another good phone interview!! Capitalization YAY. Another GOOD phone interview. Punct1 + Cap. YAY! Another GOOD phone interview! Punct2 + Cap. YAY!! Another GOOD phone interview!! Punct3 + Cap. YAY!!! Another GOOD phone interview!!! Punct3 + Cap. + Degree Mod. YAY!!! Another EXTREMELY GOOD phone in- terview!!! Table 2: Example of baseline text with eight test conditions com- prised of grammatical and syntactical variations. post synt diffe 2. Mov from from tive the tenc 3. Tec leve prod (200 4. Opi http://comp.social.gatech.edu/papers/icwsm14.vader.hutto.pdf
  51. @alsothings - Do Angry People Have Poor Grammar? mean std

    dev min 25% 50% 75% max positive 0.14381 0.15702 0 0 0.10800 0.22900 1 neutral 0.776917 0.175849 0 0.667000 0.789000 0.915000 1 0.079243 0.079243 0 0 0 0.128000 1
  52. 3. Statistical dependance?

  53. @alsothings - Do Angry People Have Poor Grammar? y =

    f ( x )
  54. @alsothings - Do Angry People Have Poor Grammar? regression! http://comp.social.gatech.edu/papers/icwsm14.vader.hutto.pdf

  55. @alsothings - Do Angry People Have Poor Grammar?

  56. @alsothings - Do Angry People Have Poor Grammar? http://comp.social.gatech.edu/papers/icwsm14.vader.hutto.pdf S

    = n X i=1 ri 2
  57. @alsothings - Do Angry People Have Poor Grammar?

  58. @alsothings - Do Angry People Have Poor Grammar? correlation testing

  59. @alsothings - Do Angry People Have Poor Grammar? Pearson’s negative

    to correlation p lints 0.0311 4.535E-249 normed lints 0.0600 2.4228E-68
  60. @alsothings - Do Angry People Have Poor Grammar? Nope.

  61. @alsothings - Do Angry People Have Poor Grammar? NOPE.

  62. conclusions!

  63. @alsothings - Do Angry People Have Poor Grammar? People on

    reddit are generally reasonable,
  64. @alsothings - Do Angry People Have Poor Grammar? but when

    they aren’t, the language isn’t any stylistically worse
  65. @alsothings - Do Angry People Have Poor Grammar? at least

    last January.
  66. Let’s have some questions !/alsothings