Visual Pipelines for Text Analysis

C93e0512fbfca1b61a9913bfceeac7ec?s=47 Data Intelligence
June 28, 2017

Visual Pipelines for Text Analysis

Benjamin Bengfort, District Data Labs
Audience level: Intermediate
Topic area: Modeling
Employing machine learning in practice is half search, half expertise, and half blind luck. In this talk we will explore how to make the luck half less blind by using visual pipelines to steer model selection from raw input to operational prediction. We will look specifically at extending transformer pipelines with visualizers for sentiment analysis and topic modeling text corpora.


Data Intelligence

June 28, 2017


  1. Visual Pipelines for Text Analysis Benjamin Bengfort District Data Labs

  2. Visual Steering of Text ML Models. - Language Aware Data

    Products - Pipelines are essential to text models. - Yellowbrick for: - Topic Models - Sentiment Analysis
  3. Natural Language Understanding (AI) Models for semantic understanding, reasoning, and

    generation of natural languages for human-computer interaction. Computational Linguistics (NLP) Approaches to demonstrate how humans interpret and understand language and show how languages evolve. Language Aware Data Products are Not Necessarily
  4. Sidebar: An Academic Context input projection hidden layer output layer

    Language Models and Perplexity Information Extraction/Retrieval
  5. Text Models

  6. Instances = Documents or Utterances (no matter their size)

  7. Text as Features in Machine Learning Is “Bag of Words”

    a bad thing? - Discourse Features - Syntactic Features - Morphological Features - N-Grams - Grammar Extracted Phrases - Named Entities - Chunks
  8. Vectorization 0 at 2 bat 1 can 0 door 1

    echolocation 0 elephant 0 of 0 open 0 potato 2 see 0 she 1 sight 1 sneeze 0 studio 1 the 0 to 1 via 0 w onder The elephant sneezed at the sight of potatoes. Bats can see via echolocation. See the bat sight sneeze! Wondering, she opened the door to the studio.
  9. High Dimensional Space! Baleen sample contains 2,021 files in 6

    categories. Structured as: 36,816 paragraphs (18.217 mean paragraphs per file) 61,597 sentences (1.673 mean sentences per paragraph). Word count of 1,365,829 with a vocabulary of 51,227 (26.662 lexical diversity). Corpus scan took 2.7726128101348877 seconds. Reductions: - Lemmatization, Stemming - Truncation - Dimensionality Reduction - Embeddings
  10. We’re doing feature engineering! pipeline = Pipeline([ ('vect', CountVectorizer()), ('tfidf',

    TfidfTransformer()), ('svd', TruncatedSVD()), ('model', SGDClassifier()), ]) parameters = { 'vect__max_df': (0.5, 0.75, 1.0), 'vect__max_features': (None, 5000, 10000), 'tfidf__use_idf': (True, False), 'tfidf__norm': ('l1', 'l2'), 'svd__n_components': (500, 1000, 2000, 5000), 'model__alpha': (0.00001, 0.000001), 'model__penalty': ('l2', 'elasticnet'), } search = GridSearchCV(pipeline, parameters), y)
  11. Data Loader Text Normalization Text Vectorization Feature Decomposition Estimator Data

    Loader Feature Union Pipeline Estimator Text Normalization Document Features Text Extraction Summary Vectorization Article Vectorization Concept Features Metadata Features Dict Vectorizer Text Pipelines
  12. - Naive Bayes (FAST) - Maximum Entropy (Logistic Regression) -

    SVMs (usually linear) or SGD - RBMs - Perceptrons Commonly Used Models - Hidden Markov Models - LDA/LSA - DBSCAN/Birch - KMeans
  13. The Model Selection Triple Arun Kumar Feature Analysis Algorithm

    Selection Hyperparameter Tuning
  14. Visual Steering

  15. The basic text visualization is … from yellowbrick.text import FrequencyVisualizer

    from sklearn.feature_extraction.text import CountVectorizer vect = CountVectorizer() docs = vect.fit_transform(docs) viz = FrequencyVisualizer( features=vect.get_feature_names() ) viz.poof()
  16. J. Heer and B. Shneiderman, “Interactive dynamics for visual analysis,”

    Queue, vol. 10, no. 2, p. 30, 2012. The Visual Analytics Mantra Overview First Zoom and Filter Details on Demand
  17. Conditional Frequency by Label from yellowbrick.text import FrequencyVisualizer from sklearn.feature_extraction.text

    import CountVectorizer vect = CountVectorizer() docs = vect.fit_transform(docs) viz = FrequencyVisualizer( features=vect.get_feature_names() ), labels) viz.poof()
  18. “TED Word Flows” Santiago Ortiz

  19. Topic Modeling

  20. Topic Modeling Training Text, Documents Feature Vectors Clustering Algorithm New

    Document Feature Vector Topic A Topic B Topic C Similarity
  21. Topic #0 â jim abenomics japan japanese rickards institution reckoning

    q please Topic #1 favoriting weidenhammer udacity daydream allo marks thrun raby seo aquafaba Topic #2 kearny secaucus crowhurst 2015â stringer loopholes skylineâ billowing newark hoboken Topic #3 megayachts mischa lovekin superyacht cw_anderson enchanted flickrallen sevigny zeppelin chloe Topic #4 yotel hubspot unplugged disrupted benioff maranhã lenã maranhenses vitormarigoduring quora ... Topic #10 github zube dewalt combinator jira scrum 42floors zendesk logins zenhub The Basic Topic Model - LDA from sklearn.feature_extraction.text import TfidfVectorizer from sklearn.decomposition import LatentDirichletAllocation as LDA model = Pipeline([ ('norm', TextNormalizer()), ('tfidf', TfidfVectorizer( tokenizer=identity, preprocessor=None, lowercase=False )), ('lda', LDA(n_topics=50)), ])
  22. Topic feature importance distributions!

  23. Visualizing Topic Space - Projections TF-IDF Vectors Frequency Vectors

  24. Comparing Topics: PyLDAViz for Interactive Exploration

  25. Visualizing Document Space - TSNE from yellowbrick.text import TSNEVisualizer vect

    = Pipeline([ ('norm', TextNormalizer()), ('tfidf', TfidfVectorizer( tokenizer=identity, preprocessor=None, lowercase=False )), ]) docs = vect.fit_transform(docs) tsne = TSNEVisualizer(), labels) tsne.poof()
  26. Spherical Topics with K-Means Text is typically very sparse. However,

    with a different distance metric that’s spherical, we can regain density: - Cosine - Jaccard - Soundex Opening up the possibility to using other clustering algorithms effectively.
  27. Selecting K: Elbow Curves from sklearn.cluster import MiniBatchKMeans from yellowbrick.cluster

    import KElbowVisualizer viz = KElbowVisualizer( MiniBatchKMeans(), k=20 ) viz.poof()
  28. Selecting K - Silhouette Scores from yellowbrick.cluster import SilhouetteVisualizer viz

    = SilhouetteVisualizer(KMeans(12)) viz.poof()
  29. Sentiment Analysis

  30. Visualizing Tokenization and Segmentation

  31. Sentiment Analysis Training Instances Training Labels Feature Vectors Classification Algorithm

    New Instance Feature Vector Predictive Model Predicted Label
  32. “Edging is one of those necessary evils if you want

    a great looking house. Blistered hands and time wasted, I decided there had to be a better way and this is surely it.”
  33. Part of Speech Tagging I used to use a really

    primitive manual edger that I inherited from my father . Blistered hands and time wasted , I decided there had to be a better way and this is surely it.Edging is one of those necessary evils if you want a great looking house . I do n't edge every time I mow . Usually I do it every other time . The first time out after a long winter , edging usually takes a little longer . After that , edging is a snap because you are basically in maintanence mode.I also use this around my landscaping and flower beds with equally great results.The blade on the Edge Hog is easily replaceable and the tell tale sign to replace it is when the edge starts to look a little rough and the machine seems slower . import nltk tokens = list( nltk.pos_tag( nltk.word_tokenize(text) )) viz = PosTagVisualizer() print(viz.transform(tokens))
  34. Syntax Analysis Edging is one of those necessary evils if

    you want a great looking house. NNP VBZ CD IN DT JJ NNS IN PRP VBP DT NN JJ VBG NP AD JP NP PP NP NP VP VP NP SB AR S
  35. Class Balance from yellowbrick.classifier import ClassBalance model = MultinomialNB() visualizer

    = ClassBalance(model), y_train) visualizer.score(X_test, y_test) visualizer.poof()
  36. Classification Reports

  37. Confusion Matrices

  38. ROC/AUC Curves

  39. Yellowbrick

  40. Yellowbrick at a Glance • Extend the Scikit-Learn API. •

    Enhance the model selection process. • Tools for feature visualization, visual diagnostics, and visual steering. • Visualize the model space
  41. The Scikit-Learn API Buitinck, Lars, et al. "API design for

    machine learning software: experiences from the scikit-learn project." arXiv preprint arXiv:1309.0238 (2013). class Estimator(object): def fit(self, X, y=None): """ Fits estimator to data. """ # set state of self return self def predict(self, X): """ Predict response of X """ # compute predictions pred return pred class Transformer(Estimator): def transform(self, X): """ Transforms the input data. """ # transform X to X_prime return X_prime class Pipeline(Transfomer): @property def named_steps(self): """ Returns a sequence of estimators """ return self.steps @property def _final_estimator(self): """ Terminating estimator """ return self.steps[-1]
  42. The Matplotlib API The matplotlib API combines two primary components:

    - pyplot: procedural descriptions of visualization. - artists: object oriented construction of visual elements. Drawing occurs globally and renders on demand.
  43. The trick: combine functional/procedural matplotlib + object-oriented Scikit-Learn. Yellowbrick

  44. Visualizers A visualizer is an estimator that produces visualizations based

    on data rather than new datasets or predictions. Visualizers are intended to work in concert with Transformers and Estimators to allow human insight into the modeling process. class Visualizer(Estimator): def draw(self): """ Draw the data """ def finalize(self): """ Complete the figure """ def poof(self): """ Show the figure """
  45. Scikit-Learn Pipelines: fit() and predict() Data Loader Transformer Transformer Estimator

    Data Loader Transformer Transformer Estimator Transformer
  46. Yellowbrick Visual Pipelines Data Loader Transformer(s) Feature Visualization Estimator fit()

    draw() predict() Data Loader Transformer(s) EstimatorCV Evaluation Visualization fit() predict() score() draw()
  47. Model Selection Pipelines Multi-Estimator Visualization Data Loader Transformer(s) Estimator Estimator

    Estimator Estimator Cross Validation Cross Validation Cross Validation Cross Validation
  48. Contributions Welcome!