jgs SER 594 Software Engineering for Machine Learning Lecture 17: Text Mining Dr. Javier Gonzalez-Sanchez [email protected] javiergs.engineering.asu.edu | javiergs.com PERALTA 230U Office Hours: By appointment
Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 5 jgs Text Mining Extracting information from text documents (natural language). § generate a search index; § text categorization into domains; § text clustering to organize a set of documents; § sentiment analysis to identify subjective information; § concept or entity extraction (people, places, organizations); § document summarization (identify important points); and § learning relations between named entities. § Spam detection in email messages, user comments, web pages, and so on
Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 6 jgs Topics Modeling § unsupervised technique that looks for patterns in a corpus of text – identify words (topics) that appear un a statistically meaningful way § analyze a large archive of text documents (blog post, an email, a tweet, a book chapter, etc.) and understand what the archive contains § The most well-known algorithm is Latent Dirichlet Allocation (LDA) 2003, Blei, Ng, & Jordan (available on Canvas)
Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 8 jgs Latent Dirichlet Allocation (LDA) § Latent: existing but not yet developed or manifest § Dirichlet, Peter Gustav Lejeune § Allocation: process of distributing something § It assumes that the author composed a piece of text by selecting words from possible baskets of words, where each basket corresponds to a topic. § It represents documents as a mixture of topics. § It represents a topic is a mixture of words. § If a word w has high probability of being in a topic t, all the documents having w will be more strongly associated with t as well. §
Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 12 jgs Algorithm | assumptions § Words like am/is/are/of/a/the/but/… don’t carry any information about the “topics” are eliminated from the documents as a preprocessing step. § Words that occur in >80% of the documents can be eliminated without losing any information. § The number of topics we have or want to is pre-defined. § Order of the words and the grammatical role of the words (subject, object, verbs, …) are not considered in the model.
Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 13 jgs Algorithm Randomly assign each word w in each document d to one of k topics For each document d, For each word w and compute : How many words belong to the topic t for a given document d. Excluding the current word. If a lot of words from d belongs to t, it is more probable that word w belongs to t. p(topic t | document d) the proportion of words in document d that are assigned to topic t. How many documents are in topic t because of word w. p(word w| topic t) the proportion of assignments to topic t over all documents that come from this word w. Update the probability for the word w belonging to topic t, p(word w with topic t) = p(topic t | document d) * p(word w | topic t)
Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 15 jgs Tools https://www.linuxlinks.com/excellent-java-natural-language-processing-tools/ (September 2019)
Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 16 jgs Library § Mallet (MAchine Learning for Language Toolkit), a Java-based package for statistical natural-language processing, document classification, clustering, topic modeling, information extraction, and other machine-learning applications to text. § Andrew McCallum, University of Massachusetts Amherst, 2002)
Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 19 jgs Data | BBC News Dataset § Raw text files § Consists of 2225 documents from the BBC news website corresponding to stories in five topical areas from 2004-2005. § Class Labels: 5 (business, entertainment, politics, sport, tech) § http://mlg.ucd.ie/datasets/bbc.html
Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 21 jgs Import Data for Training File iterator cc.mallet.pipe.iterator.FileIterator FileIterator iterator = new FileIterator( § A list of File[] directories with text files § A file filter that specifies which files to select within a directory § A pattern that is applied to a filename to produce a class label )
Javier Gonzalez-Sanchez | SER 594 | Spring 2022 | 22 jgs Import Data import cc.mallet.pipe.iterator.FileIterator; import java.io.File; public class Main { public static void main(String[] args) { File[] directories = new File[1]; directories[0] = new File ("/src/data/"); FileIterator iterator = new FileIterator( directories, new TextFileFilter(), FileIterator.LAST_DIRECTORY ); } } import java.io.File; import java.io.FileFilter; public class TextFileFilter implements FileFilter { @Override public boolean accept(File file) { return file.toString().endsWith(".txt"); } }
jgs SER 594 Software Engineering for Machine Learning Javier Gonzalez-Sanchez, Ph.D. [email protected] Spring 2022 Copyright. These slides can only be used as study material for the class CSE205 at Arizona State University. They cannot be distributed or used for another purpose.
javiergs
mugi_uno
okuzawats
afilina
nano72mkn
toricky6
knr109
texmeijin
mackey0225
zerebom
deepu105
pospome
ivargrimstad
tmm1
jrom
matthewcrist
davidbonilla
myddelton
addyosmani
morganepeng
jennybc
holman
eileencodes
sstephenson