jgs Definition § Supervised learning is the machine learning task of learning a function that maps an input to an output based on example input-output pairs.
jgs Text Mining Extracting information from text documents (natural language). § generate a search index; § text categorization into domains; § text clustering to organize a set of documents; § sentiment analysis to identify subjective information; § concept or entity extraction (people, places, organizations); § document summarization (identify important points); and § learning relations between named entities. § Spam detection in email messages,
jgs Previously … § Text Mining as unsupervised technique that looks for patterns in a corpus of text – identify words (topics) that appear un a statistically meaningful way § analyze a large archive of text documents (blog post, an email, a tweet, a book chapter, etc.) and understand what the archive contains § The most well-known algorithm is Latent Dirichlet Allocation (LDA) 2003, Blei, Ng, & Jordan (available on Canvas)
jgs Library § Mallet (MAchine Learning for Language Toolkit), a Java-based package for statistical natural-language processing, document classification, clustering, topic modeling, information extraction, and other machine-learning applications to text. § Andrew McCallum, University of Massachusetts Amherst, 2002)
jgs Data | Email Spam Dataset § 2000, Androutsopoulos et al. (reorganized - 2010, Ng) § Pre-processed emails § Consists of 350 spam and 350 non-spam (for training) § Consists of 130 spam and 130 non-spam (for testing) § http://openclassroom.stanford.edu/ MainFolder/DocumentPage.php? course=MachineLearning&doc=exercises/ex6/ex6.html § Download ex6DataEmails.zip (the version that in NOT preproseced)
jgs Import Data for Training File iterator cc.mallet.pipe.iterator.FileIterator FileIterator iterator = new FileIterator( § A list of File[] directories with text files § A file filter that specifies which files to select within a directory § A pattern that is applied to a filename to produce a class label )
Ph.D. [email protected] Spring 2022 Copyright. These slides can only be used as study material for the class CSE205 at Arizona State University. They cannot be distributed or used for another purpose.
javiergs
hanon52_
manfredsteyer
dunglas
karszawa
whatyouhide
rkaga
yui_knk
cocoeyes02
alquist4121
shin1x1
gokana
itarutomy
sonatard
wjessup
kneath
lemiorhan
lara
paigeccino
shpigford
orderedlist
paulrobertlloyd
zakiyama
sachag
chriscoyier
