Insight Project

Insight Project for classifying websites

Transcript

1. Classifying the Web with GoGuardian Christopher Rivera
2. The Company: GoGuardian GoGuardian builds software to help schools and teachers monitor student web usage.
3. The Problem: Classify Websites The web is huge and their students have already visited over 14 million different host sites.
4. The “Data”: Go Guardian provided 2.6 million previously classified urls. I collected 50,000 and created a set of 22,752 for modeling.
5. Developing the Classifier: I trained Random Forests, Naive Bayes, and SVM classifiers. Urls Parse Text Random Forest HTML Processing Vectorize TFID Chi2 75% Train (17064) 10 Fold Cross Validation 25% Test (5688)
6. Random Forest Performance Test Accuracy: 62.5% Random Accuracy: 7.01%. 7.4 x Lift.
7. The classifier classifies my favorites: To validate, I looked at several websites manually.
8. The classifier classifies my favorites: To validate, I looked at several websites manually.
9. The classifier classifies my favorites: To validate, I looked at several websites manually.
10. Sites the Students Visit: 2 weeks of records for 600 students.
11. Christopher Rivera Biochemist/Systems Biologist 0.0 0.1 0.2 0.3 0.4 0 10 20 30 Day Relative Abundance Phylum Bacteroidetes Caldiserica Chloroflexi Euryarchaeota Firmicutes Proteobacteria Spirochaetes Tenericutes Thermotogae WS6 Phyla over Time in Digester 1 0.3 nce Phylum Bacteroidetes Caldiserica Phyla over Time in Digester 2 Filament.pdf 601 x 40
12. Sites the Students Visit: 2 weeks of records for 600 students.
13. Random Forest Performance Test Accuracy: 62.5% Random Accuracy: 7.01%. 7.4 x Lift.