hybrid-vocal-classifier SciPy 2017

Transcript

1. github.com/NickleDave/hybrid-vocal-classifier @nicholdav David Nicholson graduate student, neuroscience Emory University
2. • Introduction • • • Methods (of methods talk) • Results • spoiler alerts: neural nets win
3. • a library that automates labeling vocalizations • what it is not: • Shazam for songbirds • what it is: • like speech to text but for songbirds
4. • Songbirds: a model system for understanding how the brain learns speech and similar motor skills • learn their vocalizations by social interaction, from a tutor Photo: Jon Sakata. Spectrogram: Dooling lab
5. • Songbirds: a model system for studying how the brain learns speech and similar motor skills • learn their vocalizations by social interaction, from a tutor • must acquire their song during a sensitive period in development Photo: Jon Sakata. Spectrogram: Dooling lab
6. • Songbirds: a model system for studying how the brain learns speech and similar motor skills • learn their vocalizations by social interaction, from a tutor • must acquire their song during a sensitive period in development • perfect their song through a process of sensorimotor learning Photo: Jon Sakata. Spectrogram: Dooling lab
7. • Birdsong • consists of elements called syllables • segment sound file into syllables by threshold crossings of amplitude
8. • Each bird’s song similar to song of its tutor • But each individual of a species will have a unique song • So syllable “c” for Bengalese finch #1 is not syllable “c” for Bengalese finch #2
9. adapted from Tumer Brainard 2007
12. • Adult song is actively maintained by error correction Sober Brainard 2009
13. • Adult song is actively maintained by error correction Sober Brainard 2009
14. • Problem: • Birds sing 100s of songs a day • Many more than can be labeled by hand
15. • Problem: • Birds sing 100s of songs a day • Many more than can be labeled by hand • Previous work: • Sound Analysis Pro (soundanalysispro.com): • Open source
16. • Problem: • Birds sing 100s of songs a day • Many more than can be labeled by hand • Previous work: • Sound Analysis Pro (soundanalysispro.com): • Open source • Avoids labeling, produces similarity scores similarity spectrogram 1 spectrogram 2 100% 0%
17. • Problem: • Birds sing 100s of songs a day • Many more than can be labeled by hand • Previous work: • Sound Analysis Pro (soundanalysispro.com): • Open source • Avoids labeling, produces similarity scores • Can use output to label by clustering • Clustering doesn’t work very well for some bird species similarity spectrogram 1 spectrogram 2 100% 0%
18. • Problem: • Birds sing 100s of songs a day • Many more than can be labeled by hand • Previous work: • Sound Analysis Pro • Other machine learning methods applied to Bengalese finch song • K nearest neighbors (k-NN) • Support Vector Machines (SVM) • Convolutional Neural Network (CNN)
19. • Problem: • Birds sing 100s of songs a day • Many more than can be labeled by hand • Previous work: • Sound Analysis Pro • Other machine learning methods applied to Bengalese finch song • Hard to compare different machine learning methods • not all open-source • not all well-documented software • very little in the way of publicly available repository of song
20. • • open source, Pure Python • built on numpy/scipy stack • implements previously proposed approaches: • SVM and k-NN via scikit-learn • neural nets using Keras • easy to use, run on YAML config file • released with a large data set • hand-labeled data • well-segmented song • days, ~20k data points/day
21. • • goals of library: • make it easy to label song in an automated way import hvc hvc.extract(‘extract.config.yml’) hvc.select(‘select.config.yml’) hvc.predict(‘predict.config.yml’)
22. • • goals of library: • make it easy to label song in an automated way • make it easy to: • compare different previously-proposed machine learning methods • make it easy to test new machine learning methods
23. • Comparing previous models: • k-Nearest Neighbors: http://www.utsa.edu/troyerlab/software.html https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm#/media/File:KnnClassification.svg Antti Ajanki, Wikipedia, CC 3.0
24. • Comparing previous models: • k-Nearest Neighbors: http://www.utsa.edu/troyerlab/software.html 50 ms
25. • Comparing previous models: • Support Vector Machine Tachibana et al. 2014 https://en.wikipedia.org/wiki/Support_vector_machine#/media/File:Svm_max_sep_hyperplane_with_margin.png
26. • Comparing previous models: • Support Vector Machine Tachibana et al. 2014 • features: • average spectra, cepstra plus • many from CUIDADO feature set (Peeters 2004): • spectral centroid • spectral spread • etc. https://en.wikipedia.org/wiki/Support_vector_machine#/media/File:Svm_max_sep_hyperplane_with_margin.png
27. • Comparing previous models: • Convolutional neural net • Koumura Okanoya 2016
28. • Comparing previous models: • Convolutional neural net • architecture • convolutional layer • max-pooling • “window” layer • goal: segmentation + classification
29. • Comparing previous models: • Convolutional neural net • open code + data!!! ty!
30. • For comparing models • Plot learning curves • accuracy v. training data (# of hand-labeled songs) • I want the best model for the least data
31. • For comparing models • Plot learning curves • accuracy v. training data (# of hand-labeled songs) • I want the best model for the least data • 5-fold cross validation
32. • For comparing models • Plot learning curves • accuracy v. training data (# of hand-labeled songs) • I want the best model for the least data • 5-fold cross validation • For each fold: random grab of n songs from training set, measure average accuracy across syllables
33. • For comparing models • Plot learning curves • accuracy v. training data (# of hand-labeled songs) • I want the best model for the least data • 5-fold cross validation • For each fold: random grab of n songs from training set, measure average accuracy across syllables • Fit learning curves: • c = accuracy at asymptote • = time to asymptote = (1 − −/)
34. • Using hybrid-vocal- classifier on our data set, I find: • SVM outperforms k- NN if a radial basis function is used github.com/NickleDave/ML-comparison-birdsong
35. • Using hybrid-vocal- classifier on our data set, I find: • A simple convolutional neural net with minimal training data outperforms SVM github.com/NickleDave/ML-comparison-birdsong
36. • Across birds, convolutional neural net outperforms both SVM and k-NN github.com/NickleDave/ML-comparison-birdsong
37. • Segmentation: • Convolutional nets for image segmentation • hybrid RNN-CNN • segmental RNN
38. SciPy, Sober lab, BioCompNS • Jonah Queen NickleDave/hybrid-vocal-classifier @nicholdav