JP Bader - Plug-n-Stream Player Piano: Signal Processing With Python

Digital Signal Processing and Player Piano don't normally come together in the same sentance. Player Pianos that are 100+ years old are awesome artisan artifacts, but they don't play digital formats very well. This talk will show how we take a 100+ year old technology and marry it to the digital age via Python libraries and precision lasers!

In this discussion we will cover how we are creating our own "Plug-n-Stream Player Piano". We will take a look at the different digital signal processing Python libraries, their functionality, and requirements for converting audio streams to piano playable audio files. After a brief walk through of our prototyped hardware, we will dissect the digital signal processing, converting streaming music to data for the Player Piano. With a real Player Piano in the room we will demo streaming music from our devices onto the piano.

LIVE(ish) Piano Playing!

https://us.pycon.org/2019/schedule/presentation/158/

Transcript

1. Caveats -  Ruined Music -  Bad Code -  Interesting (kludgey) fixes
2. •  Really bad at making slideshow presentations •  Actually, really really bad at documentation. No social media L •  Interactive Voice Response, Machine Learning, NLP, etc. •  Greenfield ideas
3. Background - History of building things I’m excellent at coming up with bad dad jokes, and finding awesome ways to embarrass my children. Sometimes I succeed, other times I fail and get the, “whoa dad, that’s pretty cool, but could you please stop bragging?!?” Husband, father, chicken and dog raiser Overeducated redneck, post-technical, recovering software engineer Garage band - Soloing is hard, visual learner, not aural! Build things BS - Piano performance Collect instruments Moving pianos, busking, platform iterations, Because fixing old mechanical things are awesome and fun! Nothing in this talk is novel (except the piano part, which in reality, is minor). Multifacted uses (mentioned before, including IVR). Musically – visual learner, not aural, and so the prospect of building a tool that could enable that sounded awesome. Maybe my eyes were bigger than my stomach, however
4. Warning you that there will be bad code, ruined music examples, interesting hardware (though it’s still being worked on…) Went down rabbit hole learning all the things, so please be patient :)
5. Setup: Most challenging is audio conversion Really, really hard to get computers to understand music To understand why it’s hard for computers to understand music, quick science/history lesson about (D)SP Isolating a single voice is hard, but then what do you do with multiple vocals, reverb, delays, other effects? Fundamental frequency (fo) Harmonics Unvoiced speech Sound effects Additional instruments
6. The field goes back several hundred years, digital only added recently Tools that work/don’t work, how we’re going to (maybe) solve this
8. Frequency v Cycle: Sine waves are culminations of mutliple signals. Wave looks linear, but if twisted around it’s center, shows pattern. When frequency meets cycles, we have unique signature for sound, giving notes TBH - for the 3 PhDs in the room who know DSP, don’t get angry, but for the rest of us, we’re going to gloss over this b/c nobody really cares :( Good programmers take other’s accomplishments to achieve! Quantization Pitch Velocity Harmonics
9. Tools -  OSX -  Javascript -  Jupyter -  Anaconda Techniques Blind Source Separation -> Semi-supervised Non- Negative Matrix Factorization Neural Network-based approaches STFT Voice-Activity Detection (https://en.wikipedia.org/ wiki/Voice_activity_detection)
10. NUSSL (nuzzle)
11. Next steps Python hooks New/updated code Explore the libraries Personally – wife won’t let me do this again for a while, however: Real-time audio conversion from mic? Next revs? Better hardware? Tempured silicone flanges v. solenoids Future iterations, help? Commercial ideas (happy to let you have the glory)! Peter Zieba – does PCB design and engineering, loves working on projects that take best parts of old, classic hardware and integrate them with modern controls and interfaces. Loves open source, python-based toolchains around FPGAs, and creates retrofits for Fadal Vertical Machining Centers and scanning electron microscopes. Carl Karsten – Videographer and documenter of all things OpenSource. He makes some really great software as well.