Generating Melodies from Melodic Outlines Towards an Improvisation Support Systems for Non-musicians

KICSS 2022

Transcript

1. Generating Melodies from Melodic Outlines Towards an Improvisation Support Systems for Non-musicians Tetsuro Kitahara Nihon University, Japan Twitter: @tetsurokitahara
2. Our goal To enable everyone to enjoy music improvisation including non-musicians Cmaj7 Am7 Chord progression Harmony theory Musical scale Music knowledge Do-Re-Mi Learned musical phrases Create a melody Play it
3. Our previous work: JamSketch Melodic outline User draws: Melody System generates: Intuitive, easy to draw based on a genetic algorithm Demo Video
4. Today's talk Re-developed the melody generation method using a CNN Input time Melodic outline Chord time Output Input Melody (notes)
5. Issue in designing a CNN Music is not shift-invariant Pitch axis The most important note Rarely appears in the C-major key Different notes have different roles Time axis on-beat off-beat On-beats and off-beats have different features
6. Filter design in our CNN model Melodic outline time ch pitch Chord progression time conv. conv. time ch deconv. time Pitch time pitch deconv. Pitch (cont) ch filter size (avoid shift in the pitch axis) filter size avoid that on-beats and off-beats are flipped
7. Dataset 96 Blues melodies from Weimar Jazz Database Input data Output data Melodic outlines made by smoothing melodies (a sequence of notes) Half for training, the rest for testing
8. Demo https://bit.ly/3zB2Tja
9. Evaluation: Method Melody from DB (given melody) Melodic outline Model Generated melody MP3 MP3 Rate 8 melodies randomly chosen (1–7) based on 6 criteria 30 persons having musical experience https://bit.ly/3tmDt5a Folder
10. Evaluation: Results Obtained close scores to given melodies
11. Conclusion Input Output Input JamSketch: an improvisation system where the user draws melodic outlines Developed the melody generation method with a CNN Melodic outline Melody Listening test Implies close quality to human melodies