kenny_g.rb: Making Ruby Write Smooth Jazz (Open Source Bridge 2015)

For too long, computers have been shut out of the red-hot music-to-listen-to-while-relaxing-in-the-bathtub genre. Today, that all changes. Our smooth-jazz-as-a-service startup is primed to disrupt this stale industry. All we need is a little Ruby and we'll make automated musical magic.

Transcript

1. kenny_g.rb:
   Making Ruby write smooth jazz
   Tim Krajcar
   

   View Slide

2. TimKrajcar
   https://www.flickr.com/photos/can_eldem/6929385615
   

   View Slide

3. TimKrajcar
   https://www.flickr.com/photos/can_eldem/6929385615
   

   View Slide

4. –Johnny Appleseed
   https://www.dropbox.com/s/8t7mg4gikhx0d5p/Screenshot
   %202015-06-09%2019.12.34.png?dl=0
   

   View Slide

5. –Johnny Appleseed
   

   View Slide

6. –Johnny Appleseed
   bit.ly/
   OSBridgeNRJobs
   

   View Slide

7. TimKrajcar
   Jason Clark - Testing the Multiverse
   Wednesday 3:45pm - B301
   Zoe Kay - Venturing into the Spooky Science of Ruby
   Thursday 3:45pm - B301
   Ward Cunningham - Learning and Knowing with Federated
   Wiki
   Jonan Scheffler - Hacking Minecraft
   

   View Slide

8. TimKrajcar
   Tim Krajcar's twitter handle is:
   @TimKrajcar
   

   View Slide

9. TimKrajcar
   

   View Slide

10. TimKrajcar
    story time
    

    View Slide

11. TimKrajcar
    https://www.flickr.com/photos/gruenewiese/15668094425
    

    View Slide

12. TimKrajcar
    https://www.flickr.com/photos/vironevaeh/4494702797/
    

    View Slide

13. TimKrajcar
    audience
    participation!
    

    View Slide

14. https://www.flickr.com/photos/kimli/1879624559/
    

    View Slide

15. TimKrajcar
    IS HE A
    WIZARD??
    

    View Slide

16. TimKrajcar
    https://www.flickr.com/photos/mhx/16419218393
    

    View Slide

17. TimKrajcar
    https://www.flickr.com/photos/mhx/16419218393
    I could do that
    

    View Slide

18. TimKrajcar
    can it be
    automated?
    

    View Slide

19. TimKrajcar
    can it be
    automated?
    

    View Slide

20. TimKrajcar
    can it make me lots of
    money without me
    doing any work?
    

    View Slide

21. TimKrajcar
    some
    background
    

    View Slide

22. TimKrajcar
    MIDI
    Musical Instrument Digital
    Interface
    

    View Slide

23. TimKrajcar
    MIDI = protocol for
    control
    

    View Slide

24. TimKrajcar
    MIDI ≠ audio
    

    View Slide

25. TimKrajcar
    MIDI is messages
    

    View Slide

26. TimKrajcar
    note on
    (note, velocity 0-127)
    

    View Slide

27. TimKrajcar
    note off
    (note)
    

    View Slide

28. TimKrajcar
    also lots of other things

    (program and control changes, aftertouch,
    pitch bend, sysex, ...)
    

    View Slide

29. TimKrajcar
    

    View Slide

30. View Slide

31. View Slide

32. TimKrajcar
    

    View Slide

33. TimKrajcar
    MIDI is fun!
    

    View Slide

34. –Johnny Appleseed
    

    View Slide

35. TimKrajcar
    the challenge:
    build a tool that plays delightful piano
    music based on a series of four notes
    

    View Slide

36. TimKrajcar
    the challenge:
    build a tool that plays delightful piano
    music based on a series of four notes
    

    View Slide

37. TimKrajcar
    the challenge:
    build a STARTUP that plays delightful piano
    music based on a series of four notes
    

    View Slide

38. –Johnny Appleseed
    

    View Slide

39. TimKrajcar
    first strategy:
    algorithmic harmony
    

    View Slide

40. –Johnny Appleseed
    

    View Slide

41. –Johnny Appleseed
    

    View Slide

42. TimKrajcar
    algorithmic music =
    not very relaxing
    https://www.youtube.com/watch?v=TAHamdXqTTI
    

    View Slide

43. TimKrajcar
    this is really hard
    
    

    View Slide

44. TimKrajcar
    user persona!!
    

    View Slide

45. https://www.flickr.com/photos/santheo/531632789
    

    View Slide

46. https://www.flickr.com/photos/santheo/531632789
    STEVE
    

    View Slide

47. https://www.flickr.com/photos/santheo/531632789
    STEVE
    STEVE'S DUCK
    

    View Slide

48. TimKrajcar
    what to do?
    

    View Slide

49. TimKrajcar
    strategy inspiration:
    reddit™
    (reddit doesn't like it if you use their logo without a license so I'm not using it)
    REDDIT and the ALIEN Logo are registered trademarks of reddit inc.
    

    View Slide

50. TimKrajcar
    "[...]when logged in as an admin, a third
    field appeared that allowed the team to
    enter a custom user name that would
    automatically be registered for an
    account upon hitting submit."
    http://venturebeat.com/2012/06/22/reddit-fake-users/
    

    View Slide

51. TimKrajcar
    like any good startup,
    we will CHEAT!
    

    View Slide

52. TimKrajcar
    like any good startup,
    we will CHEAT!
    

    View Slide

53. TimKrajcar
    like any good startup, we will
    utilize pre-recorded
    interactions to simulate real-
    time generative algorithms!
    

    View Slide

54. TimKrajcar
    play back pre-recorded
    "transitions" between
    chords
    

    View Slide

55. –Johnny Appleseed
    

    View Slide

56. –Johnny Appleseed
    

    View Slide

57. –Johnny Appleseed
    

    View Slide

58. –Johnny Appleseed
    

    View Slide

59. –Johnny Appleseed
    
    

    View Slide

60. –Johnny Appleseed
    we need to write
    transitions for all of
    these?!
    

    View Slide

61. TimKrajcar
    BIG MUSIC
    SECRET!!!
    

    View Slide

62. TimKrajcar
    THE SECRET:
    you just need the
    relative intervals
    

    View Slide

63. TimKrajcar
    transpose using the
    first note as "zero"
    

    View Slide

64. TimKrajcar
    we need to record a
    "phrase"
    a short bit of music that transitions
    from "zero" to an interval
    

    View Slide

65. TimKrajcar
    play back and
    repeat as needed
    

    View Slide

66. TimKrajcar
    let's do it
    

    View Slide

67. TimKrajcar
    https://github.com/tkrajcar/kenny_g.rb
    

    View Slide

68. TimKrajcar
    tech demo 1:
    all the keys
    

    View Slide

69. TimKrajcar
    require "midi"
    @output =
    UniMIDI::Output.use(:first)
    MIDI.using(@output) do
    velocity 64
    (0..127).each do |n|
    play n, 0.01
    end
    end
    

    View Slide

70. TimKrajcar
    tech demo 2:
    recording our phrases
    

    View Slide

71. TimKrajcar
    saving notes!
    sounds like a job for
    MIDI!
    

    View Slide

72. –Johnny Appleseed
    

    View Slide

73. –Johnny Appleseed
    

    View Slide

74. –Johnny Appleseed
    
    

    View Slide

75. TimKrajcar
    we'll just make up our
    own standard
    

    View Slide

76. TimKrajcar
    big news!!
    

    View Slide

77. View Slide

78. TO HACKER NEWS!!
    

    View Slide

79. TimKrajcar
    [
    {
    "note": 60,
    "velocity": 80,
    "starting_time_offset": 0.677646,
    "duration": 0.248938
    },
    {
    "note": 67,
    "velocity": 65,
    "starting_time_offset": 0.920729,
    "duration": 0.27753
    },
    {
    "note": 64,
    "velocity": 72,
    // ...
    

    View Slide

80. TimKrajcar
    tech demo 3:
    playback!
    

    View Slide

81. TimKrajcar
    MIDI.using(@output) do
    @notes = JSON.parse(File.read("sample.notejs"))
    @offset = 0
    @notes.each do |note|
    sleep(note["starting_time_offset"] - @offset)
    play note["note"], note["duration"]
    @offset += note["duration"]
    end
    end
    

    View Slide

82. TimKrajcar
    https://www.flickr.com/photos/deivis/2300233581/
    

    View Slide

83. TimKrajcar
    https://www.flickr.com/photos/deivis/2300233581/
    DO NOT SHIP IT
    

    View Slide

84. TimKrajcar
    MIDI.using(@output) do
    @notes = JSON.parse(File.read("sample.notejs"))
    @offset = 0
    @notes.each do |note|
    sleep(note["starting_time_offset"] - @offset)
    play note["note"], note["duration"]
    @offset += note["duration"]
    end
    end
    

    View Slide

85. TimKrajcar
    tech demo 4:
    playback better!
    

    View Slide

86. TimKrajcar
    MIDI.using(@output) do
    @notes = JSON.parse(File.read("sample.notejs"))
    @messages = @notes.flat_map do |note|
    [
    {type: :note_on, note: note["note"], velocity:
    note["velocity"], at: note["starting_time_offset"]},
    {type: :note_off, note: note["note"], at:
    note["starting_time_offset"] + note["duration"]}
    ]
    end
    @messages.sort! do |a, b|
    a[:at] - b[:at]
    end
    

    View Slide

87. TimKrajcar
    @offset = 0
    @messages.each do |message|
    sleep(message[:at] - @offset)
    if message[:type] == :note_on
    note message[:note], velocity: message[:velocity]
    else
    note_off message[:note]
    end
    @offset += sleep_duration
    end
    

    View Slide

88. TimKrajcar
    tech demos complete!
    almost ready for
    funding!
    

    View Slide

89. TimKrajcar
    the patent-pending
    kenny_g.rb algorithm
    

    View Slide

90. TimKrajcar
    step 1:
    convert user-provided chords to
    relative intervals, with the first
    chord as 0
    

    View Slide

91. TimKrajcar
    input:
    F, A, C, D
    relative intervals:
    [0, 4, 7, 9]
    

    View Slide

92. TimKrajcar
    step 2:
    convert list of intervals to
    list of phrase transitions
    

    View Slide

93. TimKrajcar
    input:
    [0, 4, 7, 9]
    phrase transitions:
    {0, +4}, {4, +3}, {7, +2}
    

    View Slide

94. TimKrajcar
    step 3:
    play back pre-recorded
    phrases, transposed into
    starting key
    

    View Slide

95. TimKrajcar
    DEMO!
    

    View Slide

96. TimKrajcar
    thanks!
    

    View Slide