How music works, using Ruby

Slide 1

Slide 1 text

How Music Works Thijs Cadier / / How music works RailsConf 2022 RailsConf 2022

Slide 2

Slide 2 text

•Get interested, don’t understand it •Write Code •Understand it a whole lot better •Show the outcome of this to you all! My process 00 Thijs Cadier / / How music works RailsConf 2022

Slide 3

Slide 3 text

Thijs Cadier / / How music works RailsConf 2022 What we are covering today: 01 What is music made of? 02 A very brief history of recorded music 03 Digital audio 04 Ampli fi cation 05 Making sounds 06 Mixing 07 Compression 01 What is music made of? 02 A very brief history of recorded music 03 Digital audio 04 Ampli fi cation 05 Making sounds 06 Mixing 07 Compression Thijs Cadier / / How music works

Slide 4

Slide 4 text

A waveform that we perceive as having pitch, timbre and a tempo. What is music made of? 01 Thijs Cadier / / How music works RailsConf 2022

Slide 5

Slide 5 text

A waveform Thijs Cadier / / How music works RailsConf 2022

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

From a sound source to your ear Thijs Cadier / / How music works RailsConf 2022

Slide 8

Slide 8 text

Your ear resonates with the waveform

Slide 9

Slide 9 text

From follicles to your brain

Slide 10

Slide 10 text

How does the brain map a waveform? Thijs Cadier / / How music works RailsConf 2022

Slide 11

Slide 11 text

Pitch 0s 2s Thijs Cadier / / How music works RailsConf 2022

Slide 12

Slide 12 text

Timbre Thijs Cadier / / How music works RailsConf 2022

Slide 13

Slide 13 text

Tempo Thijs Cadier / / How music works RailsConf 2022

Slide 14

Slide 14 text

0s 2s

Slide 15

Slide 15 text

Back in the day music was always live. Thijs Cadier / / How music works RailsConf 2022

Slide 16

Slide 16 text

Recorded music But then came Thijs Cadier / / How music works RailsConf 2022

Slide 17

Slide 17 text

02 A very brief history of recorded music Thijs Cadier / / How music works RailsConf 2022

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

Digital audio 03 Thijs Cadier / / How music works RailsConf 2022

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

Digital audio A lot of numbers in a sequence Thijs Cadier / / How music works RailsConf 2022

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

3 def read_wave(path) 4 # Open the input wave 5 reader = WaveFile::Reader.new(path) 6 # Prepare a buffer for all samples 7 samples = [] 8 # Loop through all buffers in the file 9 loop do 10 begin 11 # Read part of the file 12 buffer = reader.read(8198) 13 # Add it to the samples buffer 14 samples.concat(buffer.samples) 15 rescue EOFError 16 # We're at the end, return the samples buffer 17 return samples 18 end 19 end 20 end

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

22 def write_wave(path, samples) 23 WaveFile::Writer.new(path, wave_format) do |writer| 24 writer.write(WaveFile::Buffer.new(samples, wave_format)) 25 end 26 end

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

14 def write_points_image(path, samples) 15 image = ChunkyPNG::Image.new(WIDTH, HEIGHT, BLACK) 16 17 samples.each_with_index do |sample, i| 20 position = if sample > 0 21 HALFWAY - offset_from_halfway(sample) 22 else 23 HALFWAY + offset_from_halfway(sample) 24 end 25 26 image.rect(i, position, i + 8, position + 8, YELLOW, YELLOW) 27 end 28 29 separator_line(image) 30 31 image.save(path) 32 end

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

Slide 44

Slide 44 text

04 Amplification Thijs Cadier / / How music works RailsConf 2022

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

1 drum = read_wave("input/drum.wav") 2 3 def amplify(track, ratio) 4 track.map do |sample| 5 sample * ratio 6 end 7 end 8 9 louder = amplify(drum, 2.0) 10 write_wave("output/louder.wav", louder)

Slide 49

Slide 49 text

1 drum = read_wave("input/drum.wav") 2 3 def amplify(track, ratio) 4 track.map do |sample| 5 sample * ratio 6 end 7 end 8 9 clipping = amplify(drum, 4.0) 10 write_wave("output/clipping.wav", clipping)

Slide 50

Slide 50 text

Clipping

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

05 Making Sounds Thijs Cadier / / How music works RailsConf 2022

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

3 # Create output array 4 output = [] 5 6 0..SAMPLE_RATE.times do 7 output << Random.rand(-30_000..30_000) 8 end 9 10 write_wave("output/noise.wav", output)

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

1 class Square 2 include Enumerable 3 4 def next_sample 5 sample = if @position < @length / 2 6 20_000 7 else 8 -20_000 9 end 10 @position += 1 11 if @position > @length 12 @position = 0 13 end 14 sample 15 end 16 17 def each 18 loop { yield next_sample } 19 end 20 end

Slide 58

Slide 58 text

3 # Create a new oscillator 4 oscillator = Square.new(440, SAMPLE_RATE) 5 6 # Create output array 7 output = [] 8 9 oscillator.each_with_index do |sample, i| 10 break if i > SAMPLE_RATE 11 output << sample 12 end 13 14 write_wave("output/square.wav", output)

Slide 59

Slide 59 text

No content

Slide 60

Slide 60 text

1 def next_sample 2 sample = Math.sin(@position * Math::PI / 2) * 25_000 3 4 # Increment position 5 @position += @increment_by 6 7 # Reset position if over max 8 if @position >= frequency 9 @position = 0.0 10 end 11 12 # Return the sample 13 sample 14 end

Slide 61

Slide 61 text

No content

Slide 62

Slide 62 text

No content

Slide 63

Slide 63 text

3 SAMPLE_RATE = 44_100 4 5 # Create multiple oscillators 6 a_note = Sine.new(440, SAMPLE_RATE) 7 c_sharp_note = Sine.new(554.37, SAMPLE_RATE) 8 e_note = Sine.new(659.25, SAMPLE_RATE)

Slide 64

Slide 64 text

10 # Create output array 11 output = [] 12 13 0..SAMPLE_RATE.times do |i| 14 # Get the samples for all three notes 15 sample_one = a_note.next_sample / 3 16 sample_two = c_sharp_note.next_sample / 3 17 sample_three = e_note.next_sample / 3 18 19 output << sample_one + sample_two + sample_three 20 end 21 22 write_wave("output/chord.wav", output)

Slide 65

Slide 65 text

No content

Slide 66

Slide 66 text

No content

Slide 67

Slide 67 text

4 one = Sine.new(220, SAMPLE_RATE) 5 two = Square.new(220, SAMPLE_RATE) 6 7 # Create output array 8 output = [] 9 10 0..SAMPLE_RATE.times do |i| 11 # Get the samples for all three notes 12 sample_one = one.next_sample / 3 13 sample_two = two.next_sample / 3 14 15 output << sample_one + sample_two 16 end

Slide 68

Slide 68 text

06 Mixing Thijs Cadier / / How music works RailsConf 2022

Slide 69

Slide 69 text

No content

Slide 70

Slide 70 text

Mixing multiple channels Thijs Cadier / / How music works RailsConf 2022

Slide 71

Slide 71 text

Results in a complicated waveform Thijs Cadier / / How music works RailsConf 2022

Slide 72

Slide 72 text

No content

Slide 73

Slide 73 text

1 require_relative "../helpers" 2 3 bass = read_wave("input/bass.wav") 4 drum = read_wave("input/drum.wav") 5 piano = read_wave("input/piano.wav")

Slide 74

Slide 74 text

9 def sum_tracks(*tracks) 10 [].tap do |out| 11 for i in 0..tracks.first.length do 12 # Start with zero 13 summed = 0 14 15 # Loop through all the tracks and increment 16 # the summed sample with its value 17 tracks.each do |track| 18 next unless track[i] 19 # Increment the summed sample, but first 20 # make it a bit less loud so we don't clip 21 summed += track[i] / 1.5 22 end 23 24 # Append the summed sample to the output 25 out << summed 26 end 27 end 28 end

Slide 75

Slide 75 text

No content

Slide 76

Slide 76 text

Compression 07 Thijs Cadier / / How music works RailsConf 2022

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

Thijs Cadier / / How music works RailsConf 2022 0 1 2 3 4 5 6 7 8 9 10 4 8 12 16

Slide 79

Slide 79 text

Treshold Thijs Cadier / / How music works RailsConf 2022 0 1 2 3 4 5 6 7 8 9 10 4 8 12 16

Slide 80

Slide 80 text

Apply compression with a ratio Thijs Cadier / / How music works RailsConf 2022 0 1 2 3 4 5 6 7 8 9 10 4 8 12 16

Slide 81

Slide 81 text

Apply compression with a ratio Thijs Cadier / / How music works RailsConf 2022 0 1 2 3 4 5 6 7 8 9 10 4 8 12 16

Slide 82

Slide 82 text

Apply make up gain Thijs Cadier / / How music works RailsConf 2022 0 1 2 3 4 5 6 7 8 9 10 4 8 12 16

Slide 83

Slide 83 text

No content

Slide 84

Slide 84 text

5 def reduce_peaks(track, treshold, ratio) 6 [].tap do |out| 7 track.each do |sample| 8 if sample > treshold or sample < -treshold 9 out << sample / ratio 10 else 11 out << sample 12 end 13 end 14 end 15 end

Slide 85

Slide 85 text

17 def apply_makeup_gain(track, makeup) 18 track.map do |sample| 19 sample * makeup 20 end 21 end

Slide 86

Slide 86 text

23 peaks_reduced = reduce_peaks(input, 1000, 8) 24 makeup_gain_applied = apply_makeup_gain(peaks_reduced, 12.0) 25 26 # Write it to disk 27 write_wave("output/final.wav", makeup_gain_applied)