Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Rubyで音を視る
Search
Sponsored
·
Your Podcast. Everywhere. Effortlessly.
Share. Educate. Inspire. Entertain. You do you. We'll handle the rest.
→
ydah
June 06, 2026
Technology
770
1
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Rubyで音を視る
松江Ruby会議12「Rubyで音を視る」の発表スライド
https://matsue.rubyist.net/matrk12/
#matrk12
ydah
June 06, 2026
Transcript
66 slides
2026/06/06 Ruby 12 Ruby seeing sound with ruby Yudai Takada,
ydah
None
None
None
None
None
Otsu City Traditional Performing Arts Hall Saturday, July 18, 2026
None
None
None
None
None
None
2026/06/06 Ruby 12 Ruby seeing sound with ruby Yudai Takada,
ydah
None
None
vizcore https://github.com/ydah/vizcore
Vizcore visualize( ) × core( ) Audio-reactive visuals Ruby DSL
Ruby scene renderer
2D/3D OpenGL
None
None
None
None
1
3
PCM samples Mic / File samples Array<Float> [ 0.0012, 0.0031,
-0.0028, . . . ]
None
Sample Window tick window 1024 60fps tick sample 圧
Samples RMS amplitude Sample RMS 夏 圧 0 [0.5, -0.5,
0.5, -0.5] (0.5 + -0.5 + 0.5 + -0.5) / 4 = 0 RMS = 1 N N ∑ i=1 x2 i
Samples FFT magnitudes / peak_frequency FFT 圧 WN = e−2πi/N
Ek = N/2−1 ∑ m=0 x2m Wkm N/2 Ok = N/2−1 ∑ m=0 x2m+1 Wkm N/2 Xk = Ek + Wk N Ok Xk+N/2 = Ek − Wk N Ok (k = 0,1,…, N 2 − 1)
magnitudes BandSplitter sub / low / mid / high FFT
Vizcore sub, low, mid, high sub: 20Hz . . 60Hz low: 60Hz . . 250Hz mid: 250Hz . . 4000Hz high: 4000Hz . . 20000Hz
Samples frame energy BeatDetector beat FFT beat beat frame 圧
E = 1 N N ∑ i=1 x2 i
beat beat_pulse beat boolean frame true 1 frame beat 圧
1.0 beat frame 0.86 長
beat history BPMEstimator bpm / beat_phase / bar_phase BPM beat
frame lag BPM beat beat beat = beat_pulse = bpm = beat_phase = bar_phase = bpm = 60.0 * frame_rate / best_lag
bands + onset drums con fi dence 圧 band onset
kick = sublow_strength * onset # sub/low ͷڧ͞ × onset snare = mid_strength * onset # mid ͷڧ͞ × onset hihat = high_strength * onset # high ͷڧ͞ × onset
magnitudes / spectrum spectral features FFT Spectral features FFT 圧
amplitude bands spectral features centroid: / rollo ff : / fl atness: / fl ux: /
raw features smoother audio features frame を EMA EMAt =
αxt + (1 − α)EMAt−1 amplitude: @sm.smooth(:amplitude, normalized[:amplitude]) bands: @sm.smooth_hash(normalized[:bands], namespace: :bands) fft: @sm.smooth_array(normalized[:fft], namespace: :fft)
amplitude bass / low high / treble beat beat_pulse bpm
/ phase spectral_* / / / / / を 長
None
audio features MappingResolver layer params map beat_pulse, to: :radius, gain:
160.0, min: 56, max: 164, attack: 1.0, release: 0.2
beat beat_pulse map beat_pulse, to: :radius, gain: 100, min: 50,
max: 150
beat map beat_pulse, to: :radius, gain: 100, min: 50, max:
150 100
beat map beat_pulse, to: :radius, gain: 100, min: 50, max:
150 50 150
beat map beat_pulse, to: :radius, gain: 100, min: 50, max:
150 radius
layer params audio_frame audio scene layers audio frame { “audio”:
{ "amplitude": 0.42, "beat": true, "beat_pulse": 1.0, "bands": { "low": 0.8 } }, "scene": { "name": "main", "layers": [ { "name": "rings", "params": { "radius": 140 } …
audio_frame WebSocket frontend WebGL renderer Frontend audio_frame Ruby Runtime Browser
Renderer (WebGL Runtime) Web Socket
6/4
BGM
None
None
50 6 4
2026/06/06 Ruby 12 Ruby seeing sound with ruby Yudai Takada,
ydah
圧 母 /
母
母 母 F1 / F2 spectral centroid low / mid
/ high ratio
F1/F2 母 F 圧 圧 : : : / :
:
母 0.0 1.0 1 F1 F2 母
None
/ / 圧 圧 burst fl atness ZCR nasal high
ratio
burst amplitude_delta = [ current[:amplitude] - previous[:amplitude], # લϑϨʔϜ͔ΒͷԻྔ૿Ճ 0.0
].max burst_strength = [ current[:onset], # Իͷ্ཱ͕ͪΓ current[:spectral_flux], # εϖΫτϧมԽྔ current.dig(:onsets, :mid), # தҬͷ্ཱ͕ͪΓ current.dig(:onsets, :high), # ߴҬͷ্ཱ͕ͪΓ amplitude_delta, # Իྔͷ૿Ճྔ recent_onset_max # ۙ૭ͷ࠷େ্ཱ͕ͪΓ ].max
spectral fl atness s_score = weighted_average([ high_ratio, # ߴҬͷڧ͞ flatness,
# ϊΠζͬΆ͞ zcr, # ܗͷࡉ͔͍༳Ε centroid # Իͷ໌Δ͞ ]) n_score = weighted_average([ nasal_score, # ඓԻͬΆ͞ 1.0 - high_ratio, # ߴҬ͕গͳ͍ low_mid_ratio, # ʙதҬͷڧ͞ 1.0 - zcr # ༳Ε͕গͳ͍ ])
attack feature attack = { strength: attack[:attack_strength], # ग़ͩ͠ͷڧ͞ centroid:
attack[:centroid_norm], # ग़ͩ͠ͷ໌Δ͞ high: attack[:high_ratio], # ߴҬͷڧ͞ low: attack[:low_ratio], # Ҭͷڧ͞ flatness: attack[:spectral_flatness],# ϊΠζͬΆ͞ zcr: attack[:zero_crossing_rate], # ࡉ͔͍༳Ε f1 : attack[:f1_frequency], # F1 ͬΆ͍ࢁ f2 : attack[:f2_frequency], # F2 ͬΆ͍ࢁ nasal: attack_nasal_score # ඓԻͬΆ͞ }
母 × kana_score = [ vowel_score * 0.55, # ԻΒ͠͞
consonant_score * 0.35, # ࢠԻΒ͠͞ burst_bonus * 0.10 # ग़ͩ͠ͷิਖ਼ ].sum candidate = { text: kana_table[consonant][vowel], # ͔ͳจࣈ vowel: vowel, # ਪఆԻ consonant: consonant, # ਪఆࢠԻ conf i dence: kana_score # ީิͷࣗ৴ }
None
vizcore Ruby ォ Ruby
$whoami ォ ydah ( ) SmartHR Ruby 西 Ruby Ruby
( 西 ) 西 圧 OSS 2
More Decks by ydah
See All by ydah
gem install とrequire のむこうがわ
ydah
0
250
わからない話を追いかけたら、プログラミング言語を作る側にいた
ydah
3
1.2k
Liberating Ruby's Parser from Lexer Hacks
ydah
2
4.5k
ルールルルルルRubyの中身の予備知識 ── RubyKaigiの前に予習しなイカ?
ydah
1
1.1k
計算機科学をRubyと歩む 〜DFA型正規表現エンジンをつくる~
ydah
3
1k
進捗
ydah
2
370
Joy with 3D Graphics Using Ruby
ydah
1
190
構文解析器入門
ydah
7
3.9k
Regional.rb and the Kyoto City
ydah
0
110
Other Decks in Technology
See All in Technology
社内アプリ基盤「Quartz」による、 非エンジニアとAIに本番デプロイを 任せても壊れないプラットフォームの設計
staka121
PRO
1
180
エンタープライズ事業未経験のデータ屋、 AI時代にN回撃沈する.pdf
ryoskdara_
0
180
AWS Client VPNの設定でハマったポイントとその過程での気づき
amarelo_n24
1
430
Context parameters で作る!はじめての型クラス
kworkdev
PRO
1
170
飯から始まるデータエンジニア
sansantech
PRO
0
130
Gov-JAWS#9 統合監視機能のマネージドサービス化+AIによる障害時自動調査
fj_yoshimura_kosuke
0
230
ノードが消える前提で設計する!Karpenterで金融システムを止めないためのノウハウ大全
daitak
1
200
3文字略語に騙されない! 世界一わかりやすい『EDR』超入門
kamadamakoto
1
320
共通基盤の実装に Kotlin - Quarkus を選定した理由と採用したアーキテクチャ
nealle
1
160
[NW-JAWS DAY]「うちのAWS、インターネット出られないんで」― AWS外の自社サービスをPrivateLinkでシュッと公開する方法
ykimi
3
910
E2E Test というハーネス — AI時代の自動テスト民主化
daishukojima
10
5.9k
From vibe coding to reliable (agentic) development
jcabot
0
150
Featured
See All Featured
The Spectacular Lies of Maps
axbom
PRO
1
1.1k
Building AI with AI
inesmontani
PRO
1
1.4k
A Guide to Academic Writing Using Generative AI - A Workshop
ks91
PRO
1
570
A designer walks into a library…
pauljervisheath
211
25k
Navigating Algorithm Shifts & AI Overviews - #SMXNext
aleyda
1
1.7k
From π to Pie charts
rasagy
1
460
More Than Pixels: Becoming A User Experience Designer
marktimemedia
3
630
VelocityConf: Rendering Performance Case Studies
addyosmani
331
25k
Bridging the Design Gap: How Collaborative Modelling removes blockers to flow between stakeholders and teams @FastFlow conf
baasie
0
780
Primal Persuasion: How to Engage the Brain for Learning That Lasts
tmiket
0
520
Building Flexible Design Systems
yeseniaperezcruz
330
41k
How to Align SEO within the Product Triangle To Get Buy-In & Support - #RIMC
aleyda
2
1.9k