Project Naraku 外伝 —— レールロード図で照らすOnigmoのバグ

https://regional.rubykaigi.org/kansai09/presentations/aim2bpg

Transcript

1. shimokawa (@aim2bpg) @ؔ੢Rubyձٞ09 in େ௡ࢢ఻౷ܳೳձؗ / 2026೥7݄18೔ Project Naraku ֎఻ ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά
2. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) Whoami shimokawaʢ@aim2bpgʣ • ໦ଜ৘ใٕज़גࣜձࣾ γεςϜج൫ຊ෦ SRE෦ ॴଐ • FjordBootCampͰ Ruby ΛֶͼɺΩϟϦΞΛελʔτ • ਖ਼نදݱΤσΟλʮRubreeʯΛ OSS Ͱ։ൃɾӡ༻த 2
3. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) ਖ਼نදݱͱͷؔΘΓ 2021.11 — FjordBootCamp ͷνΣϦʔຊྠಡձͰ @jnchito ͞Μͷ Qiita هࣄ΍ Rubular ͱग़ձ͏ 2025.04 — RubyKaigi 2025 ͷ Lightning Talks @ydah ͞Μ ͷ railroad_diagrams gem ͱग़ձ͏ 2025.11 — FjordBootCamp Ͱࣗ࡞ΞϓϦͷਖ਼نදݱΤσΟλʮRubreeʯΛ։ൃɾϦϦʔε 2025.11 — Ruby Weekly #777 Ͱ Rubree ΛऔΓ্͍͍͛ͯͨͩͨ 2025.12 — YassLab ༷ʹ Rails νϡʔτϦΞϧͷୈ6ষͰਖ਼نදݱΤσΟλʮRubreeʯΛ঺հ͍͍ͨͩͨ 2025.12 — ߴా༤େ ஶʮRubyͰͭͬͯ͘·ͳͿ ਖ਼نදݱΤϯδϯʯͰΤϯδϯΛࣗ࡞ͯ͠Έͨ 2026.04 — RubyKaigi 2026 ͷ @makenowjust ͞ΜͷൃදͰ ࣍ੈ୅ΤϯδϯʮNarakuʯͱग़ձ͏ 2026.05 — ؔ੢Rubyձٞ09 ʹ CfP Λఏग़ͯ͠ొஃ͢ΔػձΛ͍͍ͨͩͨ 2026.06 — Onigmo ͷόάΛরΒ͠ Naraku ͷ࣮૷ʹ౿ΈࠐΉɹ˒ࠓճ͸ίίͷ͓࿩ 3
4. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) ਖ਼نදݱΛֶΜ͖͔͚ͩͬ 4 https://qiita.com/jnchito/items/893c887fbf19e17d3ff9 https://amzn.asia/d/04IFj1GB
5. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) ਖ਼نදݱͷ໊ஶ΍ Onigmo ͱͷग़ձ͍ 5 https://amzn.asia/d/0bB7InVM https://amzn.asia/d/0bEqPv5Y
6. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) ҏ౻͞ΜͷͭͿ΍͖͔Β Rubular ϦϝΠΫΛܾҙ 6 ʴ ʹ Rubree https://aim2bpg.hatenablog.com/entry/2025/11/25/083000#-%E9%96%8B%E7%99%BA%E3%81%AE%E3%81%8D%E3%81%A3%E3%81%8B%E3%81%91%E3%81%A8%E8%A3%8F%E8%A9%B1 Ϩʔϧϩʔυਤ 🙅 Ruby ඇରԠ… ↓ 3೥Ҏ্ͷ݄೔͕ܦͬͨ RubyKaigi 2025 Ͱ… OSS ։ൃʹΑΔ Rubular Ϋϩʔϯ https://regexper.com/#%20%28%3F%3Cmonth%3E%5Cd%7B1%2C2%7D%29%5C%2F%28%3F%3Cday%3E%5Cd%7B1%2C2%7D%29%5C%2F%28%3F%3Cyear%3E%5Cd%7B4%7D%29 https://rubular.com/r/4LvJadwa5LsN8v
7. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) Ruby ରԠͷϨʔϧϩʔυਤ gem ͱͷग़ձ͍ 7 https://speakerdeck.com/ydah/ruby-on-railroad-the-power-of-visualizing-cfg?slide=40
8. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) ਖ਼نදݱΤσΟλʮRubreeʯϦϦʔε 8 https://aim2bpg.github.io/rubree/?p=eyJyIjoiKD88bW9udGg-XFxkezEsMn0pXFwvKD88ZGF5PlxcZHsxLDJ9KVxcLyg_PHllYXI-XFxkezR9KSIsInQiOiJUb2RheSdzIGRhdGUgaXM6IDcvMTIvMjAyNi4iLCJzIjoiXFxrPHllYXI- L1xcazxtb250aD4vXFxrPGRheT4ifQ
9. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) Ruby Weekly #777 ܝࡌ 9 https://rubyweekly.com/issues/777#:~:text=Rubree https://rubyweekly.com/issues/777
10. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) Rails νϡʔτϦΞϧܝࡌ 10 https://note.com/yasslab/n/ncb57c9812545
11. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) ਖ਼نදݱΤϯδϯΛֶͿ͖͔͚ͬ 11 https://techbookfest.org/product/wL6BnYLGbizDfxRZ4qjMEn?productVariantID=wci6vbg8mLBDBuq6gtWpQ6 https://makenowjust.github.io/kantan-regex-book/ https://ydah.hatenablog.com/entry/2025/08/27/011441
12. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) Ruby ࣍ੈ୅ਖ਼نදݱΤϯδϯʮNarakuʯͱͷग़ձ͍ 12 https://speakerdeck.com/makenowjust/re-make-regexp-in-ruby-democratizing-internals-for-the-jit?slide=40 https://speakerdeck.com/makenowjust/re-make-regexp-in-ruby-democratizing-internals-for-the-jit
13. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) Onigmo ͷόάΛরΒ͢ • ϨʔϧϩʔυਤͰ
 ऑ఺ΛরΒ͢ রᶃ Naraku ΁ͷೖޱΛরΒ͢ • devcontainer Ͱ
 ݁քΛுΓ௚͢ রᶄ Naraku ͷՄೳੑΛরΒ͢ • ೑ମΛ૊Έସ͑ͯ
 ऑ఺Λࠀ෰͢Δ রᶅ ؔ੢Rubyձٞ09 ςʔϚʮরʯ 13
14. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶃ: Onigmo ͷόάΛরΒ͢ 14
15. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶃ-1: લఏ஌ࣝʢiϑϥάͱจࣈΫϥεͷجຊʣ 15 Onigmo Ͱͷಈ࡞ (1จࣈؚ͔͠·ͳ͍จࣈΫϥε͕ɺઌ಄ͷ৔߹͚ͩϦςϥϧͱಉҰࢹ͢Δ): Naraku ͰͷରԠ (͍͔ͳΔ৔߹Ͱ΋ύλʔϯதʹจࣈΫϥεͱͯ͠ॻ͔Εͨ΋ͷ͸จࣈΫϥεͱͯ͠ѻ͏): p /[s]s/i.match?("ß") # => true (Ϛον͢Δ) p /s[s]/i.match?("ß") # => false (Ϛον͠ͳ͍ɻࠨӈ͕ඇରশ) # ↑ ͜ͷෆҰகʢࠨӈඇରশʣࣗମ͕όά p /[s]s/i.match?("ß") # => false (Ϛον͠ͳ͍) p /s[s]/i.match?("ß") # => false (Ϛον͠ͳ͍) ϙΠϯτ: ɾiϑϥάʢେจࣈখจࣈΛ۠ผ͠ͳ͍ʣͰൺ΂ΔͱɺUnicodeͰ͸1จࣈ͕2จࣈͱͯ͠ ɹѻΘΕΔ͜ͱ͕͋Δʢྫɿß → ssʣ ɾßʢυΠπޠ: ΤεπΣοτʣ͸ɺେจࣈখจࣈΛ۠ผͤͣൺ΂Δͱʮssʯͱಉ͡ѻ͍ ɾhttps://github.com/makenowjust/naraku/blob/main/docs/ja/onigmo_bugs.md ɹৄࡉ͸ʮCF6 (BUG): case folding (i ϑϥά) + ࿈઀ʯΛࢀর
16. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶃŖ2: ໰୊ͱͳΔύλʔϯ 16 ✅ /[s]s/i =~ "ß" → 0 ʢϚονͨ͠ʣ ❌ /s[s]/i =~ “ß" → nil ʢෆҰகʣ ௚ײతʹ͸ಉ͡ύλʔϯͷ͸ͣ → RubreeͷϨʔϧϩʔυਤͰ֬ೝͯ͠ΈΔ ը૾ΫϦοΫͰ Rubree ΁ ը૾ΫϦοΫͰ Rubree ΁
17. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶃŖ3: RubreeͷϨʔϧϩʔυਤͰরΒ͢ 17 ✅ /[s]s/i — จࣈͷू߹Λͦͷ··ѻ͏ ❌ /s[s]/i — ม׵ʢfoldʣ͕ޮ͔ͳ͍ ը૾ΫϦοΫͰ Rubree ΁ ը૾ΫϦοΫͰ Rubree ΁ ਤͰ͸ಉ͡ߏ଄ͳͷʹɺͳͥڍಈ͕ҧ͏ʁ → จࣈΫϥε͸1จࣈ͔͠ফඅͰ͖ͳ͍ɿs+[s]͸2จࣈ෼ඞཁ → ß(1จࣈ)ͱ͸௕͕͞߹ΘͣϚονෆՄ
18. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶄ: Naraku ΁ͷೖޱΛরΒ͢ 18
19. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶄ−1: Project Naraku ͱ͸ ʮRubyઐ༻ʯͷਖ਼نදݱΤϯδϯΛ࡞Γ௚͢ϓϩδΣΫτ Onigmo ͸ Ruby ઐ༻Ͱ͸ͳ͘ɺRuby ଆ͕ಠࣗͷલॲཧΛԿஈ΋௥Ճ͍ͯ͠Δɻ ͦͷࠞཚΛղফ͠ɺRubyͷͨΊͷɺRuby͚ͩͷਖ਼نදݱΤϯδϯΛ࡞Δɻ 1. ϞμϯͰΫϦʔϯͳΞʔΩςΫνϟ ── goto ஍ࠈ͔Βͷ୤٫ 2. Rubyͷਖ਼نදݱ࢓༷Λਖ਼͘͠ఆٛ͢Δ ── ʮޓ׵ੑʯͱ͍͏ݬ૝Λյ͢ 3. ϢʔβʔϑϨϯυϦʔͳ৽ػೳ ── I ϑϥάʢASCIIͷΈ case foldingʣͳͲ 4. ύϑΥʔϚϯεվળʢJIT࿈ܞʁʣ ── YJIT/ZJIT ͱͷڞಆ 19
20. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶄ−2: Naraku ΁ͷೖޱΛরΒ͢ VMͱίϯύΠϥ͸·ͩࢼ࡞தͷஈ֊ Claude Code ͕͋Ε͹ɺProject Naraku ͷΰʔϧʹḷΓண͚Δ͔΋͠Εͳ͍ ͔͠͠ɺߦ͘खʹ͸։ൃ؀ڥͱ͍͏໊ͷ݁ք͕ுΒΕ͍ͯΔ Ҏ߱ɺNaraku Λ Fork ͯ͠ Project Naraku ֎఻ͱͯ͠ɺ͓ಧ͚͢Δ 20 Naraku ͷґଘؔ܎Λ੔ཧ͢Δ
21. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶄ−3: Naraku ͷґଘؔ܎Λ੔ཧ͢Δ 21 devcontainer Ͱ։ൃ؀ڥΛߏங͢Δ Ruby 4.0+ / Rust NarakuຊମɾYJITରԠ CπʔϧνΣʔϯ C࣮૷ͷίϯύΠϧ mruby submodule ςετ࣮ߦج൫ Unicode σʔλ จࣈϓϩύςΟੜ੒ Ϗϧυ rake naraku:build_mruby ςετ rake naraku:test_mruby Ruby gems ςετ࣮ߦج൫ ϕϯνϚʔΫ rake ruby_prototype:benchmark ϦϙδτϦ֎Ͱ༻ҙ ϦϙδτϦʹఆٛࡁΈ ։ൃ؀ڥʢ݁քʣ ͜ͷґଘؔ܎Λ౿·͑ͯ ։ൃ؀ڥΛߏங͢Δඞཁ͕͋ͬͨ https://github.com/makenowjust/naraku/tree/main
22. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶄ−4: devcontainer Ͱ։ൃ؀ڥΛߏங͢Δ 22 ݁քͷதʹ౿ΈࠐΈɺNaraku ͷՄೳੑΛরΒ͢ .devcontainer/setup.sh Ͱ ґଘؔ܎ͷΠϯετʔϧΛࣗಈԽ .devcontainer/devcontainer.json Ͱఆٛ ։ൃ؀ڥʢ݁քʣ Ruby 4.0+ / Rust NarakuຊମɾYJITରԠ CπʔϧνΣʔϯ C࣮૷ͷίϯύΠϧ mruby submodule ςετ࣮ߦج൫ Unicode σʔλ จࣈϓϩύςΟੜ੒ Ruby gems ςετ࣮ߦج൫ Ϗϧυ rake naraku:build_mruby ςετ rake naraku:test_mruby ϕϯνϚʔΫ rake ruby_prototype:benchmark Claude Code AIΤʔδΣϯτ .install-claude-code ϑΝΠϧ Λஔ͘͜ͱͰ༗ޮԽ https://github.com/aim2bpg/naraku/tree/kansai-rubykaigi09
23. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶅ: Naraku ͷՄೳੑΛরΒ͢ 23
24. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶅ−1: Naraku ͷՄೳੑΛরΒ͢ ݁ք಺ʹ౿ΈࠐΉͱਖ਼نදݱΤϯδϯͷຊ࣭తͳ໰͍͕଴͍ͬͯͨ Ͳ͏͢Ε͹ ReDoS ͱ͍͏໊ͷᚆؾΛߏ଄͔Βஅͪ੾ΕΔͷ͔ ෼਎ʢRubyϓϩτλΠϓʣΛख͕͔Γʹɺগͣͭ͠ࢼߦΛॏͶ͍ͯ͘ 24 Naraku ͷਖ਼نදݱΤϯδϯͷߏ଄ΛরΒ͢
25. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶅ−2: Naraku ͷਖ਼نදݱΤϯδϯͷߏ଄ΛরΒ͢ 25 ReDoSΛߏ଄͔Βஅͪ੾Δ — Pike VM ͱ͸ ᶈͷ Pike VM Ͱ ReDoS Λ ߏ଄͔Βஅͪ੾͍ͬͯΔ ᶆVMίϯύΠϥ dfa/compiler.rb ᶇNFAϓϩάϥϜ dfa/program.rb ᶈ࣮ߦ(Pike VM) dfa/program.rb ᶃύʔαʔ src/parse.c ᶄAST src/node.c ᶅޙॲཧ src/postprocess.c Ruby side C side https://github.com/makenowjust/naraku/tree/main ઌߦ࣮૷͞Ε͍ͯΔ Ruby ϓϩτλΠϓ΁
26. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) Pike VM ํࣜ͸࣮૷ࣗ༝౓ʹ੍ݶ͕͋Δ΋ͷͷɺReDoS ͕ൃੜ͠ͳ͍ রᶅ−3: ReDoSΛߏ଄͔Βஅͪ੾Δ — Pike VM ͱ͸ 26 Claude Code ͷྗͰ VM ͱίϯύΠϥΛ࣮૷͢Δ ؍఺ Onigmo(όοΫτϥοΩϯά) Naraku(Pike VM) ࣌ؒܭࢉྔ ฏۉ͸ߴ଎ɺ࠷ѱ O(2ⁿ) ৗʹ O(n × m)(n=จࣈྻ௕, m=ϓϩάϥϜαΠζ) ۭؒܭࢉྔ όοΫτϥοΫελοΫ(ೖྗґଘͰ৳ͼΔ) εϨουϦετ O(m) + Ωϟϓνϟ ReDoS ύλʔϯ࣍ୈͰൃੜ ൃੜ͠ͳ͍ ୯७ύλʔϯͷఆ਺ഒ ͍ܰ(ྺ୅ͷ࠷దԽͷ஝ੵ) εϨου؅ཧͷΦʔόʔϔου͋Γ ػೳͷ࣮૷ࣗ༝౓ ߴ͍(ޙํࢀরɾઌಡΈ͕ࣗવʹॻ͚Δ) ੍ݶ͋Γ(কདྷ͸ϋΠϒϦου౳ͷ޻෉͕ඞཁ) https://github.com/aim2bpg/naraku/blob/ddd2518e439af63f4a3a6cc91cacd0de874c905f/docs/ja/naraku_vm.md#53-%E5%AE%9F%E8%A1%8C%E6%96%B9%E5%BC%8F%E3%81%AE%E7%90%86%E8%AB%96%E6%AF%94%E8%BC%83%E3%81%A8%E5%AE%9F%E6%B8%AC
27. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶅ−4: Claude Code ͷྗͰ VM ͱίϯύΠϥΛ࣮૷͢Δ 27 Naraku ͷύϑΥʔϚϯεΛଌఆ͢Δ Claude Code (͜ͷ࢓༷௨Γʹಈ͘CίʔυΛॻ͍ͯ) ᶃύʔαʔ src/parse.c ᶄAST src/node.c ᶅޙॲཧ src/postprocess.c ᶆVMίϯύΠϥ regex_compile.c ᶇNFAϓϩάϥϜ regex_vm.c ᶈ࣮ߦ(Pike VM) regex_vm.c Ruby side C side ᶆVMίϯύΠϥ dfa/compiler.rb ᶇNFAϓϩάϥϜ dfa/program.rb ᶈ࣮ߦ(Pike VM) dfa/program.rb https://github.com/aim2bpg/naraku/tree/kansai-rubykaigi09
28. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶅ−5: Naraku ͷύϑΥʔϚϯεΛଌఆ͢Δ Onigmo ʹରͯ͠େ෯ະୡ 28 ύλʔϯ Onigmo Naraku.rb Nk.rb/Ong Watson (literal) 3,681 i/s 552 i/s 0.15x foo|bar|baz (alternation) 3,693 i/s 613 i/s 0.17x a+b (repetition) 1,683 i/s 51 i/s 0.03x (a|a)+b (ambiguous) 623 i/s 17 i/s 0.03x [a-zA-Z0-9]+ (char_class) 3,866 i/s 675 i/s 0.17x \d{4}-\d{2}-\d{2} (bounded) 3,454 i/s 501 i/s 0.15x \d{4}-\d{2}-\d{2} (non-match) 4,042 i/s — — (?:a?){30}a{30} (pathological) 808 i/s 10 i/s 0.01x δϣόϯχ|ΧϜύωϧϥ (unicode) 3,649 i/s 586 i/s 0.16x [a-zA-Z0-9]+ (6000B long) 605 i/s — — i/s: iteration-per-second https://github.com/aim2bpg/naraku/blob/ddd2518e439af63f4a3a6cc91cacd0de874c905f/docs/ja/naraku_vm.md#53- %E5%AE%9F%E8%A1%8C%E6%96%B9%E5%BC%8F%E3%81%AE%E7%90%86%E8%AB%96%E6%AF%94%E8%BC%83%E3%81%A8%E5%AE%9F%E6%B8%AC Naraku ͷύϑΥʔϚϯεΛվળ͢Δ
29. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶅ−6: Naraku ͷύϑΥʔϚϯεΛվળ͢Δ 29 https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/naraku_vm.md#6- %E3%83%91%E3%83%95%E3%82%A9%E3%83%BC%E3%83%9E%E3%83%B3%E3%82%B9%E6%94%B9%E5%96%84%E3%83%AD%E3%83%BC%E3%83%89%E3%83%9E%E3%83%83%E3%83%97 վળઓུᶃϓϦϑΟϧλ NFAΛಈ͔͢લʹߜΔ ɹɹCycle B: memmem ϓϦεΩϟϯ ɹɹCycle G−1: alt multi-memmem ɹɹCycle G-2: ඞਢόΠτ memchr ɹɹCycle L: first-byte table jump ɹީิͳ͠ → ଈNK _NO _MATCH ʢNFAΛ1Ϛε΋ಈ͔ͣ͞ऴྃʣ վળઓུᶄNFAόΠύε NFAΛ׬શεΩοϓ ɹCycle I/J: ϦςϥϧɾަସܗόΠύε ɹCycle K: ඇASCII όΠύε ɹCycle M: char-class ϧʔϓόΠύε ɹ֬ఆ → ଈNK _SUCCESS ʢNFAΛىಈͤͣ݁ՌΛฦ͢ʣ վળઓུᶅNFAߴ଎Խ ͦΕͰ΋ಈ͔͢৔߹ͷ಺෦࠷దԽ ɹCycle A: no-caps Ϟʔυ ɹCycle H/C/D/O: ɹɹascii_lookupɾϥϯεΩϟϯɾSIMD ɹCycle E/F/G-3/N/N-2/P: ɹɹchar-class ϧʔϓόΠύε ɹNK _SUCCESS / NK_NO_MATCH ʢਖ਼͠͞Λอͬͨ··ߴ଎Խʣ ᶇNFAϓϩάϥϜ ʴର৅จࣈྻ ύϑΥʔϚϯεվળ݁ՌΛ֬ೝ͢Δ ᶈ࣮ߦ(Pike VM)
30. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) রᶅ−7: ύϑΥʔϚϯεվળ݁ՌΛ֬ೝ͢Δ CRuby Ͱશ10ΧςΰϦ Onigmo ௒͑ 30 ύλʔϯ Onigmo Naraku.rb Nk.rb/Ong Naraku.c Nk.c/Ong Watson (literal) 3,681 i/s 2,551 i/s 0.69x 6,012 i/s 1.63x foo|bar|baz (alternation) 3,693 i/s 1,893 i/s 0.51x 5,166 i/s 1.40x a+b (repetition) 1,683 i/s 2,678 i/s 1.59x 4,871 i/s 2.90x (a|a)+b (ambiguous) 623 i/s 2,665 i/s 4.28x 4,857 i/s 7.80x [a-zA-Z0-9]+ (char_class) 3,866 i/s 3,329 i/s 0.86x 8,235 i/s 2.13x \d{4}-\d{2}-\d{2} (bounded) 3,454 i/s 2,307 i/s 0.67x 3,712 i/s 1.07x \d{4}-\d{2}-\d{2} (non-match) 4,042 i/s 3,145 i/s 0.78x 6,594 i/s 1.63x (?:a?){30}a{30} (pathological) 808 i/s 750 i/s 0.93x 846 i/s 1.05x δϣόϯχ|ΧϜύωϧϥ (unicode) 3,649 i/s 2,715 i/s 0.74x 5,829 i/s 1.60x [a-zA-Z0-9]+ (6000B long) 605 i/s 3,057 i/s 5.05x 4,209 i/s 6.96x i/s: iteration-per-second https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/naraku_vm.md#63-%E3%83%99%E3%83%B3%E3%83%81%E3%83%9E%E3%83%BC%E3%82%AF%E7%B5%90%E6%9E%9C
31. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) ·ͱΊ Onigmo ͷόάΛϨʔϧϩʔυਤΛ࢖ͬͯরΒ͠ग़͢͜ͱ͕Ͱ͖Δ • ר຤ʹϨʔϧϩʔυਤΛ࢖ͬͨৄࡉղઆɾσϞूͷ෇࿥Λ͝༻ҙ Naraku ͷ։ൃ؀ڥͷ݁քΛղ͘͜ͱͰɺ࣍ʹདྷΔऀͷೖޱΛ։͚Δ Naraku ͷ Pike VM ํࣜ࠾༻Ͱ ReDoS ͷᚆؾΛஅͪ੾Δ͜ͱ͕Ͱ͖Δ Naraku ͷ೑ମΛ૊Έସ͑Δ͜ͱͰɺOnigmo ͷੑೳΛ্ճΕΔ ࢛ࠢ ࢼߦͷ͔͚Β͸ɺҎԼͷϦϙδτϦʹ • https://github.com/aim2bpg/naraku/tree/kansai-rubykaigi09 31
32. ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂʂ
33. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) Appendixʢ෇࿥ʣ 33
34. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) ID ෼ྨ ಺༰ CF1 BUG case folding (i ϑϥά) + ޙಡΈ ((?<=...)) CF2 BUG case folding (i ϑϥά) + ޙํࢀর CF3 BUG case folding (i ϑϥά) + จࣈΫϥε CF4 UNINTUITIVE case folding (i ϑϥά) + จࣈΫϥεͷڞ௨෦෼ ([...&&...]) CF5 UNINTUITIVE case folding (i ϑϥά) + จࣈΫϥεͷ൱ఆ CF6 BUG case folding (i ϑϥά) + ࿈઀ CF7 BUG case folding (i ϑϥά) + \w CP1 BUG Ωϟϓνϟ + ઌಡΈ BR1 UNINTUITIVE ໊લ෇͖ޙํࢀর CC1 BUG POSIXจࣈΫϥε UP1 UNINTUITIVE UnicodeϓϩύςΟ໊ UP2 UNINTUITIVE UnicodeϓϩύςΟͱΤϯίʔσΟϯά LB1 UNINTUITIVE ແ੍ݶͷޙಡΈ GC1 UNINTUITIVE ॻهૉΫϥελ ES2 BUG Τεέʔϓγʔέϯεͷલॲཧ + ໊લ෇͖Ωϟϓνϟ Onigmoͷόάɾ௚ײతͰͳ͍ڍಈ 34 UNINTUITIVE: ௚ײతͰͳ͍ڍಈɾੑ࣭ Λࢦ͢ https://github.com/makenowjust/naraku/blob/main/docs/ja/onigmo_bugs.md ͜͜ʹܝࡌ͍ͯ͠ͳ͍ ID ΋ଘࡏ͢Δ͕ɺOnigmo ͷόάͱ͍͏؍఺Ͱ͸ผͷ΋ͷͱ൑அͯ͠লུͨ͠
35. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF1 (BUG): case folding (i ϑϥά) + ޙಡΈ ((?<=...)) ✅ /ß/i =~ "ß" → 0 ❌ /(?<=ß)/i =~ "ß" → nil ݪҼ: OnigmoͷޙಡΈ͸ݻఆ௕ύλʔϯલఏɻFull case foldͰß→ssʢ1จࣈˠ2จࣈʣʹͳΔͱ௕͕͞มΘΓɺΦϑηοτܭࢉ͕ഁ୼͢Δɻ Naraku: ແ੍ݶͷޙಡΈΛαϙʔτ͢ΔͨΊࣗવʹղܾʢޙग़ͷLB1ͱಉ͡ରԠʣɻ ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) ը૾ΫϦοΫͰ Rubree ΁ ը૾ΫϦοΫͰ Rubree ΁ 35
36. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF1 — ϨʔϧϩʔυਤͰݟΔ 36 ✅ /ß/i — case fold ల։ ❌ /(?<=ß)/i — ޙಡΈ಺ͷ case fold ӈਤͷϨʔϧϩʔυਤ͸ޙಡΈ಺Ͱ ß/ẞ/ss Λ໢ཏ͍ͯ͠ΔɻͦΕͰ΋ʮß ͷ௚ޙͷҐஔʯʹϚον͠ͳ͍ɻ ਤ͕ਖ਼ͯ͘͠΋Τϯδϯͷ࣮૷͕͍͍͚͍ͭͯͯͳ͍ɻ ը૾ΫϦοΫͰ Rubree ΁ ը૾ΫϦοΫͰ Rubree ΁
37. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF1 — Naraku Λ Fork ͯ͠վળͯ͠ΈΔ 37 Onigmo Ͱͷಈ࡞: Naraku Fork Ͱͷ֬ೝ (ޙಡΈ + Full fold ͕ਖ਼͘͠ಈ͘): p /(?<=ß)/i.match?("ß") # => false (όά: ß ʹϚονͨ͠௚ޙͷҐஔͳͷʹϚον͠ͳ͍) $ bin/mruby -e ' re = Naraku::Regexp.new("(?<=ß)", Naraku::Encoding::UTF_8, is_ignore_case: true, fold_flags: [:full]) p re.match?("ß") # => true (Onigmo Ͱ͸ nil) ' true https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#cf1-bug-case-folding-i-%E3%83%95%E3%83%A9%E3%82%B0--%E5%BE%8C%E8%AA%AD%E3%81%BF-
38. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF2 (BUG): case folding (i ϑϥά) + ޙํࢀর 38 ✅ /ßß/i =~ “ssß" → 0 ❌ /(ß)\1/i =~ "ssß" → nil ݪҼ: Ωϟϓνϟͨ͠จࣈ͕ෳ਺จࣈʹfold͞ΕΔ৔߹ɺޙํࢀরͰͷϚονϯά͕ରԠ͍ͯ͠ͳ͍ɻʮßΛssͱͯ͠ࢀর͢Δʯ͕Ͱ͖ͳ͍ɻ Naraku: ޙํࢀরϚονϯά࣌ʹcase foldΛਖ਼͘͠ॲཧ͢ΔΑ͏मਖ਼ɻ ը૾ΫϦοΫͰ Rubree ΁ ը૾ΫϦοΫͰ Rubree ΁
39. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF2 — ϨʔϧϩʔυਤͰݟΔ 39 ✅ /ßß/i — 2จࣈ෼ͷcase fold ❌ /(ß)\1/i — \1ͷfoldల։ ӈਤͰ͸\1͕ßΛΩϟϓνϟͨ͠৔߹ͷcase foldల։ʢss/ß/ẞʣ΋ਖ਼͘͠දࣔ͞Ε͍ͯΔɻਤ͸ਖ਼͍͕͠Τϯδϯ͕࣮ߦͰ͖ͳ͍ɻ ը૾ΫϦοΫͰ Rubree ΁ ը૾ΫϦοΫͰ Rubree ΁
40. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF2 — Naraku Λ Fork ͯ͠վળͯ͠ΈΔ 40 Onigmo Ͱͷಈ࡞: Naraku Fork Ͱͷ֬ೝ (ޙํࢀর + Full fold ͕ਖ਼͘͠ಈ͘): p /(ß)\1/i.match?("ssß") # => false (όά: Ϛον͢Δ΂͖͕ͩ͠ͳ͍) p /ßß/i.match?("ssß") # => true (ͪ͜Β͸ಈ͘) $ bin/mruby -e ' re1 = Naraku::Regexp.new("(ß)\\1", Naraku::Encoding::UTF_8, is_ignore_case: true, fold_flags: [:full]) p re1.match?("ssß") # => true (Onigmo Ͱ͸ nil) re2 = Naraku::Regexp.new("ßß", Naraku::Encoding::UTF_8, is_ignore_case: true, fold_flags: [:full]) p re2.match?("ssß") # => true (ͪ͜Β͸ Onigmo Ͱ΋ಈ͘) ' true true https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#cf2-bug-case-folding-i-%E3%83%95%E3%83%A9%E3%82%B0--%E5%BE%8C%E6%96%B9%E5%8F%82%E7%85%A7
41. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF3 (BUG): iϑϥά + จࣈΫϥε 41 ❌ /(x)[abcß]+\1/i =~ “xssa" ݪҼ: ß/ẞ͕ޓ͍ʹೋॏల։͞Εɺ+ͷ܁Γฦ͠ͰόοΫτϥοΩϯά͕ࢦ਺രൃ͢Δɻ Naraku: ಉҰจࣈ΁ͷೋॏల։Λ๷͗ReDoSΛղফɻ ը૾ΫϦοΫͰ Rubree ΁
42. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF3 — ϨʔϧϩʔυਤͰݟΔ 42 🔍 /(x)[abcß]+\1/i — ß/ẞͷೋॏల։ ʮßʯͱʮẞʯ͕ޓ͍ʹ fold ల։͠߹͍ͬͯΔ༷ࢠ͕Ϩʔϧϩʔυਤ͔ΒಡΈऔΕΔɻ ը૾ΫϦοΫͰ Rubree ΁
43. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF3 — Naraku Λ Fork ͯ͠վળͯ͠ΈΔ 43 Onigmo Ͱͷಈ࡞: Naraku Fork Ͱͷ֬ೝ (CF3 मਖ਼: ಉ͡ೖྗ͕ O(n) ͰऴΘΔ): require "timeout" Timeout.timeout(2) { /(x)[abcß]+\1/i =~ "x" + "ss" * 30 } # => Timeout::Error (ReDoS: όοΫτϥοΩϯά͕ࢦ਺తʹരൃ) $ bin/mruby -e ' re = Naraku::Regexp.new("(x)[abcß]+\\1", Naraku::Encoding::UTF_8, is_ignore_case: true, fold_flags: [:full]) p re.match?("x" + "ss" * 30) # => false (Onigmo Ͱ͸λΠϜΞ΢τɺNaraku ͸ॠ࣌ʹऴΘΔ) ' false https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#cf3-bug-case-folding-i-%E3%83%95%E3%83%A9%E3%82%B0--%E6%96%87%E5%AD%97%E3%82%AF%E3%83%A9%E3%82%B9
44. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF4 (UNINTUITIVE): iϑϥά + จࣈΫϥεڞ௨෦෼ 44 🤔 /[[a-zA-Z]&&[^A]]/i =~ “A" → 0ʢϚονʂʣ ݪҼ: &&ద༻ޙʹcase folding͞ΕΔͨΊɺআ֎ͨ͠A͕iϑϥάͰ࠶Ϛον͢Δɻ Naraku: σϑΥϧτ͸ݱঢ়ҡ࣋ɻvϑϥάͰू߹ԋࢉͷ֤ஈ֊ʹcase foldΛద༻ɻ ը૾ΫϦοΫͰ Rubree ΁
45. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF4 — ϨʔϧϩʔυਤͰݟΔ 45 🤔 /[[a-zA-Z]&&[^A]]/i — AΛআ͍ͨ͸͕ͣ࠶ͼϚον จࣈΫϥεͷڞ௨෦෼ʢ&&ʣͱcase foldingͷద༻ॱং͕௚ײͱٯʹͳ͍ͬͯΔɻ ը૾ΫϦοΫͰ Rubree ΁
46. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF4 — Naraku vϑϥάʹΑΔվળํ਑ 46 Onigmo Ͱͷಈ࡞: Naraku vϑϥά༗ޮ࣌ͷํ਑ʢܭըʣ: p /[[a-zA-Z]&&[^A]]/i =~ "A" # => true (௚ײʹ൓͢Δ͕ޙํޓ׵ͷͨΊσϑΥϧτͷ··ҡ࣋) จࣈΫϥεͷू߹ԋࢉʢ&&, ^ ͳͲʣͷ֤ஈ֊Ͱcase foldingΛద༻͢Δɻ ݁Ռͱͯ͠ [[a-zA-Z]&&[^A]] ͸ "A" Λਖ਼͘͠আ֎Ͱ͖ΔΑ͏ʹͳΔݟࠐΈʢະ࣮૷ɾະ֬ೝʣɻ https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#cf4-unintuitive-case-folding-i-%E3%83%95%E3%83%A9%E3%82%B0-- %E6%96%87%E5%AD%97%E3%82%AF%E3%83%A9%E3%82%B9%E3%81%AE%E5%85%B1%E9%80%9A%E9%83%A8%E5%88%86-
47. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF5 (UNINTUITIVE): iϑϥά + จࣈΫϥεͷ൱ఆ 47 ✅ /[ß]/I =~ "ss" → 0ʢظ଴௨Γʣ ❌ /[^ß]/i =~ "ss" → 0ʢͳ͔ͥϚονʂʣ ݪҼ: ൱ఆ͸1จࣈ୯ҐͰॲཧ͞Εfold͕ߟྀ͞Εͣɺ[^ß]/i͕sͷ1จࣈʹϚον͢Δɻ Naraku: σϑΥϧτ͸ݱঢ়ҡ࣋ɻvϑϥάͰ൱ఆΫϥεதʹจࣈྻ͕ݱΕΔ৔߹͸Τϥʔɻ ը૾ΫϦοΫͰ Rubree ΁ ը૾ΫϦοΫͰ Rubree ΁
48. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF5 — ϨʔϧϩʔυਤͰݟΔ 48 ✅ /[ß]/i — character set ❌ /[^ß]/i — negated character set ӈਤ͸negated character setͱදࣔ͞Ε͍ͯΔ͕ɺ಺༰ʹß/ẞ/ss͕ల։͞Ε͍ͯΔɻ൱ఆͱͷ૊Έ߹Θ͕ͤ௚ײͱҟͳΔಈ࡞ΛҾ͖ى͜͢ɻ ը૾ΫϦοΫͰ Rubree ΁ ը૾ΫϦοΫͰ Rubree ΁
49. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF5 — Naraku vϑϥάʹΑΔվળํ਑ 49 Onigmo Ͱͷಈ࡞: Naraku vϑϥά༗ޮ࣌ͷํ਑ʢܭըʣ: p /[^ß]/i =~ "ss" # => true (௚ײʹ൓͢Δ͕σϑΥϧτͷ··ҡ࣋) ൱ఆ͢ΔจࣈΫϥεͷதʹෳ਺จࣈ΁fold͞ΕΔจࣈྻ͕ଘࡏ͢Δ৔߹͸Τϥʔʹ͢Δɻ [^ß]͸”ß”͕"ss"ʹfold͞ΕΔͨΊΤϥʔͱͳΓᐆດ͞ΛഉআͰ͖ΔݟࠐΈʢະ࣮૷ɾະ֬ೝʣɻ https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#cf5-unintuitive-case-folding-i-%E3%83%95%E3%83%A9%E3%82%B0-- %E6%96%87%E5%AD%97%E3%82%AF%E3%83%A9%E3%82%B9%E3%81%AE%E5%90%A6%E5%AE%9A
50. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF6 (BUG): iϑϥά + ࿈઀ — ڸ૾ͳͷʹ݁Ռ͕ٯ 50 ✅ /[s]s/i =~ "ß" → 0 ❌ /s[s]/I =~ “ß" → nil ݪҼ: ୯ಠͷจࣈΫϥε[s]͸ҐஔʹΑΓϦςϥϧѻ͍͞Εɺß㲗ssͷcase fold͕ޮ͔͘Ͳ͏͔͕มΘΔɻ Naraku: จࣈΫϥεΛৗʹจࣈΫϥεͱͯ͠ѻ͍ɺ/[s]s/i =~ “ß"΋Ϛον͠ͳ͘ͳΔɻ ը૾ΫϦοΫͰ Rubree ΁ ը૾ΫϦοΫͰ Rubree ΁
51. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF6 — ϨʔϧϩʔυਤͰݟΔ 51 ✅ /[s]s/i — character set ͕Ϧςϥϧѻ͍ ❌ /s[s]/i — fold͕ޮ͔ͳ͍ Ϩʔϧϩʔυਤ͸΄΅ڸ૾ɻࠨ͸ character set ͕ઌ಄Ͱ1จࣈΫϥε͕Ϧςϥϧѻ͍͞ΕΔɻ ӈ͸ಉ͡ߏ଄Ͱ΋ల։͞Εͳ͍ɻ ը૾ΫϦοΫͰ Rubree ΁ ը૾ΫϦοΫͰ Rubree ΁
52. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF6 — Naraku Λ Fork ͯ͠վળͯ͠ΈΔ 52 Onigmo Ͱͷಈ࡞: Naraku Fork Ͱͷ֬ೝ (จࣈΫϥε͸ৗʹҰ؏ಈ࡞): p /[s]s/i.match?("ß") # => true (Ϛον͢Δ) p /s[s]/i.match?("ß") # => false (Ϛον͠ͳ͍ɻࠨӈ͕ඇରশ) # ↑ ͜ͷෆҰகʢࠨӈඇରশʣࣗମ͕όά $ bin/mruby -e ' re1 = Naraku::Regexp.new("[s]s", Naraku::Encoding::UTF_8, is_ignore_case: true, fold_flags: [:full]) p re1.match?("ß") # => false (จࣈΫϥε͸Ϧςϥϧѻ͍͠ͳ͍) re2 = Naraku::Regexp.new("s[s]", Naraku::Encoding::UTF_8, is_ignore_case: true, fold_flags: [:full]) p re2.match?("ß") # => false (ͪ͜Β΋ falseɻOnigmo ͷඇରশόά͕ղফ) ' false false https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#cf6-bug-case-folding-i-%E3%83%95%E3%83%A9%E3%82%B0--%E9%80%A3%E6%8E%A5
53. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF7 (BUG): iϑϥά + \w ͱͷަࠩ 53 ✅ /[k&&\w]/i =~ "K" → 0 ❌ /[a-z&&\w]/i =~ “K" → nil ݪҼ: [k&&\w]͸CF6ͱಉ͡نଇͰKelvin Sign·Ͱల։ɻ[a-z&&\w]͸\wͷӨڹͰASCII֎ͷfold͕཈੍͞ΕΔɻ Naraku: σϑΥϧτ͸ݱঢ়ҡ࣋ɻvϑϥάͰ\wΛ୯७ͳू߹௥Ճͱͯ͠ѻ͍ɺfoldల։Λ౷Ұɻ ը૾ΫϦοΫͰ Rubree ΁ ը૾ΫϦοΫͰ Rubree ΁
54. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF7 — ϨʔϧϩʔυਤͰݟΔ 54 ✅ /[k&&\w]/i — Kelvin Sign·Ͱల։ ❌ /[a-z&&\w]/i — fold͕཈੍͞ΕΔ ࠨਤͰ͸Kelvin Sign (K, U+212A)͕case foldల։ͷର৅ʹؚ·ΕΔɻӈਤͰ͸\wͷӨڹͰల։͕཈੍͞ΕɺKelvin Sign͕औΓ͜΅͞ΕΔɻ ը૾ΫϦοΫͰ Rubree ΁ ը૾ΫϦοΫͰ Rubree ΁
55. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CF7 — Naraku Λ Fork ͯ͠վળͯ͠ΈΔ 55 Onigmo Ͱͷಈ࡞: Naraku Fork Ͱͷ֬ೝ ([k&&\w]/i ͱ [a-z&&\w]/i + Kelvin Sign): https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#cf7-bug-case-folding-i-%E3%83%95%E3%83%A9%E3%82%B0--w p /[k&&\w]/i.match?("K") # => true (Ϛον͢Δ: [k] ͸୯ҰจࣈͳͷͰల։͞ΕΔ) p /[a-z&&\w]/i.match?("K") # => false (όά: \w ͷଘࡏͰ asc_cc ΁ͷ௥Ճ͕཈੍͞ΕΔ) $ bin/mruby -e ' kelvin = "\xe2\x84\xaa" # K (U+212A KELVIN SIGN) re1 = Naraku::Regexp.new("[k&&\\w]", Naraku::Encoding::UTF_8, is_ignore_case: true) p re1.match?(kelvin) # => true (Onigmo ͱಉ͡) re2 = Naraku::Regexp.new("[a-z&&\\w]", Naraku::Encoding::UTF_8, is_ignore_case: true) p re2.match?(kelvin) # => true (Onigmo Ͱ͸ nil) ' true true
56. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CP1 (BUG): Ωϟϓνϟ + ઌಡΈ 56 ❌ /((?=(a)))*/ =~ "a" → ແݶϧʔϓ (Timeout::Error) ݪҼ: ۭϚον൑ఆͷόάʹΑΓɺઌಡΈͰΩϟϓνϟঢ়ଶ͕ৼಈ͠ɺ࣮࣭ແݶϧʔϓͱͳΔɻ Naraku: ઌಡΈɾޙಡΈதʹग़ݱ͢ΔΩϟϓνϟͷঢ়ଶ͸ϧʔϓ൑ఆʹؚΊͳ͍Α͏ʹ͢Δɻ ࣮ࡍʹಈ͔͢ͱ: require 'timeout' begin Timeout.timeout(3) { /((?=(a)))*/ =~ "a" } rescue Timeout::Error p "Timeout::Error (ແݶϧʔϓ)" # => "Timeout::Error (ແݶϧʔϓ)" end
57. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CP1 — Naraku Λ Fork ͯ͠վળͯ͠ΈΔ 57 Onigmo Ͱͷಈ࡞: https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#cp1-bug-%E3%82%AD%E3%83%A3%E3%83%97%E3%83%81%E3%83%A3--%E5%85%88%E8%AA%AD%E3%81%BF Naraku Fork Ͱͷ֬ೝ (POSIX จࣈΫϥε͕ਖ਼͘͠ಈ͖ɺෆਖ਼ͳ໊લ͸ΤϥʔʹͳΔ): require 'timeout' begin Timeout.timeout(3) { /((?=(a)))*/ =~ "a" } rescue Timeout::Error p "Timeout::Error (ແݶϧʔϓ)" # => "Timeout::Error (ແݶϧʔϓ)" end $ bin/mruby -e ' re = Naraku::Regexp.new("((?=(a)))*") p re.match("a") # Onigmo Ͱ͸ແݶϧʔϓ ' #<MatchData "" 1:"" 2:nil>
58. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) BR1 (UNINTUITIVE): ໊લ෇͖ޙํࢀরͷ༏ઌॱҐ 58 ❌ $~[:n] # => nil ✅ $~[:n] # => "b" ݪҼ: ಉ໊Ωϟϓνϟ͕ෳ਺͋ΔͱɺOnigmo͸ৗʹ࠷ॳʹఆٛ͞Εͨ΋ͷΛࢀর͢ΔͨΊ$~[:n]͕nilʹͳΔɻ Naraku: ޙʹఆٛ͞ΕͨΩϟϓνϟΛ༏ઌͯ͠ฦ͢Α͏౷Ұ͠ɺ$~[:n]͸”b"ʹͳΔɻ ը૾ΫϦοΫͰ Rubree ΁ ը૾ΫϦοΫͰ Rubular ΁
59. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) BR1 — ϨʔϧϩʔυਤͰݟΔ 59 🔍 (?<n>a)|(?<n>b)\k<n> — ಉ໊άϧʔϓ͕ෳ਺͋Δߏ଄ NarakuͰ͸ޙʹఆٛ͞Εͨ(?<n>b)Λ༏ઌ͠ɺ$~[:n]͕ਖ਼͘͠”b"ʹͳΔ͜ͱΛ֬ೝͰ͖Δɻ ը૾ΫϦοΫͰ Rubree ΁
60. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) BR1 — Naraku Λ Fork ͯ͠վળͯ͠ΈΔ 60 Onigmo Ͱͷಈ࡞: Naraku Fork Ͱͷ֬ೝ (ޙʹఆٛ͞ΕͨΩϟϓνϟΛ༏ઌ): https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#br1-unintuitive-%E5%90%8D%E5%89%8D%E4%BB%98%E3%81%8D%E5%BE%8C%E6%96%B9%E5%8F%82%E7%85%A7 /(?<n>a)|(?<n>b)\k<n>/ =~ "bb" p $~[:n] # => nil (௚ײతͰͳ͍: n:"b" ʹϚονͨ͠ͷʹ࠷ॳͷ (?<n>a) ͷ nil ͕ฦΔ) $ bin/mruby -e ' re = Naraku::Regexp.new("(?<n>a)|(?<n>b)\\k<n>") md = re.match("bb") p md[0] # => "bb" p md[:n] # => "b" (Onigmo Ͱ͸ nil) ' "bb" "b"
61. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CC1 (BUG): POSIXจࣈΫϥε໊ͷޡݕग़ 61 ❌ /[[:unknownclass:]]/ → RegexpError ݪҼ: POSIXΫϥε໊ͷ൑ఆʹ20จࣈઌಡΈͱ͍͏৔౰ͨΓతͳϩδοΫΛ࢖͍ɺύʔεࣗମ͕O(n²)ʹͳΔɻ Naraku: ໊લ૸ࠪதʹ[΍]͕ग़ͨΒωετͨ͠จࣈΫϥεͱͯ͠ѻ͏࢓༷ʹ͠ɺઢܗ࣌ؒͰύʔεɻ ը૾ΫϦοΫͰ Rubree ΁
62. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) CC1 — Naraku Λ Fork ͯ͠վળͯ͠ΈΔ 62 Onigmo Ͱͷಈ࡞: Naraku Fork Ͱͷ֬ೝ (POSIX จࣈΫϥε͕ਖ਼͘͠ಈ͘): begin Regexp.new("[[:unknownclass:]]") rescue RegexpError => e p e.message # => "invalid POSIX bracket type" (όά: 20จࣈ੍ݶϩδοΫܦ༝ͰΤϥʔ) end $ bin/mruby -e ' re = Naraku::Regexp.new("[[:alpha:]][[:digit:]]+") p re.match("abc x5z")[0] # => "x5" ' "x5" $ bin/mruby -e ' begin Naraku::Regexp.new("[[:unknownclass:]]") rescue Naraku::ParseError => e p e.message.lines.first.chomp end ' "invalid POSIX character class name (at span 3...15)" https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#cc1-bug-posix%E6%96%87%E5%AD%97%E3%82%AF%E3%83%A9%E3%82%B9
63. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) UP1 (UNINTUITIVE): ະ஌ͷUnicodeϓϩύςΟ 63 🔍 /\p{Indic_Conjunct_Break=Linker}/ =~ “A" ݪҼ: ϓϩύςΟผ໊ͷαϙʔτ͕Ұ؏͓ͯ͠ΒͣɺIndic_Conjunct_BreakͳͲ௕͍ผ໊͕OnigmoͰ͸SyntaxErrorʹͳΔɻ Naraku: UCDͷPropertyAliases.txtͰผ໊Λਖ਼֬ʹαϙʔτ͠ɺϓϩύςΟ=஋ͷܗࣜ΋౷Ұతʹѻ͏ɻ ը૾ΫϦοΫͰ Rubree ΁
64. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) UP1 — Naraku Λ Fork ͯ͠վળͯ͠ΈΔ 64 Onigmo Ͱͷಈ࡞: https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#up1-unintuitive-unicode%E3%83%97%E3%83%AD%E3%83%91%E3%83%86%E3%82%A3%E5%90%8D Naraku Fork Ͱͷ֬ೝ (\p{Indic_Conjunct_Break=Linker} ͕ಈ͖ɺ\p{Lu} ͷผ໊΋࢖͑Δ): begin /\p{Indic_Conjunct_Break=Linker}/ rescue SyntaxError => e p e.message # => "...invalid Unicode Property \\p{Indic_Conjunct_Break}..." end p /\p{Lu}/.match?("A") # => true (\p{Lu} ͸ಈ͘) # \p{Uppercase_Letter} ͸ Ruby 4.0 Ҏ߱Ͱಈ͘ (3.x Ͱ͸ SyntaxError ʹͳΔ৔߹͋Γ) $ bin/mruby -e ' re1 = Naraku::Regexp.new("\\p{Indic_Conjunct_Break=Linker}") p re1.match?("A") # => false (Onigmo Ͱ͸ SyntaxError) re2 = Naraku::Regexp.new("\\p{Lu}+") p re2.match?("HELLO") # => true ' false true
65. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) UP2 (UNINTUITIVE): POSIXΫϥεʴશ֯จࣈ 65 🔍 /[[:alpha:]]/ =~ “̰” (U+FF41 શ֯a) ݪҼ: UTF-8Ͱ͸શ֯จࣈ΋[:alpha:]ʹϚον͢Δ͕ɺShift_JISͰ͸ಉ͡จࣈͰ΋ϚονͤͣΤϯίʔσΟϯάґଘʹͳΔɻ Naraku: ඇUnicodeΤϯίʔσΟϯάͰ΋Unicodeม׵Λհͯ͠ϓϩύςΟϚονΛ౷Ұɻ ը૾ΫϦοΫͰ Rubree ΁
66. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) UP2 — Naraku Λ Fork ͯ͠վળͯ͠ΈΔ 66 Onigmo Ͱͷಈ࡞: https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#up2-unintuitive- unicode%E3%83%97%E3%83%AD%E3%83%91%E3%83%86%E3%82%A3%E3%81%A8%E3%82%A8%E3%83%B3%E3%82%B3%E3%83%BC%E3%83%87%E3%82%A3%E3%83%B3%E3%82%B0 Naraku Fork Ͱͷ֬ೝ (UTF-8 Ͱ͸શ֯จࣈͰ΋ [[:alpha:]] ͕ಈ͘): p /[[:alpha:]]/.match?("̰") # => true (UTF-8: Unicode ϓϩύςΟʹج͖ͮϚον) # Shift_JIS Ͱ͸ /[[:alpha:]]/.match?("̰".encode('Shift_JIS')) => false (ະରԠ) $ bin/mruby -e ' re = Naraku::Regexp.new("[[:alpha:]]", Naraku::Encoding::UTF_8) p re.match?("\xef\xbd\x81") # ̰ (શ֯a, U+FF41) => true (Onigmo ͱಉ͡) ' true
67. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) LB1 (BUG): Մม௕ޙಡΈ͕Τϥʔ 67 ❌ /(?<=a+)b/ → RegexpError ݪҼ: Մม௕ͷޙಡΈʢlookbehindʣʹରԠ͢ΔVM໋ྩ͕ଘࡏͤͣɺྔࢦఆࢠΛؚΉޙಡΈύλʔϯ͸ίϯύΠϧΤϥʔʹͳ͍ͬͯͨɻ Naraku: ޙΖ޲͖VM໋ྩΛ௥Ճ͠ɺແ੍ݶͷޙಡΈΛαϙʔτɻ ը૾ΫϦοΫͰ Rubree ΁
68. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) LB1 — Naraku Λ Fork ͯ͠վળͯ͠ΈΔ 68 Onigmo Ͱͷಈ࡞: https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#lb1-unintuitive-%E7%84%A1%E5%88%B6%E9%99%90%E3%81%AE%E5%BE%8C%E8%AA%AD%E3%81%BF Naraku Fork Ͱͷ֬ೝ (Մม௕ޙಡΈ͕ಈ͘): begin Regexp.new("(?<=a+)b") rescue RegexpError => e p e.message # => "invalid pattern in look-behind" end $ bin/mruby -e ' re = Naraku::Regexp.new("(?<=a+)b") p re.match?("aaab") # => true p re.match?("b") # => false ' true false
69. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) GC1 (UNINTUITIVE): \X — ॻهૉΫϥελڥք 69 🔍 /\X/ =~ “abc" ݪҼ: \X͸”extended grapheme cluster”ʹల։͞ΕΔ͕ɺઐ༻ͷVM໋ྩͰ͸ͳ͍ͨΊॻهૉΫϥελڥքҎ֎͔Β΋ϚονͰ͖ͯ͠·͏ɻ Naraku: ݱঢ়ͷڍಈΛҡ࣋ʢޙಡΈதͷ\X͸Τϥʔͷ··ʣɻ ը૾ΫϦοΫͰ Rubree ΁
70. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) GC1 — Naraku Λ Fork ͯ͠վળͯ͠ΈΔ 70 Onigmo Ͱͷಈ࡞: https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#gc1-unintuitive-%E6%9B%B8%E8%A8%98%E7%B4%A0%E3%82%AF%E3%83%A9%E3%82%B9%E3%82%BF Naraku Fork Ͱͷ֬ೝ (\X ͸ݱࡏ CompileErrorɺ\X ઐ༻໋ྩ͕࣮૷͞ΕΔ·Ͱ): p /\X/.match?("abc") # => true (Onigmo Ͱ͸ಈ͘: ॻهૉΫϥελͷਖ਼نදݱʹల։) $ bin/mruby -e ' begin Naraku::Regexp.new("\\X") rescue Naraku::CompileError => e p e.message.lines.first.chomp end ' "unsupported feature in this version (at span 0...2)"
71. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) ES2 (BUG): Τεέʔϓγʔέϯεͷޡॲཧ 71 ❌ /(?<\x61>x)\k<a>/ → SyntaxError ݪҼ: Ωϟϓνϟ໊தͷ\x61౳ͷΤεέʔϓॲཧ͕ෆਖ਼Ͱɺ༗ޮͳࢀর໊Ͱ΋SyntaxErrorʹͳΔɻ Naraku: ΤεέʔϓγʔέϯεͷલॲཧΛमਖ਼ɻ ը૾ΫϦοΫͰ Rubree ΁
72. ”Project Naraku ֎఻ — ϨʔϧϩʔυਤͰরΒ͢ Onigumo ͷόά” — shimokawa (@aim2bpg) ES2 — վળํ਑ 72 ݱঢ়ͷڍಈ: վળํ਑ʢܭըʣ: /(?<\x61>x)\k<a>/ # raises SyntaxError: undefined name <a> reference # (લॲཧͰ \x61 ͸ม׵͞Εͳ͍ͨΊ) ໊લ෇͖Ωϟϓνϟͷ໊લ෦෼Ͱ΋ΤεέʔϓγʔέϯεΛύʔε͢ΔΑ͏ʹ͢Δʢະ࣮૷ɾະ֬ೝʣɻ ݁Ռͱͯ͠ /(?<\x61>x)\k<a>/ ΋ "a" ʹϚον͢ΔΑ͏ʹͳΔݟࠐΈʢޙํޓ׵ੑΛഁյ͢Δ͕Өڹ͸খ͍͞ͱ൑ அɾະ࣮૷ɾະ֬ೝʣɻ https://github.com/aim2bpg/naraku/blob/kansai-rubykaigi09/docs/ja/onigmo_bugs.md#es2-bug- %E3%82%A8%E3%82%B9%E3%82%B1%E3%83%BC%E3%83%97%E3%82%B7%E3%83%BC%E3%82%B1%E3%83%B3%E3%82%B9%E3%81%AE%E5%89%8D%E5%87%A6%E7%90%86-- %E5%90%8D%E5%89%8D%E4%BB%98%E3%81%8D%E3%82%AD%E3%83%A3%E3%83%97%E3%83%81%E3%83%A3