RubyKaigi 2018 - Create libcsv based ruby/csv compatible CSV library
Create libcsv basedruby/csv compatibleCSV libraryAsakusa.rb @284km
View Slide
http://rubykaigi.org/2018/partiesTalked in the title:“Super fast CSV parser”- libcsv based- ruby/csv compatible
# libcsv based- fast- standard
# ruby/csv compatible- small && easy to read- Backward compatibility
## Motivation- CSV is often used- Sometimes I use alarge CSV
## 284km/csv/tree/libcsv```# 100,000 linesComparison:(libcsv) quoted: 81022184 allocated(libcsv) unquoted: 82071384 allocated - 1.01x moreunquoted: 110535101 allocated - 1.36x morequoted: 133536546 allocated - 1.65x more```
## libcsv- columns↑ performance↓- rows↑ performance➘
## memory usage```# 5 columns, 10,000 linesComparison:(libcsv) quoted: 8090304 allocated(libcsv) unquoted: 9139504 allocated - 1.13x moreunquoted: 11053221 allocated - 1.37x morequoted: 13354666 allocated - 1.65x more```
## memory usage```# 5 columns, 100,000 linesComparison:(libcsv) quoted: 81022184 allocated(libcsv) unquoted: 82071384 allocated - 1.01x moreunquoted: 110535101 allocated - 1.36x morequoted: 133536546 allocated - 1.65x more```
## memory usage```# 10 columns, 10,000 linesComparison:unquoted: 13954361 allocated(libcsv) quoted: 14090304 allocated - 1.01x more(libcsv) unquoted: 15139504 allocated - 1.08x morequoted: 18555121 allocated - 1.33x more```
## memory usage```# 10 columns, 1,000,000 linesComparison:unquoted: 1396650961 allocated(libcsv) quoted: 1411636904 allocated - 1.01x more(libcsv) unquoted: 1412686104 allocated - 1.01xmorequoted: 1856651721 allocated - 1.33x more```
# What/How to do after thistalk## Continue development- libcsv multibyte support- column↑ performance→- ffi gem replace to fiddle
RubyKaigi 2018Thank you