Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Create libcsv based ruby/csv compatible CSV lib...
Search
秒速284km
May 31, 2018
Programming
0
750
Create libcsv based ruby/csv compatible CSV library
RubyKaigi 2018 - Create libcsv based ruby/csv compatible CSV library
秒速284km
May 31, 2018
Tweet
Share
More Decks by 秒速284km
See All by 秒速284km
fukuoka_ruby_2019
284km
0
160
Rubyアソシエーション開発助成成果報告会
284km
0
2k
Charty - Visualize Real-world Data with Ruby
284km
1
2.3k
Charty - Visualizing your data in Ruby
284km
0
2.2k
.so にして色々な言語から便利にのっかろう
284km
0
53
Pragmatic Charty
284km
0
2.2k
Charty with Rails
284km
1
51
Charty (RubyGrant 2018)
284km
0
2.2k
Better CSV processing with Ruby 2.6
284km
0
71
Other Decks in Programming
See All in Programming
JAWS Days 2025のインフラ
komakichi
1
130
ソフトウェアエンジニアの成長
masuda220
PRO
12
2.1k
『テスト書いた方が開発が早いじゃん』を解き明かす #phpcon_nagoya
o0h
PRO
9
2.5k
Honoのおもしろいミドルウェアをみてみよう
yusukebe
1
240
機能が複雑化しても 頼りになる FactoryBotの話
tamikof
1
230
コードを読んで理解するko build
bells17
1
110
TCAを用いたAmebaのリアーキテクチャ
dazy
0
210
Honoとフロントエンドの 型安全性について
yodaka
7
1.5k
技術を改善し続ける
gumioji
0
180
Kotlinの開発でも AIをいい感じに使いたい / Making the Most of AI in Kotlin Development
kohii00
5
1.5k
iOSでQRコード生成奮闘記
ktcryomm
2
110
Django NinjaによるAPI開発の効率化とリプレースの実践
kashewnuts
1
290
Featured
See All Featured
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
193
16k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
11
1.3k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
666
120k
Imperfection Machines: The Place of Print at Facebook
scottboms
267
13k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
27
1.6k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
45
9.4k
The Power of CSS Pseudo Elements
geoffreycrofte
75
5.5k
YesSQL, Process and Tooling at Scale
rocio
172
14k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
114
51k
Optimizing for Happiness
mojombo
377
70k
Typedesign – Prime Four
hannesfritz
41
2.5k
Rebuilding a faster, lazier Slack
samanthasiow
80
8.9k
Transcript
Create libcsv based ruby/csv compatible CSV library Asakusa.rb @284km
http://rubykaigi.org/2018/parties Talked in the title: “Super fast CSV parser” -
libcsv based - ruby/csv compatible
# libcsv based - fast - standard
# ruby/csv compatible - small && easy to read -
Backward compatibility
## Motivation - CSV is often used - Sometimes I
use a large CSV
## 284km/csv/tree/libcsv ``` # 100,000 lines Comparison: (libcsv) quoted: 81022184
allocated (libcsv) unquoted: 82071384 allocated - 1.01x more unquoted: 110535101 allocated - 1.36x more quoted: 133536546 allocated - 1.65x more ```
## libcsv - columns↑ performance↓ - rows↑ performance➘
## memory usage ``` # 5 columns, 10,000 lines Comparison:
(libcsv) quoted: 8090304 allocated (libcsv) unquoted: 9139504 allocated - 1.13x more unquoted: 11053221 allocated - 1.37x more quoted: 13354666 allocated - 1.65x more ```
## memory usage ``` # 5 columns, 100,000 lines Comparison:
(libcsv) quoted: 81022184 allocated (libcsv) unquoted: 82071384 allocated - 1.01x more unquoted: 110535101 allocated - 1.36x more quoted: 133536546 allocated - 1.65x more ```
## memory usage ``` # 10 columns, 10,000 lines Comparison:
unquoted: 13954361 allocated (libcsv) quoted: 14090304 allocated - 1.01x more (libcsv) unquoted: 15139504 allocated - 1.08x more quoted: 18555121 allocated - 1.33x more ```
## memory usage ``` # 10 columns, 1,000,000 lines Comparison:
unquoted: 1396650961 allocated (libcsv) quoted: 1411636904 allocated - 1.01x more (libcsv) unquoted: 1412686104 allocated - 1.01x more quoted: 1856651721 allocated - 1.33x more ```
# What/How to do after this talk ## Continue development
- libcsv multibyte support - column↑ performance→ - ffi gem replace to fiddle
RubyKaigi 2018 Thank you