Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Create libcsv based ruby/csv compatible CSV lib...
Search
秒速284km
May 31, 2018
Programming
0
740
Create libcsv based ruby/csv compatible CSV library
RubyKaigi 2018 - Create libcsv based ruby/csv compatible CSV library
秒速284km
May 31, 2018
Tweet
Share
More Decks by 秒速284km
See All by 秒速284km
fukuoka_ruby_2019
284km
0
130
Rubyアソシエーション開発助成成果報告会
284km
0
1.9k
Charty - Visualize Real-world Data with Ruby
284km
1
2.1k
Charty - Visualizing your data in Ruby
284km
0
2.1k
.so にして色々な言語から便利にのっかろう
284km
0
39
Pragmatic Charty
284km
0
2k
Charty with Rails
284km
1
35
Charty (RubyGrant 2018)
284km
0
2k
Better CSV processing with Ruby 2.6
284km
0
57
Other Decks in Programming
See All in Programming
dotfiles について話したい #湘なんか
stefafafan
2
290
【TID2024】模擬講義:プログラマと一緒にゲームをデザインしてみよう!
akatsukigames_tech
0
440
Why Prism?
kddnewton
4
1.4k
Boost Your Performance and Developer Productivity with Jakarta EE 11
ivargrimstad
1
1.1k
Playwrightから始めるVisual Regression Testingのススメ by とっと
totto2727
2
1.8k
Go Code Generation at newmo / 2024-08-27 #newmo_layerx_go
genkey6
0
540
LangGraphでのHuman-in-the-Loopの実装
os1ma
3
870
令和トラベルにおけるLLM活用事例:社内ツール開発から得た学びと実践
ippo012
0
120
Architecture Decision Record (ADR)
nearme_tech
PRO
1
560
これからの時代の新標準!SwiftTestingへの移行とトラブルシューティング
uetyo
0
480
Swiftコードバトル必勝法
toshi0383
0
150
rbs-inlineを導入してYARDからRBSに移行する
euglena1215
1
230
Featured
See All Featured
Java REST API Framework Comparison - PWX 2021
mraible
PRO
27
7.4k
Building Applications with DynamoDB
mza
89
5.9k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
25
1.3k
Bash Introduction
62gerente
608
210k
Designing on Purpose - Digital PM Summit 2013
jponch
113
6.8k
Teambox: Starting and Learning
jrom
131
8.7k
Infographics Made Easy
chrislema
239
18k
ParisWeb 2013: Learning to Love: Crash Course in Emotional UX Design
dotmariusz
109
6.9k
The Cost Of JavaScript in 2023
addyosmani
40
5.2k
How GitHub (no longer) Works
holman
310
140k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
131
32k
VelocityConf: Rendering Performance Case Studies
addyosmani
321
23k
Transcript
Create libcsv based ruby/csv compatible CSV library Asakusa.rb @284km
http://rubykaigi.org/2018/parties Talked in the title: “Super fast CSV parser” -
libcsv based - ruby/csv compatible
# libcsv based - fast - standard
# ruby/csv compatible - small && easy to read -
Backward compatibility
## Motivation - CSV is often used - Sometimes I
use a large CSV
## 284km/csv/tree/libcsv ``` # 100,000 lines Comparison: (libcsv) quoted: 81022184
allocated (libcsv) unquoted: 82071384 allocated - 1.01x more unquoted: 110535101 allocated - 1.36x more quoted: 133536546 allocated - 1.65x more ```
## libcsv - columns↑ performance↓ - rows↑ performance➘
## memory usage ``` # 5 columns, 10,000 lines Comparison:
(libcsv) quoted: 8090304 allocated (libcsv) unquoted: 9139504 allocated - 1.13x more unquoted: 11053221 allocated - 1.37x more quoted: 13354666 allocated - 1.65x more ```
## memory usage ``` # 5 columns, 100,000 lines Comparison:
(libcsv) quoted: 81022184 allocated (libcsv) unquoted: 82071384 allocated - 1.01x more unquoted: 110535101 allocated - 1.36x more quoted: 133536546 allocated - 1.65x more ```
## memory usage ``` # 10 columns, 10,000 lines Comparison:
unquoted: 13954361 allocated (libcsv) quoted: 14090304 allocated - 1.01x more (libcsv) unquoted: 15139504 allocated - 1.08x more quoted: 18555121 allocated - 1.33x more ```
## memory usage ``` # 10 columns, 1,000,000 lines Comparison:
unquoted: 1396650961 allocated (libcsv) quoted: 1411636904 allocated - 1.01x more (libcsv) unquoted: 1412686104 allocated - 1.01x more quoted: 1856651721 allocated - 1.33x more ```
# What/How to do after this talk ## Continue development
- libcsv multibyte support - column↑ performance→ - ffi gem replace to fiddle
RubyKaigi 2018 Thank you