Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Create libcsv based ruby/csv compatible CSV lib...
Search
秒速284km
May 31, 2018
Programming
0
740
Create libcsv based ruby/csv compatible CSV library
RubyKaigi 2018 - Create libcsv based ruby/csv compatible CSV library
秒速284km
May 31, 2018
Tweet
Share
More Decks by 秒速284km
See All by 秒速284km
fukuoka_ruby_2019
284km
0
140
Rubyアソシエーション開発助成成果報告会
284km
0
2k
Charty - Visualize Real-world Data with Ruby
284km
1
2.2k
Charty - Visualizing your data in Ruby
284km
0
2.1k
.so にして色々な言語から便利にのっかろう
284km
0
45
Pragmatic Charty
284km
0
2.1k
Charty with Rails
284km
1
40
Charty (RubyGrant 2018)
284km
0
2.1k
Better CSV processing with Ruby 2.6
284km
0
65
Other Decks in Programming
See All in Programming
RWC 2024 DICOM & ISO/IEC 2022
m_seki
0
200
Zoneless Testing
rainerhahnekamp
0
120
Scalaから始めるOpenFeature入門 / Scalaわいわい勉強会 #4
arthur1
1
300
LLM Supervised Fine-tuningの理論と実践
datanalyticslabo
3
1k
数十万行のプロジェクトを Scala 2から3に完全移行した
xuwei_k
0
260
HTTP compression in PHP and Symfony apps
dunglas
2
1.7k
Go の GC の不得意な部分を克服したい
taiyow
2
770
Effective Signals in Angular 19+: Rules and Helpers @ngbe2024
manfredsteyer
PRO
0
130
ブラウザ単体でmp4書き出すまで - muddy-web - 2024-12
yue4u
2
460
17年周年のWebアプリケーションにTanStack Queryを導入する / Implementing TanStack Query in a 17th Anniversary Web Application
saitolume
0
250
htmxって知っていますか?次世代のHTML
hiro_ghap1
0
330
Semantic Kernelのネイティブプラグインで知識拡張をしてみる
tomokusaba
0
180
Featured
See All Featured
GraphQLとの向き合い方2022年版
quramy
44
13k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
232
17k
Bootstrapping a Software Product
garrettdimon
PRO
305
110k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
2
290
Designing for Performance
lara
604
68k
Adopting Sorbet at Scale
ufuk
73
9.1k
Intergalactic Javascript Robots from Outer Space
tanoku
270
27k
Put a Button on it: Removing Barriers to Going Fast.
kastner
59
3.6k
Designing for humans not robots
tammielis
250
25k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
6
510
Imperfection Machines: The Place of Print at Facebook
scottboms
266
13k
RailsConf 2023
tenderlove
29
940
Transcript
Create libcsv based ruby/csv compatible CSV library Asakusa.rb @284km
http://rubykaigi.org/2018/parties Talked in the title: “Super fast CSV parser” -
libcsv based - ruby/csv compatible
# libcsv based - fast - standard
# ruby/csv compatible - small && easy to read -
Backward compatibility
## Motivation - CSV is often used - Sometimes I
use a large CSV
## 284km/csv/tree/libcsv ``` # 100,000 lines Comparison: (libcsv) quoted: 81022184
allocated (libcsv) unquoted: 82071384 allocated - 1.01x more unquoted: 110535101 allocated - 1.36x more quoted: 133536546 allocated - 1.65x more ```
## libcsv - columns↑ performance↓ - rows↑ performance➘
## memory usage ``` # 5 columns, 10,000 lines Comparison:
(libcsv) quoted: 8090304 allocated (libcsv) unquoted: 9139504 allocated - 1.13x more unquoted: 11053221 allocated - 1.37x more quoted: 13354666 allocated - 1.65x more ```
## memory usage ``` # 5 columns, 100,000 lines Comparison:
(libcsv) quoted: 81022184 allocated (libcsv) unquoted: 82071384 allocated - 1.01x more unquoted: 110535101 allocated - 1.36x more quoted: 133536546 allocated - 1.65x more ```
## memory usage ``` # 10 columns, 10,000 lines Comparison:
unquoted: 13954361 allocated (libcsv) quoted: 14090304 allocated - 1.01x more (libcsv) unquoted: 15139504 allocated - 1.08x more quoted: 18555121 allocated - 1.33x more ```
## memory usage ``` # 10 columns, 1,000,000 lines Comparison:
unquoted: 1396650961 allocated (libcsv) quoted: 1411636904 allocated - 1.01x more (libcsv) unquoted: 1412686104 allocated - 1.01x more quoted: 1856651721 allocated - 1.33x more ```
# What/How to do after this talk ## Continue development
- libcsv multibyte support - column↑ performance→ - ffi gem replace to fiddle
RubyKaigi 2018 Thank you