Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Create libcsv based ruby/csv compatible CSV lib...
Search
秒速284km
May 31, 2018
Programming
0
750
Create libcsv based ruby/csv compatible CSV library
RubyKaigi 2018 - Create libcsv based ruby/csv compatible CSV library
秒速284km
May 31, 2018
Tweet
Share
More Decks by 秒速284km
See All by 秒速284km
fukuoka_ruby_2019
284km
0
150
Rubyアソシエーション開発助成成果報告会
284km
0
2k
Charty - Visualize Real-world Data with Ruby
284km
1
2.3k
Charty - Visualizing your data in Ruby
284km
0
2.2k
.so にして色々な言語から便利にのっかろう
284km
0
48
Pragmatic Charty
284km
0
2.2k
Charty with Rails
284km
1
43
Charty (RubyGrant 2018)
284km
0
2.1k
Better CSV processing with Ruby 2.6
284km
0
67
Other Decks in Programming
See All in Programming
return文におけるstd::moveについて
onihusube
1
1.4k
Simple組み合わせ村から大都会Railsにやってきた俺は / Coming to Rails from the Simple
moznion
3
2.1k
Асинхронность неизбежна: как мы проектировали сервис уведомлений
lamodatech
0
1.3k
良いユニットテストを書こう
mototakatsu
11
3.6k
functionalなアプローチで動的要素を排除する
ryopeko
1
190
HTML/CSS超絶浅い説明
yuki0329
0
190
php-conference-japan-2024
tasuku43
0
430
DevinとCursorから学ぶAIエージェントメモリーの設計とMoatの考え方
itarutomy
0
130
ASP.NET Core の OpenAPIサポート
h455h1
0
110
ChatGPT とつくる PHP で OS 実装
memory1994
PRO
3
190
ゼロからの、レトロゲームエンジンの作り方
tokujiros
3
1k
知られざるDMMデータエンジニアの生態 〜かつてツチノコと呼ばれし者〜
takaha4k
1
350
Featured
See All Featured
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
3
180
Making Projects Easy
brettharned
116
6k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
28
9.2k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
49k
Building Applications with DynamoDB
mza
93
6.2k
KATA
mclloyd
29
14k
Intergalactic Javascript Robots from Outer Space
tanoku
270
27k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
Site-Speed That Sticks
csswizardry
2
270
RailsConf 2023
tenderlove
29
970
A designer walks into a library…
pauljervisheath
205
24k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
38
1.9k
Transcript
Create libcsv based ruby/csv compatible CSV library Asakusa.rb @284km
http://rubykaigi.org/2018/parties Talked in the title: “Super fast CSV parser” -
libcsv based - ruby/csv compatible
# libcsv based - fast - standard
# ruby/csv compatible - small && easy to read -
Backward compatibility
## Motivation - CSV is often used - Sometimes I
use a large CSV
## 284km/csv/tree/libcsv ``` # 100,000 lines Comparison: (libcsv) quoted: 81022184
allocated (libcsv) unquoted: 82071384 allocated - 1.01x more unquoted: 110535101 allocated - 1.36x more quoted: 133536546 allocated - 1.65x more ```
## libcsv - columns↑ performance↓ - rows↑ performance➘
## memory usage ``` # 5 columns, 10,000 lines Comparison:
(libcsv) quoted: 8090304 allocated (libcsv) unquoted: 9139504 allocated - 1.13x more unquoted: 11053221 allocated - 1.37x more quoted: 13354666 allocated - 1.65x more ```
## memory usage ``` # 5 columns, 100,000 lines Comparison:
(libcsv) quoted: 81022184 allocated (libcsv) unquoted: 82071384 allocated - 1.01x more unquoted: 110535101 allocated - 1.36x more quoted: 133536546 allocated - 1.65x more ```
## memory usage ``` # 10 columns, 10,000 lines Comparison:
unquoted: 13954361 allocated (libcsv) quoted: 14090304 allocated - 1.01x more (libcsv) unquoted: 15139504 allocated - 1.08x more quoted: 18555121 allocated - 1.33x more ```
## memory usage ``` # 10 columns, 1,000,000 lines Comparison:
unquoted: 1396650961 allocated (libcsv) quoted: 1411636904 allocated - 1.01x more (libcsv) unquoted: 1412686104 allocated - 1.01x more quoted: 1856651721 allocated - 1.33x more ```
# What/How to do after this talk ## Continue development
- libcsv multibyte support - column↑ performance→ - ffi gem replace to fiddle
RubyKaigi 2018 Thank you