Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
2016 - Dillon Niederhut - What to do when your ...
Search
Sponsored
·
Your Podcast. Everywhere. Effortlessly.
Share. Educate. Inspire. Entertain. You do you. We'll handle the rest.
→
PyBay
September 25, 2016
93
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
2016 - Dillon Niederhut - What to do when your data is large, but not big
PyBay
September 25, 2016
Transcript
9 slides
Large data in python Dillon Niederhut Introduction Motivation Strategies Closing
What to do when your data are large but not big Dillon Niederhut PyBay – the San Francisco Bay Area Python Conference 20 August 2016
Large data in python Dillon Niederhut Introduction Motivation Strategies Closing
about this talk • data at github.com/deniederhut/pybay 2016 • python libraries : celery, h5py, numpy, pandas, pymongo • other libraries : mongodb, rabbitmq, sqlite
Large data in python Dillon Niederhut Introduction Motivation Strategies Closing
about me • dlab.berkeley.edu • @DLabAtBerkeley
Large data in python Dillon Niederhut Introduction Motivation Strategies Closing
size concerns 1 1from xkcd
Large data in python Dillon Niederhut Introduction Motivation Strategies Closing
time concerns 2 2always relevant
Large data in python Dillon Niederhut Introduction Motivation Strategies Closing
code concerns 3 3thanks Randall!
Large data in python Dillon Niederhut Introduction Motivation Strategies Closing
sequential processing
Large data in python Dillon Niederhut Introduction Motivation Strategies Closing
parallel processing
Large data in python Dillon Niederhut Introduction Motivation Strategies Closing
contact • dillon.niederhut.us • @dillonniederhut
More Decks by PyBay
See All by PyBay
2017 - The Packaging Gradient
pybay
2
1.1k
2017 - Building Bridges: Stopping Python 2 without damages
pybay
0
790
2017 - Bringing Python 3 to LinkedIn
pybay
1
650
2017 - Python Debugging with PUDB
pybay
0
860
2017 - Opening up to Open Source
pybay
0
330
2017 - A Gentle Introduction to Text Classification with Deep Learning
pybay
2
260
2017 - Performant Asynchronous Programming at Quora
pybay
1
470
2017 - latus - a Personal Cloud Storage App written in Python
pybay
2
650
2017 - Everything You Ever Wanted to Know About Web Authentication in Python
pybay
3
790
Featured
See All Featured
Jess Joyce - The Pitfalls of Following Frameworks
techseoconnect
PRO
1
480
How STYLIGHT went responsive
nonsquared
100
6.4k
Test your architecture with Archunit
thirion
2
2.5k
Bash Introduction
62gerente
615
220k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
10
1.4k
Visual Storytelling: How to be a Superhuman Communicator
reverentgeek
2
750
SEOcharity - Dark patterns in SEO and UX: How to avoid them and build a more ethical web
sarafernandez
0
340
Statistics for Hackers
jakevdp
799
230k
Ecommerce SEO: The Keys for Success Now & Beyond - #SERPConf2024
aleyda
1
2.3k
The Cost Of JavaScript in 2023
addyosmani
56
10k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
25
2.2k
Claude Code どこまでも/ Claude Code Everywhere
nwiizo
68
61k