Kiba ETL: feedback on OSS / open-core sustainability for a Ruby gem

It’s pretty easy to get burned out in OSS, as we all know! Since starting working on Kiba ETL (a ruby data processing framework) in 2015, I made a number of decisions to avoid this at all cost, and make sure my research on this would be funded sustainably, in a way or another.

In this talk, I give feedback on which conscious design decisions & trade-offs I made over time and I will explain where I stand at the moment.

Transcript

1. ᇌ KIBA ETL Feedback on OSS / open-core sustainability for a Ruby gem — Thibaut Barrère, LoGeek SARL (https://www.logeek.fr)
2. ETL Extract Transform Load = a form of "Data Pipeline"
3. job = Kiba.parse do source MyCSVSource, filename: "files/input.csv" transform MyTransform, some_config transform MyOtherTransform, some_config transform { |row| ... } destination MyDatabaseDestination, database: db, table: :products end Kiba.run(job)
4. Sustainabilityies
5. Don't Compare Your Insides to Other People's Outsides
7. 1995 Demomaking. No GPU. No FPU. 486SX25
8. Funding! Support! ❤ thank you Mum & Dad Also VC Dad: "3D is nice, but will this earn money at some point?"
9. 2006 Building datawarehouses (with Ruby)
10. ❤ activewarehouse-etl Declarative DSL for ETL — in Ruby! Loads of features!
11. source :in, { :file => 'extracted/crm-changes.csv', :parser => :delimited, :skip_lines => 1 }, [:first_name, :last_name] after_read :ensure_fields_presence, { :fields => fields } transform(:id_partenaire) { |n,v,r| some_computation } destination :out, { :file => output_file, :include_headers => true, :separator => ";" }
12. 2011 Open-source Maintenance
13. Loads of Features = Big Support Cost Time / Lost Income Applicability
14. Unsustainable
15. A Strict No Burnout & No Broke Policy
16. 3 OSS DECISIONS
17. #1: Reduce Scope Lessen the support cost
18. #2: Strive for Increased Applicability
19. #3: Time-Boxing
20. 2015Kiba ETL v0.5 (LGPL)
22. Open-Source All The Thingz! No Broke, No Burnout!
23. Consulting & Internal Use
27. Smaller OSS support cost ✔ Increased OSS applicability ✔ Time-boxing of OSS activity ✔
28. Funding! Support! Sustainable ✔
29. How to Keep Innovating?
30. Have a life! (outside coding) ❤ ! "
31. Make Quality Time
32. Intellectual Property "Non Exclusive Rights"
35. A form of balance / sustainability
36. Questions? info@logeek.fr
37. Image credits https://commons.wikimedia.org/wiki/File:OricAtmos01a.jpg https://fr.wikipedia.org/wiki/Balance_Roberval#/media/Fichier:BalanceRobervalwhite.jpg https://giphy.com/gifs/sad-panda-Q3Ou3eFKMTFn2 https://commons.wikimedia.org/wiki/File:ViennaConventionroadsignB2a.svg devil vs angel -> cannot find official author to credit so far https://twitter.com/SRUofPA/status/1029711272672149506 https://www.pexels.com/photo/balance-macro-ocean-pebbles-235990/