RubyKaigi 2017: Mapping your world with Ruby

In the wake of the March 2011 earthquake, many noticed a lack of good environmental data regarding radiation. The Safecast project was born from that need and our Ruby-based infrastructure how is home to over 70 million data points.

In this talk we'll go over the basics of the project, what we've learned over the last 6 years of running a volunteer-based Ruby project, and our plans for future expansion into tracking both radiation and air quality data.

Transcript

1. Mapping Your World
 with Ruby @matschaffer Site Reliability Engineer elastic.co - safecast.org
2. Are you safe?
5. http://safecast.org/tilemap/?y=34.39104&x=132.45052&z=16&l=0&m=0
7. - the project - ruby - learnings - involvement
8. 14:46:24 JST - March 11, 2011
9. NHK News - April 22nd, 2011
10. no data
11. no sensors
12. omg
13. Experience DIY Development Action Funding
15. April 20th, 2011
17. It worked!
18. It worked! April 24th, 2011
22. bgeigie mobile, crowd-sourced sensors
23. Miles Dyson, #2964 https://www.youtube.com/watch?v=6x_5pi7v65k
25. 76 million data points
30. pointcast fixed sensors
34. solarcast fixed/mobile sensors
35. drop and forget driving principle
36. - particulate + radiation - global wireless - solar power - weatherproof
38. Dual particulate temp & humidity sensors Dual radiation sensors Battery pack and solar power controller Cellular module LoRa module GPS Accelerometer High-voltage supply On/off & external power Weatherproof IP6X box with tripod mount Air outlet Air inlet Main CPU
 w/BLE
39. Devices: we have many, lots of data • bGeigie Nano (and friends) • Pointcast • Solarcast • Large community with other devices as well
40. the infrastructure
41. api.safecast.org
42. ingest.safecast.org
43. ingest.safecast.org
44. Ruby @ Safecast
45. The pros
46. good DB hygiene RUBY: THE PROS
47. interconnectivity RUBY: THE PROS
48. approachability RUBY: THE PROS
49. The cons
50. performance (sometimes) RUBY: THE CONS
51. high rate of change RUBY: THE CONS
52. high market rate RUBY: THE CONS
53. the upgrades 6 years of them
55. - love your test suite - mind your CI - deploy frequently KEY LEARNINGS
56. the data 6 years of that too
57. - ~30G postgis db - growing rapidly
 - mostly two tables LONG TERM TIME SERIES
58. - hot/warm/cold
 - table/db sharding
 - more formats (bigger data)
 FUTURE PLANS
59. the people 6 years, 30 api committers
61. Key learnings
62. People come and go PEOPLE: KEY LEARNINGS
63. Good > perfect PEOPLE: KEY LEARNINGS
64. Work small, deliver quickly PEOPLE: KEY LEARNINGS
65. Follow interests/passion PEOPLE: KEY LEARNINGS
66. Jump on holiday opportunities PEOPLE: KEY LEARNINGS
67. Why Safecast? (for me)
68. 2013: Moving to Japan WHY SAFECAST?
69. Freedom & flexibility WHY SAFECAST?
70. Amazing connections WHY SAFECAST?
72. In summary…
73. Ruby for a cause, improve your world Anyone can do it Exciting, rewarding, fun
74. Safecast: Always open Ingest layer (ruby/golang/php): https://github.com/Safecast/safecastapi https://github.com/Safecast/ingest https://github.com/Safecast/TTServe Mapping (javascript): https://github.com/Safecast/Tilemap Hardware (C/KiCad) https://github.com/Safecast/bGeigieNanoKit https://github.com/Safecast/Solarcast
75. Questions? mat@schaffer.me github.com/matschaffer twitter.com/matschaffer ೔ຊޠ΋࿩ͤ·͢
76. Thank you! safecast.org github.com/safecast mat@schaffer.me github.com/matschaffer