Electricsearch from 0 to 400 volts at Blue-Solutions by Polyconseil

At Blue-Solutions we use Elasticsearch to store and index telemetry data from thousands of blue devices such as electric vehicles, charging points, batteries, etc. to send data to our backend applications, generate alerts and gain insight on their use. In this talk we will describe how we fully automated Elasticsearch cluster deployments (on-premise) using Ansible and how we drill down into the data in our blue devices even further.

Benoit Calvez | Sr. Developer | Polyconseil
Thierno Barry | Technical Architect | Polyconseil

Transcript

1. Benoit Calvez | Sr. Developer Thierno Barry | Technical Architect Electric {search} from 0 to 400 volts Blue Solutions by Polyconseil November 9, 2016
2. by Agenda •  Deployment automation with Ansible •  Use case #1: Network events •  Use case #2: Charging point events •  Use case #3: Primary telemetry storage •  Use case #4: Business events storage •  What’s next? 2
3. by Thierno Barry I’m a devOPS barryib @barryib _nodes/dzen,barryib/info 3 Benoit Calvez I’m a DEVops dzen @dzen
4. by 4 100% electric car sharing (Personal & Enterprise) 4000 cars, 6000 charge points on 1100 stations (Paris) Opened in Paris, Lyon, Bordeaux, London, Indianapolis and Turin by … soon in Singapore
5. by Requirements and Challenges Collect them all… 5
6. by Requirements and Challenges 6
7. by Automated Deployment From Ansible with ❤… 7
8. by 8 Automate Deployments with Ansible •  Use the official ansible-elasticsearch role https://github.com/elastic/ansible-elasticsearch •  Custom playbooks and roles to –  Activate offline resources (RAM and CPU) –  Activate Elasticsearch –  Configure security features (formerly Shield) in X- Pack –  Deploy templates –  Install and configure Curator –  Deploy backup scripts ❤
9. by 9 Automate Deployments with Ansible •  Wrote custom Ansible’s modules •  Pushing back to ansible-modules-extras –  Manage Elasticsearch plugins (Ansible 2.2 and 2.1.2) –  Manage Kibana plugins (Ansible 2.2) –  Manage users and roles using security features (formerly Shield) in X-Pack (on-going) ❤
10. by Our Use Cases You know, for {search} and more… 10
11. by Charging Point Events (BCA) 11 Syslog Collectd Security (formerly Shield) Monitoring (formerly Marvel)
12. by Network Events with Netflow 12 ne8low sflow Routeurs Switchs dvSwitchs
13. by Network Events with Netflow (Kibana’s Dashboard) 13
14. by 14 Numbers * Numbers per node BCA Netflow Nodes 3 6 Documents 1.5 M per day 80 M per day Indices 1 per day 1 per hour RAM 16 Go 64 Go HDD 300 Go 300 Go
15. by 15 Learning Curve •  First clusters we ran •  Automatization with Ansible •  Machine sizing (java) •  Cluster sizing •  Supervision / Metrology
16. by Primary Telemetry Storage (Bluecap) 16 Bluecap Charging Points Electric Vehicles BaBeries POP POP WSS AMQP AMQP
17. by Primary Telemetry Storage (GSM Signal Quality) 17
18. by Primary Telemetry Storage 18
19. by Business Events Storage (Loglady / Columbo) 19 Loglady / Columbo APP #1 APP #2 APP #3 APP #N … AMQP AMQP
20. by Business Events Storage (Columbo) 20 Keep calm and watch Columbo
21. by Business Events Storage (Columbo) 21 * Made with ❤ by WiFirst
22. by 22 Numbers * Numbers per node Bluecap Loglady/Columbo Nodes 3 3 Documents 3 M per day 15 M per day Indices 1 per day No time based indices RAM 32 Go 64 Go HDD 300 Go 300 Go
23. by 23 Learning Curve •  Re-indexation, the hard way •  Rolling upgrade •  Snapshots, thanks to Curator ❤ •  Timelion ❤
24. by Lessons Learned 24 •  Automate everything you can §  Use automation tools like Ansible, Puppet, Chef, etc. •  For number of shards, start big §  To avoid unnecessary re-sharding when adding node •  Curator 4 is awesome §  Snapshot / Restore §  Closing / Opening indices §  Deleting indices
25. by Lessons Learned 25 •  Keep re-indexation in mind and industrialize it §  Use Logstash §  How do you update your documents? o  Build your own re-indexer? §  Use SSD •  Don’t forget monitoring (Marvel vs Prometheus vs Sensu)
26. by What’s next? To 400 volts… and beyond 26
27. by What’s next Technical •  Build our own Re-indexer §  async? §  In dedicated cluster? •  Upgrade to Elastic Stack 5 §  Pipelines with ingest nodes? •  Run it on our kubernetes? 27 Business •  Reporting •  Track batteries and hardware fatigue •  Use telemetry data as feedback for hardware design •  Turn technical data into business insight… •  … and new business opportunities
28. by Time for Questions Ask us anything 28
29. www.elastic.co 29
30. Except where otherwise noted, this work is licensed under hBp://creaUvecommons.org/licenses/by-nd/4.0/ CreaUve Commons and the double C in a circle are registered trademarks of CreaUve Commons in the United States and other countries. Third party marks and brands are the property of their respecUve holders.