How Adopting Elastic at Bankwest was a Catalyst for Cultural Change

Digital disruption is driving sweeping change across many established industries and finance is no exception. In order to increase business agility and foster experimentation, Bankwest's technology strategy is focused on breaking down existing monolithic systems into smaller distributed components organized around business capability, from which a requirement for centralized logging emerged. Elastic looked like a natural fit, and whilst we had confidence it would meet our requirements, what we didn't anticipate was how it would become a catalyst for wider cultural change.

This case study will look back at the journey we've been on at Bankwest with our adoption of Elastic. We'll cover the major use cases we have for it, including how an innovative grass roots experiment led to the creation a real-time search solution across over 4 million customer records with Elastic as it's backbone. What all these use cases will demonstrate is the business value that can be unlocked from organization's existing data that previously went unrealized.

This session is suitable for anyone who is interested in gaining insight into the kind of challenges faced and takeaways from real life initiatives leveraging the Elastic Stack.

Rob Crowley l Technical Product Owner, APIs l Bankwest

Transcript

1. Rob Crowley, Technical Product Owner - APIs How Adopting Elastic at Bankwest was a Catalyst for Cultural Change Bankwest 29th Nov 2016
2. Digital disruption is driving change across established industries and finance is no exception.
3. Our customers’ adoption of technology is defining our agenda Number of interactions Revenue per interaction Experience Economy Product Economy Store Monthly decreasing Internet Monthly decreasing Mobile Daily increasing Wearables Several times a day increasing Interconnected Always on increasing
4. the species that survives is the one that is able to adapt to and adjust best to the changing environment - Charles Darwin
5. AD Producer Logstash nginx FRONT END Rabbit MQ BACK END Consumer Logstash HA Proxy (Canonical) Use Case: Centralised Logging Push to Rabbit MQ cluster at opposite site via Shovel <auth> Shield Watcher ELASTIC Kibana
6. Surprise #1 – Analytics from Kibana dashboards transformed data into information.
7. “The only way to win is to learn faster than anyone else.” - Eric Ries
8. Surprise #2 - Watcher precipitated a shift from reactive to proactive support models.
9. Encryption at rest for PII data was a challenge.
10. How you tackle it depends on your company’s risk appetite. - Elastic Search Plugin (3rd Party Vendor Options Available) - Lucene Plugin (Introduced ~25% overhead) - Disk Level (dm-crypt) - Lightweight governance (& Watcher to keep us honest)
11. Surprise #3 – Elastic use cases expanded from supporting product delivery to powering end user experiences.
12. Idea + Team + Elastic = Business Value
13. (Emergent) Use Case: Customer Search Search API F5 MQ Web API Layer System of Record OpenStack Elastic Data Tenant Cloud 2 OpenStack Elastic Data Tenant Cloud 1 Sync Agent Customer Domain Bulk Agent Customer Data Extract GET /customers/{id} HTTP/1.1 <push>
14. Cloud computing enabled a low cost iterative approach to optimising Elastic cluster design.
15. Optimizing full text queries can be non trivial. A robust automated testing process is table stakes.
16. Takeaways