Scaling a backend for a big data and blockchain environment by Rafael Ríos at Big Data Spain 2017

Scaling a backend for a big data and blockchain environment by Rafael Ríos at Big Data Spain 2017

2gether is a financial platform based on Blockchain, Big Data and Artificial Intelligence that allows interaction between users and third-party services in a single interface.

https://www.bigdataspain.org/2017/talk/scaling-a-backend-for-a-big-data-and-blockchain-environment

Big Data Spain 2017
November 16th - 17th Kinépolis Madrid

Cb6e6da05b5b943d2691ceefa3381cad?s=128

Big Data Spain

December 05, 2017
Tweet

Transcript

  1. None
  2. Scaling a backend for big data and blockchain environment

  3. 1. Introduction (The companies, project and me) 2. Backend challenge

    3. Big Data Integration 4. Blockchain Integration ·············· P. 3 ··················································· ················ P. 7 ··················································· ·············· P. 17 ··················································· ········ P. 21
  4. @ganchix Head of Blockchain / Backend Me ganchix

  5. The companies

  6. None
  7. Fecha The project • Financial platform • Marketplace • Tokenization

    • Operate with cryptocurrency • Liquidity predictive models • Credit scoring • Product recommendation
  8. Backend challenge - The evolution Dapp Smart Contract Ethereum

  9. Backend challenge - The evolution

  10. Backend challenge - The evolution Dapps problem: • Noncryptocurrency users

    • Problem with some integrations • Legal problem
  11. Backend challenge - The evolution

  12. Backend challenge - The evolution

  13. Fecha Backend challenge - Why microservices? • Migration of Dapp

    easier • Easy to scale • Polyglot Database and Languages
  14. Why don't use exclusively blockchain with a database?

  15. 1. Spring Cloud Netflix and Kubernetes • Easy to learn.

    • Nice integrations • Spring 5 reactive 2. Docker • Most adopted vendor technology for containers • Well supported 3. Kubernetes • Multi-cloud provider and on-premises data centers • Self-repair and health check capabilities • Auto-scale Backend challenge - Microservice Architecture Stack
  16. Backend challenge - Microservice Architecture Stack

  17. Backend challenge - Deployment

  18. PFM values generation from user data. Apache Spark + Cassandra

    Forecast prediction and regeneration of this models Apache Spark + Cassandra Product recommendations based on the economic profile of the user and his real needs. Apache Spark + Cassandra + Neo4j Credit scoring calculation Apache Spark + Cassandra Big Data Integration - Tasks
  19. • Tasks are hard, needs: • Time • Resources •

    Not Real Time is needed. • Event Driven Architecture. Big Data Integration - Events
  20. Big Data Integration - How?

  21. Big Data Integration - RabbitMQ vs Kafka Kafka RabbitMq •

    RabbitMQ is designed as a general purpose message broker • Support existing protocols like AMQP, STOMP, MQTT. • Finer-grained consistency control/guarantees on a peer- message. • Complex routing. • Apache Kafka is designed for high volume publish-subscribe messages and streams, meant to be durable, fast, and scalable. • Event Sourcing • Your application needs access to stream history. • No complex routing. https://content.pivotal.io/blog/understanding-when-to-use-rabbitmq-or-apache-kafka
  22. • Deployed in Kubernetes. • Only accessible by NodeJS API.

    • All keys are stored in secrets vaults. • Used for: • Tokenization • Transactions of users Blockchain Integration - Private Ethereum
  23. Blockchain Integration - Private Ethereum

  24. Blockchain Integration - Ethereum Main Net

  25. Blockchain Integration - Ethereum Main Net

  26. Blockchain Integration - Ethereum Main Net • We are the

    owner of the wallets • We use Infura to connect blockchain • Used for: • Payment • Transfers
  27. @ganchix @2getherbank @bigeeksoftwa re Thanks! ICO is coming!