Designing Data Pipelines for Machine Learning Applications

Transcript

1. DESIGNING DATA PIPELINES FOR MACHINE LEARNING APPLICATIONS (WITH KAFKA AND

   KAFKA STREAMS) Alexis Seigneurin

2. Me • Data Engineer • Kafka, Spark, AWS… • Blog:

   aseigneurin.github.io • Twitter: @aseigneurin

3. Data Pipeline?

4. Data Pipeline • It is the journey of your data

   • Ingest, transform, output… all in streaming • Kafka • Sometimes called a DAG • Apache Airflow

5. Data Pipeline - Example

6. Data Pipeline - Airflow

7. Key elements • Streaming • Transformations = Independent jobs •

   (= different technologies?) • Data can be consumed by multiple consumers

8. Machine Learning Application?

9. Machine Learning Application • Use historical data to make predictions

   on new data • Train a model on historical data • Apply the model on new data

10. Example - Real estate pricing Source: https://www.kaggle.com/shree1992/housedata#data.csv F e a

    t u r e s Label

11. A batch oriented process • Training on a batch of

    data • Can take hours (days?) • Validate + deploy the new model

12. Data Pipeline + ML?

13. Batch + Streaming • Training in batch to create a

    model • Streaming app to use the model

14. Training • Done from time to time (e.g. every other

    week) • With a fixed data set • Export a model • A new model is deployed when it has been validated by a Data Scientist

15. Streaming app • Use Kafka as a backbone • Kafka

    Streams to implement transformations • Need a way to use the model on the JVM

16. Architecture ML Model ML Training Labeled data Unlabeled data (in

    Kafka) Apply the model (in Kafka Streams) Data with predictions Batch Streaming

17. ML frameworks

18. Python libraries • scikit-learn, TensorFlow, Keras… • ⚠ Need to

    build and expose a REST API • ⚠ Scaling can be complicated

19. Spark ML • Integrated in Spark • ⚠ Have to

    use Spark Streaming

20. Cloud-hosted services • AWS SageMaker, Google Cloud Machine Learning Engine…

    • No code to write to serve the models • ⚠ Less control over how the model is served

21. H2O.ai • Java-based ML platform • Can download a POJO

22. Creating a model with H2O

23. Importing the data

24. Preparing the model • Set column types: numeric, enum… •

    Split the dataset: 70/20/10 (training/validation/test) • Algorithm: Gradient Boosting Machine

25. Model with H2O

26. Model with H2O

27. POJO of the Model

28. Using the model

29. In a simple app Model Data Prediction

30. In a simple app

31. { "date": "2014-05-02 00:00:00", "bedrooms": 2, "bathrooms": 2.0, "sqft_living": 2591,

    "sqft_lot": 5182, "floors": 0.0, "waterfront": "0", "view": "0", "condition": "0", "sqft_above": 2591, "sqft_basement": 0, "yr_built": 1911, "yr_renovated": 1995, "street": "Burke-Gilman Trail", "city": "Seattle", "statezip": "WA 98155", "country": "USA" } Stream processing { "date": "2014-05-02 00:00:00", "bedrooms": 2, "bathrooms": 2.0, "sqft_living": 2591, "sqft_lot": 5182, "floors": 0.0, "waterfront": "0", "view": "0", "condition": "0", "sqft_above": 2591, "sqft_basement": 0, "yr_built": 1911, "yr_renovated": 1995, "street": "Burke-Gilman Trail", "city": "Seattle", "statezip": "WA 98155", "country": "USA", "price": 781898.4215855601 } Kafka Streams application Kafka topic Kafka topic

32. Kafka Streams • Client library for Java and Scala •

    DSL: stream(), map(), filter(), to()... • Aggregations, joins, windowing • KStream / KTable • Simple deployment model • Allows to create "microservices”

33. Kafka Streams app Read from the source topic Apply the

    model Write to the output topic Start consuming

34. Deploying the application • Kafka Streams = a plain Java

    app • Run the app: • On a VM (e.g. EC2) • In Kubernetes • …

35. Result

36. Think of how your ML framework can serve your models

    T I P # 1

37. Serving the model • Your choice of ML framework constrains

    you! • Embedded model or REST API? • Are you ok running Python code in production? • (Spotify is not - JVM only)

38. Single app to deploy Kafka Streams application Kafka topic Kafka

    topic ML model

39. 2 apps to deploy Kafka Streams application Kafka topic Kafka

    topic ML model HTTP request REST API

40. Same feature engineering in streaming and training T I P

    # 2

41. Feature Engineering • Need to apply the same transformations in

    batch & streaming • Scaling, date conversions, text extraction… • Challenging!

42. Feature Engineering • Option 1: Use the same UDFs •

    Option 2: Featran? (github.com/spotify/featran)

43. Calculate statistics about your data (and compare) T I P

    # 3

44. Know your data • Calculate statistics from your training data

    • E.g. min, max, average, median… of the square footage • E.g. nulls?

45. Check your new data • In streaming: • Check: min,

    max, null… • In batch: • Check: average, median… ‣ Outliers → Update the model?

46. Enrich the data in your pipeline T I P #

    4

47. Enrich your data • Add features from other datasets •

    Use reference datasets: • Zip code → details about the location • User ID → details about the user • IP address → location • …

48. 2 apps to deploy Kafka Streams enrichment Kafka topic Kafka

    topic Reference data as a CDC stream Join KStream KTable Reference data

49. Design your pipeline for testing T I P # 5

50. Testing • Final test in real conditions • Production data

    • No real output

51. Pre-production testing Production application Input topic Output topic (production) ML

    model Pre-prod application Output topic (pre-prod) ML model Different consumer group Compare

52. Design your pipeline for A/B testing T I P #

    6

53. A/B testing • 2 models running in parallel • Compare

    the outputs • Split: • Embedded model → partition-based split • REST API → more control

54. A/B (pre-split) Kafka Streams application Input topic Output topic ML

    model Kafka Streams application ML model Same consumer group A B

55. A/B (post-split) Kafka Streams application Input topic Temp topic A

    ML model Kafka Streams application Temp topic B ML model Merge A B Different consumer groups Output topic

56. Design your pipeline for model updates T I P #

    7

57. Updates • Update: • Embedded model → deploy a new

    version of the app • REST API → update the API • Stop/start vs Rolling updates

58. Design your pipeline to be resilient to failures T I

    P # 8

59. High Availability • Deploy more than one instance of your

    app • Same consumer group / Different physical locations

60. High Availability Kafka broker Availability Zone 2 Kafka broker Availability

    Zone 1 Kafka broker Availability Zone 3 Kafka Streams app VM1 / Availability Zone 1 Kafka Streams app VM2 / Availability Zone 2

61. Summary & Conclusion

62. Summary • Training in batch / Predictions in streaming •

    Model: Embedded vs REST API • Kafka + Kafka Streams • Design for testing, H/A, updates… • Blog: Realtime Machine Learning predictions with Kafka and H2O.ai • https://aseigneurin.github.io/2018/09/05/realtime-machine-learning-predictions-wth-kafka-and-h2o.html

63. Thank you! @aseigneurin