Let's solve cardiac diseases together: Python meets medical imaging and machine learning

Are you thinking of applying computer vision and machine learning in your new projects? Are you excited about artificial intelligence and the future of healthcare? Then join us for this talk, and at least for a moment become a part of our start-up journey at KardioMe. Come and learn about our own hearts' function and how we use Python for automated medical image analysis to improve understanding of our cardiac anatomy from computed tomography and magnetic resonance images. Together, we will see tools making our healthcare at least a bit more efficient, tools empowering all of us to take better care of our health. I will share with you some our experiences from building prototypes to deploying machine learning systems for automated medical image analysis. I will show you how deep learning with convolutional neural nets can solve some of your computer vision challenges too and how to pick your machine learning framework and image processing library (featuring Keras with Tensorflow, MxNet, scikit-learn, OpenCV, scikit-image, and SimpleITK)

Transcript

1. LET'S SOLVE CARDIAC DISEASES TOGETHER Jan Margeta | | jan@kardio.me @jmargeta
2. HOW IS AI HELPING US TODAY?
3. EVERY 18 SECONDS
4. 45%
5. IMAGING IN CARDIOLOGY X-Ray ultrasound fluoroscopy computed tomography Kelly 2007 Carmo et al. 2010 Arnold et al. 2008 Leyva et al. 2010
6. ? ? SPOT THE DIFFERENCES
7. HEALTHY ISCHEMIC HEART FAILURE SPOT THE DIFFERENCES
8. VOLUME: 142 ML VOLUME: 212 ML MEASURE THE DIFFERENCES
9. Xue et al. 2015 SEGMENTATION 30 minutes to do manually
10. WHAT A HUMAN SEES
11. WHAT A HUMAN SEES
12. WHAT A HUMAN SEES
13. WHAT A COMPUTER SEES
14. WHAT A COMPUTER SEES
15. WELCOME TO COMPUTER VISION Margeta et al. 2013, Joint work with Inria and Microsoft Research Cambridge
16. MACHINE LEARNING SOLVING PROBLEMS WITH DATA prediction = predict (data, model)
17. IMAGE RECOGNITION IN 6 LINES OF CODE from keras.applications import imagenet_utils from keras.applications.vgg16 import VGG16 # Load and prepare input images images_raw = load_images() images = imagenet_utils.preprocess_input(images_raw) # Load a pretrained image classification model model = VGG16(include_top=True, weights='imagenet') # Do the prediction predictions = model.predict(images)
18. EXCELLENT FOR NATURAL IMAGES
19. FINDING THE IMAGE REPRESENTATION
20. EXTRACTING VISUAL FEATURES # Load a pretrained classification model source_model = VGG16(weights='imagenet') # Define feature extractor feature_layer = source_model.get_layer('conv4') feature_extractor = Model( input=fix_model.input, output=feature_layer.output) # Extract features features = feature_extractor.predict(images) Check out Deep visualization toolbox on youtube
21. USING THE FEATURES WITH SCIKIT LEARN from sklearn.linear_model import LinearRegression def reshaper(features): return features.reshape(features.shape[0], -1) X_train = reshaper(features) Y_train = load_groundtruth() classifier = LinearRegression() classifier.fit(X_train, Y_train) features_test = feature_extractor.predict(images_test) X_test = reshaper(features_test).reshape(, -1) Y_test = classifier.predict(X_test)
22. LANDMARK REGRESSION Margeta et al. 2015, Joint work with Inria and Microsoft Research Cambridge
23. START FROM SCRATCH from keras.models import Model from keras.layers import Input, Conv2D, Dense from keras.layers import GlobalAveragePooling2D num_outputs = 4 num_filters = 16 # Transformer of a gray image of any size to 16 outputs input = Input(shape=(1, None, None)) c0 = Conv2D(num_filters, 3, 3, activation='relu')(input) p = GlobalAveragePooling2D()(c0) output = Dense(num_outputs, activation='softmax')(p) model = Model(input=input, output=output)
24. TRAIN THE MODEL # Loss function - task dependent # high for bad parameters, low for good ones # e.g. for image recognition loss_function = 'sparse_categorical_crossentropy' # Compile the model and fit model.compile(loss=loss_function, optimizer='adam') model.fit(images, labels) # Save the model for reuse model.save('model.h5')
25. DEPLOY AS A WEB SERVICE FLASK (CHECKOUT TENSORFLOW-SERVING) DOCKER CONTAINERS (CHECKOUT KUBERNETES)
26. EXPOSING THE MODEL WITH FLASK *it requires a little bit more care import keras from flask import Flask, jsonify, request app = Flask(__name__) model = keras.models.load_model('model.h5') @app.route('/predict', methods=['POST']) def predict(): X = request_to_numpy(request) Y = model.predict(X) prediction = convert_prediction(Y) return jsonify(output=prediction) app.run(port=5000, threaded=False)
27. DEFINE THE DOCKERFILE FROM python:3.5 RUN mkdir -p /usr/src/app COPY server.py /usr/src/app/ COPY model.h5 /usr/src/app/ COPY requirements.txt /usr/src/app/ WORKDIR /usr/src/app RUN pip install -r requirements.txt EXPOSE 5000 CMD python server.py
28. # Build the container docker build -t kardiome/model-pycon . # Run the container docker run -d -p 5000:5000 kardiome/model-pycon # Call the service curl -X POST -F 'image=@/data/im.png' localhost:5000/predict
29. FROM 30 MINUTES TO 12 SECONDS* *ON A GPU
30. Joint work with and Inria IHU Liryc
31. ITERATE FAST ONE METRIC TO RULE THEM ALL
32. PROGRESS WITH CONFIDENCE HAVE REPEATABLE PIPELINES
33. NO GLORY IN DATA PREPARATION BUT IT MUST BE DONE
34. HAVING A SMALL DATASET? DO SOMETHING ABOUT IT https://affinelayer.com/pixsrv/
35. GOT UNLABELED DATA? DON'T BE LAZY, JUST ANNOTATE IT IF YOU CAN, THERE ARE TOOLS TO HELP YOU Margeta et at. 2015 - Joint work with Inria and Microsoft Research Cambridge Check out also Scikit learn example on Label Propagation digits active learning
36. BE PRACTICAL HAVE AND OPEN MIND
37. PYTHON + AI + MEDICINE = A⚕
38. THANKS PyCon SK, Maggie, Krissy, Karol, Inria, IHU Liryc, Microsoft Research Cambridge CONNECT WITH ME Jan Margeta | | PyData meetup 27 March 2017 jan@kardio.me @jmargeta
39. RESOURCES PhD thesis - Jan Margeta PhD thesis - Rocio Cabrera Lozoya Book From Andrew Ng Fast AI Notebooks and course Visualizing convnets Conv filter visualization Transfer learning with MNIST
40. Keras and pretrained models Detecting cancer with deep learning Dermatologist-level classification of skin cancer with deep neural networks Sunnybrook cardiac dataset Cardiac death stats Label propagation with scikit learn
41. THE VERY END
42. HUMAN HEART
43. HUMAN HEART Right lung Left lung Upper body
44. VGG NETS
45. CONVOLUTION
46. WHICH LIBRARIES?
47. COMPUTER VISION AND DATA LOADING OPENCV, SIMPLEITK, SCIKIT- IMAGE, PYDICOM
48. GENERAL MACHINE LEARNING SCIKIT LEARN AND SPARK
49. DEEP LEARNING KERAS, MXNET OR TENSORFLOW