Tensorboard: Almost a one stop shop for Machine Learning Development

This talk will focus on the areas of Machine Learning development (specifically Computer Vision problems) and will discuss how using certain tools will make your life easier. Also, discuss some areas where developing custom tools is beneficial instead of using open-source tools and the trade-off for making this choice.
We will discuss why we should be using Tensorboard and why it is the only tool you will need to do machine learning development. We will see how it compares to other model development tools such as Visdom, Netron and several other Proprietary(and expensive) tools.
We will also see some small snippets of examples of what is possible with Tensorboard and how to interpret machine learning models, monitor training stats, debug models and compare experiments. This includes using tensorboard as an interactive front-end for monitoring the models and visualising its performance on test set and how we can make it to be framework agnostic and work with Pytorch, Keras and not be dependent on tensorflow.

Key Takeaways.
1. Understand the features available in Tensorboard and why it is the only tool that you will ever need.
2. How to make it work with any machine learning framework.

Transcript

1. Tensorboard: Almost a one stop shop All you need for Machine Learning Development
3. Model development cycle Model Architecture Model Training Model Evaluation Model Debugging
4. ML Engineer wonders.. 1. How does my network actually look like 2. Is my network learning at all 3. How well is it doing on train/val set over the course of training 4. Is it better than my previous version of model 5. How does the network weights change while training 6. Ok so my model is not learning at all. How do I debug it..
5. Tools used for model development Terminal Visdom Others Tensorboard 1. Visualise model graph 2. Monitor training/validation loss 3. Monitor model performance while training 4. Evaluate model and visualise performance 5. Visualise incremental performance on test images 6. Visualise Embeddings interactively 7. Clean and interactive 8. Easy to use 9. Compare experiments easily Netron Polyaxon Hyperdash Custom None None Hydrosphere Hyperdash Valohai
6. Myth Truth Tensorboard only works with TensorFlow Tensorboard is an independent project which can be used with any machine learning framework.
7. Use open source tool Build Custom Tool Pros 1. Community Supported and well tested 2. Well documented 3. Easier to get started with Cons 1. Might not be suitable for you very specific needs 2. Bug resolution takes time 3. No control over architecture and workflow 4. Might not follow input/output format compatible with your architecture Cons 1. Will need significant time and effort to develop Pros 1. You will have absolute control over the development 2. Bug resolution is faster 3. Will be having input/output format compatible with rest of the architecture 4. Endless possibilities of features
8. Question: How does my network actually look like?
9. Tensorboard for Monitoring
10. Tensorboard for Visualising Predictions
11. Use Tensorboard Without Tensorflow - TensorboardX
12. Custom built tools for Tensorboard: Visualise confusion matrix as the model is training. Notice that the second model has not learned at all and only predicting one class