Slide 1
Slide 1 text
Building a Data-Driven
Web App That Everyone
Can Use
galuh.me | @galuhsahid
Slide 2
Slide 2 text
What is a data-driven
web application?
Slide 3
Slide 3 text
Let's start with a
problem
Slide 4
Slide 4 text
No content
Slide 5
Slide 5 text
No content
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
$ git clone
$ jupyter notebook
$ python mycoolmodel.py
Slide 9
Slide 9 text
$ git clone
$ jupyter notebook
$ python mycoolmodel.py
Slide 10
Slide 10 text
Web applications are
super cool
Slide 11
Slide 11 text
It takes a lot of
people to build one
Slide 12
Slide 12 text
It takes a lot of
people to build one
Myth
Slide 13
Slide 13 text
If you want to build one
yourself, there are so many
things that you have to learn
Slide 14
Slide 14 text
If you want to build one
yourself, there are so many
things that you have to learn
Myth
Slide 15
Slide 15 text
You can build one
yourself
Fact
Slide 16
Slide 16 text
You can build one
yourself
Fact
... and it doesn't have
to take forever
Slide 17
Slide 17 text
+
+
Slide 18
Slide 18 text
About our web app
• Allows users to input their own data
• Displays data from an outside source (e.g. a third-party API)
• Displays the prediction result of a model we've trained previously
• Displays a graph that is dynamic--based on the user input
Slide 19
Slide 19 text
About our web app
• Allows users to input their own data
• Displays data from an outside source (e.g. a third-party API)
• Displays the prediction result of a model we've trained previously
• Displays a graph that is dynamic--based on the user input
Works as a prototype, demo, or simple minimum viable product
(MVP) (making a scalable web app is a whole different story!)
Slide 20
Slide 20 text
From something like this: To something like this:
Slide 21
Slide 21 text
from flask import Flask
app = Flask(__name__)
@app.route('/')
def index():
return "Hello, world!"
app.py
$ export FLASK_APP=app.py
$ flask run
* Running on http://
127.0.0.1:5000/ (Press
CTRL+C to quit)
* Restarting with stat
Flask
Slide 22
Slide 22 text
from flask import Flask
app = Flask(__name__)
@app.route('/')
def index():
return "Hello, world!"
app.py
Slide 23
Slide 23 text
from flask import Flask
app = Flask(__name__)
@app.route('/')
def index():
return "Hello, world!"
@app.route('/result')
def get_result():
result = 100000
return str(result)
app.py
Slide 24
Slide 24 text
from flask import Flask,
render_template
app = Flask(__name__)
@app.route('/')
def index():
return
render_template("index.html")
@app.route('/result')
def get_result():
result = 100000
return str(result)
app.py
Hello, world!
templates/index.htmlSlide 25
Slide 25 text
from flask import Flask,
render_template
app = Flask(__name__)
@app.route('/')
def index():
return
render_template("index.html")
@app.route('/result')
def get_result():
result = 100000
return
render_template("result.html",
result=result)
app.py
{{ result }}
templates/result.htmlSlide 26
Slide 26 text
Getting user input
Slide 27
Slide 27 text
Campaign name:
templates/index.html Getting user input 127.0.0.1:500/result?campaign=some-campaignSlide 28
Slide 28 text
from flask import Flask,
render_template, request
...
@app.route('/result')
def get_result():
campaign =
request.args.get('campaign',
None)
return
render_template("result.html",
campaign=campaign)
app.py
Getting user input
{{ campaign }}
app.py 127.0.0.1:5000/result?campaign=some-campaign query stringSlide 29
Slide 29 text
Getting some data
Slide 30
Slide 30 text
Getting some data
example.com/api/v1/campaigns?name=some-campaign
Slide 31
Slide 31 text
import json
import urllib
...
@app.route('/result')
def get_result():
campaign = request.args.get('campaign',
None)
base_url = 'http://example.com/api/v1/
campaigns?id='
url = "{}/{}".format(base_url, campaign)
response = urllib.urlopen(url)
data = json.loads(response.read())
return render_template("result.html",
data=data)
app.py
Getting some data
Slide 32
Slide 32 text
{{ data }}
result.html
Getting some data
Slide 33
Slide 33 text
What to do with our
model?
Slide 34
Slide 34 text
What to do with our model?
Slide 35
Slide 35 text
What to do with our model?
• We only want to use one model with the best accuracy/
smallest error
Slide 36
Slide 36 text
What to do with our model?
• We only want to use one model with the best accuracy/
smallest error
• We don't want to retrain our model every time a new request
hits our web application
Slide 37
Slide 37 text
Make it persistent!
What to do with our model?
• pickle: built-in Python module to serialize and de-serialize a
Python object structure
• joblib: a library that provides utilities for pipelining Python jobs
• Or we can use each library's specific method (libsvm:
svm_save_model and svm_load_model, TensorFlow:
tf.train.Saver() class)
Slide 38
Slide 38 text
Making our model persistent
from sklearn.ensemble import
RandomForestRegressor
from sklearn.externals import joblib
rf = RandomForestRegressor(n_estimators=300)
rf.fit(X_train, y_train)
joblib.dump(rf, 'model.sav')
Your original Python script/Jupyter Notebook
Slide 39
Slide 39 text
import json
import pandas as pd
from sklearn.externals import joblib
def get_prediction(data):
df_data = pd.DataFrame([data]).astype(float)
model = joblib.load("model.sav")
predicted_amount = model.predict(df_data)[0]
target_amount = data["target_amount"]
if (target_amount > predicted_amount):
is_funded = False
else:
is_funded = True
prediction = {"amount": predicted_amount,
"is_funded": is_funded}
return json.dumps(prediction)
model.py
Making our model persistent
Slide 40
Slide 40 text
from model import get_prediction
...
@app.route('/result')
def get_result():
...
data = json.loads(response.read())
prediction = json.loads(get_prediction(data))
return render_template("result.html",
data=data,
prediction=prediction)
app.py
Making our model persistent
Slide 41
Slide 41 text
{{ data }}
{{ prediction }}
result.html
Making our model persistent
False
is_funded':
Slide 42
Slide 42 text
Some pitfalls...
Slide 43
Slide 43 text
Some pitfalls
• Make sure you're loading trusted data
Slide 44
Slide 44 text
Some pitfalls
• Make sure you're loading trusted data
• Saving a model using a particular version of a library and
loading it using another version might give unexpected
results
Slide 45
Slide 45 text
Some pitfalls
• Make sure you're loading trusted data
• Saving a model using a particular version of a library and
loading it using another version might give unexpected
results
So what to do? Keep your:
• Training data
• Source code that generates the model
• Version of the library used
• Dependencies used
• Cross validation score obtained
Slide 46
Slide 46 text
Data visualization
Slide 47
Slide 47 text
import matplotlib.pyplot as plt
y = [1, 2, 3, 4, 5]
x = [0, 2, 1, 3, 4]
plt.plot(x, y)
Your original Python script/Jupyter Notebook
Data visualization
Slide 48
Slide 48 text
Data visualization
import matplotlib.pyplot as plt
import StringIO
import base64
import json
def get_plot_url(fb_shares):
y = [1, 2, 3, 4, fb_shares]
x = [0, 2, 1, 3, 4]
plt.plot(x, y)
img = StringIO.StringIO()
plt.savefig(img, format='png')
img.seek(0)
plot_url = base64.b64encode(img.getvalue())
return json.dumps({'plot_url': plot_url})
graph.py
Slide 49
Slide 49 text
{{ data }}
{{ prediction }}
{{ graph }}
result.html
Data visualization
Slide 50
Slide 50 text
Putting everything
together
Slide 51
Slide 51 text
Campaign Success Estimator
{{ data["id"] }}
Statistics
- Story word count: {{ data["story_word_count"] }}
- Number of images: {{ data["number_of_images"] }}
- Number of videos: {{ data["number_of_videos"] }}
- Number of Facebook shares: {{ data["number_of_fb_shares"] }}
- Target amount: {{ data["target_amount"] }}
Prediction
Predicted amount: {{ prediction["amount"] }} templates/result.html Putting everything togetherSlide 52
Slide 52 text
Putting everything together
Slide 53
Slide 53 text
Campaign Success Estimator
...
templates/result.html
Putting everything together
templates/result.html
Putting everything together
Slide 54
Slide 54 text
Putting everything together
Slide 55
Slide 55 text
Flask's templating engine:
Jinja2
{% block title %}{% endblock %}
-
{% for user in users %}
- {{ user.username }} {% endfor %}
Slide 56
Slide 56 text
Jinja2 examples
jinja.pocoo.org/docs/2.10/
Slide 57
Slide 57 text
Conditionals
Campaign Success Estimator
html {
font-family: "Arial"
}
.prediction {
margin: 10px;
}
.prediction .funded {
color: #2ecc71;
}
.prediction .not-funded {
color: #e74c3c;
}
...
templates/result.html
Slide 58
Slide 58 text
Conditionals
...
Predicted amount:
{{ prediction["amount"] }}
...
templates/result.html
if is_predicted returns True
if is_predicted returns Frue
Slide 59
Slide 59 text
Conditionals
Slide 60
Slide 60 text
Conditionals
Slide 61
Slide 61 text
Custom filters
...
@app.template_filter('format_currency')
def format_currency(value):
value = int(value)
return "Rp{:,}".format(value)
...
app.py
Slide 62
Slide 62 text
Custom filters
...
Target amount: {{ data["target_amount"]|
format_currency }}
Prediction
Predicted amount:
{{ prediction["amount"]|format_currency }}
...
templates/result.html
Slide 63
Slide 63 text
Custom filters
Slide 64
Slide 64 text
What's next?
• Deploy it and share it with the world:
• Heroku
• Google App Engine
• And many other options
• Add some more functionalities:
• Flask Admin
• Flask Login
• ... and so on
Slide 65
Slide 65 text
What's next?
• Make it more interactive
• react-flask
• react-redux-flask
• flask-vuejs
• flask + d3.js
• Make it more scalable
Slide 66
Slide 66 text
Examples
Slide 67
Slide 67 text
Flask
Source code: https://github.com/galuhsahid/campaign-success-predictor
Paper: https://ieeexplore.ieee.org/document/8355046/
Campaign Success Predictor
Slide 68
Slide 68 text
Campaign Success Predictor
Flask
Paper: https://ieeexplore.ieee.org/document/8355046/
Source code: https://github.com/galuhsahid/campaign-success-predictor
Slide 69
Slide 69 text
Indonesian Word Embedding (http://indonesian-word-embedding.herokuapp.com)
Flask + Vue.js
Source code: https://github.com/galuhsahid/indonesian-word-embedding
Slide 70
Slide 70 text
Resources
• Flask documentation
• Jinja2 documentation
Slide 71
Slide 71 text
That's it.
Thanks!