August 03, 2019 · 10 mins read
Lesson 2 Part 1 - ML in Production
After my blog post on the first lesson of fast.ai yesterday (which got recognised by the founder of fast.ai!), today is time for the second lesson. Because this lesson consisted of three distinct parts, I decided to write 3 separate blog posts. The first one, the one you’re reading now is on deploying machine learning models in production. The second one is on gradient descent from scratch in Pytorch. Last but not least, the third post is on things that may go wrong while training a neural network.
In this post, I will show you how to export a machine learning model from Google Cloud Platform, download it to your computer, build a small web app, and deploy everything to Render. My project does cat vs dog classification but it can be anything you want.
Exporting the model
The first thing you need to do is get your model out of google cloud, or whatever cloud computing platform you use. This can be done using just one line of code:
learn.export()
This will generate export.pkl
in the directory where your training script, presumably a notebook, was running. If you are using a notebook you are lucky because you can just download this file by selecting it in the file browser and clicking . If you are using ssh’ed into a server you get download the file as explained here.
Creating a simple web app
Before we start creating the web app, start by creating a virtual environment and install the flask
and fastai
even if you have them installed on your system already. (installation)
# Create virtual env
virtualenv env
# Activate virtual env
source env/bin/activate
# Install dependencies
pip install flask
pip install fastai
The framework we will use is Flask. Although there might be better options out there, Flask can do everything we want and it is super simple to use. Install it using pip:
pip3 install flask
You need just a few line of codes to get your app running. First, initialise an app object. Then you create your first route, a location of the website.
from flask import Flask
# Get a new app object
app = Flask(__name__)
@app.route('/')
def index():
return 'Hello world!'
if __name__ == '__main__':
app.run(host='0.0.0.0')
By setting the host to 0.0.0.0 the app is accessible to all computers that your computer accepts public traffic from. In your LAN this could be other computers. In our case, when we are running it on a server, it means that all computers on the internet have access to this page.
If you run this code locally and visit localhost:5000 you should be able to see “Hello World!” If you don’t see this output, check if Flask caught an error or outputted another port for your app.
Sending HTML responses
While sending text to the user is interesting, we would rather use HTML. If you have never used HTML before, don’t worry: you don’t need to understand how it works - that’s not the point of this tutorial.
To send HTML documents to the user, you need to change the initial program a little. More specifically, update the following lines
from flask import Flask
# Get a new app object
app = Flask(__name__)
to
from flask import Flask, render_template
# Get a new app object
app = Flask(__name__, template_folder='templates')
Create a directory in your application’s directory called templates
. Add the following to the index.html
file (inside templates
):
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Dog / Cat Classifier</title>
</head>
<body>
<h1>Dog / Cat classifier</h1>
<form action='/' method='POST' enctype='multipart/form-data'>
<input type='file' name='file' accept="image/png, image/jpeg"><br>
<input type='submit' name='upload_btn' value='upload'>
</form>
</body>
</html>
Then update the index route to:
return render_template('index.html')
Now reload the web page and you’ll see a simple html form.
It’s not my goal to get this page to look pretty. It’s just for demonstration purposes. Feel free to add some styling yourself.
Accepting image uploads
The page allows us to upload images to the web server. If you tried to do this already, you probably got a 405 Method not allowed
error. That’s because the route currently only accepts so-called GET
requests (asking). Update the route declaration so the route accepts POST
requests (telling) as well:
@app.route('/', methods=['GET', 'POST'])
In order for us to learn more about the request (POST
requests are requests too. The naming is a little unconventional), you should import request
from flask like so (note: we also import make_response
that we’ll need in a second):
from flask import Flask, render_template, request, make_response
The request object will store information about the request. To learn what kind of request the user made, we access the method (GET
, POST
, etc.) property. Change the index()
function accordingly:
global learn
if request.method == 'GET':
return render_template('index.html')
elif request.method == 'POST':
# todo: process image
return 'you made a POST request!'
Usually it’s bad practise to have global variables. In this case however, it’s the most convenient way to share the
learn
object, which can may take some time to load, with all the users.
Finally, we can access the images sent with the request like so:
request.images
In the index()
function, replace the line that says todo with:
try:
data = request.files['file']
img = open_image(data) # convert to an image fastai can understand
except KeyError:
return make_response('Bad request', 400)
This makes sure we have an image to work with, and otherwise notifies the user he made an invalid request.
Processing the images
Now to the most fun part: integrating machine learning.
Start by importing fastai.vision
like so:
from fastai.vision import *
Start by setting the device to a CPU:
vision.defaults.device = vision.torch.device('cpu')
The reason for this (source):
“You probably want to use CPU for inference, except at massive scale (and you almost certainly don’t need to train in real-time). If you don’t have a GPU that happens automatically.”
Start by defining the path and loading the model you downloaded (you might need to move it to your application folder) using load_learner
path = vision.Path('.')
learn = vision.load_learner(path)
You can now use the learn
object in the POST part of the index route:
pred_class,pred_idx,outputs = learn.predict(img)
Finally, return the prediction along with the accuracy in JSON format:
return jsonify({
'result': str(pred_class),
'accuracy': float(outputs[pred_idx])
})
This uses another function of flask: jsonify
that you need to import as well.
This completes the web app! You can now use it on your computer.
Deploying to Render
Although running code on your computer is fun, exposing something the world is even better. To do so, we’re using Render. Render is a great service for deploying code. It is super easy to use, even if you have never deployed before and it is quite cheap as well: only $5 a month for 1 cpu and 1gb of ram. The best thing, however, is that you can pay per second so you can follow the tutorial and then shut down you server if you want to do so. You can see more information on the pricing page.
We’ll need a requirements.txt
that contains all there requirements for this project. You can create that like so (in an active virtual environment):
pip freeze > requirements.txt
Assuming you’re familiar with git and GitHub, initialize a git repository and push the code to GitHub.
If you don’t know what the above means, check out a tutorial like this one. It’ll give you superpowers!
Then create an account at render.com.
Go to this page to create a new Render project from your GitHub repository. Just select the project, give it a name and enter something like this for the following fields:
- Build Command:
pip install -r requirements.txt
- Run Command:
python3 main.py
Click create web service et voila, there’s your production server. Easy as that.
Conclusion
As you’ve seen in this tutorial, it’s easy to export a fast.ai model and load it in another app. You’ve also learned how to write a basic web application using flask and deploy it for the world to see!
You can view my code on GitHub and you should also take a look at the deployment.
This was the first of a three part series on fast.ai lesson 2. You should also read part 2, part 3 and more posts about my fast.ai journey.
A huge thanks to Sam Miserendino for proofreading this post!