Building Web APIs with Flask-RESTful

As presented at PyOhio 2014:

The Flask framework’s DIY nature makes it ideal for implementing REST APIs. Flask-RESTful is a Flask extension providing reusable, extensible behavior for common elements of REST APIs like content negotiation, input validation, and output marshalling. This talk will show how to use Flask-RESTful to build a REST API, demonstrating how to start from a blueprint to adapt it for your own use cases.

Transcript

1. Building Web APIs with Flask-RESTful
2. @skimbrel Hi. I’m Sam Kimbrel. That’s my twitter handle, it’s pronounced “skimbrel”.
3. I work at Twilio, in San Francisco, building and maintaining their public REST API.
4. Building Web APIs with Flask-RESTful Today I’ll be talking about building Web APIs with Flask-RESTful. Let’s start with some introductions.
5. REST REST stands for REpresentational State Transfer. It’s a term that gets thrown around a lot, mostly in arguments over how RESTful or not-RESTful a given API is. Roy Fielding wrote a paper outlining a bunch of principles for REST systems to obey: they should expose a uniform interface for a client-server model, be stateless, be cacheable, and so on. Mostly, I tend to think of REST as embodying the idea that the client and server exchange changes to the state of the world by exchanging representations of it in request/response pairs that are independent from each other.
6. Flask Flask is a popular microframework for writing web applications in Python. It’s simple, easy to learn, and highly extensible… which makes it a great choice for implementing web APIs.
7. Hello, Flask from flask import Flask app = Flask(__name__) @app.route('/') def hello(): return u'Hello World!' Some of you have probably seen this before, but here’s the canonical Flask hello-world application. (explain what’s going on)
8. Let’s write an API! Okay, cool! Let’s write an API with Flask! I didn’t have any sudden flashes of inspiration for exciting new toy data models, so we’re going to pretend we’re modeling a student registration system for a college or something.
9. @app.route('/students', methods=['GET']) def get_students(): students = Student.query.all() formatted_students = [] for s in students: formatted_students.append({ 'id': s.id, 'first_name': s.first_name, 'last_name': s.last_name, }) return json.dumps({'students': formatted_students}), 200, {'Content-Type': 'application/json'} Here’s how we might implement a Flask view that returns a JSON-formatted list of student resources. We start by going to the database to get the list of objects, and then loop over them and extract the attributes we care about into a list of dictionaries. Then we run that through json.dumps and hand it back with an appropriate response code and a header saying “hey you should expect some JSON now”.
10. @app.route('/students', methods=['POST']) def create_student(): kwargs = {} for field in ('first_name', 'last_name', ...'): kwargs[field] = request.form.get(field) student = Student(**kwargs) return json.dumps({ 'id': s.id, 'first_name': s.first_name, 'last_name': s.last_name, }) Creating a student gives us a little more work to do: for each of the attributes we’d like to accept, we need to look at the submitted request data (this example is an old-school HTTP form body; it’d work just as well for a JSON- or XML-encoded object) and see whether that attribute is present. In the real world, we’d do some validation here too, because SQL injection and stored XSS attacks aren’t much fun. Once we’ve extracted the input arguments, we can instantiate a new student object and save it to the database, then transform it into the response structure, serialize, and return it.
11. @app.route('/students/<id>', methods=['PUT']) def update_student(id): s = Student.query.get(id) if not s: abort(404) for field in ('first_name', 'last_name', ...): setattr(s, field, request.form.get(field, getattr(s, field))) db.commit() return json.dumps({ 'id': s.id, 'first_name': s.first_name, 'last_name': s.last_name, }) Okay, one more. Here’s our vanilla-Flask API view for updating an existing student resource (HTTP PUT). It should look pretty familiar at this point: extract our input arguments, update the underlying model object, transform, and serialize. So now we’ve outlined enough of this that it’s pretty easy to see what’s good about using Flask to implement a REST API and what’s not so good, right? The code’s pretty straightforward and easy to read, but there’s kind of a lot of it. There’s one principle of programming that I really want to call out here:
12. Don’t Repeat Yourself Don’t repeat yourself. You’ve probably heard this as the “dry” rule, or of code that doesn’t follow it as being “wet” (We Enjoy Typing). As with every other programming maxim, you can definitely take it too far, but there are some pretty clear patterns starting to emerge just from the three REST methods we’ve gone through. Let’s look at that code again.
13. @app.route('/students', methods=['GET']) def get_students(): students = Student.query.all() formatted_students = [] for s in students: formatted_students.append({ 'id': s.id, 'first_name': s.first_name, 'last_name': s.last_name, }) return json.dumps({'students': formatted_students}), 200, {'Content-Type': 'application/json'} Here’s the function for our GET to the list resource again, and I’ve highlighted all of the code that’s just dealing with transforming our internal representation into the plain-old-dict representation we want to return to the user.
14. @app.route('/students', methods=['POST']) def create_student(): kwargs = {} for field in ('first_name', 'last_name', ...'): kwargs[field] = request.form.get(field) student = Student(**kwargs) return json.dumps({ 'id': s.id, 'first_name': s.first_name, 'last_name': s.last_name, }) Here’s the POST handler. Again, highlighted code is only dealing with extracting input data or transforming data for output.
15. @app.route('/students/<id>', methods=['PUT']) def update_student(id): s = Student.query.get(id) if not s: abort(404) for field in ('first_name', 'last_name', ...): setattr(s, field, request.form.get(field, getattr(s, field))) db.commit() return json.dumps({ 'id': s.id, 'first_name': s.first_name, 'last_name': s.last_name, }) And finally, our vanilla-Flask PUT handler. Same highlighting: well over half of this code is dealing with nothing but moving data in and out of various dicts. In general, API implementations spend a lot of time dealing with a few common themes:
16. Input Input parsing and validation are a large area of concern for just about any API implementation.
17. Output Output sanitization, formatting, and serialization are also a pretty big deal, right?
18. Resources And finally, REST APIs are organized around a well-defined set of resources. If we think back to our examples from earlier, we just had this bag of functions that we distinguished from each other by applying Flask’s route decorator to the same paths with a different method allowed. We also had function names that duplicated the resource name each time.
19. Flask-RESTful Flask-RESTful offers an extension library to Flask that helps you abstract out these common pieces and write resource implementations that are concise and easy to follow. It grew out of work done at Twilio as we implemented our current generation of APIs, both internal and external, and eventually became something we felt like sharing with the open source community. Let’s go through what Flask-RESTful offers to make life easier when writing web APIs in Python.
20. Input We’ll start with input validation.
21. from flask.ext.restful import inputs from flask.ext.restful import reqparse parser = reqparse.RequestParser() parse.add_argument(u'birthdate', type=inputs.date) args = parser.parse_args() Flask-RESTful comes with a module called reqparse, which is patterned after Python’s built-in argparse module and takes responsibility for extracting input arguments from your inbound requests. In this example, we just import the reqparse module, construct a parser, and add some arguments to it. Then, just like with argparse, we ask it to parse the args and give us back the results. Reqparse has the added benefit of looking in multiple sources for request data: by default, it’ll inspect the query string parameters, the form body, and, if your request included a JSON body, it can look into JSON dictionaries. We’ll come back to this when we put everything together.
22. $ curl -d 'birthdate=foo' localhost:5000/ { 'status': 400, 'message': 'foo cannot be converted to date' } Reqparse’s parsing function works with the rest of the framework to automatically catch validation errors and turn them into user-friendly error responses. As you can see in this example, if we hand “birthdate=foo” to a Flask-RESTful API, we get back this nice JSON- serialized response with a message telling us what we did wrong.
23. Output I’d like to move on to Flask-RESTful’s output features. The framework provides us with a marshaling function that can extract data from Python data structures and transform it into almost any structure we like.
24. from flask.ext.restful import fields from flask.ext.restful import marshal student_fields = { 'id': fields.Integer, 'first_name': fields.String, 'last_name': fields.String, } output = marshal(student_fields, s) Here’s the declaration of the fields and structure we’d use to recreate our example API’s output format: we create a mapping from field name to field type. Out of the box, we get support for Integers, Strings, DateTimes, Urls, Floats… and a bunch more. Once we’ve done that, we use the marshal function to produce a formatted output dictionary from the defined fields and one of our internal data objects.
25. student_list_fields = { 'students': fields.List( fields.Nested(student_fields) ), 'count': fields.Integer, } One of the more powerful features in the marshaling library is the ability to create lists of repeated structures and to embed or nest already-defined fields structures into another one. If we take the formatting fields for a single student resource we already defined and use them with the Nested and List fields, we get a list resource structure virtually for free. So, we’ve covered how Flask-RESTful handles input parsing and validation, and how we can use its marshaling functionality to eliminate repeated sections of code for formatting responses.
26. Resources Now that we’ve dealt with our input and output concerns, let’s put everything together and re-implement our example API resources with the Flask-RESTful extension.
27. from flask.ext.restful import ( Api, marshal, Resource ) api = Api(app) class Student(Resource): def get(self, id): s = Student.query.get(id) if not s: abort(404) return marshal(student_fields, s) api.add_resource(Student, '/students/<int:id>') Here’s part of the implementation for the student instance resource, using Flask-RESTful. The boilerplate up top: we import a few extra things from the extension, and construct an Api object to wrap our Flask application in. The extension treats each resource as a class, with methods defined corresponding to the HTTP request methods the resource responds to. I didn’t have enough room on this slide to fit all four (it’s not magic), but let’s take a look at the method for handling an HTTP GET to a student resource by ID. All of the formatting logic is gone; it’s handled for us by the marshal function, using the field definitions we set forth earlier. The method is clearer and easier to understand: we retrieve an item from the database, throwing a 404 if it doesn’t exist, and then return it. The resource class we’ve used here also takes care of one other major concern we had to deal with in our standalone implementation: it automatically serializes the returned data into the appropriate content type for the client, for example JSON.
28. from flask.ext.restful import marshal_with class Students(Resource): @marshal_with(student_fields) def post(self): args = parser.parse_args() s = Student(**args) db.session.add(s) db.commit() return s api.add_resource(Students, '/students') Here’s the method to handle creating a new student resource. We use the request parser we defined outside of this resource class to parse the incoming data (remember, it’ll throw the appropriate error if something is out of place), create the object, and return it. You might notice that our return statement didn’t wrap the return value in the marshal function; we instead decorated this method with the marshal_with decorator. It’s just syntactic sugar for returning the marshalled version of the object. After we’ve declared our resource classes, we add them to the Api object at the appropriate path; the Api takes care of dispatching incoming requests to the appropriate method based on the HTTP method. So, to recap: we’ve extracted our API’s inputs and outputs and plugged them into the framework’s helper functions, we’ve moved our resource view functions into classes organized by resource, and we’ve taken advantage of the framework Resource class’s automatic serialization capability.
29. But wait! So we can already see some places where using Flask-RESTful will speed up our application development and make the resulting code easier to read and maintain. We can do more, though; I’d like to go into a couple of ways in which we can augment the framework to handle some other common needs.
30. Serializers I mentioned earlier that Flask-RESTful takes care of content-negotiation and serialization for us. That part was true, but it’s also a bit of a lie by omission: out of the box, we only get a JSON serializer. This was done by design — JSON has become the de facto standard for web APIs. It’s also intended to obey the principle of least surprise; in this case, you’re avoiding the surprise of having a user write in complaining that the XML serialization you didn’t even know your API had is broken. Fortunately, adding new serializers is really easy.
31. def output_xml(data, code, headers=None): root = etree.Element('data') # Nasty recursive XMLification goes here... xml = etree.Element.tostring(root) return make_response(xml, code) api = Api(app) api.representations['application/xml'] = output_xml The Api class comes with logic to handle content negotiation based on the Accept header in the HTTP request; all you need to do is define a function that takes a marshaled data structure and returns a Flask response object with the serialized content, and add that function to the Api object’s representations map under the appropriate MIME types.
32. Custom data types Flask-RESTful also supports custom data types for both input and output.
33. def prime_number(value): val = int(value) if not is_prime(val): raise ValueError("Value is not prime") return val reqparse.add_argument('silly', type=prime_number) Here’s a (admittedly rather contrived) custom input function: it should accept the raw input value as input, do whatever logic it likes to validate it, and either return the parsed and validated value or raise ValueError if it wasn’t valid.
34. class YellingField(fields.Raw): def format(self, value): return str(value).upper() fields = {'angry_name': YellingField} Output is pretty similar: subclass the base class (called Raw) from the fields module and expose a format method that returns the marshaled representation of the field data. This example just forces whatever it gets to a string and then upper-cases that. Kind of silly, but makes it easy to see what’s going on.
35. Error handling Finally, we might like to have some cleaner error handling than a bunch of if/else or try/ except blocks scattered through our resource methods. Thanks to a recent community contribution, Flask-RESTful now has a mechanism for this.
36. errors = { 'NotFoundError': { 'message': 'Not found', 'status': 404, } 'FrobnitzError': …, } api = Api(app, errors=errors) The Api class takes an optional errors argument that should contain a mapping from exception class names to error response objects. Here we’ve mapped a NotFoundError to a simple dict containing a message and the HTTP status. If exceptions defined in this mapping propagate to the Flask-level error handler, the Api class uses it to return the appropriate error response to the client.
37. Flask-RESTful Okay, so that concludes our tour of the Flask-RESTful framework.
38. Input validation Output formatting Resource routing Serialization Custom data types Error handlers To recap, the framework provides abstractions and library functionality to deal with input validation, output formatting, resource routing, custom serialization formats and data types, and error handling. We use Flask-RESTful at Twilio to handle the majority of our public API requests, and I hope you’ll find it useful for your projects as well.
39. Thanks!
40. pip install flask-restful github.com/twilio/flask-restful flask-restful.readthedocs.org @skimbrel