Deliver Your Software in an Envelope by Augie Fackler and Nathaniel Manista

Transcript

1. Augie: Mercurial, Python libraries. Nathaniel: Tech Lead of Melange, has contributed to Pylint. Transition: you test it, you document it, but how often have you gotten grief for regressions that didn't break tests? Deliver Your Software In An Envelope Augie Fackler & Nathaniel Manista Google, Inc. 13 April, 2014 Deliver Your Software in an Envelope http://localhost:8080/print/ 1 of 59 4/13/14, 12:57 PM
2. Have you ever changed the behavior of your software in ways that don't break any promises but still gotten grief from your clients? Image source: http://commons.wikimedia.org/wiki/File:Angry_mob_of_four.jpg Deliver Your Software in an Envelope http://localhost:8080/print/ 2 of 59 4/13/14, 12:57 PM
3. Behavioral Envelopes Deliver Your Software in an Envelope http://localhost:8080/print/ 3 of 59 4/13/14, 12:57 PM
4. Lockheed F-104A Starfighter Aviation uses a notion of “performance envelopes” for describing the valid uses of an aircraft. Deliver Your Software in an Envelope http://localhost:8080/print/ 4 of 59 4/13/14, 12:57 PM
5. We believe that you can; let's explore what that might look like. Can we do this for software? Deliver Your Software in an Envelope http://localhost:8080/print/ 5 of 59 4/13/14, 12:57 PM
6. Axes not even yet known. Vague sense that for software, like other systems, there are right ways and wrong ways to use it. Deliver Your Software in an Envelope http://localhost:8080/print/ 6 of 59 4/13/14, 12:57 PM
7. Examples Deliver Your Software in an Envelope http://localhost:8080/print/ 7 of 59 4/13/14, 12:57 PM
8. math.sqrt Deliver Your Software in an Envelope http://localhost:8080/print/ 8 of 59 4/13/14, 12:57 PM
9. Acceptable inputs are numbers greater than or equal to zero. DO NOT BOTHER TALKING ABOUT imaginary numbers. Transition to "but at another level of abstraction..." Deliver Your Software in an Envelope http://localhost:8080/print/ 9 of 59 4/13/14, 12:57 PM
10. Think of a video game. One of my favorites will run with 4G of RAM, but you'd better close your browser and other RAM hogs before you do, or it'll be a somewhat swappy experience. I've got a low end GPU, so I have to turn the texture quality down. "System Requirements" Deliver Your Software in an Envelope http://localhost:8080/print/ 10 of 59 4/13/14, 12:57 PM
11. Works at multiple levels of abstraction - an "HTTP server" can be a on object in a language interpreter, a process on a machine, or a machine itself. HTTP server (multiple levels of abstraction) Deliver Your Software in an Envelope http://localhost:8080/print/ 11 of 59 4/13/14, 12:57 PM
12. Server: class HttpServer(object): def __call__(self, request): """HttpRequest -> HttpResponse.""" Forbidden input: "the quick brown fox jumps over the lazy dog" Deliver Your Software in an Envelope http://localhost:8080/print/ 12 of 59 4/13/14, 12:57 PM
13. Large space: universe of all values; small space: all values of type HttpRequest. Deliver Your Software in an Envelope http://localhost:8080/print/ 13 of 59 4/13/14, 12:57 PM
14. Server: httpserver.exe Forbidden input: /dev/urandom Deliver Your Software in an Envelope http://localhost:8080/print/ 14 of 59 4/13/14, 12:57 PM
15. Plane is all bytestreams. Field of dots because the differentiation here is done value-by-value because we don't have types to differentiate. Deliver Your Software in an Envelope http://localhost:8080/print/ 15 of 59 4/13/14, 12:57 PM
16. Deliver Your Software in an Envelope http://localhost:8080/print/ 16 of 59 4/13/14, 12:57 PM
17. Deliver Your Software in an Envelope http://localhost:8080/print/ 17 of 59 4/13/14, 12:57 PM
18. Deliver Your Software in an Envelope http://localhost:8080/print/ 18 of 59 4/13/14, 12:57 PM
19. So far we've introduced the idea of defining an envelope for a system's domain (its inputs). It turns out that it is helpful to define one for its outputs as well, --TRANSITION-- though not so much for systems that have exact (deterministic) outputs Deliver Your Software in an Envelope http://localhost:8080/print/ 19 of 59 4/13/14, 12:57 PM
20. The utility of an expression like this is that it produces the same exact value every time. bool(None) Deliver Your Software in an Envelope http://localhost:8080/print/ 20 of 59 4/13/14, 12:57 PM
21. This too. --TRANSITION-- But great for systems that feature deliberately inexact outputs 3 * 4 Deliver Your Software in an Envelope http://localhost:8080/print/ 21 of 59 4/13/14, 12:57 PM
22. random.random() Deliver Your Software in an Envelope http://localhost:8080/print/ 22 of 59 4/13/14, 12:57 PM
23. Deliver Your Software in an Envelope http://localhost:8080/print/ 23 of 59 4/13/14, 12:57 PM
24. Systems that knowingly return fast but inaccurate answers Deliver Your Software in an Envelope http://localhost:8080/print/ 24 of 59 4/13/14, 12:57 PM
25. Systems that anticipate being attacked by an adversary. * Proprietary commercial systems exposing only an API * Game systems for which implementation details would give cheaters an advantage --TRANSITION-- So what does this add up to generally? Deliver Your Software in an Envelope http://localhost:8080/print/ 25 of 59 4/13/14, 12:57 PM
26. A wormhole between universes! Input dimensions: parameter values, assertions about the state of the world. Output dimensions: return values, raised exceptions, side effects. We will say "behavioral envelope" to mean the sum of "input envelope" and "output envelope". Deliver Your Software in an Envelope http://localhost:8080/print/ 26 of 59 4/13/14, 12:57 PM
27. You likely already are in some ways. Function input domains. Output ranges. Blocking and non-blocking semantics. Thread safety. The pressure we're putting on you here is that if you state your envelope, you're guaranteed to have thought about it first. We see a lot of code that was written without such thinking. This is how you hold off the angry mob. Weak Thesis: You should state your system's behavioral envelope. Strong Thesis: Your statement of your system's behavioral envelope should be your only statement of your system's behavior. Deliver Your Software in an Envelope http://localhost:8080/print/ 27 of 59 4/13/14, 12:57 PM
28. How? 1. Type system. 2. Machine-checked constraints. 3. Documentation. public static double sqrt(double n) { // code goes here } def sqrt(n): """Returns the square root of N. >>> sqrt(5) 2.23606797749979 >>> sqrt(-1) ValueError """ # code goes here Deliver Your Software in an Envelope http://localhost:8080/print/ 28 of 59 4/13/14, 12:57 PM
29. Same function in both languages, but Haskell's type system is another mechanism of expression available to both machines and programmers. map map: (a0 -> b0) -> [a0] -> [b0] Deliver Your Software in an Envelope http://localhost:8080/print/ 29 of 59 4/13/14, 12:57 PM
30. Also note how the input envelope for map is interesting in the way it demands a certain kind of agreement between the types of the inputs. Deliver Your Software in an Envelope http://localhost:8080/print/ 30 of 59 4/13/14, 12:57 PM
31. Tests are machine-checked constraints of your system's envelope. Now maybe this doesn't look like something you'd want to share with your clients as part of your system's definition... --TRANSITION-- Pycurracy test in something more like natural language. def testPageLoads(self): """Tests that page loads properly.""" user = profile_utils.seedNDBUser( host_for=[self.program]) profile_utils.loginNDB(user) response = self.get(_getOrgAppShowUrl(self.org)) self.assertResponseOK(response) Deliver Your Software in an Envelope http://localhost:8080/print/ 31 of 59 4/13/14, 12:57 PM
32. This is test code that you could ship to clients of your system as persuasive evidence that your system is suitable for some purpose. Given I go to GSoC Home Page When I click "Apply_Org" link and wait Then I see "test_org" title Deliver Your Software in an Envelope http://localhost:8080/print/ 32 of 59 4/13/14, 12:57 PM
33. If you need to, you can always just say "that input is not allowed". Or "this function will never...", or "overriding subclasses must...". --TRANSITION-- How do these systems interact? Are they redundant? Complementary? DO NOT COVER INTEGRATION WITH TYPE SYSTEM YET Deliver Your Software in an Envelope http://localhost:8080/print/ 33 of 59 4/13/14, 12:57 PM
34. Story of a space being defined by a space being defined by a continuous barrier on one side and a single point on the other side. Transition into exploring this as software, rather than metaphor. Deliver Your Software in an Envelope http://localhost:8080/print/ 34 of 59 4/13/14, 12:57 PM
35. Consider again an HTTP server, this time accepting and returning strings. class HttpServer(object): def __call__(client_string): """string -> string""" # code goes here Deliver Your Software in an Envelope http://localhost:8080/print/ 35 of 59 4/13/14, 12:57 PM
36. In the space of all values we can separate out and only allow strings. The edge of the envelope is smooth because we're using a type system. Deliver Your Software in an Envelope http://localhost:8080/print/ 36 of 59 4/13/14, 12:57 PM
37. Within the space of all strings, some parse as HTTP requests and some don't. We represent those as points because type systems can't make this discrimination within the string type. Deliver Your Software in an Envelope http://localhost:8080/print/ 37 of 59 4/13/14, 12:57 PM
38. The values we choose as test data in our system tests serve to illustrate the edge of our envelope that cannot be described by the type system. --TRANSITION-- to javadoc showing how these types and documentation can work together. Deliver Your Software in an Envelope http://localhost:8080/print/ 38 of 59 4/13/14, 12:57 PM
39. No tech writer or engineer put those types in that documentation - in this case the type system and documentation system are working together. Deliver Your Software in an Envelope http://localhost:8080/print/ 39 of 59 4/13/14, 12:57 PM
40. Who recognizes this text? Despite the legal disclaimer of fitness, the engineering reality is that we would like our clients to have some assurance that our software achieves something. Live, running code in the context of a demo is a great way to persuade that our software does something useful. What conducts a demonstration of live, running code? A test. So ship your tests to your clients. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. Deliver Your Software in an Envelope http://localhost:8080/print/ 40 of 59 4/13/14, 12:57 PM
41. Machine-checkable documentation wins. Good checkable-documentation formats include ([doctest](http://docs.python.org /2/library/doctest.html), [Sphinx](http://sphinx-doc.org/), Go’s [example test functions](http://golang.org/pkg/testing/), and [cram](https://pypi.python.org/pypi/cram) for shell tools, Docs are good, but explaining is losing. Bonus points if your docs are automatically tested. def sqrt(n): """Returns the square root of N. >>> sqrt(5) 2.23606797749979 >>> sqrt(-1) ValueError """ # code goes here Deliver Your Software in an Envelope http://localhost:8080/print/ 41 of 59 4/13/14, 12:57 PM
42. Fewer promises make everyone happier - it's easier to understand the library's functionality as a user, and it's less to maintain for an author. Aim for the least-committed envelope possible without crippling your clients or hampering their performance. Narrow the input side but widen the output side. Transition to examples. Good/bad envelope definition examples. Difference between what must be and what happens to be. Less (Guarantees) Is More (Freedom To Keep Those Guarantees) Deliver Your Software in an Envelope http://localhost:8080/print/ 42 of 59 4/13/14, 12:57 PM
43. Examples of Doing It Wrong Deliver Your Software in an Envelope http://localhost:8080/print/ 43 of 59 4/13/14, 12:57 PM
44. Provides a (decent-ish) http client library, right in the standard library. This is actually pretty great. Python's httplib Deliver Your Software in an Envelope http://localhost:8080/print/ 44 of 59 4/13/14, 12:57 PM
45. This state machine is taken from the docs. It doesn't fit on the slide, but that's not really what's important here. (null) | | HTTPConnection() v Idle | | putrequest() v Request-started | | ( putheader() )* endheaders() v Request-sent | | response = getresponse() v Unread-response [Response-headers-read] |\____________________ | | | response.read() | putrequest() v v Idle Req-started-unread-response ______/| / | Deliver Your Software in an Envelope http://localhost:8080/print/ 45 of 59 4/13/14, 12:57 PM
46. This means it's impossible to make any substantial changes to the internals of this package, because you've told clients how to abuse your internals. This would have been a great thing to change in Python 3, but it [didn't happen](http://hg.python.org /cpython/file/b466fd273625/Lib/http/client.py). Python 4, I guess? "HTTPResponse class does not enforce this state machine, which implies sophisticated clients may accelerate the request/response pipeline." Deliver Your Software in an Envelope http://localhost:8080/print/ 46 of 59 4/13/14, 12:57 PM
47. Dimensions of the input envelope are "all possible methods that could be exposed on Object" and the input envelope was built too widely. Java's Object methods Object.clone Object.finalize Deliver Your Software in an Envelope http://localhost:8080/print/ 47 of 59 4/13/14, 12:57 PM
48. Good Examples Deliver Your Software in an Envelope http://localhost:8080/print/ 48 of 59 4/13/14, 12:57 PM
49. HTTP (the protocol) Deliver Your Software in an Envelope http://localhost:8080/print/ 49 of 59 4/13/14, 12:57 PM
50. Only 15 status codes in http/1.0. In 1.1, many more (38) are defined. Status-Code = "200" ; OK | "201" ; Created | "202" ; Accepted | "204" ; No Content | "301" ; Moved Permanently | "302" ; Moved Temporarily | "304" ; Not Modified | "400" ; Bad Request | "401" ; Unauthorized | "403" ; Forbidden | "404" ; Not Found | "500" ; Internal Server Error | "501" ; Not Implemented | "502" ; Bad Gateway | "503" ; Service Unavailable Deliver Your Software in an Envelope http://localhost:8080/print/ 50 of 59 4/13/14, 12:57 PM
51. RFC 1945 is curious because it defines significantly fewer status codes than RFC 2616 (HTTP/1.1), which came 3 years later. 402 (Payment required), 206 (Partial content), etc all introduced later, and are safe for existing clients only because of the above defensive specification. RFC 1945: HTTP status codes are extensible[...]. HTTP applications are not required to understand the meaning of all registered status codes[...]. However, applications must understand the class of any status code, as indicated by the first digit, and treat any unrecognized response as being equivalent to the x00 status code of that class, with the exception that an unrecognized response must not be cached. Deliver Your Software in an Envelope http://localhost:8080/print/ 51 of 59 4/13/14, 12:57 PM
52. Has a backport to 2.x, and it's pretty great. The concurrent.futures package in Python 3. Deliver Your Software in an Envelope http://localhost:8080/print/ 52 of 59 4/13/14, 12:57 PM
53. Comes with two executor classes out of the box. One lets you work around the GIL, but has a limitation in that it has to pickle arguments to send them to other processes. Easy to envision something that'd use Stackless's microthreads. Maybe even one that used any PEP3156 event loop. ThreadPoolExecutor ProcessPoolExecutor Deliver Your Software in an Envelope http://localhost:8080/print/ 53 of 59 4/13/14, 12:57 PM
54. Is reassigning the value associated with a key a thread-safe operation in Python? Once came up in a Google-internal debate. It turns out that it is in the current version of Python - but as an implementation detail. There's no formal support for it. Our position in the debate was that you should program to the envelope of behavior that Python's documentation guarantees rather than some specific behavior that it implements. my_dict[my_key] = new_value Deliver Your Software in an Envelope http://localhost:8080/print/ 54 of 59 4/13/14, 12:57 PM
55. --- TRANSITION --- This is all important because when we started programming, it felt like a bunch of discrete circuits. Source Code Is The Best Documentation! Deliver Your Software in an Envelope http://localhost:8080/print/ 55 of 59 4/13/14, 12:57 PM
56. When we're learning to code, coding feels more like wiring: connecting exact bits of behavior into larger, still very precise systems. Deliver Your Software in an Envelope http://localhost:8080/print/ 56 of 59 4/13/14, 12:57 PM
57. After you start thinking and reasoning about software this way it feels more like plumbing a variable-flow system. You're still connecting outputs to downstream inputs, but when you do you're ensuring simple compatibility between them rather than making an exact match. Mature software creation is about assembling, transforming, and convolving behavioral envelopes. Deliver Your Software in an Envelope http://localhost:8080/print/ 57 of 59 4/13/14, 12:57 PM
58. Summary Provide small, well-defined envelopes. Only rely on the stated envelope others provide. Deliver Your Software in an Envelope http://localhost:8080/print/ 58 of 59 4/13/14, 12:57 PM
59. Thanks! Deliver Your Software in an Envelope http://localhost:8080/print/ 59 of 59 4/13/14, 12:57 PM