A Divided Web and the Role of Frameworks Alex Gaynor DjangoCon.AU Friday, July 5, 13 Hi everyone. I’m very excited to be able to kick off the first DjangoCon in Australia, this is a real pleasure, so thanks to all of you for coming out and making this as a reality. So this talk is titled “A divided web and the role of frameworks” and I want to talk today about two things, first the direction the web is headed, as I see it, and second about the evolving role of web frameworks, particularly Django.

Alternate title: A brief, highly inaccurate history of the web Friday, July 5, 13 But honestly, a more accurate title would be this. A lot of what I’m going to say is about the history of the web, and of computing as a whole, as I’ve perceived it. I’m wrong about a lot of this stuff, my hope is that I’m wrong in an interesting way that helps you think about these issues.

Who am I? • Django, PyPy, CPython, Topaz core developer • Former DSF board member, current PSF board member • Engineer at Rackspace Friday, July 5, 13 I’ve been hacking on Django for a good long while, I hack on a bunch of other Python stuff as well.

Friday, July 5, 13 I work for Rackspace, we just opened a datacenter in Sydney, so that’s pretty cool. I work on the developer relations group, so feel free to find me and ask questions about Rackspace later.

Takeaway: I have no clue what’s going on. Friday, July 5, 13 I like my talks to have a clear takeaway message, something I want the audience to have learned, or something I want them to do. Today’s message is: I have no clue what’s going on. So, keep that in mind.

A history of the web Friday, July 5, 13 So to understand where we are, and where we’re going I want to start with a brief history of the web.

~13.77 billion years ago Friday, July 5, 13 “In the beginning the Universe was created. This has made a lot of people very angry and been widely regarded as a bad move.” That’s a Douglas Adams quote. This was a key step in the internet coming into being.

ENIAC: ~1946 Friday, July 5, 13 So following the creation of the universe, in 1946 ENIAC was unveiled. It was the first general purpose, completely electronic computer. Also, fun fact, it computed in decimal, not binary.

1943: “I think there is a world market for maybe five computers” Friday, July 5, 13 Allegedly said by Thomas Watson in 1943 (might just be apocryphal). I don’t think this comment is reflective of being dismissive of computing, but rather indicates a very sever- sided approach; we’ll have a few giant computers and everyone will get some time on them.

1977 - Apple II Friday, July 5, 13 in 1977 the Apple II is released, which in my mind, at least, marks the beginning of personal computing. Look at this thing, it could fit on a desk! It cost about 5 grand in 2012 dollars, had 4KB of RAM, and a 1MHz processor.

1990 - World Wide Web Friday, July 5, 13 In 1990, Tim-Berners Lee invents the web as we know it (for some values of “As we know it”). This includes URLs, HTML, and HTTP. The most important detail of the web is that it wasn’t an application protocol. It was a platform. (SMTP, FTP vs. HTTP)

First, there was static content Friday, July 5, 13 And this didn’t really significantly affect anything. Really, at first we wrote entire web applications in text editors. Whatever.

And then we generated pages dynamically Friday, July 5, 13 And this made PHP and Perl insanely popular. Because before that we mostly used C to write things, and C is a terrible language for strings, and web pages are all about strings.

Anyone remember DHTML? Friday, July 5, 13 We figured out how to do stuff dynamically on the frontend. And we did awful things with it mostly.

And then Microsoft invented AJAX Friday, July 5, 13 This was in 1998, and basically the Outlook Web Access team invents what we later come to know as AJAX. And the rest is history.

Asynchronous Javascript And XML Friday, July 5, 13 I think even the name Ajax gives a really interesting view of how they were thinking about Ajax at the time. “Asynchronous” - this was about improving page times, you could show some stuff without having all the data. “Javascript” - it wasn’t nearly as known back then, so I guess this was important. “and” - a popular conjunction. “XML” - lol.

“Web 2.0” Friday, July 5, 13 (Make a joke about audible groans here). People started developing more social apps, more things that we’d identify as applications today.

This births a new wave of frameworks Friday, July 5, 13 So of course in 2004/2005 Rails and Django were released. They had as their objectives making practical dynamic websites easier to build. If you want a blast from the past, go watch “Snakes vs Rubies” presentations from Adrian Holovaty and DHH in 2005 on Django and Rails.

“Database driven applications” Friday, July 5, 13 This isn’t a term I’ve heard in 3+ years, but back when Django and Rails came out, it was what we were talking about. Literally building a website with a database was a thing we didn’t have good tools for.

“Web applications” Friday, July 5, 13 So now we build “web applications”. I honestly don’t know what this means, but it includes applications that are way more dynamic, things like gmail, rdio.

So Many Tools Friday, July 5, 13 These tools are basically in no way related to each other, or compete with each other. They don’t do the same things. But this is what I got when I asked for tools for building web applications.

Trends Friday, July 5, 13

Remote v. Local Compute We landed on Both Friday, July 5, 13 This was a pretty big trend in computing, whether or not the majority of your computing is on the server or the client. We ended up with both for web apps, and this isn’t going to change ever. You want remote so that it works anywhere, and you need locals because of the laws of physics.

Technologies die slow Friday, July 5, 13 More slowly than ever before. Tons of languages that were popular in the 70s are dead. Stuff isn’t going to die so fast anymore. There are bigger installed bases, more developers who know the tools, more code written with them. EVERYTHING popular today is going to be the COBOL of the 1950s.

Tools don’t create capabilities Friday, July 5, 13 This one is a hugely important. The tools we get don’t make it possible to do things, the ability to do thing, basically comes from the browser. It was over a decade from the creation of the web to Django. They make things easy. And this is where the divide comes in. Django made it possible to build the best websites of the day with a small team in not too long.

Django was built to make “Database driven applications” tractable Friday, July 5, 13 In 2005, when Django was released, this was a thing people were trying (and failing) to do quickly. Django made it possible. It let a 2 developer team do things in a week that giant teams couldn’t do in a year.

Can we make modern web applications with small teams? Friday, July 5, 13 I don’t know. Even in the few years I was doing web development professional, I saw the scope of projects grow hugely. The expectations of what could be built are way higher.

Web development in 2005 Relational Database Cache Application Admin tools Templates XML feeds Friday, July 5, 13 So this is a slide from Simon Willison on what the things involved in web development in 2005 were like. There’s a few things, and Django basically did them all.

Web development in 2009 Relational Database Cache Application Admin tools Templates XML feeds Datastructure servers Search index External web services Monitoring and reporting API Webhooks Message queue Offline workers Non-relational database Friday, July 5, 13 And here’s his 2009 version. The green ones are the hold overs. Tons of new stuff, Django itself doesn’t really help with any of these, some of them have great external projects like queues (celery) or search (haystack).

2013 additions •Deployment • Configuration management • Internal services (SOA) • Real time endpoints •Javascript Friday, July 5, 13 These are a few things I’d add. Of these only real-time endpoints are a thing that’s technically new, but these are all things that are picking up steam and getting more popular, and we should do more of them. Except javascript, its terrible, but its still being used more and more.

Still no tools that encompass all of these Friday, July 5, 13 And it’s getting more and more disconnected. In 2005 Django did all the things, in 2009 it did some of the things, in 2013 it does even less of the things. And there’s no reason to believe this is going to change significantly.

More platforms: will they converge Friday, July 5, 13 We’ve created more “application platforms” then ever before. Different tools for turning your web application into a mobile app, half a dozen different ways to automatically deploy your app, etc. Platforms should be standard (POSIX!), how do we get to there? Are we going to at all?

Specialization vs. Generalization Friday, July 5, 13 Django is a very general tool. Relational databases are very generalized tools. These were the tools of 2005. Now we’re using increasingly specialized tools it seems to me, and it’s not obvious if this will converge. There’s half a dozen databases to choose from, so many different JS frameworks, etc, etc. Can a general purpose framework compete? Can a developer know just one specialized tool?

No frameworks have eaten Django or Rails lunch Yet Friday, July 5, 13 So Django and Rails don’t solve any of these problems. But they’re still crazy widely used. Something’s probably going to sneak up and eat our lunch.

I have no idea where the web is going. Friday, July 5, 13 But I’m sure it’s going to be more and more complex.

One spot of good news Friday, July 5, 13 Through all this, I have seen what seems to be good news.

Deployment! Friday, July 5, 13 Deployment, which has been a bane of everyone’s existence since we started trying to deploy things more complex than static HTML or PHP; actually seems to be getting better.

A bright future My application Docker container Virtual and physical servers Heat templates Chef Friday, July 5, 13 (Explain all the moving parts)

Flip side: more parts: “as a service” Friday, July 5, 13 Is this a good thing that everything from databases, to queues, to map reduce infrastructure is now becoming “as a service”. Unclear! If it’s open source that’s at least better I think.

I repeat: I have no idea what’s going on Friday, July 5, 13 I also don’t know how to make it better.

The divide: Small vs. Big Contractor vs. In-house “App” vs. “Site” Friday, July 5, 13

Thanks! Questions? https://speakerdeck.com/alex Friday, July 5, 13