A discussion on real-time, multi-device, rich-client applications EyebrowJS A talk given by Mark Wubben at LondonJS School Trip in London, UK, on June 30th 2012. Photo by Vox Efx, http://www.flickr.com/photos/vox_efx/2873644723/, CC-BY 2.0. 

This is about architecture I’d like to talk today about application architecture. Who here has seen the new direction Twitter is taking with their website? They’re now rendering an HTML page on the server, and even shipping HTML snippets to dynamically update the pages. No more hashbangs, no more showing the homepage whilst the client fetches the tweet you really wanted to see. It’s a similar approach 37signals is taking with their new version of Basecamp. In fact, DHH tweeted this: Photo by David DeHetre, , CC- BY-2.0. 

“It's a perversion to think that responding to Ajax with HTML fragments instead of JSON is somehow dirty. It's simple, clean, and fast.” Personally I don’t like shipping HTML snippets, and that’s because I have a different view of how web applications should function. Now, I work at a startup named State, and this talk is about how we view web applications and how we’re trying to build them. 

STATE STATE At State we’re building a global opinion network. We let you state your opinions (on anything!) and connect you to interesting stories, people or debates from around the world. 

How can a web application be a website? How do we build a web application that is also a website? What do we need out of our front- end architecture? EyebrowJS is the framework we have under development that aims to solve these challenges. When it’s ready we’ll open source it. For now, let’s discuss! At State, your opinion counts ;-) Photo by Katelyn Kenderdine, http://www.flickr.com/photos/kmk7702/6434193577/, CC- BY-2.0. 

What’s an application? Applications can demand things Sites must support all the things What do we mean by website or -application? I feel as if an application can put demands on the browser – require, say, JavaScript – and pretty much runs in the browser without going to the server at all. A website has to cater to all kinds of browsers and does use server round-trips to render new pages. So if we want our web application to be like a website we need to find a way of merging these properties. 

Sites ship static HTML Allows for indexing What the WWW is build on The most obvious property is that websites ship static HTML to the client. This ensures that search engine crawlers and semantic web parsers can actually understand your site, I mean, application. 

Sites use progressive enhancement Support all the browsers Be awesome when possible Sites rely on progressive enhancement. They don’t assume JavaScript is present, or that the browser is of May 2010 vintage. Of course we’ve known this for about 10 years now. These days we have extra challenges. 

What about real-time updates? More application-y Not just streams, any kind of data that’s on the screen We need to handle real-time updates. Not just a stream, but data changing across the screen. 

Responsive design Can no longer make assumptions And our websites need to be responsive. We can no longer make assumptions about screen size or even where people use the website. 

Interactive widgets Requires an architecture or it’ll spiral out of control And finally we need websites with interactive widgets. This shouldn’t be bolted on to your application but be an integral part of it. You need to architect the widgets and the code that drives them, neglect this and your application simply spirals out of control. 

What does that mean for the application stack? Where’s the balance between site and application? How do we make things real-time, responsive and interactive? What does all this mean for the application stack? Photo by Anssi Koskinen, http://www.flickr.com/photos/ansik/2450140330/, CC-BY 2.0. 

Conditional-tier rendering Website on the server Application on the client Coined by Dion Almaer What we want is for the server to render a website, but for the client to be an application. Dion Almaer calls this [conditional-tier rendering][1]. [1]: http://functionsource.com/post/conditional-tier-rendering-the-battle-of-server- innerhtml-vs-js-mvc-json 

Conditional-tier rendering (cont.) Full page on the server Updates on the client Depending on browser support When requests hit the server we’ll ship static HTML. However navigation within the browser can be handled there, rendering the new pages purely via client-side code. The URL can be updated using pushState. Of course if the browser does not support this we’ll fall back to doing a server page fetch. 

Isomorphism (noun) corresponding or similar in form and relations. Basecamp and Twitter update the client-side view by sending HTML snippets. This means the server renders the templates. I prefer doing the rendering on the client-side, which means the templates should be isomorphic: they need to render both on the server *and* on the client. Isomorphism doesn’t extend just to the views. Routes and controllers also need to be isomorphic. 

Abstracted data access APIs, not databases Service-oriented architectures are cool! With traditional websites the server often talks directly to the database. This isn’t possible on the client-side, so you need a different way of getting the data needed to render a new view. You need an API. Having APIs is cool anyway, since it makes it easier to replace parts of your backend, and surely you’ll want other people to consume your API as well. Instead of talking to a database, the server talks to the API. So does the client. At State we’re building everything on top of our API. We’re eating our own dog food, so to speak. 

APIs make real-time easier Stream API resources via WebSockets Generic stream endpoint Let’s assume the client uses XHR to request JSON documents from the server. You could just as easily stream those same documents via WebSockets or Server-Sent Events. The stream endpoint itself can be generic: it only needs to ship JSON documents, not HTML snippets tailored to the specific client. 

Responsive UI and behavior Context Input devices Screen estate Browser support Users may have different needs from your application based on the device they’re using to interact with it. The supported input device may also lead to differences in the UI. A search menu on desktops may rely on mouse scrolling for pagination, but should have a button on touch devices where finger scrolling is obscured in the UI. Different UI elements will work differently between a phone, tablet, laptop and 27” iMac. A search menu can’t just be a drop-down on a phone, it needs to take over the whole screen. But it can’t take over the screen on a tablet, it must be a drop-down. Different browsers support different features. The experience may therefore be different between browsers. Your application must respond to all these scenarios. Can a framework provide the tools? 

OK, seriously. What’s the application stack? Let’s look at the different facets of State’s application stack and see where EyebrowJS fits in. Photo by Evan Blaser, http://www.flickr.com/photos/evanblaser/5499492306/, CC-BY-2.0. 

Server-side rendering Output HTML strings Ephemeral views Node.js We have a webserver written in Node.js. It renders the views into an HTML string for each request. This means the views are ephemeral, they only exist for as long as it takes to ship them down the wire. 

Client-side rendering Create & maintain a DOM tree Persistent views Data-binding for automatic updates The story is different on the client-side. We need to create and maintain a DOM tree, making changes when navigating between pages or when the underlying data changes. The views are persistent, not ephemeral. We use data-binding from the view to the model so model changes are automatically reflected in the DOM. 

Isomorphic rendering Render same template on client & server Made easier by server-side JavaScript The same template renders on the server and on the client. Server-side we compile into a Node-module, just straight JavaScript that renders the view when it’s executed. For the client-side we compile into an instruction-set that can efficiently create the DOM tree. Of course having server-side JavaScript makes this a lot easier, since the Eyebrow parser/ compiler as well as both runtimes are written in the same language. 

Heavy views Include logic to present the model One de inition for rendering in multiple scenarios I don’t get why people want logic-less templates. Sure, we shouldn’t write programs in our views but supporting if-statements or even variable assignment makes it easier to present the model without having separate helper code. This is especially important when rendering a view in different scenarios, like server and client. You express the UI logic only once: in your views. Let’s look at an example of “heavy views” in EyebrowJS. Note that the syntax here is pseudo- code, this talk isn’t about specifics, it’s about architecture and solutions. 

{{if user is following us}}

{{user.username}} follows you.

{{/if}} HTML and template tags are mixed. Here we use double mustaches to read values from the model. An if-statement (not actual syntax!) is used to conditionally render the paragraph: only if the user we’re looking at is following us. 

# user Arran is following us!

Arran follows you

If Arran is following us the paragraph renders. 

# user Arran is not following us :-( If he isn’t, the ASIDE is empty. If this were client-side and we got a real-time update that Arran stopped following us, the UI would have changed automatically. 

! {{for each in users}} ! !
• ! ! ! {{username}} ! !
! {{/for each}}

Or to give a different example, loop through an array and render a list. 

# users Arran, Edd, Matt

• Arran
• Edd
• Matt

The generated HTML lists the three users. 

# Add Charlotte to users

• Arran
• Edd
• Matt
• Charlotte

But client-side, if we add Charlotte to the list, it automatically gets added at the end of the UL’s child nodes. 

# Remove Edd from users

• Arran
• Matt
• Charlotte

And removing Edd will correctly remove his LI. 

Parse, compile & optimize views for specific environments The trick behind our approach is that EyebrowJS will parse, compile and optimize these views to run in specific environments. UI code that is needed on the server may not be needed on the client and vice versa. The render implementation on the server may indeed be radically different from the client implementation. The templates behind the views are still the same though! 

Models Ephemeral, per-request instances on the server Persistent, per-page-load on the client On the server the models are instantiated for each request and destroyed as soon as the request has finished. Like the server views they’re ephemeral. On the client they are persistent and may survive for as long as the page is open in the user’s browser. 

Models (cont.) Thin models on the server, essentially read-only Fat models on the client, manipulated and synced with server The server resources then are thin, essentially read-only. On the client however they are fat. They’ll contain business logic for how to update or manipulate their data. 

Models (cont.) One instance per unique resource Cache invalidation? For each unique resource we keep one object in memory. Wherever in the models this resource appears it’ll be the same object. Because views bind to the model instances we can’t simply remove them from the local datastore. Over time the application takes up more and more memory. Cache invalidation really is one of the hardest problems! Part of the solution will be to track whether a particular model (or field) is currently bound to by the view. Combined with expiry rules we should be able to keep memory under control. 

Caveat: resources must not contain contextual fields If each unique resource is stored with only one object you need to take care when modeling them. Resources should not contain fields that rely on a particular context, say via which API method they were returned, as those fields will survive when users navigate your application. 

GET /users/arran { "id": "arran" } GET /events/schooltrip/organizers { "id": "arran", "has_organized": true } In this example a User resource is returned by both endpoitns. However the second API call returns an `has_organized` field which is specific to the Schooltrip event. 

GET /users/arran { "id": "arran" } GET /events/schooltrip/organizers { "user": { "id": "arran" }, "has_organized": true } Instead we return an EventOrganizer resource which merely points to the User. 

Routing Isomorphic, too! Map a URL to a view and required API calls Prepare data for rendering in the view Isomorphic views only make sense if the application routing can be isomorphic as well. But what’s a route? In essence it maps a URL to both a view and the required API calls to construct the model context. The way we solved this at State is to define our routes in a Node-module. We then have a development-time endpoint in the server which can compile the routes into an AMD-module we can use in the browser. When we deploy to production we compile the routes directly into the application code. 

Server-side controllers Form POSTs, cookies, etc Everything you don’t want exposed as an API, or don’t want to do on the client-side Not all pages can be rendered client-side. Some form POSTs, like signup or logout are managed via server-side controllers. Same for the about page. Basically everything you don’t want exposed to the client via your API you keep to the server. 

Client-side interactions Happen in the view Need to affect the model One remaining question is how we provide interactions between the view and model layers. Of course these can only happen in the browser. My JavaScript toolkit of choice is Dojo, to which I'm a committer. Dojo has a widget library called Dijit. It uses markup to declare widgets and event bindings. Here's an example adapted from the official reference guide: 

Frankly this is just rubbish. One problem with Dijit is that it’s old. IE6 old, if not older. It’d be much nicer to build a menu using the MENU tag. Because it's based on custom DOM attributes Dijit is overly verbose and really not very flexible. That said, declarative widgets make a lot of sense! You can define them where you need them, instead of writing class names and separate code that tries to find the elements that need to become interactive. 

Declare interactions in views Already got heavy views, make them heavier Control interaction with (parts of) the view via JavaScript We’ve already got heavy views so adding declarative interactions isn’t that big of a step. We can define components and behavior, as well as bind DOM nodes into the component instances and wire up event handling. The behavior of components is kept separate from the views they affect. 

{{with user}}

{{username}}

{{component FollowButton}} {{/with}} Here’s an example. Within the context of a particular user model we render the FollowButton component. It has a view defined elsewhere that is rendered in the DOM tree where the component is declared. 

{{with user}}

{{username}}

{{behavior UserRelationship}} {{if followed by us}} Stop following {{else}} Follow {{/if}} {{/behavior}} {{/with}} In this example we directly apply behavior from a separate JavaScript file to a view. In this case the UserRelationship behavior, with an event hooked up to the button to toggle the following state. Assuming the underlying user model is updated to reflect the current relationship, the button text will automatically change. 

{{with user}}

{{username}}

{{behavior UserRelationship}} {{if followed by us}} Stop following {{else}} Follow {{/if}} {{/behavior}} {{/with}} In this example we directly apply behavior from a separate JavaScript file to a view. In this case the UserRelationship behavior, with an event hooked up to the button to toggle the following state. Assuming the underlying user model is updated to reflect the current relationship, the button text will automatically change. 

Arran

Follow CLICK! Let's click on that button… 

Arran

Stop following Behind the scenes the behavior changes the following state. The model also gets updated, which means the if-statement changes. And presto! The button now says “Stop following”. 

Views respond to state changes Data-binding, baby! With data-binding into the models and component states, the views can respond to changes. Rather than having your component manipulate the DOM directly, it can set a state value and the UI logic in the view will update automatically. 

{{behavior UserRelationship}} {{if followed by us}} Stop following {{else}} Follow {{/if}} {{/behavior}} In this example the UserRelationship behavior sets a value on its own widget model that is exposed to the view. We can now disable the button while the relationship is modified on the server. 

Arran

Follow CLICK! Let's click on that button… 

# Updating…

Arran

Follow Behind the scenes the behavior changes its "updating" state. The button gets disabled. 

# Done!

Arran

Stop following When the model has been changed the button is re-enabled and the text has changed to “Stop following”. 

Decoupling makes testing components easier Test component separately from view Test view rendering given (static) view models This decoupling makes it easier to test and develop the components, since they themselves know nothing about the DOM tree they’re controlling other than what the view decides to tell the component. Similarly all permutations of the view models can be set in a controlled environment to make sure the view responds appropriately and the correct DOM tree is rendered. 

Responsive markup Not just interactions Shared de inition for server & client, and responsive scenarios It’s not just that we want to declare interactions in our views. We also want to declare snippets that should only be rendered in particular circumstances. The view becomes a shared definition for server- and client-side rendering, as well as the various responsive scenarios we have to deal with. 

{{with user}}

{{username}}

! ! {{if we’re not on the server…}} {{behavior UserRelationship}} ! ! ! ! … ! {{/behavior}} ! ! {{/if}} {{/with}} We should only render the follow button if the view isn’t being rendered on the server. This would be because the button requires JavaScript to function, and we can’t guarantee that the users who see the server output have JavaScript enabled. 

{{behavior Menu}} … {{if we’re on a tablet device…}}

Load more

{{/if}} {{/behavior}} Or here for a Menu behavior, if we’re on a tablet device we can render a button to load more items. 

{{component UserSearch}} Declaring a search component is great, but it doesn’t help with progressive enhancement. The widget is only rendered when JavaScript is available in the browser. 

{{component UserSearch}} {{else if we can’t render the component}} Search {{/component}} With this markup we can render a fallback in case the component itself couldn’t be rendered. 

Optimize for different platforms By placing the conditions on when to render parts of the view into the markup itself we can optimize the views for different platforms. For example we could compile optimized JavaScript just for an iPhone so we don’t have to ship any code that would normally run on a desktop. 

Write in the same place, not once Even if a particular part of your application behaves differently in different scenarios, you can still write those variations in one place. You don’t need to build four different implementations with lots of duplication between them. 

Let’s talk design patterns To recap, let’s briefly talk about the various patterns in application design EyebrowJS encourages. Photo by Quinn Dombrowski, http://www.flickr.com/photos/quinnanya/4990127592/, CC- BY-SA 2.0. 

Model Model → Resource Models! Yes! I feel though that historically this talks more about databases than it does about API resources. In fact EyebrowJS encourages business logic around the data to be in the model layer. I’ll follow Charlie Robbins lead and will call models “Resources” instead. Charlie, of Nodejitsu, wrote an excellent article by the way on [isomorphic JavaScript code][sijc]. The link is in the slides. [sijc]: http://blog.nodejitsu.com/scaling-isomorphic-javascript-code 

View Views! Yes! Heavy views, with logic. 

ViewModel With the logic inside the views and the way we use components we can safely say we’re using the ViewModel pattern. 

Isomorphism A key characteristic of applications built with EyebrowJS is reusing the same templates, routes and API calls on the server and client. Isomorphism deserves a shout-out. 

Controllers And we do have a few (server-side) controllers too. 

MVC is missing something… Up to this point we’re pretty much talking MVC on the server-side and MVVM on the client side. But that doesn’t really cover our interactions and responsive markup. 

Behavior We need to draw attention to the Behavior part of EyebrowJS applications. There is actually a pattern called [Presentation-Abstraction-Control][PAC] that somewhat describes having a hierarchical structure of “model”, “view” and “controller” responsibilities which matches widgets in a DOM tree. Nobody uses it though. [PAC]: http://en.wikipedia.org/wiki/Presentation%E2%80%93abstraction%E2%80%93control 

Drumroll please… Now, if for the sake of being fancy we want to name this pattern, what would it be? 

iRVVM+B ;-) It’s isomorphic-Resource-View-ViewModel+Behavior! 

(MVVM) But if that’s too silly, let’s just say it’s MVVM. 

Register for an invite at state.com I’m @novemberborn or [email protected] Cheers! And that’s all folks. Further questions? 

Licensed under Creative Commons Attribution-Share Alike 3.0 http://creativecommons.org/licenses/by-sa/3.0/ Imagery by Vox Efx David DeHetre Katelyn Kenderdine Anssi Koskinen Evan Blaser Quinn Dombrowski Jeff Kubina Many, many thanks to the wonderful people on Flickr who licensed their photos under Creative Commons. Photo by Jeff Kubina, http://flickr.com/photos/kubina/903033693/. CC-BY-SA 2.0.