Web Dev @ ORF.at

My guest lecture under the official title "Web-Engineering @ Work" for the Web Engineering course at the University of Technology Vienna.

Transcript

1. Web Engineering
 @ ORF.at Philipp Naderer philipp.naderer@orf.at
2. We are hiring! – 32h / Week (not full time for now, sorry)
 – You know how to build a client-side JavaScript application
 – Node.js or RingoJS (even better) knowledge –> a plus, but not required
 – Basic Linux stuff is required (restarting an application, …)
 – We use Apache, so you should not be an Apache hater ;-)
 – You will be responsible for sites that millions of people use http://jobs.orf.at/job577.html
3. My Background • Last days as a student (Software Engineering) • Joined ORF.at in summer 2007 • Job description: „Software Engineer“
 Front-end (news/sport/…), CMS Back-end Developer, Web Accessibility,
 Data-Driven Journalism, Maintainer of RingoJS • Github / Twitter: @botic • Mail: philipp.naderer@orf.at
4. Outline for this talk • What is ORF.at and fundamental building blocks
 „The Stack“ – Hardware, Databases, CMS, RingoJS • Theory vs. Practice
 Topics covered by the „Web Engineering“ lecture and my daily practices • Covering Elections
 How ORF.at covers elections and what is the technical infrastructure • Q&A
5. • 50,2% market reach in (#1 of all Austrian sites)
 ~ 400 Mio. Page Impressions
 ~ 8,5 Mio. Unique Clients
 ~ 3,1 Mio. Unique Users • Own infrastructure, no cloud services involved • Custom CMS „Mojo“ • Built on top of JavaScript runtime environments Helma and RingoJS • 12 developers, company size ~ 110 employees
6. 1998 ➜ 2016
7. news.orf.at sport.orf.at oesterreich.orf.at
8. wetter.orf.at science.orf.at help.orf.at
9. oe3.orf.at fm4.orf.at 7 tage / on-demand
10. religion.orf.at debatte.orf.at der.orf.at corporate site
11. orf.at specials songcontest.orf.at orf.at/wahl
12. apps for android and ios
13. On-premise Hardware Hosted at APA’s datacenter in Vienna
14. Hardware Facts • Standard server hardware
 – around 90 external servers
 – same type of machines, easier maintenance
 – SSDs for newer generations, still some HDDs
 – typical server between 12 GB to 64 GB memory • 10 different clusters
 – a primary server mirrors files to other members
 – relatively simple setup: heartbeat + inotify • 20 GBit connection to the Internet
15. Operating Systems • Development on Windows / Macs / Linux
 – developers should use whatever they prefer
 – I use Windows 10 only at work
 – most users still have a Windows PC
 – tooling support still better on Linux and Macs
 
 
 • Customized Debian Linux on servers 
 – optimized kernel settings
 – integrated update mechanism
 – well-tested with our JVM-based stack
16. Databases Newer Products & Services Radio On-demand Service OpenStreetMap / PostGIS ORF Corporate Site runs on every dev machine Core Applications Mojo CMS Mojo-related Services
17. Mojo CMS • Core of all news websites • Runs on top of Helma • ~ 70 concurrent text & photo editors, proofreaders • > 70.000 stories per year • Wiki text editor with live preview • Client-side implementation based on jQuery plugins
22. Mojo CMS technical details • JSON-like schema-less „content“ objects • 3 levels of caching at the front-end:
 – fully rendered HTML (caching servlet)
 – rendered Wiki text for each story (content cache)
 – Helma’s JS object cache (orm / db cache) • There exists a 1:1 statified auto-updating snapshot of the dynamic Helma-based site for maintenance • Static CSS / JS resources are compressed by YUI
23. dev.ORF.at Blog with screencasts and technical details
 around Mojo by Marius Lessiak
25. History of JavaScript @ ORF.at - monolithic web framework - special HopObject
 as root prototype - objects mapped to db tables - complex code repository loader - modular runtime environment - built-in core modules - CommonJS modules - Ringo-specific modules - package support - hot code reloading
26. What is RingoJS? “Ringo is a JavaScript platform built on the JVM and optimized for server-side applications. It takes a non-dogmatic stance on things like I/O paradigms.”
27. What is RingoJS? • multi-threaded JavaScript • CommonJS compatible • ES5 + some ES6 • code repository manager • optimized to run
 web applications
28. What is RingoJS is not … • copying Node.js • compatible with npm • the fastest async io framework • used by thousands • JEE / enterprise software • single-threaded
29. webapp.js
 Module webapp.js
 Module blocking running running Event Loop
30. Multi-threaded JavaScript? • Ringo‘s concept: Workers • Every worker corresponds to a JVM thread • You can still use an EventLoop if you want • Avoid the shared global scope • Singletons instead of shared data
31. RingoJS Servlet-based
 HTTP Request Handling
32. Demo …
33. Web Scraper addToClasspath("../../../jars/jsoup.jar");
 importPackage(Packages.org.jsoup); let orfat = Jsoup.connect("http://orf.at");
 orfat.userAgent("Mozilla/5.0 …").timeout(5000).get(); orfat.select("h2 + p");
 // do whatever you want with the elements …
34. More information on RingoJS • ringojs.org – the official website of the project • packages.ringojs.org – Ringo’s package registry • https://github.com/ringo/ringojs – Sources • @ringojs on Twitter
35. Remarks on the
 Web Engineering course from my personal perspective,
 heavily influenced by my work at ORF.at
36. HTML / Markup • We try to avoid errors in the validator, but some of them are required e.g. for easier SVG fallbacks • No strict rule / style guide to use a trailing slash or not on self-closing elements • Used HTML5 very early • HTML5 video still is complicated for live streams • HTML5 video and audio will be default soon,
 we have our own oon.media library
37. CSS / SASS • Mojo projects use CSS, newer ones SASS / SCSS • Strict rule: #ids are evil, specificity only via classes • box-sizing: border-box;
 200px = 200px, not 180px = 200px – 20px paddings
 width = content width + paddings + borders • Relative units where possible (rem / % / vw / vh) • Avoid floats, use display: inline-block; or Flexbox
38. Custom typeface „ORF ON“ • „ORF ON“ font family • 19 different font styles • Special style for tabular data • Created by Ole Schäfer from
 primetype • Used in almost all ORF online sites via web fonts & in apps
 (and in this slides) • Test them: ORF.at/fonts/
39. Accessibility • This is a major topic for ORF.at as public broadcaster • WCAG 2.0 Level AA as goal since years • 4 important principles: • Information must be Perceivable • User interfaces must be Operable • Information must be Understandable • Content must be Robust
40. Accessibility • Avoid ARIA as much as possible
 – use the <button> element instead of roles
 – use landmarks for the basic site areas
 – hide inaccessible ads from screen readers • If you work on a CMS: Look at the Authoring Tool Accessibility Guidelines 2.0 (ATAG) • Dyschromatopsia (Rot-Grün-Sehschwäche) is a problem for election data (SPÖ vs. Grüne) • NVDA (NonVisual Desktop Access) is a great open source and free screen reader! http://www.nvaccess.org
41. Responsive Webdesign • We use this principle where we can • Still there is a separate mobile version • The future is responsive-only! • Don’t do mobile first, look at your content and principle site elements and then design a site • Desktop is still strong in many areas • Ads / ad format standards are still a big problem
42. JavaScript • Don’t over-engineer your sites, sometimes jQuery is enough for simple interactions • We use Marionette JS / Backbone, Angular 1 & 2, and experiment with React for future projects • gulp & webpack for client-side applications • ES6 via Babel lets us use modern JS today • Service Workers are a very strong technology for the near future for Progressive Web Applications (PWA)
43. General • JSON instead of XML
 ORF.at is JavaScript oriented for nearly 20 years(!) • Don’t be the first, try to implement it like the best • A big challenge: distribute knowledge across the team • Automate things as much as you can • The number of (internal) APIs exploded in the last years • Apps: WebViews vs. native: go native for important parts
44. Austrian Presidential Elections
45. Very broad coverage • news.ORF.at/wahl – information around the election campaigns, live updates on the election day and result overview • news.ORF.at/beingpresident – social channel, Instagram Q/A with candidates, 360° videos • ORF.at Wahl – apps for iOS and Android,
 data source was the „wahl“ channel
46. Project Management • We use JIRA for ticket management • Per election a new Kanban board • Slack for communication during the election day • Email is still a very prominent tool • We start working on the sites
 around 2 to 3 weeks before the election
47. „Wahlaggregator“ Services news.orf.at api collects stories from ORF.at tagged with „wahl16“ wahlcomputer client-side web application renders election results embedded wahlcomputer separate built of full app wahlaggregator fetches stories, videos and audios and renders HTML tvthek api videos grouped in a „topic“ ö1 podcast xml for radio broadcasts special gulp task static html
48. „Wahlcomputer“ Services ringo-wahlparser converts .csv into .json wahldaten.orf.at watched directory bmi raw data txt like in the 1970ies orf ta2 transformator converts .txt into .csv results as json ftp
 pull ftp push local copy wahlcomputer client-side web application renders election results sora projection calculated by experts wahlcrash special mini CMS json json
49. Wahlcomputer • Client-side web application
 – 3.800 lines of JS, 3.200 lines of SCSS
 – gulp (https://github.com/orfon/gulp-boilerplate)
 – SASS as CSS preprocessor
 – Marionette JS / Backbone
 – ES6 transformed with Babel to browser ES5
 – npm also for client JS modules • Renders SVG maps or embedded slippy maps • Also responsible for the SORA projection rendering
50. Globus Mapnik-based rendering of geo information
55. Presidential Election Peaks • HTTP request peaks:
 – First round: 8.100 req/s
 – Stand-off: 14.100 req/s
 – Monday: 8.600 req/s • The fill-in broadcast
 „Weißblaue Geschichten“
 was visible in the monitoring • To mitigate the effect we served a static „fallback“ version of ORF.at
56. We are hiring! – 32h / Week (not full time for now, sorry)
 – You know how to build a client-side JavaScript application
 – Node.js or RingoJS (even better) knowledge –> a plus, but not required
 – Basic Linux stuff is required (restarting an application, …)
 – We use Apache, so you should not be an Apache hater ;-)
 – You will be responsible for sites that millions of people use http://jobs.orf.at/job577.html
57. Mulțumesc!