Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Web Dev @ ORF.at

Web Dev @ ORF.at

My guest lecture under the official title "Web-Engineering @ Work" for the Web Engineering course at the University of Technology Vienna.

Philipp Naderer

May 30, 2016
Tweet

More Decks by Philipp Naderer

Other Decks in Technology

Transcript

  1. Web Engineering

    @ ORF.at
    Philipp Naderer
    [email protected]

    View full-size slide

  2. We are hiring!
    – 32h / Week (not full time for now, sorry)

    – You know how to build a client-side JavaScript application

    – Node.js or RingoJS (even better) knowledge –> a plus, but not required

    – Basic Linux stuff is required (restarting an application, …)

    – We use Apache, so you should not be an Apache hater ;-)

    – You will be responsible for sites that millions of people use
    http://jobs.orf.at/job577.html

    View full-size slide

  3. My Background
    • Last days as a student (Software Engineering)
    • Joined ORF.at in summer 2007
    • Job description: „Software Engineer“

    Front-end (news/sport/…), CMS Back-end Developer, Web Accessibility,

    Data-Driven Journalism, Maintainer of RingoJS
    • Github / Twitter: @botic
    • Mail: [email protected]

    View full-size slide

  4. Outline for this talk
    • What is ORF.at and fundamental building blocks

    „The Stack“ – Hardware, Databases, CMS, RingoJS
    • Theory vs. Practice

    Topics covered by the „Web Engineering“ lecture and my daily practices
    • Covering Elections

    How ORF.at covers elections and what is the technical infrastructure
    • Q&A

    View full-size slide

  5. • 50,2% market reach in (#1 of all Austrian sites)

    ~ 400 Mio. Page Impressions

    ~ 8,5 Mio. Unique Clients

    ~ 3,1 Mio. Unique Users
    • Own infrastructure, no cloud services involved
    • Custom CMS „Mojo“
    • Built on top of JavaScript runtime environments
    Helma and RingoJS
    • 12 developers, company size ~ 110 employees

    View full-size slide

  6. 1998 ➜ 2016

    View full-size slide

  7. news.orf.at sport.orf.at oesterreich.orf.at

    View full-size slide

  8. wetter.orf.at science.orf.at help.orf.at

    View full-size slide

  9. oe3.orf.at fm4.orf.at 7 tage / on-demand

    View full-size slide

  10. religion.orf.at debatte.orf.at der.orf.at
    corporate site

    View full-size slide

  11. orf.at specials songcontest.orf.at orf.at/wahl

    View full-size slide

  12. apps for android and ios

    View full-size slide

  13. On-premise Hardware
    Hosted at APA’s datacenter in Vienna

    View full-size slide

  14. Hardware Facts
    • Standard server hardware

    – around 90 external servers

    – same type of machines, easier maintenance

    – SSDs for newer generations, still some HDDs

    – typical server between 12 GB to 64 GB memory
    • 10 different clusters

    – a primary server mirrors files to other members

    – relatively simple setup: heartbeat + inotify
    • 20 GBit connection to the Internet

    View full-size slide

  15. Operating Systems
    • Development on Windows / Macs / Linux

    – developers should use whatever they prefer

    – I use Windows 10 only at work

    – most users still have a Windows PC

    – tooling support still better on Linux and Macs



    • Customized Debian Linux on servers 

    – optimized kernel settings

    – integrated update mechanism

    – well-tested with our JVM-based stack

    View full-size slide

  16. Databases
    Newer Products & Services
    Radio On-demand Service
    OpenStreetMap / PostGIS
    ORF Corporate Site
    runs on every dev machine
    Core Applications
    Mojo CMS
    Mojo-related Services

    View full-size slide

  17. Mojo CMS
    • Core of all news websites
    • Runs on top of Helma
    • ~ 70 concurrent text & photo editors, proofreaders
    • > 70.000 stories per year
    • Wiki text editor with live preview
    • Client-side implementation based on jQuery plugins

    View full-size slide

  18. Mojo CMS technical details
    • JSON-like schema-less „content“ objects
    • 3 levels of caching at the front-end:

    – fully rendered HTML (caching servlet)

    – rendered Wiki text for each story (content cache)

    – Helma’s JS object cache (orm / db cache)
    • There exists a 1:1 statified auto-updating snapshot of
    the dynamic Helma-based site for maintenance
    • Static CSS / JS resources are compressed by YUI

    View full-size slide

  19. dev.ORF.at
    Blog with screencasts and technical details

    around Mojo by Marius Lessiak

    View full-size slide

  20. History of JavaScript @ ORF.at
    - monolithic web framework
    - special HopObject

    as root prototype
    - objects mapped to db tables
    - complex code repository loader
    - modular runtime environment
    - built-in core modules
    - CommonJS modules
    - Ringo-specific modules
    - package support
    - hot code reloading

    View full-size slide

  21. What is RingoJS?
    “Ringo is a JavaScript platform
    built on the JVM and optimized for
    server-side applications. It takes a
    non-dogmatic stance on things like
    I/O paradigms.”

    View full-size slide

  22. What is RingoJS?
    • multi-threaded
    JavaScript
    • CommonJS compatible
    • ES5 + some ES6
    • code repository
    manager
    • optimized to run

    web applications

    View full-size slide

  23. What is RingoJS is not …
    • copying Node.js
    • compatible with npm
    • the fastest async io
    framework
    • used by thousands
    • JEE / enterprise
    software
    • single-threaded

    View full-size slide

  24. webapp.js

    Module
    webapp.js

    Module
    blocking
    running running
    Event Loop

    View full-size slide

  25. Multi-threaded JavaScript?
    • Ringo‘s concept: Workers
    • Every worker corresponds to a JVM thread
    • You can still use an EventLoop if you want
    • Avoid the shared global scope
    • Singletons instead of shared data

    View full-size slide

  26. RingoJS Servlet-based

    HTTP Request Handling

    View full-size slide

  27. Web Scraper
    addToClasspath("../../../jars/jsoup.jar");

    importPackage(Packages.org.jsoup);
    let orfat = Jsoup.connect("http://orf.at");

    orfat.userAgent("Mozilla/5.0 …").timeout(5000).get();
    orfat.select("h2 + p");

    // do whatever you want with the elements …

    View full-size slide

  28. More information on RingoJS
    • ringojs.org – the official website of the project
    • packages.ringojs.org – Ringo’s package registry
    • https://github.com/ringo/ringojs – Sources
    • @ringojs on Twitter

    View full-size slide

  29. Remarks on the

    Web Engineering course
    from my personal perspective,

    heavily influenced by my work at ORF.at

    View full-size slide

  30. HTML / Markup
    • We try to avoid errors in the validator, but some of
    them are required e.g. for easier SVG fallbacks
    • No strict rule / style guide to use a trailing slash or
    not on self-closing elements
    • Used HTML5 very early
    • HTML5 video still is complicated for live streams
    • HTML5 video and audio will be default soon,

    we have our own oon.media library

    View full-size slide

  31. CSS / SASS
    • Mojo projects use CSS, newer ones SASS / SCSS
    • Strict rule: #ids are evil, specificity only via classes
    • box-sizing: border-box;

    200px = 200px, not 180px = 200px – 20px paddings

    width = content width + paddings + borders
    • Relative units where possible (rem / % / vw / vh)
    • Avoid floats, use display: inline-block; or Flexbox

    View full-size slide

  32. Custom typeface „ORF ON“
    • „ORF ON“ font family
    • 19 different font styles
    • Special style for tabular data
    • Created by Ole Schäfer from

    primetype
    • Used in almost all ORF online
    sites via web fonts & in apps

    (and in this slides)
    • Test them: ORF.at/fonts/

    View full-size slide

  33. Accessibility
    • This is a major topic for ORF.at as public broadcaster
    • WCAG 2.0 Level AA as goal since years
    • 4 important principles:
    • Information must be Perceivable
    • User interfaces must be Operable
    • Information must be Understandable
    • Content must be Robust

    View full-size slide

  34. Accessibility
    • Avoid ARIA as much as possible

    – use the element instead of roles

    – use landmarks for the basic site areas

    – hide inaccessible ads from screen readers
    • If you work on a CMS: Look at the Authoring Tool
    Accessibility Guidelines 2.0 (ATAG)
    • Dyschromatopsia (Rot-Grün-Sehschwäche) is a problem
    for election data (SPÖ vs. Grüne)
    • NVDA (NonVisual Desktop Access) is a great open
    source and free screen reader! http://www.nvaccess.org

    View full-size slide

  35. Responsive Webdesign
    • We use this principle where we can
    • Still there is a separate mobile version
    • The future is responsive-only!
    • Don’t do mobile first, look at your content and
    principle site elements and then design a site
    • Desktop is still strong in many areas
    • Ads / ad format standards are still a big problem

    View full-size slide

  36. JavaScript
    • Don’t over-engineer your sites, sometimes jQuery is
    enough for simple interactions
    • We use Marionette JS / Backbone, Angular 1 & 2, and
    experiment with React for future projects
    • gulp & webpack for client-side applications
    • ES6 via Babel lets us use modern JS today
    • Service Workers are a very strong technology for the
    near future for Progressive Web Applications (PWA)

    View full-size slide

  37. General
    • JSON instead of XML

    ORF.at is JavaScript oriented for nearly 20 years(!)
    • Don’t be the first, try to implement it like the best
    • A big challenge: distribute knowledge across the team
    • Automate things as much as you can
    • The number of (internal) APIs exploded in the last years
    • Apps: WebViews vs. native: go native for important parts

    View full-size slide

  38. Austrian Presidential Elections

    View full-size slide

  39. Very broad coverage
    • news.ORF.at/wahl – information around the
    election campaigns, live updates on the election
    day and result overview
    • news.ORF.at/beingpresident – social channel,
    Instagram Q/A with candidates, 360° videos
    • ORF.at Wahl – apps for iOS and Android,

    data source was the „wahl“ channel

    View full-size slide

  40. Project Management
    • We use JIRA for ticket management
    • Per election a new Kanban board
    • Slack for communication during the election day
    • Email is still a very prominent tool
    • We start working on the sites

    around 2 to 3 weeks before the election

    View full-size slide

  41. „Wahlaggregator“ Services
    news.orf.at api
    collects stories from ORF.at
    tagged with „wahl16“
    wahlcomputer
    client-side web application
    renders election results
    embedded
    wahlcomputer
    separate built of full app
    wahlaggregator
    fetches stories, videos and
    audios and renders HTML
    tvthek api
    videos grouped in a „topic“
    ö1 podcast xml
    for radio broadcasts
    special gulp task
    static html

    View full-size slide

  42. „Wahlcomputer“ Services
    ringo-wahlparser
    converts .csv into .json
    wahldaten.orf.at
    watched directory
    bmi raw data
    txt like in the 1970ies
    orf ta2 transformator
    converts .txt into .csv
    results as
    json
    ftp

    pull
    ftp
    push
    local copy
    wahlcomputer
    client-side web application
    renders election results
    sora projection
    calculated by experts
    wahlcrash
    special mini CMS
    json
    json

    View full-size slide

  43. Wahlcomputer
    • Client-side web application

    – 3.800 lines of JS, 3.200 lines of SCSS

    – gulp (https://github.com/orfon/gulp-boilerplate)

    – SASS as CSS preprocessor

    – Marionette JS / Backbone

    – ES6 transformed with Babel to browser ES5

    – npm also for client JS modules
    • Renders SVG maps or embedded slippy maps
    • Also responsible for the SORA projection rendering

    View full-size slide

  44. Globus
    Mapnik-based rendering of geo information

    View full-size slide

  45. Presidential Election Peaks
    • HTTP request peaks:

    – First round: 8.100 req/s

    – Stand-off: 14.100 req/s

    – Monday: 8.600 req/s
    • The fill-in broadcast

    „Weißblaue Geschichten“

    was visible in the monitoring
    • To mitigate the effect we
    served a static „fallback“
    version of ORF.at

    View full-size slide

  46. We are hiring!
    – 32h / Week (not full time for now, sorry)

    – You know how to build a client-side JavaScript application

    – Node.js or RingoJS (even better) knowledge –> a plus, but not required

    – Basic Linux stuff is required (restarting an application, …)

    – We use Apache, so you should not be an Apache hater ;-)

    – You will be responsible for sites that millions of people use
    http://jobs.orf.at/job577.html

    View full-size slide