$30 off During Our Annual Pro Sale. View Details »

A high profile project with Symfony and API Platform: beIN SPORTS

A high profile project with Symfony and API Platform: beIN SPORTS

Speakers: Kevin Dunglas and Fabien Gasser. In a first part we will introduce semantic and linked data technologies supported by Symfony and API Platform: JSON-LD, Hydra and Schema.org, their contributions to interoperability and a web of data and how to use them, technologies until there the restricted domain of Java such as RDF, SPARQL, triple stores, ontology engines.

In a second part we will give a feedback on the strategy we employed to build a high traffic CMS platform using Symfony, API Platform and AngularJS as technical stack. We will see how an API-first strategy helped us to deliver fluently content on several channels (desktop, mobile apps, connected devices, partners…) and how that strategy helped us to make large teams working together on the same project. We will also see how to leverage Varnish and AWS to absorb gigantic traffic peaks, how pure-JS fit well with that architecture and effects on SEO

In a third part we will give you some tips in order to build an e-commerce platform based on that approach.

Kévin Dunglas

December 03, 2015
Tweet

More Decks by Kévin Dunglas

Other Decks in Programming

Transcript

  1. A high profile project with Symfony
    and API Platform: beIN SPORTS
    Symfony Con 2015

    View Slide

  2. Fabien Gasser
    twitter: @fabien_gasser
    Solution Architect at Smile
    [email protected]
    Kévin Dunglas
    twitter/github: @dunglas
    CEO at Les-Tilleuls.coop
    Symfony Core Team
    API Platform creator
    [email protected]

    View Slide

  3. Build the new beIN SPORTS Core Platform: may the force be with us !
    • Content Distributed on several channel: websites, mobile apps, TV apps, XBOX,
    PS4…
    • Multi-site: 4 territories
    • Multi-language per site: 7 languages
    • Publication workflow: 60 journalists distributed across 4 Newsrooms
    • Real-time publication
    • Strong SEO needs
    • Performant, maintainable, open
    • High traffic: up to 70M of hits and 50M video views monthly
    • 5 teams, with more than 30 people working on the project
    We created an API Centric CMS
    OUR MISSION

    View Slide

  4. Use Linked Data Luke, to enrich your data:
    • Knowledge Database: semantically structured data
    • External Knowledge database (dbpedia, private partners)
    • Content enrichment
    • API-Management: manage the data we produce to
    external consumer
    We created a Linked Data API Centric CMS
    LINKED DATA

    View Slide

  5. http://www.beinsports.com/
    In a 4 months time frame, Smile
    redeveloped from grounds up, beIN
    SPORTS digital core platform serving up
    to 70 m PV and 50 m video views
    monthly, across 30 countries where
    beIN operates 24/7, in 4 languages.
    From day-1 launch, back-end platform
    supported multilingual publishing
    workflow by a team of 60
    journalist/editorialist, distributed across
    4 Newsrooms (Miami, Paris, Doha,
    Sydney).
    This has been made possible
    thanks to Symfony.

    View Slide

  6. The API-first strategy

    View Slide

  7. An API at the heart of the Information System:
    • Central point to access data
    • Encapsulate all the business logic
    • Access same data and features from everywhere
    • Agregate heterogenous data and display coherent data
    encapsulating business-logic.
    API FIRST

    View Slide

  8. + Guzzle
    EASY TO EXPOSE, EASY TO CONSUME !
    • Core web API
    • Back-office:
    a HTML5 webapp (SPA)
    • Front-office:
    a website generating HTML
    server-side (for SEO)
    • Connected devices:
    TV apps, XBOX app, PS4
    app
    • Third part application:
    EPG, other websites

    View Slide

  9. • Centralizes R/W access to data
    • Holds all the business logic
    • Is stateless (PHP sessions make horizontal scalability
    harder)
    • Requires authentication to access to some endpoints
    (but not all) and to write data
    • Is built with PHP, Symfony and API Platform
    API

    View Slide

  10. • Single Page Application: 100% HTML5 / JavaScript /
    CSS
    • Presentation logic: login screen, publishing workflow,
    medias management...
    • Queries the API in AJAX to retrieve and modify data
    • Client-side routing (HTML5 push state)
    BACK OFFICE

    View Slide

  11. • Public pages: homepage, lists, categories, articles,
    videos, photos...
    • Server-side generated HTML (some JS too)
    • SEO: fancy URLs, structured data, breadcrumbs...
    • Queries the API in AJAX
    • Responsive design
    FRONT OFFICE

    View Slide

  12. Mobile apps, TV apps, XBOX and PS4 apps…:
    • One responsibility: display (nicely) data
    • Query the central API to retrieve data
    What about User Generated Content like comments,
    reviews?
    External services.
    OTHER CLIENTS

    View Slide

  13. Each application has its own:
    • Git repository
    • CI
    • servers (if applicable)
    • domain name (if
    applicable:
    api.example.com,
    admin.example.com,
    example.com)
    CONTINOUS INTEGRATION

    View Slide

  14. Benefits for the project:
    harder, better, faster, stronger
    • 1 app = 1 team (specialized skills)
    • No business logic duplication: all in the API
    • Easy refactoring: touching a component has no impact
    on others
    • API format = contract: documented and tested
    • Easy to add new client apps
    WHY SHOULD I USE API PLATFORM ?

    View Slide

  15. CMS AND KNOWLEDGE DATABASE
    MICROSERVICES ARCHITECTURE
    Content
    API
    Sports
    API
    EPG
    API
    Core Platform
    API
    Web App
    Mobile
    Connected devices
    REST
    REST
    REST
    Database
    Back Office
    Third part
    content
    Batch
    REST
    Core Platform
    API
    JWT
    AUTH
    DB
    DB
    DB
    DB
    Webhooks
    XML

    View Slide

  16. Formats, open standards and
    patterns

    View Slide

  17. HTTP + REST + JSON:
    • Work everywhere
    • Lightweight
    • Stateless (if done well)
    • HTTP has a powerful caching model
    • Extensible (JSON-LD, Hydra...)
    • High quality tooling
    API PATTERN

    View Slide

  18. Hypermedia as the Engine of Application State
    • Hypermedia: IRI as identifier
    • Ability to reference external data (like hypertext links)
    • Auto discoverable => Generic clients
    API FORMAT: HATEOAS

    View Slide

  19. JSON-LD: JSON for Linked Data
    • Compliant with technologies of the semantic web:
    RDF, SPARQL, triple store…
    • Standard: W3C recommendation (since 2014)
    • Easy to use: looks like a typical JSON document
    • Already used by Gmail, GitHub, BBC, Microsoft, US
    gov… and now beIN SPORTS
    Remember our Knowledge Database?
    JSON-LD

    View Slide

  20. The Schema.org vocabulary
    • Large set of elements: events, team, people, videos...
    • Created and understood by Google, Bing, Yahoo! and
    Yandex
    • Massively used, and run by the W3C (Web schemas
    group)
    • Can be used in HTML (microdata / RDFa) and JSON-LD
    • Extensible (custom vocabularies)
    SCHEMA.ORG

    View Slide

  21. HYDRA
    • Describe REST APIs in JSON-LD
    • = write support
    • = auto discoverable APIs
    • Standard for collections, paginations, errors, filters
    • Draft W3C
    HYDRA

    View Slide

  22. JSON Web Token (JWT)
    • Lightweight and simple
    • Stateless
    • token signed and verified server-side
    • then, stored client-side (web storage)
    • sent in an Authorization header in each AJAX request
    API AUTHENTICATION

    View Slide

  23. Linked Data

    View Slide

  24. LINKED DATA
    Soccer Player
    Person
    Name
    Date of Birth
    Soccer Match
    Event
    Team
    play in
    Rdf:type
    ⚽ http://dbpedia.org/ressource/Zlatan_Ibrahimovic
    Begin Date
    End Date
    Rdfs:subClassOf
    Dbpedia-owl:birth_name
    Dbpedia-owl:birthdate
    Has a
    Has a
    Has a
    Is an
    Dbpprop:Team
    Third part
    semantic engine:
    • Apache Jena
    • Apache
    Marmotta
    • Apache Stanbol
    Luke, learn more
    about linked data !

    View Slide

  25. API Platform powered

    View Slide

  26. A decoupled PHP web framework to build modern, API-first
    web projects.
    Out of the box hypermedia and Linked Data support with
    JSON-LD, Schema.org, Hydra and JWT in is heart
    $> composer create-project api-platform/api-platform my-project
    API PLATFORM

    View Slide

  27. API Platform Symfony
    • Built on top of Symfony full-stack (3.0 OK)
    • Install any existing SF bundles
    • Follow SF Best Practices
    • Work with existing SF app
    • Optional: advanced Doctrine support
    API PLATFORM AND SYMFONY

    View Slide

  28. SCHEMA GENERATOR
    Pick types and properties you need from Schema.org:
    # app/config/schema.yaml
    namespaces:
    entity: AppBundle\Entity
    types:
    Person:
    parent: false
    properties:
    name: ~
    birthDate: ~
    gender: ~
    # other Schema.org types
    $> bin/schema generate-types \
    src/app/config/schema.yml
    namespace AppBundle\Entity;
    // Use statements
    /**
    * A person (alive, dead, undead, or fictional).
    *
    * @see http://schema.org/Person Documentation on Schema.org
    *
    * @ORM\Entity
    * @Iri("http://schema.org/Person")
    */
    class Person
    {
    /**
    * @var integer
    * @ORM\Column(type="integer")
    * @ORM\Id
    * @ORM\GeneratedValue(strategy="AUTO")
    */
    private $id;
    /**
    * @var string The name of the item.
    *
    * @ORM\Column(nullable=true)
    * @Assert\Type(type="string")
    * @Iri("https://schema.org/name")
    */
    private $name;
    // Other properties, getters, setters, adders, removers….

    View Slide

  29. SCHEMA GENERATOR
    You get:
    • PHP classes, properties, getters and setters (PSR
    compliant)
    • Doctrine ORM mapping (including relations and
    mapped superclasses)
    • Validation constraints from the Validator component
    • Full PHPDoc extracted from schema human-readable
    descriptions
    • (optional) PHP interfaces
    • (optional) ResolveTargetEntity Doctrine mappings
    • (optional) JSON-LD IRI annotations (useful for the API
    bundle)

    View Slide

  30. # app/config/services.yml
    services:
    person_resource:
    parent: "api.resource"
    arguments: [ "AppBundle\Entity\Person" ]
    tags: [ { name: "api.resource" } ]
    DEFINE RESOURCES TO EXPOSE

    View Slide

  31. CREATE EASILY REST APIs
    Validation Pagination
    Errors
    serialization
    Filtering Sorting
    Awesome
    features
    Browse
    a pretty, automatically generated
    documentation.
    Specify the API and test it
    thanks to a system especially suitable for
    Behavior-Driven Development.
    Everything is extensible
    thanks to a powerful event system and
    strong OOP.

    View Slide

  32. FULL SUPPORT OF JSON-LD, HYDRA AND SCHEMA.ORG
    An auto
    discoverable
    Linked Data
    API for free!

    View Slide

  33. USER DOCUMENTATION AND SANDBOX
    NelmioApiDoc
    automatically
    detects and
    documents
    exposed
    resources.

    View Slide

  34. Software Stack

    View Slide

  35. CUSTOM APPLICATION
    API Back-office Front-office
    PHP Symfony
    API Platform
    Behat + PHPunit
    Doctrine
    MySQL
    Varnish
    NelmioApiDoc
    FosHttpCache
    FOSUserBundle
    LexikJwtAuthenticationB
    undle
    NelmioCorsBundle
    AngularJS
    Restangular
    Less
    Bootstrap
    Grunt
    Protractor
    PHP Symfony
    Guzzle
    Twig
    Behat + PHPunit
    Less
    Bootstrap
    Varnish
    FosHttpCache
    No database!

    View Slide

  36. Hosting Stack

    View Slide

  37. HOSTING ENVIRONMENT
    Static files
    S3
    SPA
    S3
    Assets front
    S3
    medias
    Browser
    API
    Autoscaling group
    Varnish
    Varnish
    EC2 -1
    EC2 -n
    Front
    Autoscaling group
    Varnish
    Varnish
    EC2 -1
    EC2 -n
    Amazon RDS
    Assets
    Other services: Route 53 (DNS), SES (mail), …

    View Slide

  38. HOSTING ENVIRONMENT
    API
    Autoscaling group
    Varnish
    Varnish
    EC2 -1
    EC2 -n
    Static files
    S3
    SPA
    S3
    Assets front
    S3
    medias
    Smartphones
    Connected devices
    Back office
    Amazon RDS
    Assets
    Other services: Route 53 (DNS), SES (mail), …

    View Slide

  39. • Request / day on front: 5,2 M
    • Request / day on API front & connected devices: 15,6 M
    468 M request per month to serve in:
    • AVG response time on front: 29ms
    • AVG response time on API: 4ms
    FEW FIGURES

    View Slide

  40. API PLATFORM EVERYWHERE
    CMS – E-COMMERCE
    Real life use case examples
    TM & © Lucasfilm Ltd. All Rights Reserved

    View Slide

  41. Monolith application are not enough to handle the
    omnichannel experience.
    E-COMMERCE
    POS
    API
    Orchestration with information system
    CMS
    B2C
    CRM
    PIM
    OMS
    Back Office
    Corpo B2B

    View Slide

  42. And now Luke, scaffold your model and expose it with API
    Platform.
    1. Scaffold your core-application API with schema.org
    2. Create an OMS to manage Cart and Order
    3. Plug external data-sources and unify it with Hydra
    keeping a unify vocabulary to describe your data
    according to their referential
    Pros domain separation, loose coupling
    Cons only for e-commerce Jedi, you don’t need this for
    classic retail shop.
    E-COMMERCE

    View Slide

  43. EVERYTHING

    View Slide

  44. Questions ?
    TM & © Lucasfilm Ltd. All Rights Reserved

    View Slide

  45. Thank you, vote for us Luke!
    Fabien Gasser
    Solution Architect
    [email protected]
    Kévin Dunglas
    API Platform creator
    [email protected]
    https://joind.in/talk/view/16486
    http://symfony.com/showcase/170

    View Slide