Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Decoupling your application

Decoupling your application

Symfony Live Paris 2014

Find this presentation at:
http://moquet.net/talks/symfony-live-2014/

Original description:
http://paris2014.live.symfony.com/speakers#session-1131

Matthieu Moquet

April 08, 2014
Tweet

More Decks by Matthieu Moquet

Other Decks in Programming

Transcript

  1. Decoupling your
    application

    …with SSO, OAuth & more

    View Slide

  2. View Slide

  3. View Slide

  4. /me
    Matthieu Moquet
    @MattKetmo
    Developer at
    …for almost 3 years

    View Slide

  5. Back to 2011…

    View Slide

  6. Covoiturage.fr
    1 monolithic PHP app
    o Core website
    o Web mobile
    o B2B platform
    o Backoffice tools
    o Web views (mobile apps)
    o External widgets

    View Slide

  7. Plain Old PHP
    No Dependency Injection
    No functional tests
    High coupling
    …hard to maintain

    View Slide

  8. View Slide

  9. # wc -l lib.trip.php
    3678
    …longest method: 1000+ lines

    View Slide

  10. View Slide

  11. Get rid of the
    Technical Debt

    View Slide

  12. Since 2012
    Start from scratch

    View Slide

  13. 2.0

    View Slide

  14. NEW ORGANISATION
    NEW FRAMEWORK

    View Slide

  15. Let’s embrace
    best practices

    View Slide

  16. 1 bundle
    BlablacarWebBundle
    BlablacarBlogBundle
    BlablacarFaqBundle
    BlablacarTripBundle
    BlablacarUserBundle
    ...
    N bundles
    OR

    View Slide

  17. Services & Controllers
    §  Should controllers access the repository?
    §  How should I split my services?
    §  Is ContainerAware that bad?
    §  How to organize my Business Layer?

    View Slide

  18. Where should I put my
    Model & Doctrine entities?

    View Slide

  19. Forms
    Should I map my entities with my forms?

    View Slide

  20. Validation
    §  Should I map validation with entities?
    §  Should I use groups?

    View Slide

  21. Translations
    §  Catalogue message?
    §  Should I use key identifiers or phrases?
    §  How to name the keys?

    View Slide

  22. Route naming
    homepage
    blablacar_homepage
    blablacar_main_homepage

    View Slide

  23. Events
    §  Listen on Doctrine events?
    §  Create your own? ACTION / PRE_ACTION / POST_ACTION ?
    §  Subscribers VS. Listeners

    View Slide

  24. Design
    When you don’t have full-time designer,
    then Bootstrap(2) FTW

    View Slide

  25. With a few developers…
    At the beginning, you need to start quickly,
    (but you take time to write conventions).
    Legacy App New App

    View Slide

  26. Then you hire new people

    View Slide

  27. How many developers to be
    more efficient?

    View Slide

  28. More code means…

    View Slide

  29. Heavy Container

    View Slide

  30. $ app/console cache:warmup
    Warming up the cache for the dev env with debug true

    View Slide

  31. View Slide

  32. $ app/console assetic:dump
    Dumping all dev assets.
    Debug mode is on.
    23:00:32 [file+] /path/to/web/css/0e781b6.css
    23:00:35 [file+] /path/to/web/css/0e781b6_part_1_bootstrap_1.css
    23:00:35 [file+] /path/to/web/css/0e781b6_part_2_buttons_1.css
    23:00:35 [file+] /path/to/web/css/0e781b6_part_2_card_2.css
    23:00:35 [file+] /path/to/web/css/0e781b6_part_2_form_3.css
    23:00:35 [file+] /path/to/web/css/0e781b6_part_2_grid_4.css
    23:00:36 [file+] /path/to/web/css/0e781b6_part_2_list_5.css
    23:00:36 [file+] /path/to/web/css/0e781b6_part_2_navbar_6.css
    23:00:36 [file+] /path/to/web/css/0e781b6_part_2_panel_7.css
    23:00:37 [file+] /path/to/web/css/0e781b6_part_2_vcard_9.css
    23:00:37 [file+] /path/to/web/js/e66598b.js
    23:00:37 [file+] /path/to/web/js/e66598b_jquery_1.js
    23:00:37 [file+] /path/to/web/js/e66598b_bootstrap_2.js
    23:00:37 [file+] /path/to/web/js/e66598b_script_3.js

    View Slide

  33. View Slide

  34. $ phpunit -c app
    ........................... .............. (63/975)
    .......................... ............... (126/975)
    .............. ........................... (189/975)
    .......F................ ................. (252/975)
    .............F....

    View Slide

  35. Changed few lines of code?
    Run the full test suite!

    View Slide

  36. View Slide

  37. Optimize test execution time
    Invest in parallelization processes
    Split TestSuite in VMs (on AWS)
    Run 1h tests in 10min

    View Slide

  38. One big projet makes your team
    less reactive
    How often do you deploy your main project?

    View Slide

  39. https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/

    View Slide

  40. Changing foundations is
    expensive
    What if we want to change…

    View Slide

  41. What if we want to change
    The backend framework?
    We don’t plan to change it, and we couldn’t

    View Slide

  42. What if we want to change
    The testing framework?
    We won’t rewrite the whole test suite.
    But we can use several frameworks at the same time (eg. Behat)

    View Slide

  43. What if we want to change
    The frontend framework?
    Well, we are stuck with Bootstrap2.
    Updating to Bootstrap3 or rewriting our own will take time.

    View Slide

  44. What if we want to change
    The assets builder?
    Assetic took too long to compile all assets.
    We moved to asset management with Grunt.

    View Slide

  45. What if we want to change
    The data layer?
    Actually frontend servers make MySQL queries.
    But in the long term, it’s not a good practice (see coming slides).

    View Slide

  46. We’ll ALWAYS have technical debt.
    We must LIMIT it as much as possible.

    View Slide

  47. Think as small as possible
    Microservices
    The secret to building large apps is never
    build large apps.
    Break your application into small pieces.
    Then, assemble those testable, bite-sized
    pieces into your big application
    — Justin Meyer

    View Slide

  48. 3 patterns
    to build a better software
    architecture…

    View Slide

  49. Request-Response

    View Slide

  50. Request-Response
    GET your resources
    synchronously
    Data Layer

    View Slide

  51. Request-Response
    $user = $this
    ->get('my.repository.user')
    ->find(1337);

    View Slide

  52. Request-Response
    mysql_query('...')

    View Slide

  53. Request-Response
    Id   Firstname   Lastname   Pseudo   Email   Birthday  
    1337   Ma'hieu   Moquet   Ma'Ketmo   ma'[email protected]   1988-­‐12-­‐17  

    View Slide

  54. Request-Response
    However front-end servers should NOT access
    the database directly
    It should fetch normalized data from an
    internal service
    API for the win!

    View Slide

  55. Request-Response
    GET /users/123
    Do you speak REST ?
    Better versioning & normalization

    View Slide

  56. Request-Response
    …or you may speak:
    § SOAP
    § XML-RPC
    § Protobuffer
    § Thrift
    § etc.

    View Slide

  57. Request-Response
    Even RabbitMQ can be used for
    synchronous requests
    scrutinizer-ci/rabbitmq
    src/Scrutinizer/RabbitMQ/Rpc
    http://www.rabbitmq.com/tutorials/tutorial-six-python.html

    View Slide

  58. Request-Response

    View Slide

  59. Command(Handler)

    View Slide

  60. Command
    $cmd = FooCommand('bar');
    $handler = FooCommandHandler();
    $handler->execute($cmd);
    Do not expect a return value

    View Slide

  61. Command
    Perfect for
    asynchronous jobs
    §  Send e-mails / SMS / PUSH notifications
    §  Image processing
    §  Data indexation
    §  Saving complex data
    §  etc.

    View Slide

  62. Command
    publish consume
    RabbitMQ  
    DIRECT  rou

    View Slide

  63. Command
    publish consume
    Easy  scaling  

    View Slide

  64. Command
    Service is accessible via a
    queueing system only (not REST)
    §  RabbitMQ
    §  ActiveMQ
    §  Beanstalkd
    §  Gearman
    §  …
    More at http://queues.io

    View Slide

  65. Sending Newsletters
    publish
    100k messages
    Create an army of
    workers in AWS
    consume
    send mails
    message payload = emails
    address + content
    App
    Scheduler (Java + Quartz)
    Worker
    NL @10am
    segment X
    Get users of
    segment X
    (Scroll ElasticSearch)

    View Slide

  66. PubSub
    Obverser / Event Dispatcher

    View Slide

  67. PubSub
    $dispatcher = new EventDispatcher();!
    !
    // Listen using an object or a callback!
    $listener = new AcmeListener();!
    $dispatcher->addListener('foobar', array($listener, 'onFoobar'));!
    !
    $dispatcher->addListener('foobar', function (Event $event) {!
    // do something else with the events!
    });!
    !
    // Dispatch event!
    $dispatcher->dispatch('foobar', $event);!
    Example with the Symfony EventDispatcher

    View Slide

  68. PubSub
    Notify your infrastructure of
    every business events
    Publish events without knowing who is listening
    Create services without any core changes

    View Slide

  69. PubSub
    pub
    RabbitMQ  
    TOPIC  rousub
    user.register
    user.edit_bio
    user.register
    user.edit_bio
    user.edit_bio

    View Slide

  70. PubSub
    Use Case
    BI

    View Slide

  71. user.register
    user.edit_bio
    user.left_rating
    user.post_trip
    ...
    Data WareHouse
    Log every business events in Hadoop

    View Slide

  72. user.register
    user.edit_bio
    user.left_rating
    user.post_trip
    ...
    user.register
    user.post_trip
    ...
    Meanwhile…
    RealTime Dashboard
    Log every business events in ElasticSearch

    View Slide

  73. View Slide

  74. ReqRes — Command — PubSub
    Message
    Broker
    RabbitMQ removes hard link between services

    View Slide

  75. Now we have the keys to start a distributed architecture
    let’s start decoupling our application…
    Front-end desktop + mobile
    Hard to split in several projects (need to delegate jobs)
    API
    For mobile apps & partners
    Backoffice
    Set of administration tools
    Workers
    Already decoupled from the core app

    View Slide

  76. Backoffice
    A set of independent tools
    Easy to split

    View Slide

  77. Backoffice — CRUD
    Some tools are just data manipulation:
    – User Management
    – Blog
    – FAQ
    GET /users
    PUT /users/123
    Data Layer

    View Slide

  78. Backoffice — Moderation
    Manage user « data » which need to be moderated
    user.upload_avatar
    user.edit_bio
    user.left_rating
    UI to check
    data manually
    Auto detect
    spam & non
    compliant data
    Machine Learning data.received
    data.treated
    send mails

    View Slide

  79. Backoffice — URL Shortener
    Some tools are completely independent
    https://www.blablacar.com/register SHORTEN
    No data shared with core business

    View Slide

  80. Authentication
    Splitting our backoffice in many apps
    should not be a pain for the UX

    View Slide

  81. Let’s build a
    Single Sign-On service

    View Slide

  82. View Slide

  83. « OAuth is an open standard for authorization
    […] to access server resources on behalf of a
    resource owner »
    — Wikipedia

    View Slide

  84. OAuth2 Grant Types
    Authorization Code
    Connect-like workflow
    Implicit Grant (Direct Token)
    Usefull for JS app
    Password flow
    Trusted app (client credentials)
    Client Credentials
    Basic (use of client id + secret)

    View Slide

  85. OAuth2
    Client App
    Use token to call
    API
    Server App

    View Slide

  86. Do It Yourself

    View Slide

  87. From LDAP to OAuth
    Given I am on the login page
    Then I should login with my LDAP credentials
    App
    LDAP
    Login

    View Slide

  88. THERE IS A BUNDLE
    FOR THAT!

    View Slide

  89. FR3DLdapBundle

    View Slide

  90. FR3DLdapBundle/config.yml
    # LDAP Configuration!
    fr3d_ldap:!
    driver:!
    host: "%ldap_host%"!
    username: "%ldap_username%"!
    password: "%ldap_password%"!
    baseDn: "%ldap_username%"!
    user:!
    baseDn: "%ldap_user_dn%"!
    filter: "%ldap_user_filter%"!
    attributes:!
    - { ldap_attr: "uid", user_method: "setUsername" }!
    - { ldap_attr: "cn", user_method: "setName" }!
    - { ldap_attr: "mail", user_method: "setEmail" }!
    service:!
    user_manager: "acme.user_manager"!

    View Slide

  91. From LDAP to OAuth
    Given I am on "/me" with "user" access token
    Then I should get "user" resources in JSON
    App
    LDAP OAuth2
    API

    View Slide

  92. THERE IS A BUNDLE
    FOR THAT!

    View Slide

  93. FOSOAuthServerBundle

    View Slide

  94. FOSOAuthServerBundle/security.yml
    firewalls:!
    api:!
    pattern: ^/api!
    fos_oauth: true!
    stateless: true  

    View Slide

  95. From LDAP to OAuth
    App
    LDAP OAuth2
    API
    Login

    View Slide

  96. Enforce security
    PO said,
    this SSO entry-point should be a
    top-notch secure app

    View Slide

  97. THERE ARE BUNDLES
    FOR THAT!

    View Slide

  98. SchebTwoFactorBundle
    SpomkyIpFilterBundle
    CCDNUserSecurityBundle
    NelmioSecurityBundle

    View Slide

  99. App
    LDAP OAuth2
    2FA
    FW

    View Slide

  100. What about the client?

    View Slide

  101. THERE IS A BUNDLE
    FOR THAT!

    View Slide

  102. HWIOAuthBundle

    View Slide

  103. View Slide

  104. # HWI OAuth Configuration!
    hwi_oauth:!
    firewall_name: "main"!
    resource_owners:!
    acme_sso:!
    type: "oauth2"!
    client_id: "%client_id%"!
    client_secret: "%client_secret%"!
    access_token_url: "%base_url%/oauth/v2/token"!
    authorization_url: "%base_url%/oauth/v2/auth"!
    infos_url: "%base_url%/api/me”!
    paths:!
    identifier: "id"!
    nickname: "username"!
    realname: "name"!
    email: "email"!

    View Slide

  105. UX — We don’t really need this login form

    View Slide

  106. UX — We don’t really need a one-button page

    View Slide

  107. 302 Found
    UX — Get transparent login process

    View Slide

  108. OAuth is all about
    authorization,
    not authentication
    Watch out for weaknesses & attacks

    View Slide

  109. GET /api/me
    {!
    "id": 1337,!
    "firstname": "Matthieu",!
    "lastname": "Moquet",!
    "nickname": "MattKetmo"!
    }!
    (the cheat)

    View Slide

  110. CSRF
    http://example.org?redirect_uri=xxxx&state=yyyy
    session[state] === params[state].
    http://tools.ietf.org/html/draft-ietf-oauth-v2-27#section-10.12

    View Slide

  111. id_token
    Multiple Response Type Encoding Practices.
    Provides an assertion of the identity of the
    Resource Owner.
    http://openid.bitbucket.org/oauth-v2-multiple-response-types-1_0.html

    View Slide

  112. However if OAuth is not designed to
    be an SSO protocol, what should I use?

    View Slide

  113. SAML
    (Security Assertion Markup Language)
    SAML is an XML-based protocol that uses
    security tokens containing assertions to pass
    information about a principal between an
    identity provider, and a consumer.
    There are bundles/lib for that (but not maintained, see impl.)
    §  pdias/FOSSamlBundle
    §  aerialship/SamlSPBundle
    §  chtitux/sfSAMLPlugin (symfony1)

    View Slide

  114. JWT
    (JSON Web Token)
    Payload signed server-side with a
    JSON Web Signature (JWS).
    auth.  
    signed token
    request

    View Slide

  115. JWT
    (JSON Web Token)
    Sign payload using a secret key
    auth.  
    signed token
    request

    View Slide

  116. firebase/php-jwt
    curl –H "Authorization: Bearer eyJ0eXAiOiJKV..." example.org
    $key = "s3cr3t_key";
    $token = array(
    "sub" => "mattketmo",
    "aud" => "http://example.com",
    "exp" => 1356999524,
    );
    $jwt = JWT::encode($token, $key);  // eyJ0eXAiOiJKV...

    View Slide

  117. namshi/jose
    OpenSSL + cookies
    // Auth (SSO)
    $privateKey = openssl_pkey_get_private("file://private.key");
    $jws = new JWS('RS256');
    $jws->setPayload([$userId]);
    $jws->sign($privateKey)
    setcookie('identity', $jws->getTokenString());
    // Client App
    $jws = JWS::load($_COOKIE['identity']);
    $publicKey = openssl_pkey_get_public("/public.key");
    if ($jws->isValid($publicKey)) {
    $payload = $jws->getPayload();
    $userId = $payload['id'];
    }

    View Slide

  118. OAuth2 + JWT
    http://tools.ietf.org/html/draft-ietf-oauth-jwt-bearer-08

    View Slide

  119. Of course, there are Symfony bundles:
    –  formapro/FpOpenIdBundle
    –  KainHaart/OpenIDAuthBundle
    –  etc.

    View Slide

  120. Hawk
    Shared secret between client & server
    by Eran Hammer
    Read more at
    http://alexbilbie.com/2012/11/hawk-a-new-http-authentication-scheme/

    View Slide

  121. CAS
    (Central Authentication Service)
    Example of client implementation
    BeSimple/BeSimpleSsoAuthBundle

    View Slide

  122. h"ps://github.com/BeSimple/BeSimpleSsoAuthBundle/blob/master/Resources/doc/protocols.md  

    View Slide

  123. View Slide

  124. Core app is now lightweight
    Tiny container (less services/listeners)
    Test suite is smaller

    View Slide

  125. Choose the right tools
    for the task
    Backend / Frontend / Datastore / …

    View Slide

  126. Make experiments

    View Slide

  127. Work faster
    No more bottleneck & dependencies
    with other dev teams

    View Slide

  128. BE EFFICIENT
    AVOID REPETITIVE PROCESSES

    View Slide

  129. Install dev environment
    in less than 1h

    View Slide

  130. Bootstrap everything
    composer create-project blablacar/backoffice-app
    git clone [email protected]/worker-skeleton.git
    via git
    or via composer

    View Slide

  131. Enforce reusable components
    {
    "require": {
    "blablacar/monolog": "~1.0",
    "blablacar/scheduler-client": "~1.1",
    "blablacar/redis-client": "~1.2",
    "blablacar/rabbit-mq-admin-toolkit": "dev-master"
    }
    }

    View Slide

  132. Be ready for production
    Don’t loose time in configuration setup
    Parameters
    for $ENV
    Config template
    for $PROJECT
    Config file for
    $PROJECT/$ENV
    Centralized build tool to generate a project configuration
    file for any environment (local / dev / staging / prod)

    View Slide

  133. Never miss a log

    View Slide

  134. Don’t be lost in translations
    Open-sourced a tool to manage your projects’
    translations easily

    View Slide

  135. More information at
    openl10n.io

    View Slide

  136. Thank you
    Slides available at
    moquet.net/talks/symfony-live-2014
    Leave feedbacks @MattKetmo
    we’re hiring!

    View Slide