$30 off During Our Annual Pro Sale. View Details »

SSO & SOA

SSO & SOA

Matthieu Moquet

June 23, 2014
Tweet

More Decks by Matthieu Moquet

Other Decks in Programming

Transcript

  1. SOA
    SSO
    & @MattKetmo
    PHPTour
    Lyon 2014

    View Slide

  2. Service
    Oriented
    Architecture
    “Developing a single application as a suite of small
    services, each running in its own process and
    communicating with lightweight mechanisms”
    #EventDriven #DistributedSystem

    View Slide

  3. View Slide

  4. /me
    Matthieu Moquet
    @MattKetmo
    Developer at
    …for almost 3 years

    View Slide

  5. Back to 2011…

    View Slide

  6. Covoiturage.fr
    1 monolithic PHP app
    o Core website
    o Web mobile
    o B2B platform
    o Backoffice tools
    o Web views (mobile apps)
    o External widgets

    View Slide

  7. Plain Old PHP
    No Dependency Injection
    No functional tests
    High coupling
    …hard to maintain

    View Slide

  8. View Slide

  9. # wc -l lib.trip.php
    3678
    …longest method: 1000+ lines

    View Slide

  10. View Slide

  11. get rid of the
    TECHNICAL DEBT

    View Slide

  12. started from scratch
    since 2012

    View Slide

  13. 2.0

    View Slide

  14. Let’s embrace
    best practices

    View Slide

  15. 1 bundle
    BlablacarAppBundle
    BlablacarBlogBundle
    BlablacarFaqBundle
    BlablacarTripBundle
    BlablacarUserBundle
    ...
    N bundles
    OR

    View Slide

  16. Services & Controllers
    §  Should controllers access the repository?
    §  How should I split my services?
    §  Is ContainerAware that bad?
    §  How to organize my Business Layer?

    View Slide

  17. Where should I put my
    Model & Doctrine entities?

    View Slide

  18. Forms
    Should I map my entities with my forms?

    View Slide

  19. Validation
    §  Should I map validation with entities?
    §  Should I use groups?

    View Slide

  20. Translations
    §  Catalogue message?
    §  Should I use key identifiers or phrases?
    §  How to name the keys?

    View Slide

  21. Route naming
    homepage
    blablacar_homepage
    blablacar_main_homepage

    View Slide

  22. Events
    §  Listen on Doctrine events?
    §  Create your own? ACTION / PRE_ACTION / POST_ACTION ?
    §  Subscribers VS. Listeners

    View Slide

  23. Design
    When you don’t have full-time designer,
    then Bootstrap(2) FTW

    View Slide

  24. With a few developers…
    At the beginning, you need to start quickly,
    (but you take time to write conventions).
    Legacy App New App

    View Slide

  25. Then you hire new people

    View Slide

  26. How many developers to be
    more efficient?

    View Slide

  27. More code means…

    View Slide

  28. Heavy Container

    View Slide

  29. $ app/console cache:warmup
    Warming up the cache for the dev env with debug true

    View Slide

  30. View Slide

  31. $ app/console assetic:dump
    Dumping all dev assets.
    Debug mode is on.
    23:00:32 [file+] /path/to/web/css/0e781b6.css
    23:00:35 [file+] /path/to/web/css/0e781b6_part_1_bootstrap_1.css
    23:00:35 [file+] /path/to/web/css/0e781b6_part_2_buttons_1.css
    23:00:35 [file+] /path/to/web/css/0e781b6_part_2_card_2.css
    23:00:35 [file+] /path/to/web/css/0e781b6_part_2_form_3.css
    23:00:35 [file+] /path/to/web/css/0e781b6_part_2_grid_4.css
    23:00:36 [file+] /path/to/web/css/0e781b6_part_2_list_5.css
    23:00:36 [file+] /path/to/web/css/0e781b6_part_2_navbar_6.css
    23:00:36 [file+] /path/to/web/css/0e781b6_part_2_panel_7.css
    23:00:37 [file+] /path/to/web/css/0e781b6_part_2_vcard_9.css
    23:00:37 [file+] /path/to/web/js/e66598b.js
    23:00:37 [file+] /path/to/web/js/e66598b_jquery_1.js
    23:00:37 [file+] /path/to/web/js/e66598b_bootstrap_2.js
    23:00:37 [file+] /path/to/web/js/e66598b_script_3.js

    View Slide

  32. View Slide

  33. $ phpunit -c app
    ........................... .............. (63/975)
    .......................... ............... (126/975)
    .............. ........................... (189/975)
    .......F................ ................. (252/975)
    .............F....

    View Slide

  34. Changed few lines of code?
    Run the full test suite!

    View Slide

  35. View Slide

  36. Optimize test execution time
    Invest in parallelization processes
    Split TestSuite in VMs (on AWS)
    Run 1h tests in 10min

    View Slide

  37. One big projet makes your team
    less reactive
    How often do you deploy your main project?

    View Slide

  38. https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/

    View Slide

  39. Changing foundations is
    expensive
    What if we want to change…

    View Slide

  40. What if we want to change
    The backend framework?
    We don’t plan to change it, and we couldn’t

    View Slide

  41. What if we want to change
    The testing framework?
    We won’t rewrite the whole test suite.
    But we can use several frameworks at the same time (eg. Behat)

    View Slide

  42. What if we want to change
    The frontend framework?
    Well, we are stuck with Bootstrap2.
    Updating to Bootstrap3 or rewriting our own will take time.

    View Slide

  43. What if we want to change
    The assets builder?
    Assetic took too long to compile all assets.
    We moved to asset management with Grunt.

    View Slide

  44. What if we want to change
    The data layer?
    Actually frontend servers make MySQL queries.
    But in the long term, it’s not a good practice (see coming slides).

    View Slide

  45. We’ll ALWAYS have technical debt.
    We must LIMIT it as much as possible.

    View Slide

  46. Think as small as possible
    µServices
    The secret to building large apps is never
    build large apps.
    Break your application into small pieces.
    Then, assemble those testable, bite-sized
    pieces into your big application
    — Justin Meyer

    View Slide

  47. 3 patterns
    to build a better software
    architecture…

    View Slide

  48. Request-Response

    View Slide

  49. Request-Response
    GET your resources
    synchronously
    Data Layer

    View Slide

  50. Request-Response
    $user = $this
    ->get('my.repository.user')
    ->find(1337);

    View Slide

  51. Request-Response
    $mysql->query('...')

    View Slide

  52. Request-Response
    Id   Firstname   Lastname   Pseudo   Email   Birthday  
    1337   Ma'hieu   Moquet   Ma'Ketmo   ma'[email protected]   1988-­‐12-­‐17  

    View Slide

  53. Request-Response
    However front-end servers should NOT access
    the database directly
    It should fetch normalized data from an
    internal service
    API for the win!

    View Slide

  54. Request-Response
    GET /users/123
    Do you speak REST ?
    Better versioning & normalization

    View Slide

  55. Request-Response
    …or you may speak:
    § SOAP
    § XML-RPC
    § Protobuffer
    § Thrift
    § etc.

    View Slide

  56. Request-Response
    Even RabbitMQ can be used for
    synchronous requests
    scrutinizer-ci/rabbitmq
    src/Scrutinizer/RabbitMQ/Rpc
    http://www.rabbitmq.com/tutorials/tutorial-six-python.html

    View Slide

  57. Request-Response

    View Slide

  58. Command(Handler)

    View Slide

  59. Command
    $cmd = FooCommand('bar');
    $handler = FooCommandHandler();
    $handler->execute($cmd);
    Do not expect a return value

    View Slide

  60. Command
    Perfect for
    asynchronous jobs
    §  Send e-mails / SMS / PUSH notifications
    §  Image processing
    §  Data indexation
    §  Saving complex data
    §  etc.

    View Slide

  61. Command
    publish consume
    RabbitMQ  
    DIRECT  rou

    View Slide

  62. Command
    publish consume
    Easy  scaling  

    View Slide

  63. Command
    Service is accessible via a
    queueing system only (not REST)
    §  RabbitMQ
    §  ActiveMQ
    §  Beanstalkd
    §  Gearman
    §  …
    More at http://queues.io

    View Slide

  64. Sending Newsletters
    publish
    100k messages
    Create an army of
    workers in AWS
    consume
    send mails
    message payload = emails
    address + content
    App
    Scheduler (Java + Quartz)
    Worker
    NL @10am
    segment X
    Get users of
    segment X
    (Scroll ElasticSearch)

    View Slide

  65. PubSub
    Obverser / Event Dispatcher

    View Slide

  66. PubSub
    $dispatcher = new EventDispatcher();!
    !
    // Listen using an object or a callback!
    $listener = new AcmeListener();!
    $dispatcher->addListener('foobar', array($listener, 'onFoobar'));!
    !
    $dispatcher->addListener('foobar', function (Event $event) {!
    // do something else with the events!
    });!
    !
    // Dispatch event!
    $dispatcher->dispatch('foobar', $event);!
    Example with the Symfony EventDispatcher

    View Slide

  67. PubSub
    Notify your infrastructure of
    every business events
    Publish events without knowing who is listening
    Create services without any core changes

    View Slide

  68. PubSub
    pub
    RabbitMQ  
    TOPIC  rousub
    user.register
    user.edit_bio
    user.register
    user.edit_bio
    user.edit_bio

    View Slide

  69. PubSub
    Use Case
    BI

    View Slide

  70. user.register
    user.edit_bio
    user.left_rating
    user.post_trip
    ...
    Data Warehouse
    Log every business events in Hadoop

    View Slide

  71. user.register
    user.edit_bio
    user.left_rating
    user.post_trip
    ...
    user.register
    user.post_trip
    ...
    Meanwhile…
    RealTime Dashboard
    Log every business events in ElasticSearch

    View Slide

  72. View Slide

  73. ReqRes — Command — PubSub
    Message
    Broker
    RabbitMQ removes hard link between services

    View Slide

  74. Now we have the keys to start a distributed architecture
    let’s start decoupling our application…
    Front-end desktop + mobile
    Hard to split in several projects (need to delegate jobs)
    API
    For mobile apps & partners
    Backoffice
    Set of administration tools
    Workers
    Already decoupled from the core app

    View Slide

  75. Backoffice
    A set of independent tools
    Easy to split

    View Slide

  76. Backoffice — CRUD
    Some tools are just data manipulation:
    – User Management
    – Blog
    – FAQ
    GET /users
    PUT /users/123
    Data Layer

    View Slide

  77. Backoffice — Moderation
    Manage user « data » which need to be moderated
    user.upload_avatar
    user.edit_bio
    user.left_rating
    UI to check
    data manually
    Auto detect
    spam & non
    compliant data
    Machine Learning data.received
    data.treated
    send mails

    View Slide

  78. Backoffice — URL Shortener
    Some tools are completely independent
    https://www.blablacar.com/register SHORTEN
    No data shared with core business

    View Slide

  79. Authentication
    Splitting our backoffice in many apps
    should not be a pain for the UX

    View Slide

  80. Let’s build a
    Single Sign-On service

    View Slide

  81. Single
    Sign
    On
    “SSO is a method allowing a user to access
    multiple applications, making only a single
    authentication.”

    View Slide

  82. STARTING POINT
    LDAP

    View Slide

  83. Single Sign ONCE
    example.org/app
    app.example.org
    Simple Case: Shared Host / Shared Domain

    View Slide

  84. View Slide

  85. « OAuth is an open standard for authorization
    […] to access server resources on behalf of a
    resource owner »
    — Wikipedia

    View Slide

  86. OAuth2 Grant Types
    Authorization Code
    Connect-like workflow
    Implicit Grant (Direct Token)
    Usefull for JS app
    Password flow
    Trusted app (client credentials)
    Client Credentials
    Basic (use of client id + secret)

    View Slide

  87. OAuth2

    View Slide

  88. Do It Yourself
    with

    View Slide

  89. From LDAP to OAuth
    Given I am on the login page
    Then I should login with my LDAP credentials
    App
    LDAP
    Login

    View Slide

  90. There is a bundle for that

    View Slide

  91. FR3DLdapBundle

    View Slide

  92. FR3DLdapBundle/config.yml
    # LDAP Configuration!
    fr3d_ldap:!
    driver:!
    host: "%ldap_host%"!
    username: "%ldap_username%"!
    password: "%ldap_password%"!
    baseDn: "%ldap_username%"!
    user:!
    baseDn: "%ldap_user_dn%"!
    filter: "%ldap_user_filter%"!
    attributes:!
    - { ldap_attr: "uid", user_method: "setUsername" }!
    - { ldap_attr: "cn", user_method: "setName" }!
    - { ldap_attr: "mail", user_method: "setEmail" }!
    service:!
    user_manager: "acme.user_manager"!

    View Slide

  93. From LDAP to OAuth
    Given I am on "/me" with "user" access token
    Then I should get "user" resources in JSON
    App
    LDAP OAuth2
    API

    View Slide

  94. There is a bundle for that

    View Slide

  95. FOSOAuthServerBundle

    View Slide

  96. FOSOAuthServerBundle/
    security.yml
    firewalls:!
    api:!
    pattern: ^/api!
    fos_oauth: true!
    stateless: true  

    View Slide

  97. From LDAP to OAuth
    App
    LDAP OAuth2
    API
    Login

    View Slide

  98. Enforce security
    PO said,
    this SSO entry-point should be a
    top-notch secure app

    View Slide

  99. There ARE bundles for THAT

    View Slide

  100. SchebTwoFactorBundle
    SpomkyIpFilterBundle
    CCDNUserSecurityBundle
    NelmioSecurityBundle

    View Slide

  101. App
    LDAP OAuth2
    2FA
    FW

    View Slide

  102. What about the client?

    View Slide

  103. Of course,
    there is a bundle for that

    View Slide

  104. HWIOAuthBundle

    View Slide

  105. View Slide

  106. # HWI OAuth Configuration!
    hwi_oauth:!
    firewall_name: "main"!
    resource_owners:!
    acme_sso:!
    type: "oauth2"!
    client_id: "%client_id%"!
    client_secret: "%client_secret%"!
    access_token_url: "%base_url%/oauth/v2/token"!
    authorization_url: "%base_url%/oauth/v2/auth"!
    infos_url: "%base_url%/api/me”!
    paths:!
    identifier: "id"!
    nickname: "username"!
    realname: "name"!
    email: "email"!

    View Slide

  107. UX — We don’t really need this login form

    View Slide

  108. UX — We don’t really need a one-button page

    View Slide

  109. 302 Found
    UX — Get transparent login process

    View Slide

  110. OAuth is all about
    authorization,
    not authentication
    Watch out for weaknesses & attacks

    View Slide

  111. GET /api/me
    {
    "id": 1337,
    "firstname": "Matthieu",
    "lastname": "Moquet",
    "nickname": "MattKetmo"
    }
    (the cheat)

    View Slide

  112. CSRF
    http://example.org?redirect_uri=xxxx&state=yyyy
    session[state] === params[state].
    http://tools.ietf.org/html/draft-ietf-oauth-v2-27#section-10.12

    View Slide

  113. Cover Redirect
    (is a joke)
    http://homakov.blogspot.fr/2014/05/covert-redirect-faq.html

    View Slide

  114. id_token
    Multiple Response Type Encoding Practices.
    Provides an assertion of the identity of the
    Resource Owner.
    http://openid.bitbucket.org/oauth-v2-multiple-response-types-1_0.html

    View Slide

  115. However if OAuth2 is not designed to be
    an SSO protocol, what should I use?

    View Slide

  116. OAuth1.a

    View Slide

  117. It seems Symfony have a
    bundle for everything…

    View Slide

  118. BazingaOAuthServerBundle

    View Slide

  119. View Slide

  120. SAML
    (Security Assertion Markup Language)
    SAML is an XML-based protocol that uses
    security tokens containing assertions to pass
    information about a principal between an
    identity provider, and a consumer.
    There are bundles/lib for that (but not maintained, see impl.)
    §  pdias/FOSSamlBundle
    §  aerialship/SamlSPBundle
    §  chtitux/sfSAMLPlugin (symfony1)

    View Slide

  121. CAS
    (Central Authentication Service)
    Example of client implementation
    BeSimple/BeSimpleSsoAuthBundle

    View Slide

  122. h"ps://github.com/BeSimple/BeSimpleSsoAuthBundle/blob/master/Resources/doc/protocols.md  

    View Slide

  123. JWT
    (JSON Web Token)
    Payload signed server-side with a
    JSON Web Signature (JWS).
    auth.  
    signed token
    request

    View Slide

  124. JWT
    (JSON Web Token)
    Sign payload using a secret key
    auth.  
    signed token
    request

    View Slide

  125. firebase/php-jwt
    curl –H "Authorization: Bearer eyJ0eXAiOiJKV..." example.org
    $key = "s3cr3t_key";
    $token = array(
    "sub" => "mattketmo",
    "aud" => "http://example.com",
    "exp" => 1356999524,
    );
    $jwt = JWT::encode($token, $key);  // eyJ0eXAiOiJKV...

    View Slide

  126. namshi/jose
    OpenSSL + cookies
    // Auth (SSO)
    $privateKey = openssl_pkey_get_private("file://private.key");
    $jws = new JWS('RS256');
    $jws->setPayload([$userId]);
    $jws->sign($privateKey)
    setcookie('identity', $jws->getTokenString());
    // Client App
    $jws = JWS::load($_COOKIE['identity']);
    $publicKey = openssl_pkey_get_public("/public.key");
    if ($jws->isValid($publicKey)) {
    $payload = $jws->getPayload();
    $userId = $payload['id'];
    }

    View Slide

  127. OAuth2 + JWT
    http://tools.ietf.org/html/draft-ietf-oauth-jwt-bearer-08

    View Slide

  128. OpenID Connect
    Feb. 2014 — openid.net/connect/

    View Slide

  129. View Slide

  130. Core app is now lightweight
    o Tiny container (less services/listeners)
    o Test suite is smaller
    o Forget about technical debt

    View Slide

  131. Choose the right tools
    for the task
    Backend / Frontend / Datastore / …

    View Slide

  132. Make experiments

    View Slide

  133. Work faster
    No more bottleneck & dependencies
    with other dev teams

    View Slide

  134. BE EFFICIENT
    AVOID REPETITIVE PROCESSES

    View Slide

  135. Install dev environment
    in less than 1h

    View Slide

  136. Bootstrap everything
    composer create-project blablacar/backoffice-app
    git clone [email protected]/worker-skeleton.git
    via git
    or via composer

    View Slide

  137. Enforce reusable components
    {
    "require": {
    "blablacar/monolog": "~1.0",
    "blablacar/scheduler-client": "~1.1",
    "blablacar/redis-client": "~1.2",
    "blablacar/rabbit-mq-admin-toolkit": "dev-master"
    }
    }

    View Slide

  138. Be ready for production
    Don’t loose time in configuration setup
    Parameters
    for $ENV
    Config template
    for $PROJECT
    Config file for
    $PROJECT/$ENV
    Centralized build tool to generate a project configuration
    file for any environment (local / dev / staging / prod)

    View Slide

  139. Never miss a log

    View Slide

  140. Don’t be lost in translations
    Open-sourced a tool to manage your project
    translations easily
    More information tomorrow, 9:45

    View Slide

  141. Thank you
    Slides available at
    moquet.net/talks/phptour-2014-sso-soa
    Leave feedbacks @MattKetmo

    View Slide