Upgrade to Pro — share decks privately, control downloads, hide ads and more …



Matthieu Moquet

June 23, 2014

More Decks by Matthieu Moquet

Other Decks in Programming


  1. SOA SSO & @MattKetmo PHPTour Lyon 2014

  2. Service Oriented Architecture “Developing a single application as a suite

    of small services, each running in its own process and communicating with lightweight mechanisms” #EventDriven #DistributedSystem
  3. None
  4. /me Matthieu Moquet @MattKetmo Developer at …for almost 3 years

  5. Back to 2011…

  6. Covoiturage.fr 1 monolithic PHP app o Core website o Web mobile o B2B

    platform o Backoffice tools o Web views (mobile apps) o External widgets
  7. Plain Old PHP No Dependency Injection No functional tests High

    coupling …hard to maintain
  8. None
  9. # wc -l lib.trip.php 3678 …longest method: 1000+ lines

  10. None
  11. get rid of the TECHNICAL DEBT

  12. started from scratch since 2012

  13. 2.0

  14. Let’s embrace best practices

  15. 1 bundle BlablacarAppBundle BlablacarBlogBundle BlablacarFaqBundle BlablacarTripBundle BlablacarUserBundle ... N bundles

  16. Services & Controllers §  Should controllers access the repository? § 

    How should I split my services? §  Is ContainerAware that bad? §  How to organize my Business Layer?
  17. Where should I put my Model & Doctrine entities?

  18. Forms Should I map my entities with my forms?

  19. Validation §  Should I map validation with entities? §  Should

    I use groups?
  20. Translations §  Catalogue message? §  Should I use key identifiers

    or phrases? §  How to name the keys?
  21. Route naming homepage blablacar_homepage blablacar_main_homepage

  22. Events §  Listen on Doctrine events? §  Create your own?

    ACTION / PRE_ACTION / POST_ACTION ? §  Subscribers VS. Listeners
  23. Design When you don’t have full-time designer, then Bootstrap(2) FTW

  24. With a few developers… At the beginning, you need to

    start quickly, (but you take time to write conventions). Legacy App New App
  25. Then you hire new people

  26. How many developers to be more efficient?

  27. More code means…

  28. Heavy Container

  29. $ app/console cache:warmup Warming up the cache for the dev

    env with debug true
  30. None
  31. $ app/console assetic:dump Dumping all dev assets. Debug mode is

    on. 23:00:32 [file+] /path/to/web/css/0e781b6.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_1_bootstrap_1.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_2_buttons_1.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_2_card_2.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_2_form_3.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_2_grid_4.css 23:00:36 [file+] /path/to/web/css/0e781b6_part_2_list_5.css 23:00:36 [file+] /path/to/web/css/0e781b6_part_2_navbar_6.css 23:00:36 [file+] /path/to/web/css/0e781b6_part_2_panel_7.css 23:00:37 [file+] /path/to/web/css/0e781b6_part_2_vcard_9.css 23:00:37 [file+] /path/to/web/js/e66598b.js 23:00:37 [file+] /path/to/web/js/e66598b_jquery_1.js 23:00:37 [file+] /path/to/web/js/e66598b_bootstrap_2.js 23:00:37 [file+] /path/to/web/js/e66598b_script_3.js
  32. None
  33. $ phpunit -c app ........................... .............. (63/975) .......................... ............... (126/975)

    .............. ........................... (189/975) .......F................ ................. (252/975) .............F....
  34. Changed few lines of code? Run the full test suite!

  35. None
  36. Optimize test execution time Invest in parallelization processes Split TestSuite

    in VMs (on AWS) Run 1h tests in 10min
  37. One big projet makes your team less reactive How often

    do you deploy your main project?
  38. https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/

  39. Changing foundations is expensive What if we want to change…

  40. What if we want to change The backend framework? We

    don’t plan to change it, and we couldn’t
  41. What if we want to change The testing framework? We

    won’t rewrite the whole test suite. But we can use several frameworks at the same time (eg. Behat)
  42. What if we want to change The frontend framework? Well,

    we are stuck with Bootstrap2. Updating to Bootstrap3 or rewriting our own will take time.
  43. What if we want to change The assets builder? Assetic

    took too long to compile all assets. We moved to asset management with Grunt.
  44. What if we want to change The data layer? Actually

    frontend servers make MySQL queries. But in the long term, it’s not a good practice (see coming slides).
  45. We’ll ALWAYS have technical debt. We must LIMIT it as

    much as possible.
  46. Think as small as possible µServices The secret to building

    large apps is never build large apps. Break your application into small pieces. Then, assemble those testable, bite-sized pieces into your big application — Justin Meyer
  47. 3 patterns to build a better software architecture…

  48. Request-Response

  49. Request-Response GET your resources synchronously Data Layer

  50. Request-Response $user = $this ->get('my.repository.user') ->find(1337);

  51. Request-Response $mysql->query('...')

  52. Request-Response Id   Firstname   Lastname   Pseudo   Email

      Birthday   1337   Ma'hieu   Moquet   Ma'Ketmo   ma'hieu@moquet.net   1988-­‐12-­‐17  
  53. Request-Response However front-end servers should NOT access the database directly

    It should fetch normalized data from an internal service API for the win!
  54. Request-Response GET /users/123 Do you speak REST ? Better versioning

    & normalization
  55. Request-Response …or you may speak: § SOAP § XML-RPC § Protobuffer § Thrift § etc.

  56. Request-Response Even RabbitMQ can be used for synchronous requests scrutinizer-ci/rabbitmq

    src/Scrutinizer/RabbitMQ/Rpc http://www.rabbitmq.com/tutorials/tutorial-six-python.html
  57. Request-Response

  58. Command(Handler)

  59. Command $cmd = FooCommand('bar'); $handler = FooCommandHandler(); $handler->execute($cmd); Do not

    expect a return value
  60. Command Perfect for asynchronous jobs §  Send e-mails / SMS

    / PUSH notifications §  Image processing §  Data indexation §  Saving complex data §  etc.
  61. Command publish consume RabbitMQ   DIRECT  rou<ng  

  62. Command publish consume Easy  scaling  

  63. Command Service is accessible via a queueing system only (not

    REST) §  RabbitMQ §  ActiveMQ §  Beanstalkd §  Gearman §  … More at http://queues.io
  64. Sending Newsletters publish 100k messages Create an army of workers

    in AWS consume send mails message payload = emails address + content App Scheduler (Java + Quartz) Worker NL @10am segment X Get users of segment X (Scroll ElasticSearch)
  65. PubSub Obverser / Event Dispatcher

  66. PubSub $dispatcher = new EventDispatcher();! ! // Listen using an

    object or a callback! $listener = new AcmeListener();! $dispatcher->addListener('foobar', array($listener, 'onFoobar'));! ! $dispatcher->addListener('foobar', function (Event $event) {! // do something else with the events! });! ! // Dispatch event! $dispatcher->dispatch('foobar', $event);! Example with the Symfony EventDispatcher
  67. PubSub Notify your infrastructure of every business events Publish events

    without knowing who is listening Create services without any core changes
  68. PubSub pub RabbitMQ   TOPIC  rou<ng   sub user.register user.edit_bio

    user.register user.edit_bio user.edit_bio
  69. PubSub Use Case BI

  70. user.register user.edit_bio user.left_rating user.post_trip ... Data Warehouse Log every business

    events in Hadoop
  71. user.register user.edit_bio user.left_rating user.post_trip ... user.register user.post_trip ... Meanwhile… RealTime

    Dashboard Log every business events in ElasticSearch
  72. None
  73. ReqRes — Command — PubSub Message Broker RabbitMQ removes hard

    link between services
  74. Now we have the keys to start a distributed architecture

    let’s start decoupling our application… Front-end desktop + mobile Hard to split in several projects (need to delegate jobs) API For mobile apps & partners Backoffice Set of administration tools Workers Already decoupled from the core app
  75. Backoffice A set of independent tools Easy to split

  76. Backoffice — CRUD Some tools are just data manipulation: – User

    Management – Blog – FAQ GET /users PUT /users/123 Data Layer
  77. Backoffice — Moderation Manage user « data » which need

    to be moderated user.upload_avatar user.edit_bio user.left_rating UI to check data manually Auto detect spam & non compliant data Machine Learning data.received data.treated send mails
  78. Backoffice — URL Shortener Some tools are completely independent https://www.blablacar.com/register

    SHORTEN No data shared with core business
  79. Authentication Splitting our backoffice in many apps should not be

    a pain for the UX
  80. Let’s build a Single Sign-On service

  81. Single Sign On “SSO is a method allowing a user

    to access multiple applications, making only a single authentication.”

  83. Single Sign ONCE example.org/app app.example.org Simple Case: Shared Host /

    Shared Domain
  84. None
  85. « OAuth is an open standard for authorization […] to

    access server resources on behalf of a resource owner » — Wikipedia
  86. OAuth2 Grant Types Authorization Code Connect-like workflow Implicit Grant (Direct

    Token) Usefull for JS app Password flow Trusted app (client credentials) Client Credentials Basic (use of client id + secret)
  87. OAuth2

  88. Do It Yourself with

  89. From LDAP to OAuth Given I am on the login

    page Then I should login with my LDAP credentials App LDAP Login
  90. There is a bundle for that

  91. FR3DLdapBundle

  92. FR3DLdapBundle/config.yml # LDAP Configuration! fr3d_ldap:! driver:! host: "%ldap_host%"! username: "%ldap_username%"!

    password: "%ldap_password%"! baseDn: "%ldap_username%"! user:! baseDn: "%ldap_user_dn%"! filter: "%ldap_user_filter%"! attributes:! - { ldap_attr: "uid", user_method: "setUsername" }! - { ldap_attr: "cn", user_method: "setName" }! - { ldap_attr: "mail", user_method: "setEmail" }! service:! user_manager: "acme.user_manager"!
  93. From LDAP to OAuth Given I am on "/me" with

    "user" access token Then I should get "user" resources in JSON App LDAP OAuth2 API
  94. There is a bundle for that

  95. FOSOAuthServerBundle

  96. FOSOAuthServerBundle/ security.yml firewalls:! api:! pattern: ^/api! fos_oauth: true! stateless: true

  97. From LDAP to OAuth App LDAP OAuth2 API Login

  98. Enforce security PO said, this SSO entry-point should be a

    top-notch secure app
  99. There ARE bundles for THAT

  100. SchebTwoFactorBundle SpomkyIpFilterBundle CCDNUserSecurityBundle NelmioSecurityBundle …

  101. App LDAP OAuth2 2FA FW

  102. What about the client?

  103. Of course, there is a bundle for that

  104. HWIOAuthBundle

  105. None
  106. # HWI OAuth Configuration! hwi_oauth:! firewall_name: "main"! resource_owners:! acme_sso:! type:

    "oauth2"! client_id: "%client_id%"! client_secret: "%client_secret%"! access_token_url: "%base_url%/oauth/v2/token"! authorization_url: "%base_url%/oauth/v2/auth"! infos_url: "%base_url%/api/me”! paths:! identifier: "id"! nickname: "username"! realname: "name"! email: "email"!
  107. UX — We don’t really need this login form

  108. UX — We don’t really need a one-button page

  109. 302 Found UX — Get transparent login process

  110. OAuth is all about authorization, not authentication Watch out for

    weaknesses & attacks
  111. GET /api/me { "id": 1337, "firstname": "Matthieu", "lastname": "Moquet", "nickname":

    "MattKetmo" } (the cheat)
  112. CSRF http://example.org?redirect_uri=xxxx&state=yyyy session[state] === params[state]. http://tools.ietf.org/html/draft-ietf-oauth-v2-27#section-10.12

  113. Cover Redirect (is a joke) http://homakov.blogspot.fr/2014/05/covert-redirect-faq.html

  114. id_token Multiple Response Type Encoding Practices. Provides an assertion of

    the identity of the Resource Owner. http://openid.bitbucket.org/oauth-v2-multiple-response-types-1_0.html
  115. However if OAuth2 is not designed to be an SSO

    protocol, what should I use?
  116. OAuth1.a

  117. It seems Symfony have a bundle for everything…

  118. BazingaOAuthServerBundle

  119. None
  120. SAML (Security Assertion Markup Language) SAML is an XML-based protocol

    that uses security tokens containing assertions to pass information about a principal between an identity provider, and a consumer. There are bundles/lib for that (but not maintained, see impl.) §  pdias/FOSSamlBundle §  aerialship/SamlSPBundle §  chtitux/sfSAMLPlugin (symfony1)
  121. CAS (Central Authentication Service) Example of client implementation BeSimple/BeSimpleSsoAuthBundle

  122. h"ps://github.com/BeSimple/BeSimpleSsoAuthBundle/blob/master/Resources/doc/protocols.md  

  123. JWT (JSON Web Token) Payload signed server-side with a JSON

    Web Signature (JWS). auth.   signed token request
  124. JWT (JSON Web Token) Sign payload using a secret key

    auth.   signed token request
  125. firebase/php-jwt curl –H "Authorization: Bearer eyJ0eXAiOiJKV..." example.org $key = "s3cr3t_key";

    $token = array( "sub" => "mattketmo", "aud" => "http://example.com", "exp" => 1356999524, ); $jwt = JWT::encode($token, $key);  // eyJ0eXAiOiJKV...
  126. namshi/jose OpenSSL + cookies // Auth (SSO) $privateKey = openssl_pkey_get_private("file://private.key");

    $jws = new JWS('RS256'); $jws->setPayload([$userId]); $jws->sign($privateKey) setcookie('identity', $jws->getTokenString()); // Client App $jws = JWS::load($_COOKIE['identity']); $publicKey = openssl_pkey_get_public("/public.key"); if ($jws->isValid($publicKey)) { $payload = $jws->getPayload(); $userId = $payload['id']; }
  127. OAuth2 + JWT http://tools.ietf.org/html/draft-ietf-oauth-jwt-bearer-08

  128. OpenID Connect Feb. 2014 — openid.net/connect/

  129. None
  130. Core app is now lightweight o Tiny container (less services/listeners) o Test

    suite is smaller o Forget about technical debt
  131. Choose the right tools for the task Backend / Frontend

    / Datastore / …
  132. Make experiments

  133. Work faster No more bottleneck & dependencies with other dev


  135. Install dev environment in less than 1h

  136. Bootstrap everything composer create-project blablacar/backoffice-app git clone git@git.example.org/worker-skeleton.git via git

    or via composer
  137. Enforce reusable components { "require": { "blablacar/monolog": "~1.0", "blablacar/scheduler-client": "~1.1",

    "blablacar/redis-client": "~1.2", "blablacar/rabbit-mq-admin-toolkit": "dev-master" } }
  138. Be ready for production Don’t loose time in configuration setup

    Parameters for $ENV Config template for $PROJECT Config file for $PROJECT/$ENV Centralized build tool to generate a project configuration file for any environment (local / dev / staging / prod)
  139. Never miss a log

  140. Don’t be lost in translations Open-sourced a tool to manage

    your project translations easily More information tomorrow, 9:45
  141. Thank you Slides available at moquet.net/talks/phptour-2014-sso-soa Leave feedbacks @MattKetmo