Decoupling your application

Decoupling your application

Symfony Live Paris 2014

Find this presentation at:
http://moquet.net/talks/symfony-live-2014/

Original description:
http://paris2014.live.symfony.com/speakers#session-1131

F135ce7f204af6fac8075b469685c91d?s=128

Matthieu Moquet

April 08, 2014
Tweet

Transcript

  1. Decoupling your application …with SSO, OAuth & more

  2. None
  3. None
  4. /me Matthieu Moquet @MattKetmo Developer at …for almost 3 years

  5. Back to 2011…

  6. Covoiturage.fr 1 monolithic PHP app o Core website o Web mobile o B2B

    platform o Backoffice tools o Web views (mobile apps) o External widgets
  7. Plain Old PHP No Dependency Injection No functional tests High

    coupling …hard to maintain
  8. None
  9. # wc -l lib.trip.php 3678 …longest method: 1000+ lines

  10. None
  11. Get rid of the Technical Debt

  12. Since 2012 Start from scratch

  13. 2.0

  14. NEW ORGANISATION NEW FRAMEWORK

  15. Let’s embrace best practices

  16. 1 bundle BlablacarWebBundle BlablacarBlogBundle BlablacarFaqBundle BlablacarTripBundle BlablacarUserBundle ... N bundles

    OR
  17. Services & Controllers §  Should controllers access the repository? § 

    How should I split my services? §  Is ContainerAware that bad? §  How to organize my Business Layer?
  18. Where should I put my Model & Doctrine entities?

  19. Forms Should I map my entities with my forms?

  20. Validation §  Should I map validation with entities? §  Should

    I use groups?
  21. Translations §  Catalogue message? §  Should I use key identifiers

    or phrases? §  How to name the keys?
  22. Route naming homepage blablacar_homepage blablacar_main_homepage

  23. Events §  Listen on Doctrine events? §  Create your own?

    ACTION / PRE_ACTION / POST_ACTION ? §  Subscribers VS. Listeners
  24. Design When you don’t have full-time designer, then Bootstrap(2) FTW

  25. With a few developers… At the beginning, you need to

    start quickly, (but you take time to write conventions). Legacy App New App
  26. Then you hire new people

  27. How many developers to be more efficient?

  28. More code means…

  29. Heavy Container

  30. $ app/console cache:warmup Warming up the cache for the dev

    env with debug true
  31. None
  32. $ app/console assetic:dump Dumping all dev assets. Debug mode is

    on. 23:00:32 [file+] /path/to/web/css/0e781b6.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_1_bootstrap_1.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_2_buttons_1.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_2_card_2.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_2_form_3.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_2_grid_4.css 23:00:36 [file+] /path/to/web/css/0e781b6_part_2_list_5.css 23:00:36 [file+] /path/to/web/css/0e781b6_part_2_navbar_6.css 23:00:36 [file+] /path/to/web/css/0e781b6_part_2_panel_7.css 23:00:37 [file+] /path/to/web/css/0e781b6_part_2_vcard_9.css 23:00:37 [file+] /path/to/web/js/e66598b.js 23:00:37 [file+] /path/to/web/js/e66598b_jquery_1.js 23:00:37 [file+] /path/to/web/js/e66598b_bootstrap_2.js 23:00:37 [file+] /path/to/web/js/e66598b_script_3.js
  33. None
  34. $ phpunit -c app ........................... .............. (63/975) .......................... ............... (126/975)

    .............. ........................... (189/975) .......F................ ................. (252/975) .............F....
  35. Changed few lines of code? Run the full test suite!

  36. None
  37. Optimize test execution time Invest in parallelization processes Split TestSuite

    in VMs (on AWS) Run 1h tests in 10min
  38. One big projet makes your team less reactive How often

    do you deploy your main project?
  39. https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/

  40. Changing foundations is expensive What if we want to change…

  41. What if we want to change The backend framework? We

    don’t plan to change it, and we couldn’t
  42. What if we want to change The testing framework? We

    won’t rewrite the whole test suite. But we can use several frameworks at the same time (eg. Behat)
  43. What if we want to change The frontend framework? Well,

    we are stuck with Bootstrap2. Updating to Bootstrap3 or rewriting our own will take time.
  44. What if we want to change The assets builder? Assetic

    took too long to compile all assets. We moved to asset management with Grunt.
  45. What if we want to change The data layer? Actually

    frontend servers make MySQL queries. But in the long term, it’s not a good practice (see coming slides).
  46. We’ll ALWAYS have technical debt. We must LIMIT it as

    much as possible.
  47. Think as small as possible Microservices The secret to building

    large apps is never build large apps. Break your application into small pieces. Then, assemble those testable, bite-sized pieces into your big application — Justin Meyer
  48. 3 patterns to build a better software architecture…

  49. Request-Response

  50. Request-Response GET your resources synchronously Data Layer

  51. Request-Response $user = $this ->get('my.repository.user') ->find(1337);

  52. Request-Response mysql_query('...')

  53. Request-Response Id   Firstname   Lastname   Pseudo   Email

      Birthday   1337   Ma'hieu   Moquet   Ma'Ketmo   ma'hieu@moquet.net   1988-­‐12-­‐17  
  54. Request-Response However front-end servers should NOT access the database directly

    It should fetch normalized data from an internal service API for the win!
  55. Request-Response GET /users/123 Do you speak REST ? Better versioning

    & normalization
  56. Request-Response …or you may speak: § SOAP § XML-RPC § Protobuffer § Thrift § etc.

  57. Request-Response Even RabbitMQ can be used for synchronous requests scrutinizer-ci/rabbitmq

    src/Scrutinizer/RabbitMQ/Rpc http://www.rabbitmq.com/tutorials/tutorial-six-python.html
  58. Request-Response

  59. Command(Handler)

  60. Command $cmd = FooCommand('bar'); $handler = FooCommandHandler(); $handler->execute($cmd); Do not

    expect a return value
  61. Command Perfect for asynchronous jobs §  Send e-mails / SMS

    / PUSH notifications §  Image processing §  Data indexation §  Saving complex data §  etc.
  62. Command publish consume RabbitMQ   DIRECT  rou<ng  

  63. Command publish consume Easy  scaling  

  64. Command Service is accessible via a queueing system only (not

    REST) §  RabbitMQ §  ActiveMQ §  Beanstalkd §  Gearman §  … More at http://queues.io
  65. Sending Newsletters publish 100k messages Create an army of workers

    in AWS consume send mails message payload = emails address + content App Scheduler (Java + Quartz) Worker NL @10am segment X Get users of segment X (Scroll ElasticSearch)
  66. PubSub Obverser / Event Dispatcher

  67. PubSub $dispatcher = new EventDispatcher();! ! // Listen using an

    object or a callback! $listener = new AcmeListener();! $dispatcher->addListener('foobar', array($listener, 'onFoobar'));! ! $dispatcher->addListener('foobar', function (Event $event) {! // do something else with the events! });! ! // Dispatch event! $dispatcher->dispatch('foobar', $event);! Example with the Symfony EventDispatcher
  68. PubSub Notify your infrastructure of every business events Publish events

    without knowing who is listening Create services without any core changes
  69. PubSub pub RabbitMQ   TOPIC  rou<ng   sub user.register user.edit_bio

    user.register user.edit_bio user.edit_bio
  70. PubSub Use Case BI

  71. user.register user.edit_bio user.left_rating user.post_trip ... Data WareHouse Log every business

    events in Hadoop
  72. user.register user.edit_bio user.left_rating user.post_trip ... user.register user.post_trip ... Meanwhile… RealTime

    Dashboard Log every business events in ElasticSearch
  73. None
  74. ReqRes — Command — PubSub Message Broker RabbitMQ removes hard

    link between services
  75. Now we have the keys to start a distributed architecture

    let’s start decoupling our application… Front-end desktop + mobile Hard to split in several projects (need to delegate jobs) API For mobile apps & partners Backoffice Set of administration tools Workers Already decoupled from the core app
  76. Backoffice A set of independent tools Easy to split

  77. Backoffice — CRUD Some tools are just data manipulation: – User

    Management – Blog – FAQ GET /users PUT /users/123 Data Layer
  78. Backoffice — Moderation Manage user « data » which need

    to be moderated user.upload_avatar user.edit_bio user.left_rating UI to check data manually Auto detect spam & non compliant data Machine Learning data.received data.treated send mails
  79. Backoffice — URL Shortener Some tools are completely independent https://www.blablacar.com/register

    SHORTEN No data shared with core business
  80. Authentication Splitting our backoffice in many apps should not be

    a pain for the UX
  81. Let’s build a Single Sign-On service

  82. None
  83. « OAuth is an open standard for authorization […] to

    access server resources on behalf of a resource owner » — Wikipedia
  84. OAuth2 Grant Types Authorization Code Connect-like workflow Implicit Grant (Direct

    Token) Usefull for JS app Password flow Trusted app (client credentials) Client Credentials Basic (use of client id + secret)
  85. OAuth2 Client App Use token to call API Server App

  86. Do It Yourself

  87. From LDAP to OAuth Given I am on the login

    page Then I should login with my LDAP credentials App LDAP Login
  88. THERE IS A BUNDLE FOR THAT!

  89. FR3DLdapBundle

  90. FR3DLdapBundle/config.yml # LDAP Configuration! fr3d_ldap:! driver:! host: "%ldap_host%"! username: "%ldap_username%"!

    password: "%ldap_password%"! baseDn: "%ldap_username%"! user:! baseDn: "%ldap_user_dn%"! filter: "%ldap_user_filter%"! attributes:! - { ldap_attr: "uid", user_method: "setUsername" }! - { ldap_attr: "cn", user_method: "setName" }! - { ldap_attr: "mail", user_method: "setEmail" }! service:! user_manager: "acme.user_manager"!
  91. From LDAP to OAuth Given I am on "/me" with

    "user" access token Then I should get "user" resources in JSON App LDAP OAuth2 API
  92. THERE IS A BUNDLE FOR THAT!

  93. FOSOAuthServerBundle

  94. FOSOAuthServerBundle/security.yml firewalls:! api:! pattern: ^/api! fos_oauth: true! stateless: true  

  95. From LDAP to OAuth App LDAP OAuth2 API Login

  96. Enforce security PO said, this SSO entry-point should be a

    top-notch secure app
  97. THERE ARE BUNDLES FOR THAT!

  98. SchebTwoFactorBundle SpomkyIpFilterBundle CCDNUserSecurityBundle NelmioSecurityBundle …

  99. App LDAP OAuth2 2FA FW

  100. What about the client?

  101. THERE IS A BUNDLE FOR THAT!

  102. HWIOAuthBundle

  103. None
  104. # HWI OAuth Configuration! hwi_oauth:! firewall_name: "main"! resource_owners:! acme_sso:! type:

    "oauth2"! client_id: "%client_id%"! client_secret: "%client_secret%"! access_token_url: "%base_url%/oauth/v2/token"! authorization_url: "%base_url%/oauth/v2/auth"! infos_url: "%base_url%/api/me”! paths:! identifier: "id"! nickname: "username"! realname: "name"! email: "email"!
  105. UX — We don’t really need this login form

  106. UX — We don’t really need a one-button page

  107. 302 Found UX — Get transparent login process

  108. OAuth is all about authorization, not authentication Watch out for

    weaknesses & attacks
  109. GET /api/me {! "id": 1337,! "firstname": "Matthieu",! "lastname": "Moquet",! "nickname":

    "MattKetmo"! }! (the cheat)
  110. CSRF http://example.org?redirect_uri=xxxx&state=yyyy session[state] === params[state]. http://tools.ietf.org/html/draft-ietf-oauth-v2-27#section-10.12

  111. id_token Multiple Response Type Encoding Practices. Provides an assertion of

    the identity of the Resource Owner. http://openid.bitbucket.org/oauth-v2-multiple-response-types-1_0.html
  112. However if OAuth is not designed to be an SSO

    protocol, what should I use?
  113. SAML (Security Assertion Markup Language) SAML is an XML-based protocol

    that uses security tokens containing assertions to pass information about a principal between an identity provider, and a consumer. There are bundles/lib for that (but not maintained, see impl.) §  pdias/FOSSamlBundle §  aerialship/SamlSPBundle §  chtitux/sfSAMLPlugin (symfony1)
  114. JWT (JSON Web Token) Payload signed server-side with a JSON

    Web Signature (JWS). auth.   signed token request
  115. JWT (JSON Web Token) Sign payload using a secret key

    auth.   signed token request
  116. firebase/php-jwt curl –H "Authorization: Bearer eyJ0eXAiOiJKV..." example.org $key = "s3cr3t_key";

    $token = array( "sub" => "mattketmo", "aud" => "http://example.com", "exp" => 1356999524, ); $jwt = JWT::encode($token, $key);  // eyJ0eXAiOiJKV...
  117. namshi/jose OpenSSL + cookies // Auth (SSO) $privateKey = openssl_pkey_get_private("file://private.key");

    $jws = new JWS('RS256'); $jws->setPayload([$userId]); $jws->sign($privateKey) setcookie('identity', $jws->getTokenString()); // Client App $jws = JWS::load($_COOKIE['identity']); $publicKey = openssl_pkey_get_public("/public.key"); if ($jws->isValid($publicKey)) { $payload = $jws->getPayload(); $userId = $payload['id']; }
  118. OAuth2 + JWT http://tools.ietf.org/html/draft-ietf-oauth-jwt-bearer-08

  119. Of course, there are Symfony bundles: –  formapro/FpOpenIdBundle –  KainHaart/OpenIDAuthBundle

    –  etc.
  120. Hawk Shared secret between client & server by Eran Hammer

    Read more at http://alexbilbie.com/2012/11/hawk-a-new-http-authentication-scheme/
  121. CAS (Central Authentication Service) Example of client implementation BeSimple/BeSimpleSsoAuthBundle

  122. h"ps://github.com/BeSimple/BeSimpleSsoAuthBundle/blob/master/Resources/doc/protocols.md  

  123. None
  124. Core app is now lightweight Tiny container (less services/listeners) Test

    suite is smaller
  125. Choose the right tools for the task Backend / Frontend

    / Datastore / …
  126. Make experiments

  127. Work faster No more bottleneck & dependencies with other dev

    teams
  128. BE EFFICIENT AVOID REPETITIVE PROCESSES

  129. Install dev environment in less than 1h

  130. Bootstrap everything composer create-project blablacar/backoffice-app git clone git@git.example.org/worker-skeleton.git via git

    or via composer
  131. Enforce reusable components { "require": { "blablacar/monolog": "~1.0", "blablacar/scheduler-client": "~1.1",

    "blablacar/redis-client": "~1.2", "blablacar/rabbit-mq-admin-toolkit": "dev-master" } }
  132. Be ready for production Don’t loose time in configuration setup

    Parameters for $ENV Config template for $PROJECT Config file for $PROJECT/$ENV Centralized build tool to generate a project configuration file for any environment (local / dev / staging / prod)
  133. Never miss a log

  134. Don’t be lost in translations Open-sourced a tool to manage

    your projects’ translations easily
  135. More information at openl10n.io

  136. Thank you Slides available at moquet.net/talks/symfony-live-2014 Leave feedbacks @MattKetmo we’re

    hiring!