Decoupling your application

Symfony Live Paris 2014

Find this presentation at:
http://moquet.net/talks/symfony-live-2014/

Original description:
http://paris2014.live.symfony.com/speakers#session-1131

Transcript

1. Decoupling your application …with SSO, OAuth & more
4. /me Matthieu Moquet @MattKetmo Developer at …for almost 3 years
5. Back to 2011…
6. Covoiturage.fr 1 monolithic PHP app o Core website o Web mobile o B2B platform o Backoffice tools o Web views (mobile apps) o External widgets
7. Plain Old PHP No Dependency Injection No functional tests High coupling …hard to maintain
9. # wc -l lib.trip.php 3678 …longest method: 1000+ lines
11. Get rid of the Technical Debt
12. Since 2012 Start from scratch
13. 2.0
14. NEW ORGANISATION NEW FRAMEWORK
15. Let’s embrace best practices
16. 1 bundle BlablacarWebBundle BlablacarBlogBundle BlablacarFaqBundle BlablacarTripBundle BlablacarUserBundle ... N bundles OR
17. Services & Controllers §  Should controllers access the repository? §  How should I split my services? §  Is ContainerAware that bad? §  How to organize my Business Layer?
18. Where should I put my Model & Doctrine entities?
19. Forms Should I map my entities with my forms?
20. Validation §  Should I map validation with entities? §  Should I use groups?
21. Translations §  Catalogue message? §  Should I use key identifiers or phrases? §  How to name the keys?
22. Route naming homepage blablacar_homepage blablacar_main_homepage
23. Events §  Listen on Doctrine events? §  Create your own? ACTION / PRE_ACTION / POST_ACTION ? §  Subscribers VS. Listeners
24. Design When you don’t have full-time designer, then Bootstrap(2) FTW
25. With a few developers… At the beginning, you need to start quickly, (but you take time to write conventions). Legacy App New App
26. Then you hire new people
27. How many developers to be more efficient?
28. More code means…
29. Heavy Container
30. $ app/console cache:warmup Warming up the cache for the dev env with debug true
32. $ app/console assetic:dump Dumping all dev assets. Debug mode is on. 23:00:32 [file+] /path/to/web/css/0e781b6.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_1_bootstrap_1.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_2_buttons_1.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_2_card_2.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_2_form_3.css 23:00:35 [file+] /path/to/web/css/0e781b6_part_2_grid_4.css 23:00:36 [file+] /path/to/web/css/0e781b6_part_2_list_5.css 23:00:36 [file+] /path/to/web/css/0e781b6_part_2_navbar_6.css 23:00:36 [file+] /path/to/web/css/0e781b6_part_2_panel_7.css 23:00:37 [file+] /path/to/web/css/0e781b6_part_2_vcard_9.css 23:00:37 [file+] /path/to/web/js/e66598b.js 23:00:37 [file+] /path/to/web/js/e66598b_jquery_1.js 23:00:37 [file+] /path/to/web/js/e66598b_bootstrap_2.js 23:00:37 [file+] /path/to/web/js/e66598b_script_3.js
34. $ phpunit -c app ........................... .............. (63/975) .......................... ............... (126/975) .............. ........................... (189/975) .......F................ ................. (252/975) .............F....
35. Changed few lines of code? Run the full test suite!
37. Optimize test execution time Invest in parallelization processes Split TestSuite in VMs (on AWS) Run 1h tests in 10min
38. One big projet makes your team less reactive How often do you deploy your main project?
39. https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook/
40. Changing foundations is expensive What if we want to change…
41. What if we want to change The backend framework? We don’t plan to change it, and we couldn’t
42. What if we want to change The testing framework? We won’t rewrite the whole test suite. But we can use several frameworks at the same time (eg. Behat)
43. What if we want to change The frontend framework? Well, we are stuck with Bootstrap2. Updating to Bootstrap3 or rewriting our own will take time.
44. What if we want to change The assets builder? Assetic took too long to compile all assets. We moved to asset management with Grunt.
45. What if we want to change The data layer? Actually frontend servers make MySQL queries. But in the long term, it’s not a good practice (see coming slides).
46. We’ll ALWAYS have technical debt. We must LIMIT it as much as possible.
47. Think as small as possible Microservices The secret to building large apps is never build large apps. Break your application into small pieces. Then, assemble those testable, bite-sized pieces into your big application — Justin Meyer
48. 3 patterns to build a better software architecture…
49. Request-Response
50. Request-Response GET your resources synchronously Data Layer
51. Request-Response $user = $this ->get('my.repository.user') ->find(1337);
52. Request-Response mysql_query('...')
53. Request-Response Id   Firstname   Lastname   Pseudo   Email   Birthday   1337   Ma'hieu   Moquet   Ma'Ketmo   ma'hieu@moquet.net   1988-­‐12-­‐17  
54. Request-Response However front-end servers should NOT access the database directly It should fetch normalized data from an internal service API for the win!
55. Request-Response GET /users/123 Do you speak REST ? Better versioning & normalization
56. Request-Response …or you may speak: § SOAP § XML-RPC § Protobuffer § Thrift § etc.
57. Request-Response Even RabbitMQ can be used for synchronous requests scrutinizer-ci/rabbitmq src/Scrutinizer/RabbitMQ/Rpc http://www.rabbitmq.com/tutorials/tutorial-six-python.html
58. Request-Response
59. Command(Handler)
60. Command $cmd = FooCommand('bar'); $handler = FooCommandHandler(); $handler->execute($cmd); Do not expect a return value
61. Command Perfect for asynchronous jobs §  Send e-mails / SMS / PUSH notifications §  Image processing §  Data indexation §  Saving complex data §  etc.
62. Command publish consume RabbitMQ   DIRECT  rou<ng  
63. Command publish consume Easy  scaling  
64. Command Service is accessible via a queueing system only (not REST) §  RabbitMQ §  ActiveMQ §  Beanstalkd §  Gearman §  … More at http://queues.io
65. Sending Newsletters publish 100k messages Create an army of workers in AWS consume send mails message payload = emails address + content App Scheduler (Java + Quartz) Worker NL @10am segment X Get users of segment X (Scroll ElasticSearch)
66. PubSub Obverser / Event Dispatcher
67. PubSub $dispatcher = new EventDispatcher();! ! // Listen using an object or a callback! $listener = new AcmeListener();! $dispatcher->addListener('foobar', array($listener, 'onFoobar'));! ! $dispatcher->addListener('foobar', function (Event $event) {! // do something else with the events! });! ! // Dispatch event! $dispatcher->dispatch('foobar', $event);! Example with the Symfony EventDispatcher
68. PubSub Notify your infrastructure of every business events Publish events without knowing who is listening Create services without any core changes
69. PubSub pub RabbitMQ   TOPIC  rou<ng   sub user.register user.edit_bio user.register user.edit_bio user.edit_bio
70. PubSub Use Case BI
71. user.register user.edit_bio user.left_rating user.post_trip ... Data WareHouse Log every business events in Hadoop
72. user.register user.edit_bio user.left_rating user.post_trip ... user.register user.post_trip ... Meanwhile… RealTime Dashboard Log every business events in ElasticSearch
74. ReqRes — Command — PubSub Message Broker RabbitMQ removes hard link between services
75. Now we have the keys to start a distributed architecture let’s start decoupling our application… Front-end desktop + mobile Hard to split in several projects (need to delegate jobs) API For mobile apps & partners Backoffice Set of administration tools Workers Already decoupled from the core app
76. Backoffice A set of independent tools Easy to split
77. Backoffice — CRUD Some tools are just data manipulation: – User Management – Blog – FAQ GET /users PUT /users/123 Data Layer
78. Backoffice — Moderation Manage user « data » which need to be moderated user.upload_avatar user.edit_bio user.left_rating UI to check data manually Auto detect spam & non compliant data Machine Learning data.received data.treated send mails
79. Backoffice — URL Shortener Some tools are completely independent https://www.blablacar.com/register SHORTEN No data shared with core business
80. Authentication Splitting our backoffice in many apps should not be a pain for the UX
81. Let’s build a Single Sign-On service
83. « OAuth is an open standard for authorization […] to access server resources on behalf of a resource owner » — Wikipedia
84. OAuth2 Grant Types Authorization Code Connect-like workflow Implicit Grant (Direct Token) Usefull for JS app Password flow Trusted app (client credentials) Client Credentials Basic (use of client id + secret)
85. OAuth2 Client App Use token to call API Server App
86. Do It Yourself
87. From LDAP to OAuth Given I am on the login page Then I should login with my LDAP credentials App LDAP Login
88. THERE IS A BUNDLE FOR THAT!
89. FR3DLdapBundle
90. FR3DLdapBundle/config.yml # LDAP Configuration! fr3d_ldap:! driver:! host: "%ldap_host%"! username: "%ldap_username%"! password: "%ldap_password%"! baseDn: "%ldap_username%"! user:! baseDn: "%ldap_user_dn%"! filter: "%ldap_user_filter%"! attributes:! - { ldap_attr: "uid", user_method: "setUsername" }! - { ldap_attr: "cn", user_method: "setName" }! - { ldap_attr: "mail", user_method: "setEmail" }! service:! user_manager: "acme.user_manager"!
91. From LDAP to OAuth Given I am on "/me" with "user" access token Then I should get "user" resources in JSON App LDAP OAuth2 API
92. THERE IS A BUNDLE FOR THAT!
93. FOSOAuthServerBundle
94. FOSOAuthServerBundle/security.yml firewalls:! api:! pattern: ^/api! fos_oauth: true! stateless: true  
95. From LDAP to OAuth App LDAP OAuth2 API Login
96. Enforce security PO said, this SSO entry-point should be a top-notch secure app
97. THERE ARE BUNDLES FOR THAT!
98. SchebTwoFactorBundle SpomkyIpFilterBundle CCDNUserSecurityBundle NelmioSecurityBundle …
99. App LDAP OAuth2 2FA FW
100. What about the client?
101. THERE IS A BUNDLE FOR THAT!
102. HWIOAuthBundle
104. # HWI OAuth Configuration! hwi_oauth:! firewall_name: "main"! resource_owners:! acme_sso:! type: "oauth2"! client_id: "%client_id%"! client_secret: "%client_secret%"! access_token_url: "%base_url%/oauth/v2/token"! authorization_url: "%base_url%/oauth/v2/auth"! infos_url: "%base_url%/api/me”! paths:! identifier: "id"! nickname: "username"! realname: "name"! email: "email"!
105. UX — We don’t really need this login form
106. UX — We don’t really need a one-button page
107. 302 Found UX — Get transparent login process
108. OAuth is all about authorization, not authentication Watch out for weaknesses & attacks
109. GET /api/me {! "id": 1337,! "firstname": "Matthieu",! "lastname": "Moquet",! "nickname": "MattKetmo"! }! (the cheat)
110. CSRF http://example.org?redirect_uri=xxxx&state=yyyy session[state] === params[state]. http://tools.ietf.org/html/draft-ietf-oauth-v2-27#section-10.12
111. id_token Multiple Response Type Encoding Practices. Provides an assertion of the identity of the Resource Owner. http://openid.bitbucket.org/oauth-v2-multiple-response-types-1_0.html
112. However if OAuth is not designed to be an SSO protocol, what should I use?
113. SAML (Security Assertion Markup Language) SAML is an XML-based protocol that uses security tokens containing assertions to pass information about a principal between an identity provider, and a consumer. There are bundles/lib for that (but not maintained, see impl.) §  pdias/FOSSamlBundle §  aerialship/SamlSPBundle §  chtitux/sfSAMLPlugin (symfony1)
114. JWT (JSON Web Token) Payload signed server-side with a JSON Web Signature (JWS). auth.   signed token request
115. JWT (JSON Web Token) Sign payload using a secret key auth.   signed token request
116. firebase/php-jwt curl –H "Authorization: Bearer eyJ0eXAiOiJKV..." example.org $key = "s3cr3t_key"; $token = array( "sub" => "mattketmo", "aud" => "http://example.com", "exp" => 1356999524, ); $jwt = JWT::encode($token, $key);  // eyJ0eXAiOiJKV...
117. namshi/jose OpenSSL + cookies // Auth (SSO) $privateKey = openssl_pkey_get_private("file://private.key"); $jws = new JWS('RS256'); $jws->setPayload([$userId]); $jws->sign($privateKey) setcookie('identity', $jws->getTokenString()); // Client App $jws = JWS::load($_COOKIE['identity']); $publicKey = openssl_pkey_get_public("/public.key"); if ($jws->isValid($publicKey)) { $payload = $jws->getPayload(); $userId = $payload['id']; }
118. OAuth2 + JWT http://tools.ietf.org/html/draft-ietf-oauth-jwt-bearer-08
119. Of course, there are Symfony bundles: –  formapro/FpOpenIdBundle –  KainHaart/OpenIDAuthBundle –  etc.
120. Hawk Shared secret between client & server by Eran Hammer Read more at http://alexbilbie.com/2012/11/hawk-a-new-http-authentication-scheme/
121. CAS (Central Authentication Service) Example of client implementation BeSimple/BeSimpleSsoAuthBundle
122. h"ps://github.com/BeSimple/BeSimpleSsoAuthBundle/blob/master/Resources/doc/protocols.md  
124. Core app is now lightweight Tiny container (less services/listeners) Test suite is smaller
125. Choose the right tools for the task Backend / Frontend / Datastore / …
126. Make experiments
127. Work faster No more bottleneck & dependencies with other dev teams
128. BE EFFICIENT AVOID REPETITIVE PROCESSES
129. Install dev environment in less than 1h
130. Bootstrap everything composer create-project blablacar/backoffice-app git clone git@git.example.org/worker-skeleton.git via git or via composer
131. Enforce reusable components { "require": { "blablacar/monolog": "~1.0", "blablacar/scheduler-client": "~1.1", "blablacar/redis-client": "~1.2", "blablacar/rabbit-mq-admin-toolkit": "dev-master" } }
132. Be ready for production Don’t loose time in configuration setup Parameters for $ENV Config template for $PROJECT Config file for $PROJECT/$ENV Centralized build tool to generate a project configuration file for any environment (local / dev / staging / prod)
133. Never miss a log
134. Don’t be lost in translations Open-sourced a tool to manage your projects’ translations easily
135. More information at openl10n.io
136. Thank you Slides available at moquet.net/talks/symfony-live-2014 Leave feedbacks @MattKetmo we’re hiring!