Development Workflow at BlaBlaCar

Symfony Live Paris 2013

Find this presentation at:
http://moquet.net/talks/symfony-live-2013/

Original description:
http://paris2013.live.symfony.com/speakers#session-882

Transcript

1. Development workflow
2. $developer Matthieu Moquet MattKetmo The quality guy Yoann Brault Opium84
3. What is BlaBlaCar ?
5. …and more than 200 white-label websites.
7. Benefits Less CO2 Social++ Less traffic Low cost
8. 3 000 000 members A large community
9. 8 countries § France § Spain § Italy § UK § Poland § Portugal § Netherlands § Benelux
10. 50 millions 25 millions January 2008 January 2013 Growth Page views.
11. Infrastructure ü 2  physical  fronts   ü 1  MySQL  master  +  4  slaves  SSD   ü 1  private  cloud  (KVM  +  Open  vSwitch)   o  Redis   o  Memcache   o  RabbitMQ/workers   ü 1  cluster  ElasEcSearch  
12. Why migrate to Symfony?
13. …quand je dois corriger un bug dans du code "historique" Source: http://lesjoiesducode.tumblr.com
14. Why migrate to Symfony2 ? §  « Separation of concerns » §  Services isolation (Bundles/Components) §  Unit & functional tests §  Scalability/maintainability o Modern Framework PHP 5.3+ o Active community o SensioLabs expertise
15. How to migrate to Symfony?
17. Migration Strategy 2 applications with same database
18. OLD   NEW   Progressive Migration
19. Behind the scene…
20. Development Workflow
21. A day at BlaBlaCar… •  Up to 15 deployments per days •  Default environment is localhost •  Common development server •  Fast reaction time
23. Workflow git Branching  model:   ü  dev ü  preprod ü  master   §  Cherry-­‐pick  to  deploy   §  Branch  for  long  features  
24. Workflow git : the bad parts •  Discontinuous workflow à May cause inconsistencies •  It complicates long features •  Intermediate branches are boring
26. Git Flow Consistent workflow, but not adapted to our needs
27. GitHub Flow « We typically deploy dozens of times per day and ship new features regularly » — Brian Doll http://scottchacon.com/2011/08/31/github-flow.html
28. Tools (before) Issue tracker Repository manager Pull Request Code Review Continuous Integration Trac gitolite (Internal) pmsipilot/Crew Jenkins
31. Tools (now) Issue tracker Repository manager Pull Request Code Review Continuous Integration Jira Stash Stash Stash Bamboo
33. Internal organization ü Product team conceives features and creates Jira tickets ü Tickets are distributed to developers each week ü Developers resolve those tickets (captain obvious) ü Product team validates the implementation ü Q/A team checks everything works ü Minidevs are for Friday
34. ü How does the product team validate the changes? ü How does the Q/A team check everything works as expected? GitHub Flow, That’s cool, but…
35. Prototypes
36. Prototypes GOAL Make available any branch on our development servers HOW? – API REST Symfony2 – Javascript client – PHP Workers
37. How Prototypes works? $ git checkout -b hipster-feature # commit code... $ git push origin hipster-feature
39. Environments $debug !$debug DEV PROD
40. Loads of time saved ü Easy access to dev or prod database ü Product & Q/A team can test a branch directly with « real » data ü We can debug production with the Symfony WDT (Web Debug Toolbar)
41. Summary Code à Test à Review à Deploy à Validate à Merge
42. …quand j’ai une idée de feature Source: http://lesjoiesducode.tumblr.com
43. Translations
44. Translations workflow RULE #1 o Translations must be editable at any moment o Developers are not required to update translations à  Save translations in database
45. Translations workflow RULE #2 •  Do NOT "SELECT * From translations" for each HTTP request •  Keep translations app cache in each frontal server à  Only keep token in Redis
46. Translations workflow HTTP   Request   TranslaEon   Service   isFresh()   false   SELECT translations   Update   app/cache   $lastModified >= $redis->get('last_modif') ? true : false;  
47. Translations workflow ü  Once  the  feature  is  finished,  the  branch  is  waiEng  for   the  product  &  Q/A  team  validaEon   ü  Meanwhile,  internaEonal  team  translates  all  the  new   strings     ü  Before  merging  to  master,  the  new  strings  have  already   been  translated   ü  At  any  moment  they  can  be  updated,  even  aVer  a   deployment  
48. ü  FULLTEXT search tool ü  Debug mode How to find the right translations keys?
49. Translator DebugMode // AcmeBundle/Translator.php class Translator extends BaseTranslator { protected $debugMode; public function trans($id, $parameters, $domain = 'messages', $locale = null) { if ($this->debugMode) { return $id; } return parent::trans($id, $parameters, $domain, $locale); } } // AcmeBundle/DebugListener.php public function onKernelRequest(GetResponseEvent $e) { $request = $e->getRequest(); if (/* put your own logic here */) { $this->translator->setDebug(true); } }
51. Emailing
52. Rendering emails GOAL •  Display any email without having to perform the associated actions •  See the rendering with different locales or with DebugMode à  Let’s make an interface for that…
53. Email Renderer Interface interface EmailRendererInterface { /** * @return string */ public function getTemplate(); /** * @return Form */ public function getForm(); } public function renderAction(Request $request) { $renderer = // ... $form = $renderer->getForm(); if ('POST' === $request->getMethod() && $form->bind($request)->isValid()) { return $this->render($renderer->getTemplate(), $form->getData()) } return $this->render('EmailRenderer:index.html.twig'); }
56. Front-end development
57. Front-end development GOAL •  Allow front-end developers to easily create Twig templates •  Do not interfere with existing templates <form action="{{ path('foo') }}" {{ form_enctype(form) }}> {{ form_widget(form) }} <input type="submit" /> </form> class AcmeExtension extends \Twig_Extension { public function getFilters() { return array( 'price' => new \Twig_Filter_Method( $this, 'priceFilter'), ); } }
58. Front-end development SOLUTION •  Create a dedicated folder for front-end templates •  List those templates in development environment only # AcmeDevBundle/Resources/views Design "## index.html.twig $## templates "## demo % $## example.html.twig "## profile % "## annonces.html.twig % "## dashboard.html.twig % "## vehicles.html.twig % $## verifications.html.twig "## registration % "## phone-fill.html.twig % $## register.html.twig "## search % "## empty.html.twig % "## no-result.html.twig % $## search.html.twig "## static % "## apps.html.twig % "## howto.html.twig % $## trust.html.twig $## widget $## widget.html.twig
59. Front-end development class DesignController { public function indexAction() { $files = Finder::create()->files()->in($this->templateDirectory); return $this->render('AcmeCoreBundle:Design:index.html.twig', array( 'files' => $files )); } public function showAction(Request $request, $template) { $filename = $this->templateDirectory.'/'.$template; if (!file_exists($filename)) { throw $this->createNotFoundException(); } return $this->render( 'AcmeCoreBundle:Design:templates/'.$template, $request->query->all(); // convert parameters from the request ); } }
63. Tests
64. Continuous Integration
65. Yoann Brault Opium84 Meet Yoann, not a developer The quality guy
66. Quality Assurance 6 years experience in web and mobile application testing
67. My role in the BlaBlaCar team The product and marketing teams write specifications Developers implement them The Q/A. verifies that the specifications are respected
68. Problematic of Q.A. Ensure that the entire site works on all browsers, for all languages ​​ and avoid regression
69. The adopted solution Automation for the Scenario execution on production environment on available prototypes
70. An example for the search functionnality on the BlaBlacar website First we define the function to be tested Then we define the scenario
71. Paris Reims We have what is expected Submit the form An example for the search functionnality on the BlaBlacar website
72. We have what was expected An example for the search functionnality on the BlaBlacar website
73. Here, the result of the executed scenario An example for the search functionnality on the BlaBlacar website
74. Here is the result of the tests. Those are launched two to three times a day. Below the 34 scenarios were all successful Below, 1 failed scenario, resulting in 7 ignored steps. An example for the search functionnality on the BlaBlacar website
75. Screenshots are always usefull to better understand why the test failed An example for the search functionnality on the BlaBlacar website
76. List of current automatic tests I can register a new account I can't use an email already in use I can create a new alert I can check the lowest/max price I can add preference I can post a new trip I can check the ladies only option and check if the trip is available for men I create a new roundtrip I Can check the sharing options and change them I can access the alerts tab I Can signup with a FB account on Landing Pages 3 I can verify a phone number I can perform a classic search I can perform a search only with departure city I can perform a search only with arrival city I can connect with Facebook, […] I check for private message notification […] I can delete a trip I can add a car to my profile I can remove a car from my profile I Can signup with a FB account on Landing Pages 1 I can add a bio to my profile I can check if the photo notification is available I can create a trip with stepover I can post a new trip, make a return and duplicate the trip I can sign up in with a FB account I can search each subtrip from a trip with stepover I can access to the post trip page from the homepage I can request a new password I can erase an account I Can signup with a FB account on Landing Pages 4 I can perform a search from the homepage I can check some short urlAcces to the backoffice to verify that the short link page is ok ...
77. Future of automatic tests We have developed a tool to make things easier and faster
78. Deployments
79. …quand je fais une mise en prod Source: http://lesjoiesducode.tumblr.com
80. How to MEP? $ ./mep.sh
81. How to MEP? Yo dude! it’s time to deploy [RSYNC] Gimme the code
82. Monitoring
83. More details on our tech blog http://tech.covoiturage.fr
84. What is YOUR development workflow?
85. Thank you! Slides available at http://moquet.net/talks/symfony-live-2013 We’re  hiring!   Leave feedbacks @MattKetmo