Building an Open-Source Campaign Platform for the new President of France

This case study will take you to the technical backdrop of Emmanuel Macron's victorious campaign in the French presidential elections of 2017. We will reveal how the En-Marche and SensioLabs teams worked closely together to develop a collaborative, Open-Source and citizen platform with PHP 7 and Symfony to structure the movement. In addition to the technical and political issues involved in this project, we'll present solutions such as continuous deployment with Docker under Kubernetes, continuous integration with Circle CI, SCRUM and Kanban project management, use of Rabbitmq, Symfony 3 new features, and more. We'll demonstrate the importance of choosing these solutions to meet the agility and scalability needs of the highlights of the campaign.

Transcript

1. Building an Open-Source Campaign Platform for the President of France . SymfonyCon 2017 / Nov. 17th / Cluj / Romania Hugo Hamon
2. Hugo Hamon Senior Software Developer 15 years of PHP experience 10 years of Symfony experience Conferences speaker @hhamon on social networks Books (co) author
3. Titouan Galopin Enrolled in the En-Marche movement Initiated the Symfony migration Full-Stack Developer Setup the new technical stack @titouangalopin on Twitter
4. Innovative e-learning solution targeting Symfony teams. https://university.sensiolabs.com
5. The Context
6. Why is the Presidential election such a challenging environment?
7. Why building a Citizen Collaborative Platform?
8. Accept donations Register adherents Facilitate communication Structure movement
9. April 2016
11. Summer 2016
12. +
14. October 2016
16. https://github.com/enmarche/en-marche.fr
17. https://en-marche.fr
18. Response Web Design Full featured Admin area Users management Fined grained permissions Http caching Images lazy loading Content Management Unit & functional tests Mass emails management Maps & address geocoding Social networks integration Procurations management Media management Field actions reports Legislatives elections support Logging Search engine Etc.
19. End of Year 2016
20. Emmanuel Macron formalizes his candidacy for the presidential election.
21. Symfony 3.2 also comes out!
22. Coincidence? J
25. # app/config/parameters.yml parameters: env(DATABASE_HOST): db env(DATABASE_PORT): 3306 env(DATABASE_SOCKET): null env(DATABASE_NAME): enmarche env(DATABASE_USER): root env(DATABASE_PASSWORD): root
27. class AssetsController extends Controller { /** * @Route("/assets/{path}", name="asset_url") * @Cache(maxage=900, smaxage=900) */ public function assetAction(Request $request, string $path): Response { // ... $glide = $this->get('app.glide'); $glide->setResponseFactory(new SymfonyResponseFactory($request)); try { return $glide->getImageResponse($path, $request->query->all()); } catch (FileNotFoundException $e) { throw $this->createNotFoundException('', $e); } } }
28. January 2017
30. Week n People at En-Marche write the User Stories and introduce them at SensioLabs on Thursday. Week n+1 Poker planning session on Tuesday with the SensioLabs team. Week n+2 New coding sprint starts on Monday. Demo session is on Friday afternoon.
31. trait EntityIdentityTrait { /** * @ORM\Id * @ORM\Column(type="integer") * @ORM\GeneratedValue */ private $id; /** * @ORM\Column(type="uuid") * @var \Ramsey\Uuid\UuidInterface */ private $uuid; // ... + getters } Custom UUIDs for Doctrine entities.
32. class MembershipController extends Controller { // ... public function registerAction(Request $request): Response { $membership = MembershipRequest::createWithCaptcha(...); $form = $this ->createForm(MembershipRequestType::class, $membership) ->handleRequest($request); if ($form->isSubmitted() && $form->isValid()) { $this->get('app.membership')->handle($membership); return $this->redirectToRoute('app_membership_donate'); } return $this->render('...', [ 'form' => $form->createView(), // ... ]); } } Thin & dumb controllers.
33. src/Mailjet/ ├── ApiClient.php ├── ClientInterface.php ├── EmailTemplate.php ├── EmailTemplateFactory.php ├── Event/ │ ├── MailjetEvent.php │ └── MailjetEvents.php ├── EventSubscriber/ │ └── EmailPersisterEventSubscriber.php ├── Exception/ │ └── MailjetException.php ├── MailjetService.php ├── MailjetUtils.php ├── Message/ │ ├── AdherentAccountActivationMessage.php │ ├── AdherentAccountConfirmationMessage.php │ ├── ... │ └── TonMacronFriendMessage.php └── Transport/ ├── ApiTransport.php ├── RabbitMQTransport.php └── TransportInterface.php
34. class MailjetService { // ... public function sendMessage(MailjetMessage $message): void { $email = $this->factory->createFromMailjetMessage($message); $event = new MailjetEvent($message, $email); try { $this->dispatch(MailjetEvents::DELIVERY_MESSAGE, $event); $this->transport->sendTemplateEmail($email); $this->dispatch(MailjetEvents::DELIVERY_SUCCESS, $event); } catch (MailjetException $e) { $this->dispatch( MailjetEvents::DELIVERY_ERROR, new MailjetEvent($message, $email, $e) ); } } }
35. class MembershipRequestHandler { // ... public function handle(MembershipRequest $membershipRequest) { $adherent = $this->factory->createFromMembershipRequest($membershipRequest); $token = AdherentActivationToken::generate($adherent); $this->manager->persist($adherent); $this->manager->persist($token); $this->manager->flush(); $message = AdherentAccountActivationMessage::createFromAdherent( $adherent, $this->generateMembershipActivationUrl($adherent, $token) ); $this->mailjet->sendMessage($message); $e = new AdherentAccountWasCreatedEvent($adherent); $this->dispatch(..., $e); } }
36. February 2017
38. class CommitteeManagementAuthority { // ... function followCommittee(Adherent $adherent, Committee $committee): void { $this->manager->followCommittee($adherent, $committee); if (!$hosts = $this->manager->getCommitteeHosts($committee)) { return; } $this->mailjet->sendMessage(CommitteeNewFollowerMessage::create( $committee, $hosts, $adherent, $this->urlGenerator->getUrl('...', $committee) )); } } Service Layer
39. Service Layer Custom voters src/Committee/Voter ├─ AbstractCommitteeVoter.php ├─ CreateCommitteeVoter.php ├─ FollowCommitteeVoter.php ├─ HostCommitteeVoter.php ├─ ShowCommitteeVoter.php └─ SuperviseCommitteeVoter.php
40. class Geocoder implements GeocoderInterface { // ... public function geocode(string $address): Coordinates { try { $addresses = $this->geocoder->geocode($address); } catch (\Exception $exception) { throw GeocodingException::create($address, $exception); } if (!$geo = $addresses->first()) { throw GeocodingException::create($address); } return new Coordinates( $geo->getLatitude(), $geo->getLongitude() ); } } Geocoding addresses Based on Google Maps
41. class EntityAddressGeocodingSubscriber implements EventSubscriberInterface { // ... public function onCommitteeCreated(CommitteeWasCreatedEvent $event) { $this->updateGeocodableEntity($event->getCommittee()); } private function updateGeocodableEntity(GeoPointInterface $geocodable) { if ($coords = $this->geocode($geocodable->getGeocodableAddress())) { $geocodable->updateCoordinates($coords); $this->manager->flush(); } } } Doctrine listener to update coordinates of any Geocodable entity when it’s saved to the database.
44. trait NearbyTrait { public function getNearbyExpression(): string { return '(6371 * acos(cos(radians(:latitude)) * cos(radians(n.postAddress.latitude)) * cos(radians(n.postAddress.longitude) - radians(:longitude)) + sin(radians(:latitude)) * sin(radians(n.postAddress.latitude))))'; } }
45. trait NearbyTrait { function createNearbyQueryBuilder(Coordinates $coordinates, bool $hidden): { $hidden = $hidden ? 'hidden' : ''; return $this ->createQueryBuilder('n') ->addSelect($this->getNearbyExpression().' as '.$hidden.' dst_between') ->setParameter('latitude', $coordinates->getLatitude()) ->setParameter('longitude', $coordinates->getLongitude()) ->where('n.postAddress.latitude IS NOT NULL') ->andWhere('n.postAddress.longitude IS NOT NULL') ->orderBy('dst_between', 'asc') ; } }
46. /** * @group functional */ class CommitteeControllerTest extends MysqlWebTestCase { // ... function testAnonymousUserIsNotAllowedToFollowCommittee() { $crawler = $this->client->request('GET', ...); $this->assertResponseStatusCode(200, ...); $this->assertFalse($this->seeFollowLink($crawler)); $this->assertFalse($this->seeUnfollowLink($crawler)); $this->assertTrue($this->seeRegisterLink($crawler)); } } Functional Testing PHPUnit
47. Continuous Integration Pull-Requests SensioLabsInsight Style CI Circle CI
48. March 2017
49. Program announced! +6,000 users live +300,000 daily
50. <script type="application/ld+json"> { "@context": "http://schema.org", "@type": "WebSite", "url": "http://en-marche.fr", "name": "En Marche !", "image": "http://en-marche.fr/images/default_sharer.jpg", "description": "Pour ceux qui ... l'Europe.", "funder": { "@type": "Person", "givenName": "Emmanuel", "familyName": "Macron", "jobTitle": "President of France" } } </script> Schema.org JSON LD Metadata in HTML code
51. Optimizing natural indexing in search engines.
52. # app/config/parameters.yml parameters: # Staging configuration env(ENABLE_CANARY): 1 # Production configuration env(ENABLE_CANARY): 0 Feature Flags Env vars
53. class PageController extends Controller { // ... public function ellesMarchentAction() { if (!$this->getParameter('enable_canary'))) { throw $this->createNotFoundException(); } return $this->render('... '); } } Feature Flags
54. Procuration System
55. Asynchronous Tasks use AppBundle\Mailer\EmailTemplate; use OldSound\RabbitMqBundle\RabbitMq\Producer; class MailerProducer extends Producer implements MailerProducerInterface { public function scheduleEmail(EmailTemplate $email): void { $this->publish(json_encode([ 'uuid' => (string) $email->getUuid(), ])); } }
56. // ... abstract class AbstractMailerConsumer extends AbstractConsumer { // ... protected function doExecute(array $data): int { try { if (!$message = $this->repository->findOneByUuid($data['uuid'])) { $this->logger->error('Email not found', $data); return self::MSG_ACK; } if ($delivered = $this->sendEmail($message->getRequestPayloadJson())) { $this->getEmailRepository()->setDelivered($message, $delivered); } return $delivered ? self::MSG_ACK : self::MSG_REJECT_REQUEUE; } catch (...) { ... } } }
57. April 2017
58. Official regulations require candidates to stop proactive propaganda 24h before each ballot.
59. Notifications Committees & events Make the website read-only! Contents publication Emails
60. Who’s trully concerned?
61. https://commons.wikimedia.org/wiki/File%3AMapadefrancia.svg
62. UTC -3 Guyana Saint Pierre and Miquelon UTC -4 St. Martin St. Barthelemy Guadeloupe Martinique UTC -8 Clipperton UTC -9 / UTC -9,5 / UTC -10 French Polynesia https://commons.wikimedia.org/wiki/File%3AMapadefrancia.svg
63. UTC +3 Mayotte Europa Island https://commons.wikimedia.org/wiki/File%3AMapadefrancia.svg UTC +4 Crozet Islands Reunion Glorious Islands Tromelin Island Juan de Nova UTC +5 St. Paul and Amsterdam Kerguelen Islands UTC +10 Adelie Land UTC +10 New Caledonia UTC +12 Wallis and Futuna
64. How to make the platform fullfil the official regulations?
65. {% if not app.request.attributes.get('_campaign_expired') and not user_is_adherent %} <a href="{{ hwi_oauth_login_url('auth') }}" class="btn b__nudge--right-nano"> Connexion </a> <a href="{{ auth_register_url }}" class="btn b__nudge--right"> Inscription </a> {% endif %} Disabling links in Twig
66. class ArticleController extends Controller { /** * @Route( * "/articles/{category}/{page}", * requirements={"category"="\w+", "page"="\d+"}, * defaults={ "_enable_campaign_silence": true } * ) */ public function actualitesAction(...): Response { ... } } Marking a mutable action
67. class CampaignSilenceSubscriber implements EventSubscriberInterface { // ... public function onKernelController(FilterControllerEvent $event) { // ... $expired = $this->processor->isCampaignExpired($request); $request->attributes->set('_campaign_expired', $expired); if (!$expired || $request->attributes->get('_enable_campaign_silence', false)) { return; } $event->setController(function () { return new Response($this->twig->render('campaign_silent.html.twig')); }); } }
68. # app/config/config_prod.yml services: app.mailjet.transport.transactional: class: AppBundle\Mailjet\Transport\MailjetNullTransport arguments: ['@?logger'] public: false app.mailjet.transport.campaign: class: AppBundle\Mailjet\Transport\MailjetNullTransport arguments: ['@?logger'] public: false
69. April 23rd 1st Ballot
70. ~500,000 connections to the website.
71. May 2017
72. # ... http { # ... server { # ... # Block WP Pingback DDoS attacks if ($http_user_agent ~* "WordPress") { return 403; } # ... } } WordPress pingback DDoS attacks
73. class AssetsController extends Controller { /** * @Route("/assets/{path}") * @Method("GET") * @Cache(maxage=900, smaxage=900) */ public function assetAction(string $path, ...) { // ... } } Http caching for static assets
74. May 7th 2nd Ballot
75. ~650,000 connections to the website.
76. Wrap Up
77. Thank you! https://www.flickr.com/photos/enmarchefr/31534490113/ @titouangalopin @hhamon @sensiolabs