Decathlon Sport Meeting - Grails, a new sport discipline - Greach'15

http://greachconf.com/speakers/antonio-de-la-torre-alonso-torres-decathlon-sportmeeting-sports-a-new-grails-discipline/

Talk given in
Greach 2015
2015 April 10th
Madrid

Speakers:
Antonio de la Torre @adelatorrefoss
Alonso Torres @alotor

Abstract:

Decathlon’s mission is to make sport accessible to more people. Decathlon SportMeeting, its new social network, was created to take this one step further, allowing everyone to find people who share their sport and their passion.

DSM was defined from scratch to support the actual traffic with more than 100k registered users, 1000 active sport proposals for more than 30 sports.

This web platform is entirely built with Groovy & Grails but there are also applications in Android and iOS that use its RESTful API. Along the development process several plugins were created and open-sourced to the community.

In this talk Kaleidos will explain how the development of this platform was, some of the technical decisions that were made, lessons learned, pitfalls or how the infrastructure has been evolving for almost 3 years, and much more.

Transcript

1. Decathlon Sport Meeting Alonso Torres, DEVELOPER Sports, a new Grails discipline Antonio de la Torre, DEVELOPER Kaleidos
2. Introduction
3. What’s Decathlon Sport Meeting? Intro
4. Intro Why want Decathlon to develop a Social Network? • Decathlon’s mission is to make sports widely available • Sport Meeting was created to take this one step further
5. Intro Who developed it? Kaleidos
6. What did it contributed to the community? • 3 Pull Request to Grails GRAILS-11647, GRAILS-11323, GRAILS-10763 • 6 open sourced plugins Intro
7. Intro How is DSM? 100k registered users 1000 active sport proposals 35 sports
8. Development Decisions «The risk of a wrong decision is preferable to the terror of indecision.» Maimónides (12th c.)
9. Development decisions Divided in three applications • WEB, JOBS, API and a plugin with business logic and common code • CORE JOBS WEB API CORE
10. Development decisions Command Objects • Responsibility • Validate objects ◦ accessing model to validate data
11. Development decisions Command Objects Check if is a type and a valid domain object
12. Development decisions Tests • Start with Integration tests << due to odd behaviour • and moving to Unit tests << less dependency
13. Development decisions Asynchronous processes and background jobs • Asynchronous processes (platform-core) ◦ Immediate email sending ◦ Push notifications to mobile devices
14. Development decisions Asynchronous processes and background jobs • Background jobs (quartz) ◦ Quartz scheduler library ◦ Tasks that are processed overnight ▪ Proposals clean up ▪ Mailing with interesting stuff (users, proposals)
15. API Decisions
16. API decisions Trait with helpers to create responses: 4XX
17. API decisions Command Objects accepts input JSON < Magic happens
18. API decisions Command Objects accepts input JSON < Magic happens
19. API decisions Command Objects accepts input JSON < Magic happens Binding Lists
20. API decisions Data Oriented Design (RESTful) over Screen Oriented Design
21. Infrastructure decisions
22. Infrastructure decisions PostgreSQL as our main database • Open-Source and very mature • Highly scalability • Widely used
23. Infrastructure decisions MongoDB as a secondary storage for denormalized data • Optimize read and searches by storing documents • Filter, sorting, querying • Highly availability
24. Infrastructure decisions MongoDB to store statistics and usage data • Fire and forget model • Map-reduce to extract the information
25. Infrastructure decisions MongoDB for geolocalized data • Great support of geolocalization • Aggregation pipeline
26. Infrastructure decisions Terracotta as a Hibernate second-level cache and HTTP sessions • Dual licensing (community and commercial) • Problems scaling (on the community product) • Decided to change to Redis
27. Infrastructure decisions Releases upload to a Nexus repository • Upload all the software revisions • “release plugin” can manage this • Could be easily automated for a continuous delivery
28. Awesome Tools «If I have seen further it is only by standing on the shoulders of giants.» Isaac Newton
29. Awesome tools Java Melody Plugin • Allows monitoring of requests and JVM state • Saved us in several occasions
31. Awesome tools Build Test Data Plugin • Creates complex objects from your constraints
32. Awesome tools Build Test Data Plugin • Creates complex objects from your constraints
33. Awesome tools Build Test Data Plugin • Creates complex objects from your constraints Check constraints and relationships and creates the necessary objects
34. PostgresSQL Extensions plugin Awesome tools • Disclaimer: created by @ilopmar & @alotor inside Kaleidos • Provides native access to PSQL native types: HSTORE, Arrays, JSON...
35. PostgresSQL Extensions plugin Awesome tools
36. Awesome tools Platform Core plugin • Asynchronous task is a bliss • Maintained by the Grails-core team
37. Awesome tools Platform Core plugin
38. Know your tools
39. Awesome tools GPars • Easy parallelization inside the Groovy core • Provides DSL’s and an easy API to parallelize expensive tasks • Sometimes difficult to work with GORM
40. Awesome tools GPars
41. Awesome tools GPars Five threads at the same time
42. Awesome tools GPars we need a transaction within the new thread
43. Awesome tools Groovy Metaprogramming • Sometimes the magic is fun :) • We developed a system with metaprogramming to easily create new types of notification
44. Awesome tools
45. Awesome tools Grails & Spring • Problem: we wanted different implementations in development and “real” environments • Examples: notifications, file storage...
46. Awesome tools
47. Awesome tools Inject a generic “fileService”
48. Awesome tools Alias the necessary bean per environment
49. Difficult Challenges
50. Difficult Challenges Migration Grails 2.1.5 → 2.4.4
51. Difficult Challenges Migration Grails 2.1.5 → 2.4.4 • Upgrade Grails and it’s dependencies • Package changes, API changes… • DataBinding and Command objects changes
52. Difficult Challenges URL’s internationalization • Grails doesn’t support out-of-the-box multi-language URL’s
53. Difficult Challenges Tests pollution • With 1500+ tests if you’re polluting one of your tests is a big issue • If this happens to you check: by @tednaleid http://bit.ly/1FlJs75
54. Difficult Challenges API Documentation • There is no easy way to keep an up-to-date API documentation with Grails REST support • Our solution: the documentation as GSP’s allow us to reuse certain common part’s (like messages structure)
55. Difficult Challenges MongoDB Plugin • Mongodb’s GORM plugin had unexpected behaviour • We finally decided to use the low-level API when accessing MongoDB
56. Going Forward
57. Going forward... • Currently the application is only available for spanish users • Decathlon is present in 27 countries Travel abroad!
58. Going forward... • Decathlon Sport Meeting will be opening their API • We hope to create an application ecosystem around the social network Opening the API
59. Going forward... • First application using DSM API • Was done during the ΠWEEK Sport Spot
60. Going forward...
61. See you on track!