Sharing code among multiple projects: the case of private libraries

As tech companies grow, they often find the need to develop multiple projects or applications in order to better address their challenges. Along with its main app, Deezer has also released Deezer for Creators, Audiobooks by Deezer, Android-TV apps, and (soon) a public SDK. Many other businesses have expanded as well - you can think of Uber with Uber Eats or BlaBlaCar with BlaBlaCar Daily for instance.

These applications share a lot of common code, from authentication to playback engine, to Design System. Is there a way to factorise it?

At Deezer, we chose to publish our code in libraries, which led us to reflect on:
- the content of these libraries,
- the extraction of features and components into a library,
- the organisation of the libraries (e.g. single or multiple libraries, library modules),
- the versioning and the handling of repositories (monorepo vs multiple repos),
- the release of libraries and their integration into projects.

Our return on experience will help you discover possible solutions, better understand the pitfalls to avoid (as we fell into some of them), and see how private libraries could benefit your own organisation.

Transcript

1. Sharing code among multiple projects
   
   
   thanks to private libraries at Deezer
   Jean-Baptiste VINCEY
   
   
   @jbvincey
   

   View Slide

2. View Slide

3. More Deezer apps
   

   View Slide

4. The story of Audiobooks by Deezer
   

   View Slide

5. Authentication
   
   
   Player
   
   
   Request engine
   
   
   Synchronization / cache
   
   
   UI components
   
   
   Business models
   
   
   Business logic
   

   View Slide

6. • Be easy to integrate
   
   • Induce no integration delay
   
   • Allow frequent change / API change
   
   • Be appropriate for apps with various pace
   Shared code should:
   

   View Slide

7. Libraries!
   

   View Slide

8. • Libraries at Deezer
   
   
   Modules
   
   
   Content
   
   
   Organization
   
   
   Distribution
   
   
   Dev process
   
   
   • Documentation
   
   
   • Learnings
   

   View Slide

9. Libraries
   Libraries as modules
   Content
   
   Organization
   
   Distribution
   
   Dev process
   

   View Slide

10. View Slide

11. Sharing modules
    Source code or compiled artifacts?
    VS
    

    View Slide

12. 2017
    Android TV
    developed
    with common
    modules
    2018
    Android TV
    paused
    2019
    Mobile app
    goes on with
    API changes
    on common
    modules
    2020
    Android TV
    update required
    2 months
    just to compile
    again
    

    View Slide

13. Source
    
    code
    Compiled
    artifact
    Easy to integrate ++ -
    No integration
    delay
    ++ -
    Frequent change /
    API change
    ++ +
    Appropriate for
    apps with di
    ff
    .
    timelines
    --- +++
    

    View Slide

14. Libraries
    Libraries as modules
    
    Content
    Organization
    
    Distribution
    
    Dev process
    

    View Slide

15. • Only share what is common
    
    • Keep client speci
    fi
    cities on client side
    
    • No over con
    fi
    guration
    
    • Extend features with speci
    fi
    c behaviours on client side
    
    • Be generic enough (but not too much)
    Any type, but should comply with criteria:
    What sort of content?
    

    View Slide

16. View Slide

17. View Slide

18. ?
    

    View Slide

19. Core auth Design System
    

    View Slide

20. Core auth Design System
    

    View Slide

21. Libraries
    Libraries as modules
    
    Content
    
    Organization
    Distribution
    
    Dev process
    

    View Slide

22. Core Design System
    Features
    

    View Slide

23. com.deezer.designsystem
    

    View Slide

24. libsVersion = "2.3.+"
    
    
    implementation "com.deezer.designsystem:buttons:$libsVersion"
    
    
    implementation "com.deezer.designsystem:colors:$libsVersion"
    
    
    implementation "com.deezer.designsystem:icons:$libsVersion"
    

    View Slide

25. Scripts
    dependencies.gradle
    Scripts
    dependencies.gradle
    Scripts
    dependencies.gradle
    

    View Slide

26. Corelib 2.4
    dependencies.gradle
    Features 1.2
    dependencies.gradle
    Design System 1.8
    dependencies.gradle
    

    View Slide

27. Scripts
    dependencies.gradle
    Scripts
    material:1.5
    
    
    Scripts
    material:1.2
    
    
    

    View Slide

28. Scripts
    dependencies.gradle
    

    View Slide

29. Libraries
    Libraries as modules
    
    Content
    
    Organization
    
    Distribution
    Dev process
    

    View Slide

30. • [major] major change (min Android SDK etc.)
    
    • [minor] default new version
    
    • [patch] urgent bug
    fi
    x
    
    • Unique version for all libs
    → all libs are released together
    
    → one lib can only depend on the same version of an other lib
    [major].[minor].[patch]
    Versioning
    

    View Slide

31. Release git flow
    Release train every 2 weeks
    

    View Slide

32. Automated publication
    

    View Slide

33. Publishing
    Script steps:
    Script
    1: resolve version
    2: publishes Core
    3: publishes Design System
    4: publish Features
    

    View Slide

34. Libraries
    Libraries as modules
    
    Content
    
    Organization
    
    Distribution
    
    Dev process
    

    View Slide

35. • Be easy to integrate
    • Induce no integration delay
    • Allow frequent change / API change
    • Be appropriate for apps with various pace
    Shared code should:
    

    View Slide

36. Git flow
    Deezer app
    Deezer libs
    develop
    libs-snapshot
    master
    stable-2.2 stable-2.3
    2 weeks
    master = future 2.3 master = future 2.4
    Use libs snapshot
    
    (master = future 2.3)
    Use libs snapshot
    
    (master = future 2.4)
    Use libs stable-2.2 Use libs stable-2.3
    

    View Slide

37. Publishing to maven local
    To work with local lib changes on client projects
    Script
    1: resolve version
    2: publish Core
    3: publish Design System
    4: publish Features
    

    View Slide

38. Publishing to maven local
    To work with local lib changes on client projects
    Suffix on every libs package name: _local
    
    Example: com.deezer.designsystem_local
    
    → prevents clash with Nexus
    
    → enforces usage of local version
    

    View Slide

39. project.ext.set("useAndroidLibsLocal", "true")
    local_overrides.gradle
    

    View Slide

40. androidLibsVersion = "2.3.+"
    
    
    androidLibsGroupSuffix = project.hasProperty("useAndroidLibsLocal")
    
    
    && useAndroidLibsLocal == ''true'' ? ''_local'' : ''''
    
    
    designsystemGroup = ''com.deezer.designsystem'' + androidLibsGroupSuffix
    
    
    libraries = [
    
    
    design_buttons: "${designsystemGroup}:buttons:${androidLibsVersion}"
    
    
    ]
    build.gradle
    

    View Slide

41. View Slide

42. 1. Add new icon on Design System locally
    (based on branch master)
    

    View Slide

43. Script
    1. Add new icon on Design System locally (based
    on branch master)
    
    2. Publish libs locally
    

    View Slide

44. 1. Add new icon on Design System locally (based
    on branch master)
    
    2. Publish libs locally
    
    3. Switch to use local libs
    project.ext.set("useAndroidLibsLocal", "true")
    local_overrides.gradle
    

    View Slide

45. 1. Add new icon on Design System locally (based
    on branch master)
    
    2. Publish libs locally
    
    3. Switch to use local libs
    
    4. Develop feature on app locally (based on
    branch libs-snapshot)
    

    View Slide

46. 1. Add new icon on Design System locally (based
    on branch master)
    
    2. Publish libs locally
    
    3. Switch to use local libs
    
    4. Develop feature on app locally (based on
    branch libs-snapshot)
    
    5. Create Pull Request on Design System
    

    View Slide

47. 1. Add new icon on Design System locally (based
    on branch master)
    
    2. Publish libs locally
    
    3. Switch to use local libs
    
    4. Develop feature on app locally (based on
    branch libs-snapshot)
    
    5. Create Pull Request on Design System
    
    6. Create Pull Request on app for feature,
    including the link to the PR on Design
    System

    

    View Slide

48. Script
    1. Add new icon on Design System locally (based
    on branch master)
    
    2. Publish libs locally
    
    3. Switch to use local libs
    
    4. Develop feature on app locally (based on
    branch libs-snapshot)
    
    5. Create Pull Request on Design System
    
    6. Create Pull Request on app for feature,
    including the link to the PR on Design System

    7. Once validated, merge Pull Request on
    Design System and publish snapshot of the
    libs
    

    View Slide

49. 1. Add new icon on Design System locally (based
    on branch master)
    
    2. Publish libs locally
    
    3. Switch to use local libs
    
    4. Develop feature on app locally (based on branch
    libs-snapshot)
    
    5. Create Pull Request on Design System
    
    6. Create Pull Request on app for feature, including
    the link to the PR on Design System

    7. Once validated, merge Pull Request on Design
    System and publish snapshot of the libs
    
    8. Once validate, merge Pull Request on app side

    

    

    View Slide

50. Documentation
    

    View Slide

51. Documentation website based on MkDocs
    

    View Slide

52. Sample apps
    Core Design System Features
    

    View Slide

53. Release notes
    

    View Slide

54. • Min Android SDK

    • Metadata (e.g. *.kotlin_module
    fi
    les)

    • R8 consumer rules

    • Resource visibility

    • Kotlin explicit API mode
    Library development considerations
    

    View Slide

55. Learnings
    

    View Slide

56. View Slide

57. KMP at Deezer
    More on libraries at Deezer
    

    View Slide

58. • Libraries as modules
    
    • Source code: easier to integrate
    
    • Compiled: supports versioning, better for projects with various timelines
    
    • The organisation of libraries is key to foster reusability
    
    • Distribution enables easy integration on client projects
    
    • Git
    fl
    ow and local publication can help reduce the integration delay
    
    • Sample apps and documentation are essential to ease libraries usage
    Takeaways
    

    View Slide

59. Thank you
    
    Jean-Baptiste VINCEY
    
    
    @jbvincey
    Deezer jobs
    Slides
    

    View Slide