Successfully Publishing a Tested, Upgradeable and Documented Open Source Library

You’ve written some code, and you’d like to release it as Open Source. Congratulations!

Publishing an Open Source library goes beyond just releasing your code on Github. New users with different use cases means preparing for the unpredictable. How do you ensure your code is safe and efficient? Is your code easy to read? Do you have a plan for upgrades in the future?

This talk will look at best practices and showcase a checklist to follow when writing and publishing an Open Source Library, from the documentation to ensuring your library works as intended and doesn’t break applications using it.

Transcript

1. How to write
   a safe and reliable
   Open Source Library
   

   View Slide

2. Xavier F. Gouchet
   Senior Software Engineer
   at Datadog
   @xgouchet / @datadoghq
   2
   

   View Slide

3. View Slide

4. 0
   Introduction
   4
   

   View Slide

5. github.com/DataDog/dd-sdk-android
   5
   commit 0db48d14dea27ac02e8ddeb9ccf7413feaacc20e
   Author: Xavier F. Gouchet
   Date: Mon Nov 4 13:38:30 2019 +0100
   Initial drop with base gradle config
   

   View Slide

6. Repository statistics
   - 2400+ commits
   - 3 developers
   - 20+ contributors
   - (internal and community)
   - 30 published versions
   - 10 libraries
   6
   

   View Slide

7. 1
   MAKING IT SAFE
   7
   

   View Slide

8. Reliability is key
   8
   ▪ Library lives in hundreds apps…
   ▪ … deployed on 100k+ devices
   

   View Slide

9. Notice potential issues early
   9
   Compilation CI Code Review Dogfooding Production
   Cost of
   fixing a bug
   Time
   

   View Slide

10. Ensure the code is safe
    10
    ▪ objective zero crash
    ▫ Static Analysis
    ▫ Code review
    ▫ *-tests
    ▪ Real life use cases
    

    View Slide

11. Static analysis (Detekt)
    ▪ Forbid throwing exceptions
    ▪ Forbid specific Kotlin syntax
    ▫ check
    ▫ require
    ▫ !!
    ▪ List all third party methods throwing
    exceptions
    11
    

    View Slide

12. Mandatory Code Review
    ▪ Increase readability
    ▪ Generate discussion
    ▫ Data structure
    ▫ Design Patterns
    ▫ Third party libraries
    12
    

    View Slide

13. “Testing is doubting!”
    13
    

    View Slide

14. Unit/Integration/End-to-end Testing
    ▪ Strict policy on the coverage
    ▪ Deal with flaky test ASAP
    ▪ Stay confident in our tests
    14
    

    View Slide

15. Real life use cases
    ▪ Dogfooding in our own apps
    ▪ Regular tests using OSS apps
    ▫ Wikipedia
    ▫ Reddit client
    ▫ HackerNews client
    15
    

    View Slide

16. Respect privacy
    16
    ▪ Leverage Sandbox storage / Cache
    ▪ Add GDPR option
    ▪ Write doc about Data Collection
    

    View Slide

17. 2
    MAKING IT EFFICIENT
    17
    

    View Slide

18. Limit library performance impact
    18
    ▪ Offload heavy computation to background
    threads/workers
    ▪ Track Memory Leaks
    

    View Slide

19. Monitor library overhead
    19
    ▪ Compute the impact of the library on
    ▫ CPU
    ▫ Memory
    ▫ Network
    ▫ …
    

    View Slide

20. Keep a focus on performance
    20
    ▪ Track the performance impact for each
    update
    

    View Slide

21. 3
    MAKING IT USABLE
    21
    

    View Slide

22. Opt-in or opt-out by default
    22
    ▪ Balance between “it works out of the box”
    and “it doesn’t do anything that I don’t
    know of”
    ▪ Onboarding should work in 15 minutes
    

    View Slide

23. Every thing can be configured
    ▪ Allow enabling/disabling individual
    features
    ▪ Allow tweaking sampling rates,
    thresholds, …
    23
    

    View Slide

24. Use sensible defaults
    24
    ▪ New customers don’t need to configure
    everything right now…
    ▪ … or ever
    

    View Slide

25. Communicate
    ▪ Log any actionable “issue”
    ▫ configuration error
    ▫ feature misuse
    ▫ key internal events
    ▪ Add actions or documentation links in the
    message
    ▪ Provide information about what the
    library is doing
    25
    

    View Slide

26. 4
    MAKING IT FUTURE PROOF
    26
    

    View Slide

27. Keep track of API surface
    27
    ▪ Prevent any API breaking change
    ▪ Provide new option but keep the old ones
    working
    ▫ with @Deprecated
    ▫ with a LogCat log
    ▪ Provide migration guide
    

    View Slide

28. Monitor dependencies
    28
    ▪ Review changelogs
    ▪ Be mindful of transitive dependencies
    

    View Slide

29. Monitor dependencies
    $ ./gradlew :my-lib:dependencies
    releaseRuntimeClasspath
    +- project :my-lib
    | +- org.jetbrains.kotlin:kotlin-stdlib:1.3.61
    | +- com.example.something:core-lib:1.2.0
    | | \- org.jetbrains.kotlin:kotlin-stdlib:1.3.61 → 1.5.31 (*)
    29
    

    View Slide

30. Monitor dependencies
    $ ./gradlew :my-lib:dependencies
    releaseRuntimeClasspath
    +- project :my-lib
    | +- org.jetbrains.kotlin:kotlin-stdlib:1.5.31
    | +- com.example.something:core-lib:1.2.0
    | | \- org.jetbrains.kotlin:kotlin-stdlib:1.3.61 → 1.5.31 (*)
    30
    

    View Slide

31. Protect experimental features
    ▪ Use -alpha and -beta releases
    ▪ Use Kotlin’s opt-in annotations
    31
    

    View Slide

32. Kotlin Opt-in annotations
    32
    @RequiresOptIn(message = "This API is experimental.")
    @Retention(AnnotationRetention.BINARY)
    @Target(AnnotationTarget.CLASS, AnnotationTarget.FUNCTION)
    annotation class MyCustomOptIn
    

    View Slide

33. Kotlin Opt-in annotations
    33
    @MyCustomOptIn
    class MyExperimentalClass {
    // …
    }
    

    View Slide

34. Kotlin Opt-in annotations
    34
    @OptIn(MyCustomOptIn::class)
    fun doSomething {
    val x = MyExperimentalClass()
    // …
    }
    

    View Slide

35. 5
    MAKING IT OPEN SOURCE
    35
    

    View Slide

36. Why an Open Source Library
    36
    ▪ It lives within our customer’s code
    ▫ Readable
    ▫ Editable
    ▪ Allows direct contribution
    ▪ Keeps the engineering team closer to our
    users
    ▪ We also use Open Source every day
    

    View Slide

37. Contributing.md
    37
    ▪ Developer environment (IDE, tools, …)
    ▪ How to build and run the code
    ▪ How to run the tests / static analysis
    ▪ How to contribute
    ▫ Issues
    ▫ PRs
    ▪ Coding conventions
    

    View Slide

38. PR & Issue templates
    38
    ▪ Dedicated files
    ▫ .github/ISSUE_TEMPLATE/bug.md
    ▫ .github/PULL_REQUEST_TEMPLATE.md
    ▪ Custom checklist for everyone
    

    View Slide

39. LICENSE
    39
    ▪ Use an existing license
    ▫ https://choosealicense.com/
    ▪ Apache / MIT
    ▫ Compatible with your dependencies
    

    View Slide

40. 6
    ANDROID GUIDELINES
    40
    

    View Slide

41. AndroidManifest.xml
    41
    
    android:name="android.permission.INTERNET" />
    android:name="android.permission.ACCESS_NETWORK_STATE" />
    
    
    

    View Slide

42. build.gradle[.kts]
    42
    android {
    minSDK = 19
    targetSdk = 31
    defaultConfig {
    aarMetadata {
    minCompileSdk = 29
    }
    }
    }
    

    View Slide

43. build.gradle[.kts]
    43
    android {
    // …
    testFixtures {
    // uses src/testFixtures
    enable = true
    }
    }
    

    View Slide

44. Publish to Maven
    ▪ Maven Central
    ▪ Custom Maven Server
    44
    

    View Slide

45. Google Play SDK Index
    45
    

    View Slide

46. CONCLUSION
    46
    

    View Slide

47. Keep an Open Mindset
    47
    ▪ Every mistake is a lesson
    ▪ Look at other Open Source Libraries
    ▫ http://github.com/Datadog/dd-sdk-android
    

    View Slide

48. Thank you!
    ANY QUESTIONS?
    @xgouchet
    @datadoghq
    48
    

    View Slide

49. CREDITS
    Special thanks to all the people who made and released
    these awesome resources for free:
    ▪ Presentation template by SlidesCarnival
    ▪ Photographs by Unsplash
    49
    

    View Slide