Successfully Publishing a Tested, Upgradeable and Documented Open Source Library

Slide 1

Slide 1 text

How to write a safe and reliable Open Source Library

Slide 2

Slide 2 text

Xavier F. Gouchet Senior Software Engineer at Datadog @xgouchet / @datadoghq 2

Slide 3

Slide 3 text

Slide 4

Slide 4 text

0 Introduction 4

Slide 5

Slide 5 text

github.com/DataDog/dd-sdk-android 5 commit 0db48d14dea27ac02e8ddeb9ccf7413feaacc20e Author: Xavier F. Gouchet Date: Mon Nov 4 13:38:30 2019 +0100 Initial drop with base gradle config

Slide 6

Slide 6 text

Repository statistics - 2400+ commits - 3 developers - 20+ contributors - (internal and community) - 30 published versions - 10 libraries 6

Slide 7

Slide 7 text

1 MAKING IT SAFE 7

Slide 8

Slide 8 text

Reliability is key 8 ▪ Library lives in hundreds apps… ▪ … deployed on 100k+ devices

Slide 9

Slide 9 text

Notice potential issues early 9 Compilation CI Code Review Dogfooding Production Cost of fixing a bug Time

Slide 10

Slide 10 text

Ensure the code is safe 10 ▪ objective zero crash ▫ Static Analysis ▫ Code review ▫ *-tests ▪ Real life use cases

Slide 11

Slide 11 text

Static analysis (Detekt) ▪ Forbid throwing exceptions ▪ Forbid specific Kotlin syntax ▫ check ▫ require ▫ !! ▪ List all third party methods throwing exceptions 11

Slide 12

Slide 12 text

Mandatory Code Review ▪ Increase readability ▪ Generate discussion ▫ Data structure ▫ Design Patterns ▫ Third party libraries 12

Slide 13

Slide 13 text

“Testing is doubting!” 13

Slide 14

Slide 14 text

Unit/Integration/End-to-end Testing ▪ Strict policy on the coverage ▪ Deal with flaky test ASAP ▪ Stay confident in our tests 14

Slide 15

Slide 15 text

Real life use cases ▪ Dogfooding in our own apps ▪ Regular tests using OSS apps ▫ Wikipedia ▫ Reddit client ▫ HackerNews client 15

Slide 16

Slide 16 text

Respect privacy 16 ▪ Leverage Sandbox storage / Cache ▪ Add GDPR option ▪ Write doc about Data Collection

Slide 17

Slide 17 text

2 MAKING IT EFFICIENT 17

Slide 18

Slide 18 text

Limit library performance impact 18 ▪ Offload heavy computation to background threads/workers ▪ Track Memory Leaks

Slide 19

Slide 19 text

Monitor library overhead 19 ▪ Compute the impact of the library on ▫ CPU ▫ Memory ▫ Network ▫ …

Slide 20

Slide 20 text

Keep a focus on performance 20 ▪ Track the performance impact for each update

Slide 21

Slide 21 text

3 MAKING IT USABLE 21

Slide 22

Slide 22 text

Opt-in or opt-out by default 22 ▪ Balance between “it works out of the box” and “it doesn’t do anything that I don’t know of” ▪ Onboarding should work in 15 minutes

Slide 23

Slide 23 text

Every thing can be configured ▪ Allow enabling/disabling individual features ▪ Allow tweaking sampling rates, thresholds, … 23

Slide 24

Slide 24 text

Use sensible defaults 24 ▪ New customers don’t need to configure everything right now… ▪ … or ever

Slide 25

Slide 25 text

Communicate ▪ Log any actionable “issue” ▫ configuration error ▫ feature misuse ▫ key internal events ▪ Add actions or documentation links in the message ▪ Provide information about what the library is doing 25

Slide 26

Slide 26 text

4 MAKING IT FUTURE PROOF 26

Slide 27

Slide 27 text

Keep track of API surface 27 ▪ Prevent any API breaking change ▪ Provide new option but keep the old ones working ▫ with @Deprecated ▫ with a LogCat log ▪ Provide migration guide

Slide 28

Slide 28 text

Monitor dependencies 28 ▪ Review changelogs ▪ Be mindful of transitive dependencies

Slide 29

Slide 29 text

Monitor dependencies $ ./gradlew :my-lib:dependencies releaseRuntimeClasspath +- project :my-lib | +- org.jetbrains.kotlin:kotlin-stdlib:1.3.61 | +- com.example.something:core-lib:1.2.0 | | \- org.jetbrains.kotlin:kotlin-stdlib:1.3.61 → 1.5.31 (*) 29

Slide 30

Slide 30 text

Monitor dependencies $ ./gradlew :my-lib:dependencies releaseRuntimeClasspath +- project :my-lib | +- org.jetbrains.kotlin:kotlin-stdlib:1.5.31 | +- com.example.something:core-lib:1.2.0 | | \- org.jetbrains.kotlin:kotlin-stdlib:1.3.61 → 1.5.31 (*) 30

Slide 31

Slide 31 text

Protect experimental features ▪ Use -alpha and -beta releases ▪ Use Kotlin’s opt-in annotations 31

Slide 32

Slide 32 text

Kotlin Opt-in annotations 32 @RequiresOptIn(message = "This API is experimental.") @Retention(AnnotationRetention.BINARY) @Target(AnnotationTarget.CLASS, AnnotationTarget.FUNCTION) annotation class MyCustomOptIn

Slide 33

Slide 33 text

Kotlin Opt-in annotations 33 @MyCustomOptIn class MyExperimentalClass { // … }

Slide 34

Slide 34 text

Kotlin Opt-in annotations 34 @OptIn(MyCustomOptIn::class) fun doSomething { val x = MyExperimentalClass() // … }

Slide 35

Slide 35 text

5 MAKING IT OPEN SOURCE 35

Slide 36

Slide 36 text

Why an Open Source Library 36 ▪ It lives within our customer’s code ▫ Readable ▫ Editable ▪ Allows direct contribution ▪ Keeps the engineering team closer to our users ▪ We also use Open Source every day

Slide 37

Slide 37 text

Contributing.md 37 ▪ Developer environment (IDE, tools, …) ▪ How to build and run the code ▪ How to run the tests / static analysis ▪ How to contribute ▫ Issues ▫ PRs ▪ Coding conventions

Slide 38

Slide 38 text

PR & Issue templates 38 ▪ Dedicated files ▫ .github/ISSUE_TEMPLATE/bug.md ▫ .github/PULL_REQUEST_TEMPLATE.md ▪ Custom checklist for everyone

Slide 39

Slide 39 text

LICENSE 39 ▪ Use an existing license ▫ https://choosealicense.com/ ▪ Apache / MIT ▫ Compatible with your dependencies

Slide 40

Slide 40 text

6 ANDROID GUIDELINES 40

Slide 41

Slide 41 text

AndroidManifest.xml 41

Slide 42

Slide 42 text

build.gradle[.kts] 42 android { minSDK = 19 targetSdk = 31 defaultConfig { aarMetadata { minCompileSdk = 29 } } }

Slide 43

Slide 43 text

build.gradle[.kts] 43 android { // … testFixtures { // uses src/testFixtures enable = true } }

Slide 44

Slide 44 text

Publish to Maven ▪ Maven Central ▪ Custom Maven Server 44

Slide 45

Slide 45 text

Google Play SDK Index 45

Slide 46

Slide 46 text

CONCLUSION 46

Slide 47

Slide 47 text

Keep an Open Mindset 47 ▪ Every mistake is a lesson ▪ Look at other Open Source Libraries ▫ http://github.com/Datadog/dd-sdk-android

Slide 48

Slide 48 text

Thank you! ANY QUESTIONS? @xgouchet @datadoghq 48

Slide 49

Slide 49 text

CREDITS Special thanks to all the people who made and released these awesome resources for free: ▪ Presentation template by SlidesCarnival ▪ Photographs by Unsplash 49