A comprehensive guide to tracker protection on Android

The vast majority of mobile applications leverage the use of third party libraries. Besides their main purpose, many of them will also collect and send your private information, using your data for their own benefit. For the past 18 months, we've been working on a solution that blocks all those trackers from any app in your device. This talk describes in detail the technology behind it, what Android APIs are used and lessons learnt through the whole process.

Transcript

1. Comprehensive guide to tracking protection on Android David González - Software Engineer DuckDuckGo Øredev - 8 Nov 2022
2. Agenda 1. What is a tracker? 2. Web vs App trackers 3. Why should I care? 4. How to block them
3. Did you know that 90% of Websites and apps are tracking you?
4. What is a tracker?
5. Piece of software whose task is to gather information on the person using the application or website
6. Types of trackers 1. Crash reporters 2. Analytics
7. Types of trackers 1. Crash reporters 2. Analytics 3. Profiling
8. Types of trackers 1. Crash reporters 2. Analytics 3. Profiling 4. Identification
9. Types of trackers 1. Crash reporters 2. Analytics 3. Profiling 4. Identification 5. Ads
10. Types of trackers 1. Crash reporters 2. Analytics 3. Profiling 4. Identification 5. Ads 6. Location
11. Web trackers
13. App trackers
15. Why should I care?
18. (https://www.washingtonpost.com/technology/2022/09/22/health-apps-privacy/)
19. Knowing what to block
20. T R A C K E R R A D A R • DuckDuckGo Tracker Radar is a data set about trackers that is automatically generated and maintained through continuous crawling and analysis. • This data set is publicly available in (https:// github.com/duckduckgo/tracker-radar) to use for research and for generating tracker block lists. And, the code behind it is open source
21. Generating a blocklist for Android 21 • Top 300 apps from https://androidrank.org/ • Install root certi fi cate • Start using apps • Record tra ff i c • Looks for inputs
22. Blocking web trackers
23. Block web trackers class BrowserWebViewClient() : WebViewClient() { override fun shouldInterceptRequest( webView: WebView, request: WebResourceRequest ) : WebResourceResponse? { private fun blockRequest( trackingEvent: TrackingEvent, request: WebResourceRequest, webViewClientListener: WebViewClientListener? ) : WebResourceResponse { trackingEvent.surrogateId ?. let { surrogateId -> val surrogate = resourceSurrogates.get(surrogateId) if (surrogate.responseAvailable) { Timber.d("Surrogate found for ${request.url}") webViewClientListener ?. surrogateDetected(surrogate) return WebResourceResponse(surrogate.mimeType, "UTF-8", surrogate.jsFunction.byteInputStream()) } } Timber.d("Blocking request ${request.url}") privacyProtectionCountDao.incrementBlockedTrackerCount() return WebResourceResponse(null, null, null) }
24. Block web trackers class BrowserWebViewClient() : WebViewClient() { override fun shouldInterceptRequest( webView: WebView, request: WebResourceRequest ) : WebResourceResponse? { private fun blockRequest( trackingEvent: TrackingEvent, request: WebResourceRequest, webViewClientListener: WebViewClientListener? ) : WebResourceResponse { trackingEvent.surrogateId ?. let { surrogateId -> val surrogate = resourceSurrogates.get(surrogateId) if (surrogate.responseAvailable) { Timber.d("Surrogate found for ${request.url}") webViewClientListener ?. surrogateDetected(surrogate) return WebResourceResponse(surrogate.mimeType, "UTF-8", surrogate.jsFunction.byteInputStream()) } } Timber.d("Blocking request ${request.url}") privacyProtectionCountDao.incrementBlockedTrackerCount() return WebResourceResponse(null, null, null) }
25. Block web trackers class BrowserWebViewClient() : WebViewClient() { private fun blockRequest( trackingEvent: TrackingEvent, request: WebResourceRequest, webViewClientListener: WebViewClientListener? ) : WebResourceResponse { trackingEvent.surrogateId ?. let { surrogateId -> val surrogate = resourceSurrogates.get(surrogateId) if (surrogate.responseAvailable) { Timber.d("Surrogate found for ${request.url}") webViewClientListener ?. surrogateDetected(surrogate) return WebResourceResponse(surrogate.mimeType, "UTF-8", surrogate.jsFunction.byteInputStream()) } } Timber.d("Blocking request ${request.url}") privacyProtectionCountDao.incrementBlockedTrackerCount() return WebResourceResponse(null, null, null) } }
26. Block web trackers class BrowserWebViewClient() : WebViewClient() { private fun blockRequest( trackingEvent: TrackingEvent, request: WebResourceRequest, webViewClientListener: WebViewClientListener? ) : WebResourceResponse { trackingEvent.surrogateId ?. let { surrogateId -> val surrogate = resourceSurrogates.get(surrogateId) if (surrogate.responseAvailable) { Timber.d("Surrogate found for ${request.url}") webViewClientListener ?. surrogateDetected(surrogate) return WebResourceResponse(surrogate.mimeType, "UTF-8", surrogate.jsFunction.byteInputStream()) } } Timber.d("Blocking request ${request.url}") privacyProtectionCountDao.incrementBlockedTrackerCount() return WebResourceResponse(null, null, null) } }
27. Block web trackers class BrowserWebViewClient() : WebViewClient() { private fun blockRequest( trackingEvent: TrackingEvent, request: WebResourceRequest, webViewClientListener: WebViewClientListener? ) : WebResourceResponse { trackingEvent.surrogateId ?. let { surrogateId -> val surrogate = resourceSurrogates.get(surrogateId) if (surrogate.responseAvailable) { Timber.d("Surrogate found for ${request.url}") webViewClientListener ?. surrogateDetected(surrogate) return WebResourceResponse(surrogate.mimeType, "UTF-8", surrogate.jsFunction.byteInputStream()) } } Timber.d("Blocking request ${request.url}") privacyProtectionCountDao.incrementBlockedTrackerCount() return WebResourceResponse(null, null, null) } }
28. Block web trackers class BrowserWebViewClient() : WebViewClient() { private fun blockRequest( trackingEvent: TrackingEvent, request: WebResourceRequest, webViewClientListener: WebViewClientListener? ) : WebResourceResponse { trackingEvent.surrogateId ?. let { surrogateId -> val surrogate = resourceSurrogates.get(surrogateId) if (surrogate.responseAvailable) { Timber.d("Surrogate found for ${request.url}") webViewClientListener ?. surrogateDetected(surrogate) return WebResourceResponse(surrogate.mimeType, "UTF-8", surrogate.jsFunction.byteInputStream()) } } Timber.d("Blocking request ${request.url}") privacyProtectionCountDao.incrementBlockedTrackerCount() return WebResourceResponse(null, null, null) } }
29. Blocking third party cookies
30. Block third party cookies private suspend fun processThirdPartyCookiesSetting( webView: WebView, uri: Uri ) { val host = uri.host ?: return val domain = authCookiesAllowedDomainsRepository.getDomain(host) withContext(dispatchers.main()) { if (domain != null && hasUserIdCookie()) { Timber.d("Cookies enabled for $uri") cookieManagerProvider.get().setAcceptThirdPartyCookies(webView, true) } else { Timber.d("Cookies disabled for $uri") cookieManagerProvider.get().setAcceptThirdPartyCookies(webView, false) } domain ?. let { deleteHost(it) } } }
31. Block third party cookies private suspend fun processThirdPartyCookiesSetting( webView: WebView, uri: Uri ) { val host = uri.host ?: return val domain = authCookiesAllowedDomainsRepository.getDomain(host) withContext(dispatchers.main()) { if (domain != null && hasUserIdCookie()) { Timber.d("Cookies enabled for $uri") cookieManagerProvider.get().setAcceptThirdPartyCookies(webView, true) } else { Timber.d("Cookies disabled for $uri") cookieManagerProvider.get().setAcceptThirdPartyCookies(webView, false) } domain ?. let { deleteHost(it) } } }
32. Block third party cookies private suspend fun processThirdPartyCookiesSetting( webView: WebView, uri: Uri ) { val host = uri.host ?: return val domain = authCookiesAllowedDomainsRepository.getDomain(host) withContext(dispatchers.main()) { if (domain != null && hasUserIdCookie()) { Timber.d("Cookies enabled for $uri") cookieManagerProvider.get().setAcceptThirdPartyCookies(webView, true) } else { Timber.d("Cookies disabled for $uri") cookieManagerProvider.get().setAcceptThirdPartyCookies(webView, false) } domain ?. let { deleteHost(it) } } }
34. App Tracking Protection
35. Identify calling app private fun getPackageIdForUid(uid: Int) : String { val packages: Array<String>? try { packages = packageManager.getPackagesForUid(uid) } catch (e: SecurityException) { Timber.e(e, "Failed to get package ID for UID : $uid due to security violation.") return "unknown" } if (packages.isNullOrEmpty()) { Timber.w("Failed to get package ID for UID : $uid") return "unknown" } return packages.f i rst() }
36. Identify calling app private fun getPackageIdForUid(uid: Int) : String { val packages: Array<String>? try { packages = packageManager.getPackagesForUid(uid) } catch (e: SecurityException) { Timber.e(e, "Failed to get package ID for UID : $uid due to security violation.") return "unknown" } if (packages.isNullOrEmpty()) { Timber.w("Failed to get package ID for UID : $uid") return "unknown" } return packages.f i rst() }
37. Blocking an app tracker
38. Blocking an app tracker // Called from native code private fun isAddressAllowed(packet: Packet) : Allowed? { packet.allowed = (callback.get() ?. isAddressBlocked(packet.toAddressRR()) == false) return if (packet.allowed) Allowed() else null }
39. Blocking an app tracker /** * Called by the VPN network to know if a particular IP address is blocked or not. * This can be combined with the [onDnsResolved] callback, to get the hostname of the * [addressRR] and then decide whether that hostname should be blocked or not * @param addressRR is the address record */ fun isAddressBlocked(addressRR : AddressRR) : Boolean
40. Blocking an app tracker override fun isAddressBlocked(addressRR : AddressRR) : Boolean { val hostname = addressLookupLruCache[addressRR.address] ?: return false val domainAllowed = shouldAllowDomain(hostname, addressRR.uid) return !domainAllowed }
41. Blocking an app tracker private fun shouldAllowDomain(name: String, uid: Int) : Boolean { val packageId = getPackageIdForUid(uid) val type = appTrackerRepository.f i ndTracker(name, packageId) if (type is AppTrackerType.ThirdParty && !isTrackerInExceptionRules(packageId = packageId, hostname = name)) { val trackingApp = appNamesCache[packageId] ?: appNameResolver.getAppNameForPackageId(packageId) // if the app name is unknown, do not block if (trackingApp.isUnknown()) return true VpnTracker( trackerCompanyId = type.tracker.trackerCompanyId, domain = type.tracker.hostname, trackingApp = TrackingApp(trackingApp.packageId, trackingApp.appName) ).run { appTrackerRecorder.insertTracker(this) } return false } return true }
42. Blocking an app tracker private fun shouldAllowDomain(name: String, uid: Int) : Boolean { val packageId = getPackageIdForUid(uid) val type = appTrackerRepository.f i ndTracker(name, packageId) if (type is AppTrackerType.ThirdParty && !isTrackerInExceptionRules(packageId = packageId, hostname = name)) { val trackingApp = appNamesCache[packageId] ?: appNameResolver.getAppNameForPackageId(packageId) // if the app name is unknown, do not block if (trackingApp.isUnknown()) return true VpnTracker( trackerCompanyId = type.tracker.trackerCompanyId, domain = type.tracker.hostname, trackingApp = TrackingApp(trackingApp.packageId, trackingApp.appName) ).run { appTrackerRecorder.insertTracker(this) } return false } return true }
43. Blocking an app tracker private fun shouldAllowDomain(name: String, uid: Int) : Boolean { val packageId = getPackageIdForUid(uid) val type = appTrackerRepository.f i ndTracker(name, packageId) if (type is AppTrackerType.ThirdParty && !isTrackerInExceptionRules(packageId = packageId, hostname = name)) { val trackingApp = appNamesCache[packageId] ?: appNameResolver.getAppNameForPackageId(packageId) // if the app name is unknown, do not block if (trackingApp.isUnknown()) return true VpnTracker( trackerCompanyId = type.tracker.trackerCompanyId, domain = type.tracker.hostname, trackingApp = TrackingApp(trackingApp.packageId, trackingApp.appName) ).run { appTrackerRecorder.insertTracker(this) } return false } return true }
45. App breakage
47. github.com/duckduckgo/Android
48. Thank you