Xamarin Source Quest

.NET Fringe Japan 2016

Transcript

1. Xamarin Source Quest
   

   View Slide

2. $ whoami
   Atsushi Eno
   Android dev. @ Xamarin (Microsoft)
   (ex.) Mono dev.
   

   View Slide

3. What is this session for ?
   ◉ You get better understanding on Xamarin/Mono sources
   and become able to
   ○ track Xamarin issues, or
   ○ understand how Xamarin frameworks work
   ◉ Xamarin got OSS-ed, but hard to read them
   ◉ I don't want to keep my knowledge on Xamarin things
   only to myself
   

   View Slide

4. You'd like to read sources
   when....
   ◉ You're facing stack traces while debugging
   ◉ You're wondering how the framework works
   ◉ You're exploring new features
   You'd like to check the sources
   

   View Slide

5. Are Xamarin sources difficult?
   ◉ It is. Because...
   ○ there are many modules and module dependencies
   ○ sources gap between product versions
   ○ for some parts, complete build is impossible (!?)
   

   View Slide

6. Available on github
   ◉ mono/mono
   ◉ mono/monodevelop
   ◉ xamarin/xamarin-macios
   ◉ xamarin/xamarin-android
   ◉ xamarin/Xamarin.Forms
   ◉ many Xamarin Components
   Which sources are available ?
   N/A
   ◉ IDE addins (XS, VS)
   ◉ inspector, workbooks
   ◉ profiler
   ◉ UITest (test cloud)
   

   View Slide

7. Xamarin Source Quest
   Find exact sources
   

   View Slide

8. Getting precise build versions
   Sources are different if versions don't match...
   though, XiOS/XA rev. != OSS repo rev.
   

   View Slide

9. Getting precise build branch
   Xamarin release "cycles" (cycle7, cycle8, cycle9 ...)
   https://developer.xamarin.com/releases/current/
   ◉ xamarin-macios, xamarin-android: cycle
   https://github.com/xamarin/xamarin-macios/branches/cycle8
   ◉ mono, monodevelop: historical versions
   https://github.com/mono/mono/branches/mono-4-8-0-branch
   ◉ Components, Forms: independent versioning
   

   View Slide

10. Getting precise stack trace
    ◉ Know your runtime. It only applies to mono
    ○ Apps on iOS, Android, Mac
    ○ Windows - most unlikely, depends on code.
    ◉ Useful mono runtime options
    ○ --debug to get line numbers.
    ○ -O=-all to get precise line numbers
    ■ disables optimizations
    ■ should not matter, but it often does...
    ● longstanding issue... JIT is difficult
    

    View Slide

11. diagnose: mono --trace
    ◉ AssemblyName
    ◉ T:Full.Type.Name
    ◉ N:Namespace.Name
    ◉ M:Full.Type.Name:Method
    (use colon)
    ◉ E:Full.Exception.Type.Name
    (or "all")
    $ mono --help-trace
    $ mono --trace=System.Xml.Linq,N:System.Xml.XPath,E:all x.exe
    Note that sometimes method call is JITted away...
    

    View Slide

12. Specify runtime options
    ◉ Desktop executable (console, Xamarin.Mac)
    ○ mono [options] {foobar.exe}
    ◉ monodevelop, Xamarin Studio
    ○ MONO_OPTIONS=[options] monodevelop
    ◉ Xamarin.Android
    ○ adb shell setprop debug.mono.env [options]
    ○ adb shell setprop debug.mono.debug 1 (environment)
    ◉ Xamarin.iOS
    ○ 'Run' option on the IDE (MONO_TRACE=... etc.)
    

    View Slide

13. Xamarin Source Quest
    Mono
    

    View Slide

14. building mono from source
    ◉ Prepare *nix environment (macosx, linux, WSL, cygwin)
    ○ Windows binaries: cross build or cygwin
    ◉ git clone --recursive ; ./autogen.sh; make; make install
    ○ --disable-nls on cygwin
    ◉ incremental builds - be careful, it's been broken (a while)
    ○ BK: run "make;make;make" to make sure to build(!?)
    

    View Slide

15. mono source diretories
    mono
    mono [C runtime]
    mcs [C#]
    mcs [compiler] (being deprecated)
    class [libraries]
    tools
    class
    lib
    {assembly, sometimes group}
    {namespace}
    {class}.cs
    Test
    referencesource
    Facades
    

    View Slide

16. framework profiles
    different profiles / set of assemblies in mcs/class/lib:
    ◉ basic - minimum build to build "build" profile
    ◉ monolite - downloadable binaries of "basic" equivalents
    ◉ build - the profile which is used to build everything
    ◉ net4_x - desktop profile
    ◉ xbuild_12, xbuild_14 - profile to run xbuild
    ◉ monotouch, monodroid - if you are building mobile
    basic: built by system (preinstalled) mono / build: built by basic
    

    View Slide

17. dll.sources: the list of sources
    > because it's not in the list? They are per profile.
    corlib.dll.sources -> default
    net_4_x_corlib.dll.sources -> net_4_x
    often #include-d in those files
    > checked #if MONOTOUCH, #if MONODROID, #if MOBILE ?
    "I'm making changes but it does not get compiled!"
    

    View Slide

18. making changes in mono
    ◉ classlibs (./mcs/class)
    ○ `make [PROFILE=xxx]` => mcs/class/lib/xxx
    ○ `make run-test` in the assembly dir for minimum tests.
    ◉ compiler (./mcs/mcs)
    ○ see mcs/tests and mcs/errors (check expected errors)
    ◉ runtime (./mono)
    ○ `make` in mono
    ◉ full tests: `make check` at top-level dir.
    ○ (yet, jenkins build should run for each mono PRs)
    make changes, run tests, create a PR!
    cf. Inside Xamarin #3
    

    View Slide

19. useful mono hacking tips
    ◉ MONO_PATH={topdir}/mcs/class/lib/net_4_x
    {topdir}/mono/runtime/mono-wrapper [YourApp.exe]
    ... to try your local changes without installing.
    ◉ mobile builds: ./configure
    --with-monotouch, --with-monodroid, --with-xammac etc.
    ◉ Mono framework assemblies for mobiles can be copied to:
    ○ $(installed_dir)/lib/xbuild-frameworks/{product}/v1.0
    ■ `Facades` subdir too.
    

    View Slide

20. Xamarin Source Quest
    MSBuild quick tour
    

    View Slide

21. MSBuild: .NET build engine
    ◉ Why MSBuild now?
    ○ Xamarin Platform "SDK" = console dev. tools and Fx.
    ○ xamarin-android, xamarin-macios and Xamarin.Forms
    depend on MSBuild system.
    ■ like Android does on Gradle, Apple does no xcodebuild
    ◉ xbuild on Mono so far (`msbuild` in the future)
    ◉ XA, XiOS, XM and XF extend each set of MSBuild tasks.
    

    View Slide

22. MSBuild project files
    users have *.csproj:
    
    
    
    

    View Slide

23. MSBuild tasks and targets
    Fx. devs maintain *.targets:
    custom s: xbuild /t:SignAndroidPackage
    property overrides: $(BuildDependsOn), ...
    custom properties: xbuild /p:XamMacArch=x86_64
    List of Xamarin targets & props ("Build Process")
    Tasks DLLs: should contain Task implementation classses.
    *.targets and *.dlls: placed in a subdir under $(MSBuildExtensionsPath)
    Task Writing: https://msdn.microsoft.com/en-us/library/t9883dzc.aspx
    

    View Slide

24. Xamarin Source Quest
    Apple Platforms
    

    View Slide

25. Xamarin.iOS / Xamarin.Mac
    What are they?
    ◉ Framework assemblies (objc interop with iOS, macOS etc.)
    ◉ native runtime "libmonotouch"
    ◉ MSBuild extensions
    ◉ App build toolchains: mtouch etc.
    ◉ Binding build toolchains: btouch etc. Objective-sharpie
    ◉ Visual Studio/MonoDevelop addins to support
    ◉ Inspector, Profiler etc.
    Inside Xamarin (5) (Xamarin.iOSのソフトウェア構成要素) 10/4
    

    View Slide

26. xamarin-macios: frameworks
    configure && make && make install (or install-system)
    => built outputs: _ios-build and _mac-build
    Xamarin.iOS.sln, Xamarin.Mac.sln : framework
    src: iOS/watchOS/tvOS/macOS assemblies, and btouch etc.
    tools: packaging tools (mtouch, mmp) etc.
    runtime: libmonotouch, using external/(watch-)mono
    

    View Slide

27. xamarin-macios: build tools
    ◉ might be helpful when investigating build failures...
    ○ Build logging: diagnostic (or, xbuild /v:diag)
    ◉ mtouch (Xamarin.iOS.sln), mmp (Xamarin.Mac.sln)
    ○ the app build tool - linking, AOT, packaging, signing
    ○ IDEs also do those jobs too (e.g. mac codesign)
    ◉ MSBuild Tasks (msbuild/Xamarin.MacDev.Tasks.sln)
    ○ msbuild/Xamarin.{iOS|Mac}.Tasks - apps
    ○ Xamarin.MacDev (external/Xamarin.MacDev)
    ■ .NET API to manipulate SDKs
    ○ msbuild/Xamarin.ObjCBinding.Tasks - bindings
    

    View Slide

28. Xamarin Source Quest
    Xamarin.Android
    

    View Slide

29. Xamarin.Android
    What are they?
    ◉ Framework assemblies: Mono.Android.dll etc.
    ◉ native runtime "libmonodroid"
    ◉ MSBuild extensions
    ◉ Binding build toolchains: generator etc.
    ◉ Debugging support (shared runtime, Xamarin.Android.Debugging.Tasks.dll)
    ◉ Visual Studio/MonoDevelop addins to support
    ◉ Inspector, Profiler etc.
    grimoire of Android
    オープンソース化された Xamarin.Androidの概要
    Inside Xamarin #7
    

    View Slide

30. Xamarin.Android platform
    make prepare & make
    all in Xamarin.Android.sln (but not really editable...)
    ◉ src/Mono.Android (Mono.Android.dll)
    ○ (kind of) standard binding project
    ○ partly hand-written, API XML prebuilt and "api-merge"d
    ○ lots of metadata fixup in .csv files
    ◉ external/Java.Interop/src/Java.Interop (Java.Interop.dll)
    ○ JNIEnv and co.
    ○ *may* work with desktop Java
    

    View Slide

31. Java Android Platform
    ◉ investigate underlying Android Fx behavior
    ○ platform/frameworks/base
    ■ android.googlesource.com (original)
    ■ github/Android (mirrors)
    ■ tools.oesf.biz (OESF Android code search)
    ○ android-sdk-[windows|macosx|linux]/sources
    

    View Slide

32. Android MSBuild Tasks
    MSBuild Tasks (src/Xamarin.Android.Build.Tasks)
    ◉ Xamarin.Android.Common.targets for App projects
    ◉ tasks: GenerateResourceDesigner, LinkAssemblies, CompileToDalvik etc.
    ◉ remember: no debugging support (no Fast Deployment)
    ◉ Xamarin.Android.Bindings.targets for Java binding projects
    

    View Slide

33. Android binding generator
    in java.interop:
    ◉ converts *.jar/*.aar -> *.xml -> *.cs -> *.dll
    ◉ API parsing:
    ○ ~cycle8: "jar2xml" (not in OSS repo)
    ○ cycle9~: "class-parse"
    ■ src/Xamarin.Android.Tools.Bytecode
    ■ src/Xamarin.Android.Tools.ApiXmlAdjuster
    ◉ API XML fixup and C# generation: tools/generator
    

    View Slide

34. Xamarin Source Quest
    Xamarin Libraries
    

    View Slide

35. XamarinComponents
    Start from github/xamarin/XamarinComponents...
    ◉ AndroidSupportComponents
    ○ branches per m2repository rev.
    ◉ GooglePlayServicesComponents
    ○ (ditto)
    ◉ ...
    mostly normal iOS/Android libs.
    

    View Slide

36. Xamarin.Forms
    https://github.com/xamarin/Xamarin.Forms/
    Basically a set of mobile libraries
    and MSBuild tasks (nupkg-ed) to run xamlg, xamlc
    Xamarin.Forms.Core
    Xamarin.Forms.Platform.*
    Xamarin.Forms.Xaml - its own parser impl.
    

    View Slide

37. Xamarin Source Quest
    Case Study #1
    

    View Slide

38. Fixing Xamarin.Forms async
    

    View Slide

39. tracking sources
    If the current thread is the UI thread, then
    the action is executed immediately. If the
    current thread is not the UI thread, the
    action is posted to the event queue of the
    UI thread.
    Xamarin.Forms.Core/Device.cs
    Xamarin.Forms.Platform.Android/Forms.cs
    android-sdk-*/sources/android-24/android/app/Activity.java
    

    View Slide

40. creating a PR
    

    View Slide

41. Xamarin Source Quest
    Case Study #2
    

    View Slide

42. Where is ur HttpClient from?
    Is Xamarin affected?
    mono/mcs/class/System.Net.Http
    System.Net.HttpClient.dll.sources
    says
    System.Net.Http/HttpClient.cs
    (NOT referencesource)
    

    View Slide

43. Is mono HttpClient alright?
    HttpClient.Dispose() -> HttpMessageInvoker.Dispose() ->
    {HttpMessageHandler implementation}.Dispose()
    HttpClient() ... #if !XAMARIN_MODERN in HttpClient.cs
    

    View Slide

44. Is Android HttpClient alright?
    HttpClient.Android.cs ...GetDefaultHandler()
    ...Wat?
    Type type = Type.GetType("Android.Runtime.AndroidEnvironment,
    Mono.Android");
    ...
    MethodInfo method = type.GetMethod ("GetHttpMessageHandler",
    BindingFlags.Static | BindingFlags.NonPublic);
    ...
    object ret = method.Invoke (null, null);
    if (ret == null)
    return GetFallback ("Xamarin.Android returned no custom HttpClientHandler");
    

    View Slide

45. Is Android HttpClient alright?
    github/xamarin/xamarin-android...
    AndroidEnvironment.GetHttpMessageHandler()
    ... returns null (by default)
    default HttpClientHandler's Dispose():
    if (disposing && !disposed) {
    Volatile.Write (ref disposed, true);
    ServicePointManager.CloseConnectionGroup (connectionGroupName);
    }
    

    View Slide

46. How about Mac/iOS?
    (homework!)
    

    View Slide

47. Xamarin Source Quest
    How to contribute
    

    View Slide

48. Many ways to contribute
    It's not just code
    ◉ code (github)
    ○ bugfixes
    ○ implement a new feature
    ◉ bugzilla.xamarin.com
    ○ github issues - second citizens
    ◉ components/bindings, Xam plugins, IDE addins
    ◉ other general stuff (blogs, writing apps, libs, meetups...)
    

    View Slide

49. How to contribute to repos
    ◉ github flow: send pull requests
    ○ git branch [bname]
    ○ git checkout [bname]
    ○
    ○ git commit
    ○ fork on github
    ○ git remote add [youralias] [forkURL]
    ○ git push [youralias] [bname]
    ○ open repo web and create PR (you'll be navigated)
    ◉ MS CLA (contributors license agreement) is required
    

    View Slide

50. coding conventions
    ◉ depends on project
    ○ Mostly mono coding guidelines rule
    ○ Xamarin.Forms follows VS style
    ○ Xamarin Studio has code formats for both
    ◉ Adding tests for better contribution
    ○ We use NUnit almost everywhere
    ○ NUnit is similar to MSTest / xunit
    

    View Slide

51. questions?
    not now, but I'll be around
    Thanks
    

    View Slide