AppCode under the hood

Each developer sooner or later thinks about how his IDE is arranged. Features such as auto-completion, navigation to the declaration, syntax highlighting, search for call points no longer seem magic when you use them. But the development of these important features is challenging and exciting activity. Don't forget that this is about supporting several languages at once.
AppCode is an IDE for development under macOS and iOS with a very rich history and interesting technological solutions. In this talk, we will find out what problems await you if you decide to write your development environment, how to solve some of them and what undocumented features of Xcode will help you in this difficult task.
Agenda:
1. Introduction and history;
2. How AppCode works with code: code modifications, indexing, SourceKit usage, etc;
3. Building, running, and debugging;
4. Conclusion.

Transcript

1. Aydar Mukhametzyanov, JetBrains AppCode under the hood
2. 👋 Introduction and AppCode history ⌨ Programming language support 🛠 Build, run and debug 👤 Team and processes
3. 👋 Introduction and AppCode history ⌨ Programming language support 🛠 Build, run and debug 👤 Team and processes
4. 7 years in iOS development 2019 г. started at JetBrains (AppCode) Introduction About me
5. Supports Swift, Objective-C/C++, C/C++ IDE for macOS and iOS development IntelliJ platform Introduction About AppCode
6. AppCode history From the idea to the implementation 2008 idea of an IDE for Objective-C 2009 Objective-C parser
7. CLion Objective-C Objective-C++ C/C++ ReSharper C++ AppCode history From idea to implementation
8. 2011 first public version AppCode history From idea to implementation
9. Challenges on the way Xcode 4 Simultaneous releases 🎁 🎁 🎁 AppCode history
10. Interface Builder 2014 UI Designer AppCode history Challenges on the way
12. Interface Builder 2014 UI Designer AppCode history Challenges on the way
13. Highlighting Grammar Full support AppCode history Challenges on the way
14. 2011 first public version 2014 UIDesigner Swift 2008 idea AppCode history
15. 👋 Introduction and AppCode history ⌨ Programming language support 🛠 Build, run and debug 👤 Team and processes
16. Intentions if (b > 0 ? b * 4 : b / 5) != a { //... if a != (b > 0 ? b * 4 : b / 5) { //... Cmd+C, Cmd+V, Cmd+C, Cmd+V
17. Alt+Enter, Enter Intentions
18. if (b > 0 ? b * 4 : b / 5) != a { //... 😰 Intentions
19. Code processing Intro if (b > 0 ? b * 4 : b / 5) != a { //... Text if ( b > 0 ? b * 4 … Lexems Abstract syntax tree if expression if condition code block parenthesised expression != a
20. PSI tree Program Structure Interface (PSI) Code processing
21. PSI tree PsiViewer https://plugins.jetbrains.com/plugin/227-psiviewer Code processing
22. Language grammar BNF Backus–Naur form https://docs.swift.org/swift-book/ReferenceManual/zzSummaryOfTheGrammar.html while-statement ::= while condition-list code-block Code processing
23. GrammarKit https://github.com/JetBrains/Grammar-Kit We update Swift.bnf GrammarKit generates the parser code Parser builds a PSI-tree Code processing
24. Intentions if (b > 0 ? b * 4 : b / 5) != a { //... 1. Update PSI 2. Apply formatting settings 3. Generate text Code processing
25. Inspections class MyObject: AnotherObject { @objc override func foo() {} @objc override func bar() {} } Alt+Enter —> Remove redundant @objc attribute Code processing
26. Inspections class MyObject: AnotherObject { @objc override func foo() {} @objc override func bar() {} } class AnotherObject: NSObject { @objc func foo() {} func bar() {} } Code processing
27. ❓❓❓ 😞 
 PSI class MyObject: AnotherObject { @objc override func foo() {} @objc override func bar() {} } ❓ Code processing Inspections
28. Finding declarations Applications Navigation Highlighting Refactorings Inspections Intentions …
29. Building symbol tables class Person { let name = "John" let surname = "Doe" let age = 42 !// Very important function func greet() { print("Hi \(name)!") } } class Person let name: String let surname: String let age: Int func greet() #-> Void Symbol table Finding declarations
30. Person.swift AppDelegate.swift LoginViewController.swift DataManager.swift DefaultPerson.swift PurchaseManager.swift Item.swift Car.swift Training.swift Symbol tables cache Finding declarations Building symbol tables
31. CoreAudio SwiftUI Accelerate StoreKit Foundation MapKit UIKit AdSupport Modules symbol tables cache import Foundation import StoreKit import CoreAudio import Accelerate import SwiftUI import MapKit import AdSupport … Finding declarations Building symbol tables
32. Symbol tables invalidation class Person { let name = "John" let surname = "Doe" let age = 42 !// Very important function func greet() { print("Hi \(name)!") } } class Person let name: String let surname: String let age: Int func greet() #-> Void Symbol table Finding declarations
33. class Person { let name = "John" let surname = "Doe" let dateOfBirth: Date? = nil !// Very important function func sayHello() { print("Hi \(name)!") } } class Person let name: String let surname: String let age: Int func greet() #-> Void Symbol table 🕸 🕷 Finding declarations Symbol tables invalidation
34. Objective-C/Objective-C++ C/C++ Finding declarations Symbol tables invalidation
35. #include "SwiftASTManager.h" #include "SwiftEditorDiagConsumer.h" #include "SwiftLangSupport.h" !//!!... #ifdef ##... Finding declarations Symbol tables invalidation
36. clang -E main.cpp > preprocessed.cpp Finding declarations Symbol tables invalidation
37. Finding declarations Symbol tables invalidation
38. A C B F D E G H K L N O P Q M D C A Finding declarations Symbol tables invalidation
39. #if DEBUG print("Hi \(name)!") #else print("Hello \(name)!") #endif Finding declarations Symbol tables invalidation
40. Finding type declarations ❓❓❓ class MyObject: AnotherObject { @objc override func foo() {} @objc override func bar() {} } Finding declarations
41. func someFunction() { !// !!... class MyObject: AnotherObject { @objc override func foo() {} @objc override func bar() {} } !// !!... } Finding declarations Finding type declarations
42. class SomeClass { !// !!... func someFunction() {##...} !// !!... } extension SomeClass { !// !!... } Finding declarations Finding type declarations
43. !// !!... class SomeClass {} !// !!... extension SomeClass {} !// !!... MyFile.swift Finding declarations Finding type declarations
44. MyFile.swift ImportantFile.swift Util.swift Network.swift Data.swift etc##... MyModule Finding declarations Finding type declarations
45. import Foundation import StoreKit import CoreAudio import Accelerate import SwiftUI import MapKit import MyLibrary Finding declarations Finding type declarations
46. func bar() {} func bar(_ v: Int = 0) {} bar() Finding declarations Finding function declarations
47. func bar(_ v: String = "") {} private func someFunction() { func bar(_ v: Int) {} bar("hello") } extension Int: ExpressibleByStringLiteral {##...} Finding declarations Finding function declarations
48. Swift and Objective-C Finding declarations
49. Swift and Objective-C class MyObject: AnotherObject {} @interface AnotherObject : NSObject !//!!... @end objc class AnotherObject: NSObject Finding declarations
50. Swift and Objective-C class MyObject: AnotherObject {} @interface AnotherObject : NSObject !//!!... @end objc @interface MyObject : NSObject objc Finding declarations
51. class MyObject: AnotherObject { @objc override func foo() {} @objc override func bar() {} } Finding declarations
52. Indexing##... ❌ Highlighting (partially) ❌ Inspections ✅ Annotations ✅ Navigation ✅ Completion
53. SourceKit usage What is SourceKit SourceKit request response Navigation Highlighting Completion …
54. SourceKit LSP Language Server Protocol SourceKit usage What is SourceKit
55. SourceKit LSP Language Server Protocol IDE Text Editor Tool Another Tool SourceKit usage What is SourceKit
56. AppCode SourceKit SourceKit usage
57. Development SourceKit ❓❓❓ 😞 😞 😞 🤔 SourceKit usage
58. $ SOURCEKIT_LOGGING=3 /Applications/Xcode.app/Contents/MacOS/Xcode >_ SourceKit usage Development
59. SOURCEKIT_LOGGING=3 /Applications/Xcode.app/Contents/MacOS/Xcode 2> logs.txt SourceKit usage Completion
60. 2020-04-02 15:46:10.180 Xcode[94422:5859798] SourceKit-client: [2:request:112131: 7.4167] [31] { key.request: source.request.codecomplete, key.compilerargs: [<...>], key.offset: 0, key.sourcefile: "/Users/aydar.mukhametzyanov/Documents/iOS/EmptyMac/EmptyMac/main.swift", key.sourcetext: "le", key.toolchains: [ "com.apple.dt.toolchain.XcodeDefault" ] } SourceKit usage Completion
61. 2020-04-02 15:46:11.664 Xcode[94422:5859799] SourceKit-client: [2:response:112131: 8.7030] [31] { key.results: [ { key.kind: source.lang.swift.keyword, key.name: "let", key.sourcetext: "let", key.description: "let", key.typename: "", key.context: source.codecompletion.context.none, key.num_bytes_to_erase: 0, key.substructure: { key.nameoffset: 0, key.namelength: 3 } }, <...> ] SourceKit usage Completion
62. key.kind: source.lang.swift.decl.function.method.instance, key.name: "addChild(:)", key.sourcetext: "?.addChild(<#T##childController: UIViewController##UIViewController#>)", key.description: "addChild(childController: UIViewController)", key.typename: "Void", key.modulename: "UIKit.UIViewController", key.context: source.codecompletion.context.thisclass, key.num_bytes_to_erase: 1, key.associated_usrs: "c:objc(cs)UIViewController(im)addChildViewController:" SourceKit usage Completion
63. PSI Swift Parsing Inspections SourceKit Intentions ✅
64. 👋 Introduction and AppCode history ⌨ Programming language support 🛠 Build, run and debug 👤 Team and processes
65. Build xcodebuild $ xcodebuild build -workspace MyApp.xcworkspace -scheme prod >_
66. xcodebuild Compiler arguments calculation Building xcodebuild Xcode AppCode Build
67. Derived Data Derived Data Build
68. Derived Data Derived Data Xcode 9 Build
69. Derived Data Derived Data Xcode 10 “This is prohibited to prevent build state inconsistencies.” Build
70. Derived Data Derived Data Derived Data Xcode 10 https://youtrack.jetbrains.com/issue/OC-17862 Build
71. Run on simulator simctl $ xcrun simctl list ❌
72. AppCode approach CoreSimulator.framework ✅ CoreSimulator.framework/Resources/bin/simctl Run on simulator
73. idb https://github.com/facebook/idb FBSimulatorControl Run on simulator
74. ∅ Run on device
75. MobileDevice.framework Run on device
76. idb https://github.com/facebook/idb FBDeviceControl Run on device
77. Debug On simulator LLDB
78. LLDB 👈 Debug On device
79. /Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/13.4/ DeveloperDiskImage.dmg ☑ Use for Development /usr/bin/debugserver MobileDevice.framework Debug On device
80. debugserver MobileDevice.framework LLDB socket Debug On device
81. 👋 Introduction and AppCode history ⌨ Programming language support 🛠 Build, run and debug 👤 Team and processes
82. How we develop AppCode Kotlin, Java, C++, Objective-C IDEA, AppCode, CLion Monorepo 3 releases per year YouTrack Upsource TeamCity
83. Tests, tests, tests TeamCity duty Push to trunk Release branches for EAP (Early Access Program) How we develop AppCode
84. Who develops AppCode Developers ☕ ☕ + 🍏 🍏 🍏 + ☕
85. Switch from iOS development: personal experience 🍏 🍏 + ☕ Huge project Longer release cycle Big research tasks null 🤔 nil Who develops AppCode
86. QA engineers Manual testing experience Ability to move a project forward independently Who develops AppCode
87. Communication channels Twitter: @appcode https://youtrack.jetbrains.com/issues/OC Who develops AppCode
88. 👤 Team and processes • How we develop AppCode • Who develops AppCode 👋 Introduction and AppCode history • Introduction • AppCode history ⌨ Programming language support • Code processing • Finding declarations • SourceKit usage 🛠 Build, run and debug • Build • Run on simulator • Run on device • Debug
89. Conclusion Making an IDE is hard, but possible AppCode doesn’t contain black magic Everybody can help to make AppCode better
90. Thank you! https://youtrack.jetbrains.com/issues/OC Twitter: @appcode @aydarmukh