Making your own Code Formatter in Swift

iOSConf SG 2019 (Jan 19, 2019)
https://2019.iosconf.sg

Library: https://github.com/inamiy/SwiftRewriter
Video: https://learntalks.com/2019/01/make-your-own-code-formatter-in-swift-ios-conf-sg-2019/

Transcript

1. Making your ownɹɹɹɹ Code Formatter in Swiftɹɹɹɹɹ 2019/01/19 iOSConf SG 2019 Yasuhiro Inami / @inamiy
4. 2016 → 2019 4 → 14ɹɹ ! developers ɹɹ ɹ ɹ
5. ! 14 developers ! 14 coding styles
6. ! Programming Styles 1. Language (Swift, Objective-C, C++, etc) 2. Typing (Object-oriented, functional) or untyping (oh my!) 3. Architecture (MVC, MVVM, Reactive Programming, etc) 4. Variable / Function Naming 5. Comments / Documentation 6. Whitespaces (tabs V.S. spaces, newline)
9. We need the Law 1. ! Coding Style Guide • https://github.com/raywenderlich/swift-style-guide • https://google.github.io/swift/ 2. " Linter / Formatter • realm/SwiftLint • nicklockwood/SwiftFormat • Both solves syntactic problems
10. SwiftLint / SwiftFormat • realm/SwiftLint • Powered by SourceKit(ten) • Syntax structure = untyped dictionary tree • Some rules are autocorrect-able • nicklockwood/SwiftFormat • Scope-aware, built-in tokenizer • Syntax structure = token array
13. SwiftSyntax https://github.com/apple/swift-syntax
14. SwiftSyntax • Swift bindings for apple/swift's libSyntax • Type-safe, AST-level structured editing library • Includes trivias (whitespaces & comments) • Attaches trivias to leading / trailing of token • Similar to .NET (Roslyn) / TypeScript Syntax API • Internally calls swiftc -frontend -emit-syntax and passes JSON for "C++ to Swift" bridging (in Swift 4.2)
16. Swift AST Explorer https://swift-ast-explorer.kishikawakatsumi.com
17. Q. Where do Swift syntaxes come from?
18. Swift 4.2 Formal grammar About the Language Reference — The Swift Programming Language (Swift 4.2)
19. top-level-declaration ! [statements] statements ! statement [statements] statement ! expression [;] | declaration [;] | loop-statement [;] | branch-statement [;] | labeled-statement [;] | control-transfer-statement [;] | defer-statement [;] | do-statement [;] | compiler-control-statement
20. // Note: Pseudo-code struct TopLevelDeclaration { let statements: [Statement] } // Simple statements, compiler control, and control flow. enum Statement { case expression(Expression, Semicolon?) case declaration(Declaration, Semicolon?) case loop(LoopStatement, Semicolon?) case branch(BranchStatement, Semicolon?) case labeled(LabeledStatement, Semicolon?) case controlTransfer(ControlTransferStatement, Semicolon?) case `defer`(DeferStatement, Semicolon?) case `do`(DoStatement, Semicolon?) case compilerControl(CompilerControlStatement) }
21. declaration ! import-declaration | constant-declaration | variable-declaration | typealias-declaration | function-declaration | enum-declaration | struct-declaration | class-declaration | protocol-declaration | initializer-declaration | deinitializer-declaration | extension-declaration | subscript-declaration | operator-declaration | precedence-group-declaration
22. /// Introduces a new name or construct, e.g. functions, structs, variables. enum Declaration { case `import`(ImportDeclaration) case constant(ConstantDeclaration) case variable(VariableDeclaration) case `typealias`(TypealiasDeclaration) case function(FunctionDeclaration) case `enum`(EnumDeclaration) case `struct`(StructDeclaration) case `class`(ClassDeclaration) case `protocol`(ProtocolDeclaration) case initializer(InitializerDeclaration) case deinitializer(DeinitializerDeclaration) case `extension`(ExtensionDeclaration) case `subscript`(SubscriptDeclaration) case `operator`(OperatorDeclaration) case precedenceGroup(PrecedenceGroupDeclaration) }
23. expression ! [try-operator] prefix-expression [binary-expressions] prefix-expression ! [prefix-operator] postfix-expression | in-out-expression postfix-expression ! primary-expression | function-call-expression | initializer-expression | l binary-expressions ! binary-expression [binary-expressions] binary-expression ! binary-operator prefix-expression | assignment-operator [try-operator] prefix-expression | conditional-operator [try-operator] prefix-expression | type-casting-operator
24. /// Can evaluate & return a value and/or cause a side effect. struct Expression { let tryOperator: TryOperator? let prefixExpression: PrefixExpression let binaryExpressions: [BinaryExpression] } enum PrefixExpression { case prefixOperator(PrefixOperator?, PostfixExpression) case `inout`(InoutExpression) } enum PostfixExpression { case primary(PrimaryExpression) case functionCall(FunctionCallExpression) case initializer(InitializerExpression) ... }
25. whitespace ! whitespace-item [whitespace] whitespace-item ! line-break | comment | multiline-comment | U+0000, U+0009, U+000B, U+000C, or U+0020 line-break ! U+000A | U+000D | U+000D followed by U+000A comment ! // comment-text line-break multiline-comment ! /* multiline-comment-text */ comment-text ! comment-text-item [comment-text] comment-text-item ! Any Unicode scalar value except U+000A or U+000D multiline-comment-text ! multiline-comment-text-item [multiline-comment-text] multiline-comment-text-item ! multiline-comment | comment-text-item | Any Unicode scalar value except /* or */
26. identifier ! identifier-head [identifier-characters] | ` identifier-head [identifier-characters] ` | implicit-parameter-name identifier-list ! identifier | identifier , identifier-list identifier-head ! A-Z | a-z | _ | U+00A8 | U+00AA | ... identifier-character ! 0-9 | U+0300–U+036F | U+1DC0–U+1DFF | ... | identifier-head identifier-characters ! identifier-character [identifier-characters]
27. literal ! numeric-literal | string-literal | boolean-literal | nil-literal numeric-literal ! [-] integer-literal | [-] floating-point-literal boolean-literal ! true | false nil-literal ! nil
29. SwiftSyntax Example
30. iOSConfSG = 2019; ↓ iOSConfSG = 2019
31. open class SemicolonTrimmer: SyntaxRewriter { open override func visit(_ syntax: CodeBlockItemSyntax) -> Syntax { if let nextToken = syntax.nextToken, nextToken.leadingTrivia.contains(where: { $0.isNewline }) || syntax.isLastChild { let noSemicolon = syntax.withSemicolon(nil) return super.visit(noSemicolon) } else { return super.visit(syntax) } } }
32. iOSConfSG = 2019 ↓ iOSConfSG = 2_019
33. open override func visit(_ token: TokenSyntax) -> Syntax { guard case var .integerLiteral(text) = token.tokenKind else { return token } if text.hasPrefix("0x") || text.hasPrefix("0b") || text.hasPrefix("0o") { return token } text = text.replacingOccurrences(of: "_", with: "") let index1 = text.index(after: text.startIndex) var index = text.endIndex while text.formIndex(&index, offsetBy: -3, limitedBy: index1) { text.insert("_", at: index) } return token.withKind(.integerLiteral(text)) }
34. struct Foo { - struct Bar { - struct Baz {} - } } + +extension Foo { + struct Bar {} +} + +extension Foo.Bar { + struct Baz {} +}
35. func foo(_ value: Int?) throws -> Int { - if let value = value { - if value >= 0 { - return value - } else { - throw MyError.negative - } - } else { - throw MyError.none - } + guard let value = value else { throw MyError.none } + guard value >= 0 else { throw MyError.negative } + return value }
36. SwiftSyntax types • SyntaxRewriter / SyntaxVisitor • Trivia (leading / trailing) • Comments & whitespaces are stored in TokenSyntax • SyntaxFactory • Creates a new Syntax nodes • Diagnostic • Data types for showing diagnosed Note & FixIt
37. try! Swift NYC 2017 - Improving Swift Tools with libSyntax apple/swift/lib/Syntax
39. Speeding up SwiftSyntax by using the parser directly - Swift Forums apple/swift#21762 apple/swift-syntax#59
40. SwiftSyntax + Optics
44. If visiting descendants manually... // Get grandchild (2 getters). var rightParen = syntax.parameters.rightParen // Modify grandchild (e.g. changing trivia). rightParen = modify(rightParen) // Set new child. let parameters = syntax.parameters .withRightParen(rightParen) // Set new `syntax` ( ❌ many more code needed if deeply nested) return syntax.withParameters(parameters)
45. Use ! Lens! override func visit(_ syntax: InitializerDeclSyntax) -> DeclSyntax { return _modify(lens: .parameters >>> .rightParen, syntax: syntax) } private func _modify<Whole, Part>( lens: Lens<Whole, Part>, syntax: Whole ) -> Whole where Whole: Syntax, Part: Syntax { let part = lens.getter(syntax) let part2 = modify(part) return lens.setter(syntax, part2) }
46. Lens.parameters >>> Lens.rightParen
49. .wowɹɹ ɹ >>> .suchɹ >>> .nestɹ >>> .muchɹ >>> .deepɹ
50. ! Lens: Functional getter & setter struct Lens<Whole, Part> { let getter: (Whole) -> Part let setter: (Whole, Part) -> Whole static func >>> <Part2>( l: Lens<Whole, Part>, r: Lens<Part, Part2> ) -> Lens<Whole, Part2> { return Lens<Whole, Part2>( getter: { r.getter(l.getter($0)) }, setter: { a, c in l.setter(a, r.setter(l.getter(a), c)) } ) } }
56. ! Prism: Lens for sum types struct Prism<Whole, Part> { let tryGet: (Whole) -> Part? let inject: (Part) -> Whole static func >>> <Part2>( l: Prism<Whole, Part>, r: Prism<Part, Part2> ) -> Prism<Whole, Part2> { return Prism<Whole, Part2>( tryGet: { l.tryGet($0).flatMap(r.tryGet) }, inject: { l.inject(r.inject($0)) } ) } }
58. >>> : (Lens<A, B>, Lens<B, C>) -> Lens<A, C> >>> : (Prism<A, B>, Prism<B, C>) -> Prism<A, C> >>> : (Lens<A, B>, Prism<B, C>) -> ??? >>> : (Prism<A, B>, Lens<B, C>) -> ??? Q. How to compose Lens and Prism together?
59. >>> : (Lens<A, B>, Lens<B, C>) -> Lens<A, C> >>> : (Prism<A, B>, Prism<B, C>) -> Prism<A, C> >>> : (Lens<A, B>, Prism<B, C>) -> ??? >>> : (Prism<A, B>, Lens<B, C>) -> ??? ɹɹɹɹɹˣˣˣɹLift to AffineTraversal!ɹˣˣˣ init(lens:) : (Lens<A, B>) -> AffineTraversal<A, B> init(prism:) : (Prism<A, B>) -> AffineTraversal<A, B> >>> : (AffineTraversal<A, B>, AffineTraversal<B, C>) -> AffineTraversal<A, C>
61. Lens.trailingClosure >>> Prism.some >>> Lens.rightParen
62. ↔ Affine Traversal: Lens + Prism struct AffineTraversal<Whole, Part> { let tryGet: (Whole) -> Part? let setter: (Whole, Part) -> Whole init(lens: Lens<Whole, Part>) { self.init(tryGet: lens.getter, setter: lens.setter) } init(prism: Prism<Whole, Part>) { self.init(tryGet: prism.tryGet, setter: { prism.inject($1) }) } }
63. SwiftRewriter https://github.com/inamiy/SwiftRewriter
64. Recap • SwiftSyntax: Type-safe & lossless AST editing library • Many incoming improvements toward Swift 5.0 or later • Swift AST Explorer: Useful AST visualizer for debugging • Formal grammar: Syntax production rules • Optics: Functional approach to traverse data structure • Let's make your own code formatter using SwiftSyntax!
65. References (Swift AST) • try! Swift NYC 2017 - Improving Swift Tools with libSyntax • try! Swift Tokyo 2018 - AST Meta-programming • apple/swift/lib/Syntax • Speeding up SwiftSyntax by using the parser directly • About the Language Reference (Swift 4.2) • SwiftFormat (Part 1 of 3) - Schibsted Bytes • google/swift (format branch)
66. References (Optics) • Lenses in Swift - Chris Eidhof • Brandon Williams - Lenses in Swift • Lenses and Prisms in Swift: a pragmatic approach | Fun iOS • Lenses, Folds, and Traversals - Edward Kmett • Oleg's gists - Glassery • Profunctor Optics: Modular Data Accessors
67. Thanks! Yasuhiro Inami @inamiy