Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Making your own Code Formatter in Swift

Making your own Code Formatter in Swift

Yasuhiro Inami

January 19, 2019
Tweet

More Decks by Yasuhiro Inami

Other Decks in Programming

Transcript

  1. ! Programming Styles 1. Language (Swift, Objective-C, C++, etc) 2.

    Typing (Object-oriented, functional) or untyping (oh my!) 3. Architecture (MVC, MVVM, Reactive Programming, etc) 4. Variable / Function Naming 5. Comments / Documentation 6. Whitespaces (tabs V.S. spaces, newline)
  2. We need the Law 1. ! Coding Style Guide •

    https://github.com/raywenderlich/swift-style-guide • https://google.github.io/swift/ 2. " Linter / Formatter • realm/SwiftLint • nicklockwood/SwiftFormat • Both solves syntactic problems
  3. SwiftLint / SwiftFormat • realm/SwiftLint • Powered by SourceKit(ten) •

    Syntax structure = untyped dictionary tree • Some rules are autocorrect-able • nicklockwood/SwiftFormat • Scope-aware, built-in tokenizer • Syntax structure = token array
  4. SwiftSyntax • Swift bindings for apple/swift's libSyntax • Type-safe, AST-level

    structured editing library • Includes trivias (whitespaces & comments) • Attaches trivias to leading / trailing of token • Similar to .NET (Roslyn) / TypeScript Syntax API • Internally calls swiftc -frontend -emit-syntax and passes JSON for "C++ to Swift" bridging (in Swift 4.2)
  5. Swift 4.2 Formal grammar About the Language Reference — The

    Swift Programming Language (Swift 4.2)
  6. top-level-declaration ! [statements] statements ! statement [statements] statement ! expression

    [;] | declaration [;] | loop-statement [;] | branch-statement [;] | labeled-statement [;] | control-transfer-statement [;] | defer-statement [;] | do-statement [;] | compiler-control-statement
  7. // Note: Pseudo-code struct TopLevelDeclaration { let statements: [Statement] }

    // Simple statements, compiler control, and control flow. enum Statement { case expression(Expression, Semicolon?) case declaration(Declaration, Semicolon?) case loop(LoopStatement, Semicolon?) case branch(BranchStatement, Semicolon?) case labeled(LabeledStatement, Semicolon?) case controlTransfer(ControlTransferStatement, Semicolon?) case `defer`(DeferStatement, Semicolon?) case `do`(DoStatement, Semicolon?) case compilerControl(CompilerControlStatement) }
  8. declaration ! import-declaration | constant-declaration | variable-declaration | typealias-declaration |

    function-declaration | enum-declaration | struct-declaration | class-declaration | protocol-declaration | initializer-declaration | deinitializer-declaration | extension-declaration | subscript-declaration | operator-declaration | precedence-group-declaration
  9. /// Introduces a new name or construct, e.g. functions, structs,

    variables. enum Declaration { case `import`(ImportDeclaration) case constant(ConstantDeclaration) case variable(VariableDeclaration) case `typealias`(TypealiasDeclaration) case function(FunctionDeclaration) case `enum`(EnumDeclaration) case `struct`(StructDeclaration) case `class`(ClassDeclaration) case `protocol`(ProtocolDeclaration) case initializer(InitializerDeclaration) case deinitializer(DeinitializerDeclaration) case `extension`(ExtensionDeclaration) case `subscript`(SubscriptDeclaration) case `operator`(OperatorDeclaration) case precedenceGroup(PrecedenceGroupDeclaration) }
  10. expression ! [try-operator] prefix-expression [binary-expressions] prefix-expression ! [prefix-operator] postfix-expression |

    in-out-expression postfix-expression ! primary-expression | function-call-expression | initializer-expression | l binary-expressions ! binary-expression [binary-expressions] binary-expression ! binary-operator prefix-expression | assignment-operator [try-operator] prefix-expression | conditional-operator [try-operator] prefix-expression | type-casting-operator
  11. /// Can evaluate & return a value and/or cause a

    side effect. struct Expression { let tryOperator: TryOperator? let prefixExpression: PrefixExpression let binaryExpressions: [BinaryExpression] } enum PrefixExpression { case prefixOperator(PrefixOperator?, PostfixExpression) case `inout`(InoutExpression) } enum PostfixExpression { case primary(PrimaryExpression) case functionCall(FunctionCallExpression) case initializer(InitializerExpression) ... }
  12. whitespace ! whitespace-item [whitespace] whitespace-item ! line-break | comment |

    multiline-comment | U+0000, U+0009, U+000B, U+000C, or U+0020 line-break ! U+000A | U+000D | U+000D followed by U+000A comment ! // comment-text line-break multiline-comment ! /* multiline-comment-text */ comment-text ! comment-text-item [comment-text] comment-text-item ! Any Unicode scalar value except U+000A or U+000D multiline-comment-text ! multiline-comment-text-item [multiline-comment-text] multiline-comment-text-item ! multiline-comment | comment-text-item | Any Unicode scalar value except /* or */
  13. identifier ! identifier-head [identifier-characters] | ` identifier-head [identifier-characters] ` |

    implicit-parameter-name identifier-list ! identifier | identifier , identifier-list identifier-head ! A-Z | a-z | _ | U+00A8 | U+00AA | ... identifier-character ! 0-9 | U+0300–U+036F | U+1DC0–U+1DFF | ... | identifier-head identifier-characters ! identifier-character [identifier-characters]
  14. literal ! numeric-literal | string-literal | boolean-literal | nil-literal numeric-literal

    ! [-] integer-literal | [-] floating-point-literal boolean-literal ! true | false nil-literal ! nil
  15. open class SemicolonTrimmer: SyntaxRewriter { open override func visit(_ syntax:

    CodeBlockItemSyntax) -> Syntax { if let nextToken = syntax.nextToken, nextToken.leadingTrivia.contains(where: { $0.isNewline }) || syntax.isLastChild { let noSemicolon = syntax.withSemicolon(nil) return super.visit(noSemicolon) } else { return super.visit(syntax) } } }
  16. open override func visit(_ token: TokenSyntax) -> Syntax { guard

    case var .integerLiteral(text) = token.tokenKind else { return token } if text.hasPrefix("0x") || text.hasPrefix("0b") || text.hasPrefix("0o") { return token } text = text.replacingOccurrences(of: "_", with: "") let index1 = text.index(after: text.startIndex) var index = text.endIndex while text.formIndex(&index, offsetBy: -3, limitedBy: index1) { text.insert("_", at: index) } return token.withKind(.integerLiteral(text)) }
  17. struct Foo { - struct Bar { - struct Baz

    {} - } } + +extension Foo { + struct Bar {} +} + +extension Foo.Bar { + struct Baz {} +}
  18. func foo(_ value: Int?) throws -> Int { - if

    let value = value { - if value >= 0 { - return value - } else { - throw MyError.negative - } - } else { - throw MyError.none - } + guard let value = value else { throw MyError.none } + guard value >= 0 else { throw MyError.negative } + return value }
  19. SwiftSyntax types • SyntaxRewriter / SyntaxVisitor • Trivia (leading /

    trailing) • Comments & whitespaces are stored in TokenSyntax • SyntaxFactory • Creates a new Syntax nodes • Diagnostic • Data types for showing diagnosed Note & FixIt
  20. Speeding up SwiftSyntax by using the parser directly - Swift

    Forums apple/swift#21762 apple/swift-syntax#59
  21. If visiting descendants manually... // Get grandchild (2 getters). var

    rightParen = syntax.parameters.rightParen // Modify grandchild (e.g. changing trivia). rightParen = modify(rightParen) // Set new child. let parameters = syntax.parameters .withRightParen(rightParen) // Set new `syntax` ( ❌ many more code needed if deeply nested) return syntax.withParameters(parameters)
  22. Use ! Lens! override func visit(_ syntax: InitializerDeclSyntax) -> DeclSyntax

    { return _modify(lens: .parameters >>> .rightParen, syntax: syntax) } private func _modify<Whole, Part>( lens: Lens<Whole, Part>, syntax: Whole ) -> Whole where Whole: Syntax, Part: Syntax { let part = lens.getter(syntax) let part2 = modify(part) return lens.setter(syntax, part2) }
  23. ! Lens: Functional getter & setter struct Lens<Whole, Part> {

    let getter: (Whole) -> Part let setter: (Whole, Part) -> Whole static func >>> <Part2>( l: Lens<Whole, Part>, r: Lens<Part, Part2> ) -> Lens<Whole, Part2> { return Lens<Whole, Part2>( getter: { r.getter(l.getter($0)) }, setter: { a, c in l.setter(a, r.setter(l.getter(a), c)) } ) } }
  24. ! Prism: Lens for sum types struct Prism<Whole, Part> {

    let tryGet: (Whole) -> Part? let inject: (Part) -> Whole static func >>> <Part2>( l: Prism<Whole, Part>, r: Prism<Part, Part2> ) -> Prism<Whole, Part2> { return Prism<Whole, Part2>( tryGet: { l.tryGet($0).flatMap(r.tryGet) }, inject: { l.inject(r.inject($0)) } ) } }
  25. >>> : (Lens<A, B>, Lens<B, C>) -> Lens<A, C> >>>

    : (Prism<A, B>, Prism<B, C>) -> Prism<A, C> >>> : (Lens<A, B>, Prism<B, C>) -> ??? >>> : (Prism<A, B>, Lens<B, C>) -> ??? Q. How to compose Lens and Prism together?
  26. >>> : (Lens<A, B>, Lens<B, C>) -> Lens<A, C> >>>

    : (Prism<A, B>, Prism<B, C>) -> Prism<A, C> >>> : (Lens<A, B>, Prism<B, C>) -> ??? >>> : (Prism<A, B>, Lens<B, C>) -> ??? ɹɹɹɹɹˣˣˣɹLift to AffineTraversal!ɹˣˣˣ init(lens:) : (Lens<A, B>) -> AffineTraversal<A, B> init(prism:) : (Prism<A, B>) -> AffineTraversal<A, B> >>> : (AffineTraversal<A, B>, AffineTraversal<B, C>) -> AffineTraversal<A, C>
  27. ↔ Affine Traversal: Lens + Prism struct AffineTraversal<Whole, Part> {

    let tryGet: (Whole) -> Part? let setter: (Whole, Part) -> Whole init(lens: Lens<Whole, Part>) { self.init(tryGet: lens.getter, setter: lens.setter) } init(prism: Prism<Whole, Part>) { self.init(tryGet: prism.tryGet, setter: { prism.inject($1) }) } }
  28. Recap • SwiftSyntax: Type-safe & lossless AST editing library •

    Many incoming improvements toward Swift 5.0 or later • Swift AST Explorer: Useful AST visualizer for debugging • Formal grammar: Syntax production rules • Optics: Functional approach to traverse data structure • Let's make your own code formatter using SwiftSyntax!
  29. References (Swift AST) • try! Swift NYC 2017 - Improving

    Swift Tools with libSyntax • try! Swift Tokyo 2018 - AST Meta-programming • apple/swift/lib/Syntax • Speeding up SwiftSyntax by using the parser directly • About the Language Reference (Swift 4.2) • SwiftFormat (Part 1 of 3) - Schibsted Bytes • google/swift (format branch)
  30. References (Optics) • Lenses in Swift - Chris Eidhof •

    Brandon Williams - Lenses in Swift • Lenses and Prisms in Swift: a pragmatic approach | Fun iOS • Lenses, Folds, and Traversals - Edward Kmett • Oleg's gists - Glassery • Profunctor Optics: Modular Data Accessors