Pro Yearly is on sale from $80 to $50! »

Making your own Code Formatter in Swift

Making your own Code Formatter in Swift

Eac0bf787b5279aca5e699ece096956e?s=128

Yasuhiro Inami

January 19, 2019
Tweet

Transcript

  1. Making your ownɹɹɹɹ Code Formatter in Swiftɹɹɹɹɹ 2019/01/19 iOSConf SG

    2019 Yasuhiro Inami / @inamiy
  2. None
  3. None
  4. 2016 → 2019 4 → 14ɹɹ ! developers ɹɹ ɹ

    ɹ
  5. ! 14 developers ! 14 coding styles

  6. ! Programming Styles 1. Language (Swift, Objective-C, C++, etc) 2.

    Typing (Object-oriented, functional) or untyping (oh my!) 3. Architecture (MVC, MVVM, Reactive Programming, etc) 4. Variable / Function Naming 5. Comments / Documentation 6. Whitespaces (tabs V.S. spaces, newline)
  7. None
  8. None
  9. We need the Law 1. ! Coding Style Guide •

    https://github.com/raywenderlich/swift-style-guide • https://google.github.io/swift/ 2. " Linter / Formatter • realm/SwiftLint • nicklockwood/SwiftFormat • Both solves syntactic problems
  10. SwiftLint / SwiftFormat • realm/SwiftLint • Powered by SourceKit(ten) •

    Syntax structure = untyped dictionary tree • Some rules are autocorrect-able • nicklockwood/SwiftFormat • Scope-aware, built-in tokenizer • Syntax structure = token array
  11. None
  12. None
  13. SwiftSyntax https://github.com/apple/swift-syntax

  14. SwiftSyntax • Swift bindings for apple/swift's libSyntax • Type-safe, AST-level

    structured editing library • Includes trivias (whitespaces & comments) • Attaches trivias to leading / trailing of token • Similar to .NET (Roslyn) / TypeScript Syntax API • Internally calls swiftc -frontend -emit-syntax and passes JSON for "C++ to Swift" bridging (in Swift 4.2)
  15. None
  16. Swift AST Explorer https://swift-ast-explorer.kishikawakatsumi.com

  17. Q. Where do Swift syntaxes come from?

  18. Swift 4.2 Formal grammar About the Language Reference — The

    Swift Programming Language (Swift 4.2)
  19. top-level-declaration ! [statements] statements ! statement [statements] statement ! expression

    [;] | declaration [;] | loop-statement [;] | branch-statement [;] | labeled-statement [;] | control-transfer-statement [;] | defer-statement [;] | do-statement [;] | compiler-control-statement
  20. // Note: Pseudo-code struct TopLevelDeclaration { let statements: [Statement] }

    // Simple statements, compiler control, and control flow. enum Statement { case expression(Expression, Semicolon?) case declaration(Declaration, Semicolon?) case loop(LoopStatement, Semicolon?) case branch(BranchStatement, Semicolon?) case labeled(LabeledStatement, Semicolon?) case controlTransfer(ControlTransferStatement, Semicolon?) case `defer`(DeferStatement, Semicolon?) case `do`(DoStatement, Semicolon?) case compilerControl(CompilerControlStatement) }
  21. declaration ! import-declaration | constant-declaration | variable-declaration | typealias-declaration |

    function-declaration | enum-declaration | struct-declaration | class-declaration | protocol-declaration | initializer-declaration | deinitializer-declaration | extension-declaration | subscript-declaration | operator-declaration | precedence-group-declaration
  22. /// Introduces a new name or construct, e.g. functions, structs,

    variables. enum Declaration { case `import`(ImportDeclaration) case constant(ConstantDeclaration) case variable(VariableDeclaration) case `typealias`(TypealiasDeclaration) case function(FunctionDeclaration) case `enum`(EnumDeclaration) case `struct`(StructDeclaration) case `class`(ClassDeclaration) case `protocol`(ProtocolDeclaration) case initializer(InitializerDeclaration) case deinitializer(DeinitializerDeclaration) case `extension`(ExtensionDeclaration) case `subscript`(SubscriptDeclaration) case `operator`(OperatorDeclaration) case precedenceGroup(PrecedenceGroupDeclaration) }
  23. expression ! [try-operator] prefix-expression [binary-expressions] prefix-expression ! [prefix-operator] postfix-expression |

    in-out-expression postfix-expression ! primary-expression | function-call-expression | initializer-expression | l binary-expressions ! binary-expression [binary-expressions] binary-expression ! binary-operator prefix-expression | assignment-operator [try-operator] prefix-expression | conditional-operator [try-operator] prefix-expression | type-casting-operator
  24. /// Can evaluate & return a value and/or cause a

    side effect. struct Expression { let tryOperator: TryOperator? let prefixExpression: PrefixExpression let binaryExpressions: [BinaryExpression] } enum PrefixExpression { case prefixOperator(PrefixOperator?, PostfixExpression) case `inout`(InoutExpression) } enum PostfixExpression { case primary(PrimaryExpression) case functionCall(FunctionCallExpression) case initializer(InitializerExpression) ... }
  25. whitespace ! whitespace-item [whitespace] whitespace-item ! line-break | comment |

    multiline-comment | U+0000, U+0009, U+000B, U+000C, or U+0020 line-break ! U+000A | U+000D | U+000D followed by U+000A comment ! // comment-text line-break multiline-comment ! /* multiline-comment-text */ comment-text ! comment-text-item [comment-text] comment-text-item ! Any Unicode scalar value except U+000A or U+000D multiline-comment-text ! multiline-comment-text-item [multiline-comment-text] multiline-comment-text-item ! multiline-comment | comment-text-item | Any Unicode scalar value except /* or */
  26. identifier ! identifier-head [identifier-characters] | ` identifier-head [identifier-characters] ` |

    implicit-parameter-name identifier-list ! identifier | identifier , identifier-list identifier-head ! A-Z | a-z | _ | U+00A8 | U+00AA | ... identifier-character ! 0-9 | U+0300–U+036F | U+1DC0–U+1DFF | ... | identifier-head identifier-characters ! identifier-character [identifier-characters]
  27. literal ! numeric-literal | string-literal | boolean-literal | nil-literal numeric-literal

    ! [-] integer-literal | [-] floating-point-literal boolean-literal ! true | false nil-literal ! nil
  28. None
  29. SwiftSyntax Example

  30. iOSConfSG = 2019; ↓ iOSConfSG = 2019

  31. open class SemicolonTrimmer: SyntaxRewriter { open override func visit(_ syntax:

    CodeBlockItemSyntax) -> Syntax { if let nextToken = syntax.nextToken, nextToken.leadingTrivia.contains(where: { $0.isNewline }) || syntax.isLastChild { let noSemicolon = syntax.withSemicolon(nil) return super.visit(noSemicolon) } else { return super.visit(syntax) } } }
  32. iOSConfSG = 2019 ↓ iOSConfSG = 2_019

  33. open override func visit(_ token: TokenSyntax) -> Syntax { guard

    case var .integerLiteral(text) = token.tokenKind else { return token } if text.hasPrefix("0x") || text.hasPrefix("0b") || text.hasPrefix("0o") { return token } text = text.replacingOccurrences(of: "_", with: "") let index1 = text.index(after: text.startIndex) var index = text.endIndex while text.formIndex(&index, offsetBy: -3, limitedBy: index1) { text.insert("_", at: index) } return token.withKind(.integerLiteral(text)) }
  34. struct Foo { - struct Bar { - struct Baz

    {} - } } + +extension Foo { + struct Bar {} +} + +extension Foo.Bar { + struct Baz {} +}
  35. func foo(_ value: Int?) throws -> Int { - if

    let value = value { - if value >= 0 { - return value - } else { - throw MyError.negative - } - } else { - throw MyError.none - } + guard let value = value else { throw MyError.none } + guard value >= 0 else { throw MyError.negative } + return value }
  36. SwiftSyntax types • SyntaxRewriter / SyntaxVisitor • Trivia (leading /

    trailing) • Comments & whitespaces are stored in TokenSyntax • SyntaxFactory • Creates a new Syntax nodes • Diagnostic • Data types for showing diagnosed Note & FixIt
  37. try! Swift NYC 2017 - Improving Swift Tools with libSyntax

    apple/swift/lib/Syntax
  38. None
  39. Speeding up SwiftSyntax by using the parser directly - Swift

    Forums apple/swift#21762 apple/swift-syntax#59
  40. SwiftSyntax + Optics

  41. None
  42. None
  43. None
  44. If visiting descendants manually... // Get grandchild (2 getters). var

    rightParen = syntax.parameters.rightParen // Modify grandchild (e.g. changing trivia). rightParen = modify(rightParen) // Set new child. let parameters = syntax.parameters .withRightParen(rightParen) // Set new `syntax` ( ❌ many more code needed if deeply nested) return syntax.withParameters(parameters)
  45. Use ! Lens! override func visit(_ syntax: InitializerDeclSyntax) -> DeclSyntax

    { return _modify(lens: .parameters >>> .rightParen, syntax: syntax) } private func _modify<Whole, Part>( lens: Lens<Whole, Part>, syntax: Whole ) -> Whole where Whole: Syntax, Part: Syntax { let part = lens.getter(syntax) let part2 = modify(part) return lens.setter(syntax, part2) }
  46. Lens.parameters >>> Lens.rightParen

  47. None
  48. None
  49. .wowɹɹ ɹ >>> .suchɹ >>> .nestɹ >>> .muchɹ >>> .deepɹ

  50. ! Lens: Functional getter & setter struct Lens<Whole, Part> {

    let getter: (Whole) -> Part let setter: (Whole, Part) -> Whole static func >>> <Part2>( l: Lens<Whole, Part>, r: Lens<Part, Part2> ) -> Lens<Whole, Part2> { return Lens<Whole, Part2>( getter: { r.getter(l.getter($0)) }, setter: { a, c in l.setter(a, r.setter(l.getter(a), c)) } ) } }
  51. None
  52. None
  53. None
  54. None
  55. None
  56. ! Prism: Lens for sum types struct Prism<Whole, Part> {

    let tryGet: (Whole) -> Part? let inject: (Part) -> Whole static func >>> <Part2>( l: Prism<Whole, Part>, r: Prism<Part, Part2> ) -> Prism<Whole, Part2> { return Prism<Whole, Part2>( tryGet: { l.tryGet($0).flatMap(r.tryGet) }, inject: { l.inject(r.inject($0)) } ) } }
  57. None
  58. >>> : (Lens<A, B>, Lens<B, C>) -> Lens<A, C> >>>

    : (Prism<A, B>, Prism<B, C>) -> Prism<A, C> >>> : (Lens<A, B>, Prism<B, C>) -> ??? >>> : (Prism<A, B>, Lens<B, C>) -> ??? Q. How to compose Lens and Prism together?
  59. >>> : (Lens<A, B>, Lens<B, C>) -> Lens<A, C> >>>

    : (Prism<A, B>, Prism<B, C>) -> Prism<A, C> >>> : (Lens<A, B>, Prism<B, C>) -> ??? >>> : (Prism<A, B>, Lens<B, C>) -> ??? ɹɹɹɹɹˣˣˣɹLift to AffineTraversal!ɹˣˣˣ init(lens:) : (Lens<A, B>) -> AffineTraversal<A, B> init(prism:) : (Prism<A, B>) -> AffineTraversal<A, B> >>> : (AffineTraversal<A, B>, AffineTraversal<B, C>) -> AffineTraversal<A, C>
  60. None
  61. Lens.trailingClosure >>> Prism.some >>> Lens.rightParen

  62. ↔ Affine Traversal: Lens + Prism struct AffineTraversal<Whole, Part> {

    let tryGet: (Whole) -> Part? let setter: (Whole, Part) -> Whole init(lens: Lens<Whole, Part>) { self.init(tryGet: lens.getter, setter: lens.setter) } init(prism: Prism<Whole, Part>) { self.init(tryGet: prism.tryGet, setter: { prism.inject($1) }) } }
  63. SwiftRewriter https://github.com/inamiy/SwiftRewriter

  64. Recap • SwiftSyntax: Type-safe & lossless AST editing library •

    Many incoming improvements toward Swift 5.0 or later • Swift AST Explorer: Useful AST visualizer for debugging • Formal grammar: Syntax production rules • Optics: Functional approach to traverse data structure • Let's make your own code formatter using SwiftSyntax!
  65. References (Swift AST) • try! Swift NYC 2017 - Improving

    Swift Tools with libSyntax • try! Swift Tokyo 2018 - AST Meta-programming • apple/swift/lib/Syntax • Speeding up SwiftSyntax by using the parser directly • About the Language Reference (Swift 4.2) • SwiftFormat (Part 1 of 3) - Schibsted Bytes • google/swift (format branch)
  66. References (Optics) • Lenses in Swift - Chris Eidhof •

    Brandon Williams - Lenses in Swift • Lenses and Prisms in Swift: a pragmatic approach | Fun iOS • Lenses, Folds, and Traversals - Edward Kmett • Oleg's gists - Glassery • Profunctor Optics: Modular Data Accessors
  67. Thanks! Yasuhiro Inami @inamiy