Swiftly Swift manipulation... with Swift

This presentation seeks to acquaint users of the Swift language tooling. Parsing, generating and manipulating Swift source code in order to build useful devtool for Swift, in Swift.

Video: https://youtu.be/cJkiCEJ7zIY

Transcript

1. Swiftly Swift manipulation... with Swift
   FrenchKit 2019
   Marcin Krzyzanowski
   

   View Slide

2. Agenda
   • Problem
   
   • Introduce SwiftSyntax
   
   • github.com/krzyzanowskim/reorder
   
   • IndexSourceDB
   

   View Slide

3. PROBLEM
   https://twitter.com/krzyzanowskim/status/1162306459075588096
   

   View Slide

4. import Foundation
   public class Conference {
   public var name: String
   public let venue: String
   INPUT
   

   View Slide

5. public class Conference {
   public var name: String
   public let venue: String
   internal typealias Line = String
   private var attendees: Set
   private var sponsors: Set
   private var costs: Array
   internal func add(attendee: String)
   internal func add(sponsor: String)
   deinit
   internal func add(cost: Decimal)
   private init()
   public func totalCosts() -> Decimal
   private func statement() -> Array
   internal func printStatement()
   public init(name: String, venue: String)
   }
   API
   

   View Slide

6. public class Conference {
   public var name: String
   public let venue: String
   internal typealias Line = String
   private var attendees: Set
   private var sponsors: Set
   private var costs: Array
   public init(name: String, venue: String)
   private init()
   deinit
   public func totalCosts() -> Decimal
   internal func add(attendee: String)
   internal func add(sponsor: String)
   internal func add(cost: Decimal)
   internal func printStatement()
   private func statement() -> Array
   }
   API Initializer
   Public
   Internal
   Private
   

   View Slide

7. public class Conference {
   public var name: String
   public let venue: String
   internal typealias Line = String
   private var attendees: Set
   private var sponsors: Set
   private var costs: Array
   internal func add(attendee: String)
   internal func add(sponsor: String)
   deinit
   internal func add(cost: Decimal)
   private init()
   public func totalCosts() -> Decimal
   private func statement() -> Array
   internal func printStatement()
   public init(name: String, venue: String)
   }
   public class Conference {
   public var name: String
   public let venue: String
   internal typealias Line = String
   private var attendees: Set
   private var sponsors: Set
   private var costs: Array
   public init(name: String, venue: String)
   private init()
   deinit
   public func totalCosts() -> Decimal
   internal func add(attendee: String)
   internal func add(sponsor: String)
   internal func add(cost: Decimal)
   internal func printStatement()
   private func statement() -> Array
   }
   →
   

   View Slide

8. HOW?
   move lines by hand
   use editor plugin
   use existing command line tool
   write a tool by my own
   

   View Slide

9. HOW?
   move lines by hand
   use editor plugin
   use existing command line tool
   write a tool by my own
   

   View Slide

10. HOW?
    move lines by hand
    use editor plugin
    use existing command line tool
    write a tool by my own
    

    View Slide

11. HOW?
    move lines by hand
    use editor plugin
    use existing command line tool
    write a tool by my own
    nicklockwood/SwiftFormat
    google/swift (format)
    inamiy/SwiftRewriter
    

    View Slide

12. HOW?
    move lines by hand
    use editor plugin
    use existing command line tool
    write a tool by my own
    

    View Slide

13. HOW? HOW?
    write a tool by my own
    Brain + Time + Computer + Programming Language + Library
    + ⏲ + + C +
    

    View Slide

14. you have brain
    

    View Slide

15. The Internet
    

    View Slide

16. SwiftSyntax
    github.com/apple/swift-syntax
    Source Code
    text/plain
    Abstract Syntax Tree
    

    View Slide

17. SwiftSyntax
    github.com/apple/swift-syntax
    Abstract Syntax Tree
    Syntax
    Syntax Syntax
    Syntax Syntax
    Syntax
    

    View Slide

18. SwiftSyntax
    Syntax
    Syntax Syntax
    Syntax Syntax
    Syntax
    Visitor Design Pattern
    The visitor design pattern is a way of
    separating an algorithm from an object
    structure on which it operates
    

    View Slide

19. SwiftSyntax
    import SwiftSyntax
    struct Visitor: SyntaxVisitor {
    mutating func visit(_ node: InitializerDeclSyntax) -> SyntaxVisitorContinueKind {
    self.process(node)
    return .skipChildren
    }
    }
    

    View Slide

20. SwiftSyntax
    class Rewriter: SyntaxRewriter {
    init() { }
    override func visit(_ node: DeinitializerDeclSyntax) -> DeclSyntax {
    return SyntaxFactory.makeBlankClassDecl()
    }
    }
    

    View Slide

21. PLAN
    1. Search for all declarations (func, init, deinit)
    
    2. Sort functions by access level and kind
    
    3. Rewrite source with new order
    
    4. Output source code
    

    View Slide

22. class Indexer: SyntaxVisitor {
    private var functions: Array = []
    func visit(_ node: DeinitializerDeclSyntax) -> SyntaxVisitorContinueKind{
    self.functions.append(node)
    return .skipChildren
    }
    func visit(_ node: InitializerDeclSyntax) -> SyntaxVisitorContinueKind {
    self.functions.append(node)
    return .skipChildren
    }
    func visit(_ node: FunctionDeclSyntax) -> SyntaxVisitorContinueKind {
    self.functions.append(node)
    return .skipChildren
    }
    }
    Indexer
    1. Search for all declarations (func, init, deinit)
    

    View Slide

23. func sorted() -> Array {
    Sort
    2. Sort functions by access level and kind
    

    View Slide

24. Rewrite
    class Rewriter: SyntaxRewriter {
    private let indexer: Indexer
    private var sortedFunctions: Array
    init(_ indexer: Indexer) {
    self.indexer = indexer
    self.sortedFunctions = indexer.sorted()
    }
    override func visit(_ node: DeinitializerDeclSyntax) -> DeclSyntax {
    sortedFunctions.removeFirst()
    }
    override func visit(_ node: InitializerDeclSyntax) -> DeclSyntax {
    sortedFunctions.removeFirst()
    }
    override func visit(_ node: FunctionDeclSyntax) -> DeclSyntax {
    sortedFunctions.removeFirst()
    }
    }
    3. Rewrite source with new order
    

    View Slide

25. public func process(_ data: Data) throws -> String {
    guard let source = String(data: data, encoding: .utf8) else {
    throw Error.invalidInput
    }
    let parsed: SourceFileSyntax = try SyntaxParser.parse(source: source)
    // Gather information about the file
    var visitor = Indexer()
    parsed.walk(&visitor)
    // Rewrite file using visitor
    let rewritten = Rewriter(visitor).visit(parsed)
    var output = StringOutputStream()
    rewritten.write(to: &output)
    return output.string
    }
    Processor
    4. Output source code
    

    View Slide

26. //
    // swift run reorder -file Sources/Conference/Conference.swift > output.swift
    //
    guard let executablePath = CommandLine.arguments.first else {
    fatalError("Missing executable")
    }
    let args = Array(CommandLine.arguments.dropFirst())
    let commands = stride(from: 0, to: args.count - 1, by: 2).map { (command:
    args[$0].dropFirst(), value: args[$0+1]) }
    var err = StderrOutputStream()
    var out = StdoutOutputStream()
    for (cmd, value) in commands {
    switch cmd {
    case "file":
    err.write("Processing \(value)\n")
    let content = try Data(contentsOf: URL(fileURLWithPath: value))
    let result = try Processor().process(content)
    out.write(result)
    break
    default:
    err.write("Error: Unknown command: \(cmd)\n")
    $ ./reorder -file Conference.swift
    

    View Slide

27. import Foundation
    public class Conference {
    public var name: String
    public let venue: String
    internal typealias Line = String
    private var attendees: Set
    private var sponsors: Set
    private var costs: Array
    public init(name: String, venue: String) {
    self.name = name
    self.venue = venue
    self.attendees = []
    self.sponsors = []
    self.costs = []
    }
    private init() {
    self.name = "FrenchKit"
    self.venue = "Bâtiment"
    self.attendees = []
    $ ./reorder -file Conference.swift
    

    View Slide

28. View Slide

29. • Sort class/struct properties
    
    • Sort within extension scope
    
    • Sort static func/property
    HOMEWORK
    github.com/krzyzanowskim/reorder
    Submit Pull Request!
    

    View Slide

30. ONE MORE THING
    

    View Slide

31. IndexStoreDB
    IndexStoreDB is a source code indexing library. It
    provides a composable and efficient query API for
    looking up source code symbols, symbol
    occurrences, and relations.
    github.com/apple/indexstore-db
    

    View Slide

32. IndexStoreDB
    IndexStoreDB is a source code indexing library. It
    provides a composable and efficient query API for
    looking up source code symbols, symbol
    occurrences, and relations.
    github.com/apple/indexstore-db
    swift build --enable-index-store
    .build/x86_64-apple-macosx/debug/index/store
    

    View Slide

33. IndexStoreDB
    github.com/apple/indexstore-db
    $ swiftc -index-store-path index -index-file Sources/Conference/Conference.swift
    

    View Slide

34. StoreIndexer
    public struct StoreIndexer {
    public typealias Path = String
    public struct Function {
    public let symbol: Symbol
    public let location: SymbolLocation
    }
    private let indexURL: URL
    

    View Slide

35. IndexStoreDB
    1. StoreIndexer fetch all files where
    Conference occurs
    
    2. SwiftSyntax process files and output
    

    View Slide

36. View Slide

37. ONE MISSING THING
    //===-------------------------------------------------------===//
    // SourceKit README
    //===-------------------------------------------------------===//
    Welcome to SourceKit! SourceKit is a framework for supporting IDE features like
    indexing, syntax-coloring, code-completion, etc. In general it provides the
    infrastructure that an IDE needs for excellent language support.
    SourceKit currently only supports the Swift language.
    

    View Slide

38. LINKS
    Building a Compiler in Swift - Nick Lockwood
    
    https://www.youtube.com/watch?v=uQNkrV0F07Q
    N. Hawes & A. Lorenz “Adding Index‐While‐Building and ..."
    
    https://www.youtube.com/watch?v=jGJhnIT-D2M
    Creating Refactoring Transformations for Swift
    
    https://www.skilled.io/u/swiftsummit/creating-refactoring-transformations-for-swift
    NSHipster - Swift Syntax
    
    https://nshipster.com/swiftsyntax/
    reorder
    
    https://github.com/krzyzanowskim/reorder
    

    View Slide

39. THANK YOU
    @krzyzanowskim
    github.com/krzyzanowskim
    https://swift.best
    

    View Slide