Swiftly Swift manipulation... with Swift

Swiftly Swift manipulation... with Swift

This presentation seeks to acquaint users of the Swift language tooling. Parsing, generating and manipulating Swift source code in order to build useful devtool for Swift, in Swift.

Video: https://youtu.be/cJkiCEJ7zIY

7b2fdba8077c8495b3caa6f36d0928da?s=128

Marcin Krzyzanowski

October 07, 2019
Tweet

Transcript

  1. Swiftly Swift manipulation... with Swift FrenchKit 2019 Marcin Krzyzanowski

  2. Agenda • Problem • Introduce SwiftSyntax • github.com/krzyzanowskim/reorder • IndexSourceDB

  3. PROBLEM https://twitter.com/krzyzanowskim/status/1162306459075588096

  4. import Foundation public class Conference { public var name: String

    public let venue: String INPUT
  5. public class Conference { public var name: String public let

    venue: String internal typealias Line = String private var attendees: Set<String> private var sponsors: Set<String> private var costs: Array<Decimal> internal func add(attendee: String) internal func add(sponsor: String) deinit internal func add(cost: Decimal) private init() public func totalCosts() -> Decimal private func statement() -> Array<Line> internal func printStatement() public init(name: String, venue: String) } API
  6. public class Conference { public var name: String public let

    venue: String internal typealias Line = String private var attendees: Set<String> private var sponsors: Set<String> private var costs: Array<Decimal> public init(name: String, venue: String) private init() deinit public func totalCosts() -> Decimal internal func add(attendee: String) internal func add(sponsor: String) internal func add(cost: Decimal) internal func printStatement() private func statement() -> Array<Line> } API Initializer Public Internal Private
  7. public class Conference { public var name: String public let

    venue: String internal typealias Line = String private var attendees: Set<String> private var sponsors: Set<String> private var costs: Array<Decimal> internal func add(attendee: String) internal func add(sponsor: String) deinit internal func add(cost: Decimal) private init() public func totalCosts() -> Decimal private func statement() -> Array<Line> internal func printStatement() public init(name: String, venue: String) } public class Conference { public var name: String public let venue: String internal typealias Line = String private var attendees: Set<String> private var sponsors: Set<String> private var costs: Array<Decimal> public init(name: String, venue: String) private init() deinit public func totalCosts() -> Decimal internal func add(attendee: String) internal func add(sponsor: String) internal func add(cost: Decimal) internal func printStatement() private func statement() -> Array<Line> } →
  8. HOW? move lines by hand use editor plugin use existing

    command line tool write a tool by my own
  9. HOW? move lines by hand use editor plugin use existing

    command line tool write a tool by my own
  10. HOW? move lines by hand use editor plugin use existing

    command line tool write a tool by my own
  11. HOW? move lines by hand use editor plugin use existing

    command line tool write a tool by my own nicklockwood/SwiftFormat google/swift (format) inamiy/SwiftRewriter
  12. HOW? move lines by hand use editor plugin use existing

    command line tool write a tool by my own
  13. HOW? HOW? write a tool by my own Brain +

    Time + Computer + Programming Language + Library + ⏲ + + C +
  14. you have brain

  15. The Internet

  16. SwiftSyntax github.com/apple/swift-syntax Source Code text/plain Abstract Syntax Tree

  17. SwiftSyntax github.com/apple/swift-syntax Abstract Syntax Tree Syntax Syntax Syntax Syntax Syntax

    Syntax
  18. SwiftSyntax Syntax Syntax Syntax Syntax Syntax Syntax Visitor Design Pattern

    The visitor design pattern is a way of separating an algorithm from an object structure on which it operates
  19. SwiftSyntax import SwiftSyntax struct Visitor: SyntaxVisitor { mutating func visit(_

    node: InitializerDeclSyntax) -> SyntaxVisitorContinueKind { self.process(node) return .skipChildren } }
  20. SwiftSyntax class Rewriter: SyntaxRewriter { init() { } override func

    visit(_ node: DeinitializerDeclSyntax) -> DeclSyntax { return SyntaxFactory.makeBlankClassDecl() } }
  21. PLAN 1. Search for all declarations (func, init, deinit) 2.

    Sort functions by access level and kind 3. Rewrite source with new order 4. Output source code
  22. class Indexer: SyntaxVisitor { private var functions: Array<DeclSyntax> = []

    func visit(_ node: DeinitializerDeclSyntax) -> SyntaxVisitorContinueKind{ self.functions.append(node) return .skipChildren } func visit(_ node: InitializerDeclSyntax) -> SyntaxVisitorContinueKind { self.functions.append(node) return .skipChildren } func visit(_ node: FunctionDeclSyntax) -> SyntaxVisitorContinueKind { self.functions.append(node) return .skipChildren } } Indexer 1. Search for all declarations (func, init, deinit)
  23. func sorted() -> Array<DeclSyntax> { Sort 2. Sort functions by

    access level and kind
  24. Rewrite class Rewriter: SyntaxRewriter { private let indexer: Indexer private

    var sortedFunctions: Array<DeclSyntax> init(_ indexer: Indexer) { self.indexer = indexer self.sortedFunctions = indexer.sorted() } override func visit(_ node: DeinitializerDeclSyntax) -> DeclSyntax { sortedFunctions.removeFirst() } override func visit(_ node: InitializerDeclSyntax) -> DeclSyntax { sortedFunctions.removeFirst() } override func visit(_ node: FunctionDeclSyntax) -> DeclSyntax { sortedFunctions.removeFirst() } } 3. Rewrite source with new order
  25. public func process(_ data: Data) throws -> String { guard

    let source = String(data: data, encoding: .utf8) else { throw Error.invalidInput } let parsed: SourceFileSyntax = try SyntaxParser.parse(source: source) // Gather information about the file var visitor = Indexer() parsed.walk(&visitor) // Rewrite file using visitor let rewritten = Rewriter(visitor).visit(parsed) var output = StringOutputStream() rewritten.write(to: &output) return output.string } Processor 4. Output source code
  26. // // swift run reorder -file Sources/Conference/Conference.swift > output.swift //

    guard let executablePath = CommandLine.arguments.first else { fatalError("Missing executable") } let args = Array(CommandLine.arguments.dropFirst()) let commands = stride(from: 0, to: args.count - 1, by: 2).map { (command: args[$0].dropFirst(), value: args[$0+1]) } var err = StderrOutputStream() var out = StdoutOutputStream() for (cmd, value) in commands { switch cmd { case "file": err.write("Processing \(value)\n") let content = try Data(contentsOf: URL(fileURLWithPath: value)) let result = try Processor().process(content) out.write(result) break default: err.write("Error: Unknown command: \(cmd)\n") $ ./reorder -file Conference.swift
  27. import Foundation public class Conference { public var name: String

    public let venue: String internal typealias Line = String private var attendees: Set<String> private var sponsors: Set<String> private var costs: Array<Decimal> public init(name: String, venue: String) { self.name = name self.venue = venue self.attendees = [] self.sponsors = [] self.costs = [] } private init() { self.name = "FrenchKit" self.venue = "Bâtiment" self.attendees = [] $ ./reorder -file Conference.swift
  28. None
  29. • Sort class/struct properties • Sort within extension scope •

    Sort static func/property HOMEWORK github.com/krzyzanowskim/reorder Submit Pull Request!
  30. ONE MORE THING

  31. IndexStoreDB IndexStoreDB is a source code indexing library. It provides

    a composable and efficient query API for looking up source code symbols, symbol occurrences, and relations. github.com/apple/indexstore-db
  32. IndexStoreDB IndexStoreDB is a source code indexing library. It provides

    a composable and efficient query API for looking up source code symbols, symbol occurrences, and relations. github.com/apple/indexstore-db swift build --enable-index-store .build/x86_64-apple-macosx/debug/index/store
  33. IndexStoreDB github.com/apple/indexstore-db $ swiftc -index-store-path index -index-file Sources/Conference/Conference.swift

  34. StoreIndexer public struct StoreIndexer { public typealias Path = String

    public struct Function { public let symbol: Symbol public let location: SymbolLocation } private let indexURL: URL
  35. IndexStoreDB 1. StoreIndexer fetch all files where Conference occurs 2.

    SwiftSyntax process files and output
  36. None
  37. ONE MISSING THING //===-------------------------------------------------------===// // SourceKit README //===-------------------------------------------------------===// Welcome to

    SourceKit! SourceKit is a framework for supporting IDE features like indexing, syntax-coloring, code-completion, etc. In general it provides the infrastructure that an IDE needs for excellent language support. SourceKit currently only supports the Swift language.
  38. LINKS Building a Compiler in Swift - Nick Lockwood https://www.youtube.com/watch?v=uQNkrV0F07Q

    N. Hawes & A. Lorenz “Adding Index‐While‐Building and ..." https://www.youtube.com/watch?v=jGJhnIT-D2M Creating Refactoring Transformations for Swift https://www.skilled.io/u/swiftsummit/creating-refactoring-transformations-for-swift NSHipster - Swift Syntax https://nshipster.com/swiftsyntax/ reorder https://github.com/krzyzanowskim/reorder
  39. THANK YOU @krzyzanowskim github.com/krzyzanowskim https://swift.best