Swiftly Swift manipulation... with Swift

This presentation seeks to acquaint users of the Swift language tooling. Parsing, generating and manipulating Swift source code in order to build useful devtool for Swift, in Swift.

Video: https://youtu.be/cJkiCEJ7zIY

Transcript

1. Swiftly Swift manipulation... with Swift FrenchKit 2019 Marcin Krzyzanowski
2. Agenda • Problem • Introduce SwiftSyntax • github.com/krzyzanowskim/reorder • IndexSourceDB
3. PROBLEM https://twitter.com/krzyzanowskim/status/1162306459075588096
4. import Foundation public class Conference { public var name: String public let venue: String INPUT
5. public class Conference { public var name: String public let venue: String internal typealias Line = String private var attendees: Set<String> private var sponsors: Set<String> private var costs: Array<Decimal> internal func add(attendee: String) internal func add(sponsor: String) deinit internal func add(cost: Decimal) private init() public func totalCosts() -> Decimal private func statement() -> Array<Line> internal func printStatement() public init(name: String, venue: String) } API
6. public class Conference { public var name: String public let venue: String internal typealias Line = String private var attendees: Set<String> private var sponsors: Set<String> private var costs: Array<Decimal> public init(name: String, venue: String) private init() deinit public func totalCosts() -> Decimal internal func add(attendee: String) internal func add(sponsor: String) internal func add(cost: Decimal) internal func printStatement() private func statement() -> Array<Line> } API Initializer Public Internal Private
7. public class Conference { public var name: String public let venue: String internal typealias Line = String private var attendees: Set<String> private var sponsors: Set<String> private var costs: Array<Decimal> internal func add(attendee: String) internal func add(sponsor: String) deinit internal func add(cost: Decimal) private init() public func totalCosts() -> Decimal private func statement() -> Array<Line> internal func printStatement() public init(name: String, venue: String) } public class Conference { public var name: String public let venue: String internal typealias Line = String private var attendees: Set<String> private var sponsors: Set<String> private var costs: Array<Decimal> public init(name: String, venue: String) private init() deinit public func totalCosts() -> Decimal internal func add(attendee: String) internal func add(sponsor: String) internal func add(cost: Decimal) internal func printStatement() private func statement() -> Array<Line> } →
8. HOW? move lines by hand use editor plugin use existing command line tool write a tool by my own
9. HOW? move lines by hand use editor plugin use existing command line tool write a tool by my own
10. HOW? move lines by hand use editor plugin use existing command line tool write a tool by my own
11. HOW? move lines by hand use editor plugin use existing command line tool write a tool by my own nicklockwood/SwiftFormat google/swift (format) inamiy/SwiftRewriter
12. HOW? move lines by hand use editor plugin use existing command line tool write a tool by my own
13. HOW? HOW? write a tool by my own Brain + Time + Computer + Programming Language + Library + ⏲ + + C +
14. you have brain
15. The Internet
16. SwiftSyntax github.com/apple/swift-syntax Source Code text/plain Abstract Syntax Tree
17. SwiftSyntax github.com/apple/swift-syntax Abstract Syntax Tree Syntax Syntax Syntax Syntax Syntax Syntax
18. SwiftSyntax Syntax Syntax Syntax Syntax Syntax Syntax Visitor Design Pattern The visitor design pattern is a way of separating an algorithm from an object structure on which it operates
19. SwiftSyntax import SwiftSyntax struct Visitor: SyntaxVisitor { mutating func visit(_ node: InitializerDeclSyntax) -> SyntaxVisitorContinueKind { self.process(node) return .skipChildren } }
20. SwiftSyntax class Rewriter: SyntaxRewriter { init() { } override func visit(_ node: DeinitializerDeclSyntax) -> DeclSyntax { return SyntaxFactory.makeBlankClassDecl() } }
21. PLAN 1. Search for all declarations (func, init, deinit) 2. Sort functions by access level and kind 3. Rewrite source with new order 4. Output source code
22. class Indexer: SyntaxVisitor { private var functions: Array<DeclSyntax> = [] func visit(_ node: DeinitializerDeclSyntax) -> SyntaxVisitorContinueKind{ self.functions.append(node) return .skipChildren } func visit(_ node: InitializerDeclSyntax) -> SyntaxVisitorContinueKind { self.functions.append(node) return .skipChildren } func visit(_ node: FunctionDeclSyntax) -> SyntaxVisitorContinueKind { self.functions.append(node) return .skipChildren } } Indexer 1. Search for all declarations (func, init, deinit)
23. func sorted() -> Array<DeclSyntax> { Sort 2. Sort functions by access level and kind
24. Rewrite class Rewriter: SyntaxRewriter { private let indexer: Indexer private var sortedFunctions: Array<DeclSyntax> init(_ indexer: Indexer) { self.indexer = indexer self.sortedFunctions = indexer.sorted() } override func visit(_ node: DeinitializerDeclSyntax) -> DeclSyntax { sortedFunctions.removeFirst() } override func visit(_ node: InitializerDeclSyntax) -> DeclSyntax { sortedFunctions.removeFirst() } override func visit(_ node: FunctionDeclSyntax) -> DeclSyntax { sortedFunctions.removeFirst() } } 3. Rewrite source with new order
25. public func process(_ data: Data) throws -> String { guard let source = String(data: data, encoding: .utf8) else { throw Error.invalidInput } let parsed: SourceFileSyntax = try SyntaxParser.parse(source: source) // Gather information about the file var visitor = Indexer() parsed.walk(&visitor) // Rewrite file using visitor let rewritten = Rewriter(visitor).visit(parsed) var output = StringOutputStream() rewritten.write(to: &output) return output.string } Processor 4. Output source code
26. // // swift run reorder -file Sources/Conference/Conference.swift > output.swift // guard let executablePath = CommandLine.arguments.first else { fatalError("Missing executable") } let args = Array(CommandLine.arguments.dropFirst()) let commands = stride(from: 0, to: args.count - 1, by: 2).map { (command: args[$0].dropFirst(), value: args[$0+1]) } var err = StderrOutputStream() var out = StdoutOutputStream() for (cmd, value) in commands { switch cmd { case "file": err.write("Processing \(value)\n") let content = try Data(contentsOf: URL(fileURLWithPath: value)) let result = try Processor().process(content) out.write(result) break default: err.write("Error: Unknown command: \(cmd)\n") $ ./reorder -file Conference.swift
27. import Foundation public class Conference { public var name: String public let venue: String internal typealias Line = String private var attendees: Set<String> private var sponsors: Set<String> private var costs: Array<Decimal> public init(name: String, venue: String) { self.name = name self.venue = venue self.attendees = [] self.sponsors = [] self.costs = [] } private init() { self.name = "FrenchKit" self.venue = "Bâtiment" self.attendees = [] $ ./reorder -file Conference.swift
29. • Sort class/struct properties • Sort within extension scope • Sort static func/property HOMEWORK github.com/krzyzanowskim/reorder Submit Pull Request!
30. ONE MORE THING
31. IndexStoreDB IndexStoreDB is a source code indexing library. It provides a composable and efficient query API for looking up source code symbols, symbol occurrences, and relations. github.com/apple/indexstore-db
32. IndexStoreDB IndexStoreDB is a source code indexing library. It provides a composable and efficient query API for looking up source code symbols, symbol occurrences, and relations. github.com/apple/indexstore-db swift build --enable-index-store .build/x86_64-apple-macosx/debug/index/store
33. IndexStoreDB github.com/apple/indexstore-db $ swiftc -index-store-path index -index-file Sources/Conference/Conference.swift
34. StoreIndexer public struct StoreIndexer { public typealias Path = String public struct Function { public let symbol: Symbol public let location: SymbolLocation } private let indexURL: URL
35. IndexStoreDB 1. StoreIndexer fetch all files where Conference occurs 2. SwiftSyntax process files and output
37. ONE MISSING THING //===-------------------------------------------------------===// // SourceKit README //===-------------------------------------------------------===// Welcome to SourceKit! SourceKit is a framework for supporting IDE features like indexing, syntax-coloring, code-completion, etc. In general it provides the infrastructure that an IDE needs for excellent language support. SourceKit currently only supports the Swift language.
38. LINKS Building a Compiler in Swift - Nick Lockwood https://www.youtube.com/watch?v=uQNkrV0F07Q N. Hawes & A. Lorenz “Adding Index‐While‐Building and ..." https://www.youtube.com/watch?v=jGJhnIT-D2M Creating Refactoring Transformations for Swift https://www.skilled.io/u/swiftsummit/creating-refactoring-transformations-for-swift NSHipster - Swift Syntax https://nshipster.com/swiftsyntax/ reorder https://github.com/krzyzanowskim/reorder
39. THANK YOU @krzyzanowskim github.com/krzyzanowskim https://swift.best