Generating Code And Other Mischief With Swift Package Manager Plugins - Server-Side Swift, London, December 2022

It’s the build tooling step everyone hates: “Now, add a new Run Script Build Phase.” The Swift and Xcode teams have worked to try to make things at least a little bit better for things which need to happen at build time by adding Build Tool Plugins and Command Plugins to Swift Package manager.

In this talk, you’ll get a look at how to set these up to generate code and documentation, plus a look at some other silly things you can make it do.

Note: This talk has a section that refers to the 360iDev version of the talk. Slides of that version are available here.

Sample App: https://github.com/designatednerd/RadioactiveCat
Sample Plugin: https://github.com/designatednerd/DatedImageGenerator

Transcript

1. GENERATING CODE AND OTHER MISCHIEF WITH SWIFT PACKAGE MANAGER PLUGINS SEVER-SIDE SWIFT | LONDON | DECEMBER 2022 ELLEN SHAPIRO | @DESIGNATEDNERD | GUSTO.COM
2. GUSTO != GOUSTO
6. [SCENE MISSING]
8. AN AMERICAN PAYROLL / HR COMPANY
9. @GUSTOHQ
11. A UK MEAL PREP COMPANY
12. @GOUSTOCOOKING
13. PLEASE STOP SENDING AMERICANS WHO CAN'T DO ANYTHING ABOUT IT PHOTOS OF YOUR MOULDY FOOD
14. PLEASE STOP SENDING AMERICANS WHO CAN'T DO ANYTHING ABOUT IT PHOTOS OF YOUR MOULDY FOOD Cheers!
15. ! SPM PLUGINS!
20. ALCATRAZ
32. ! XCODE GHOST
33. ! XCODE GHOST
34. ! STRAWHORSE
37. !
39. XCODE SOURCE EXTENSIONS
40. REQUIRED HOST MAC APP
41. REQUIRED HOST MAC APP ▸ ! Can now be distributed easily on the Mac App store
42. REQUIRED HOST MAC APP ▸ ! Can now be distributed easily on the Mac App store ▸ " Sandboxing restrictions
43. REQUIRED HOST MAC APP ▸ ! Can now be distributed easily on the Mac App store ▸ " Sandboxing restrictions ▸ # Often winds up with an empty app
44. func perform(with invocation: XCSourceEditorCommandInvocation, completionHandler: @escaping (Error?) -> Void) { // Actual code of plugin }
46. ONLY SINGLE-FILE TEXT BUFFER ACCESS
47. ONLY SINGLE-FILE TEXT BUFFER ACCESS ▸ ! Does allow at least some sorting, linting, formatting
48. ONLY SINGLE-FILE TEXT BUFFER ACCESS ▸ ! Does allow at least some sorting, linting, formatting ▸ ☹ Only ever the current file
49. ONLY SINGLE-FILE TEXT BUFFER ACCESS ▸ ! Does allow at least some sorting, linting, formatting ▸ ☹ Only ever the current file ▸ # No access to the AST
50. ONLY SINGLE-FILE TEXT BUFFER ACCESS ▸ ! Does allow at least some sorting, linting, formatting ▸ ☹ Only ever the current file ▸ # No access to the AST ▸ $ No access to file or project structure
51. ONLY SINGLE-FILE TEXT BUFFER ACCESS ▸ ! Does allow at least some sorting, linting, formatting ▸ ☹ Only ever the current file ▸ # No access to the AST ▸ $ No access to file or project structure ▸ $ No user interface permitted
52. FURTHER INVOCATION LIMITATIONS
53. FURTHER INVOCATION LIMITATIONS ▸ ! Cannot run in background
54. FURTHER INVOCATION LIMITATIONS ▸ ! Cannot run in background ▸ " Must be actively invoked by the user
55. FURTHER INVOCATION LIMITATIONS ▸ ! Cannot run in background ▸ " Must be actively invoked by the user ▸ # No default key binding
61. SWIFT PACKAGE MANAGER ! PLUGINS
66. ✅ SWIFT 5.6
67. ✅ SWIFT 5.6 ! XCODE 14
68. 1. COMMAND PLUGINS 2. BUILD TOOL PLUGINS
69. COMMAND PLUGINS
70. CODE FROM "CREATE SWIFT PACKAGE PLUGINS" HTTPS://DEVELOPER.APPLE.COM/VIDEOS/PLAY/WWDC2022/110401/
71. COMMAND PLUGINS ▸ Can ask permission to write in the package directory
72. // Package.swift .plugin( name: "GenerateContributors", capability: .command( intent: .custom(verb: "regenerate-contributors-list", description: "Generates the CONTRIBUTORS.txt file based on Git logs"), permissions: [ .writeToPackageDirectory(reason: "This command write the new CONTRIBUTORS.txt to the source root.") ] )),
73. // Package.swift .plugin( name: "GenerateContributors", capability: .command( intent: .custom(verb: "regenerate-contributors-list", description: "Generates the CONTRIBUTORS.txt file based on Git logs"), permissions: [ .writeToPackageDirectory(reason: "This command write the new CONTRIBUTORS.txt to the source root.") ] )),
74. COMMAND PLUGINS ▸ Can ask permission to write in the package directory ▸ Can be invoked at the command line
75. // Get all plugins available on a particular package swift package plugin --list // Run a specific plugin, in this case GenerateContributors swift package regenerate-contributors-list
77. COMMAND PLUGINS ▸ Can ask permission to write in the package directory ▸ Can be invoked at the command line ▸ Can be invoked from a menu item in Xcode
83. COMMAND PLUGINS ▸ Can ask permission to write in the package directory ▸ Can be invoked at the command line ▸ Can be invoked from a menu item Xcode ▸ Either way, must be invoked separately from the build process
84. BUILD TOOL PLUGINS
85. BUILD TOOL PLUGINS RUN AUTOMATICALLY ON EVERY BUILD
86. BUILD TOOL PLUGINS (THE THING I REALLY WANT TO USE)
87. THE MORE COMPLICATED BUT LESS COMPLICATED WAY: EMBED A PACKAGE IN YOUR .XCODEPROJ
88. MORE COMPLICATED (YOU HAVE TO ADD A WHOLE SWIFT PACKAGE TO YOUR PROJECT)
89. LESS COMPLICATED (IT WORKS WAY MORE RELIABLY)
90. SAMPLE APP https://github.com/designatednerd/ RadioactiveCat
91. via https://energyeducation.ca/encyclopedia/Half_life
95. BUILD TOOL EASY MODE BRING IN SOMEONE ELSE'S PLUGIN
96. DO NIL DISTURB HTTPS://GITHUB.COM/ICANZILB/ DONILDISTURBPLUGIN
97. // Package.swift .dependencies: [ .package(url: "https://github.com/icanzilb/ DoNilDisturbPlugin.git", from: "0.0.5"), ]
98. // Package.swift .target( name: "Cats", dependencies: [ ], plugins: [ .plugin(name: "DoNilDisturbPlugin", package: "DoNilDisturbPlugin") ] ),
100. DO NOT BLINDLY CLICK TRUST & ENABLE ALL
106. BUILD TOOL HARD MODE: CREATE YOUR OWN PLUGIN
107. DON'T BUILD YOUR PLUGIN AND THE PACKAGE YOU'RE TESTING IT AGAINST IN THE SAME REPO
108. GITHUB.COM/DESIGNATEDNERD/ DATEDIMAGEGENERATOR
109. PLUGIN VS. EXECUTABLE
110. // File structure of the Swift Lib ! DatedImageGenerator ! Plugins ! DatedImageGenerator " DatedImageGenerator.swift ! Sources ! DatedImage " DatedImage.swift " ExifDateFormatter.swift ! DatedImageGeneratorExecutable " DatedImageGeneratorExecutable.swift ! Tests ! DatedImageGeneratorTests " DatedImageGeneratorExecutable.swift " Media.xcassets
111. // File structure of the Swift Lib ! DatedImageGenerator ! Plugins ! DatedImageGenerator " DatedImageGenerator.swift ! Sources ! DatedImage " DatedImage.swift " ExifDateFormatter.swift ! DatedImageGeneratorExecutable " DatedImageGeneratorExecutable.swift ! Tests ! DatedImageGeneratorTests " DatedImageGeneratorExecutable.swift " Media.xcassets
112. // File structure of the Swift Lib ! DatedImageGenerator ! Plugins ! DatedImageGenerator " DatedImageGenerator.swift ! Sources ! DatedImage " DatedImage.swift " ExifDateFormatter.swift ! DatedImageGeneratorExecutable " DatedImageGeneratorExecutable.swift ! Tests ! DatedImageGeneratorTests " DatedImageGeneratorExecutable.swift " Media.xcassets
113. // File structure of the Swift Lib ! DatedImageGenerator ! Plugins ! DatedImageGenerator " DatedImageGenerator.swift ! Sources ! DatedImage " DatedImage.swift " ExifDateFormatter.swift ! DatedImageGeneratorExecutable " DatedImageGeneratorExecutable.swift ! Tests ! DatedImageGeneratorTests " DatedImageGeneratorExecutable.swift " Media.xcassets
114. // File structure of the Swift Lib ! DatedImageGenerator ! Plugins ! DatedImageGenerator " DatedImageGenerator.swift ! Sources ! DatedImage // <-- Exported library used in executable and main app " DatedImage.swift " ExifDateFormatter.swift ! DatedImageGeneratorExecutable " DatedImageGeneratorExecutable.swift ! Tests ! DatedImageGeneratorTests " DatedImageGeneratorExecutable.swift " Media.xcassets
115. // File structure of the Swift Lib ! DatedImageGenerator ! Plugins ! DatedImageGenerator " DatedImageGenerator.swift ! Sources ! DatedImage // <-- Exported Framework used in executable and main app " DatedImage.swift " ExifDateFormatter.swift ! DatedImageGeneratorExecutable // <-- The thing that does all the work " DatedImageGeneratorExecutable.swift ! Tests ! DatedImageGeneratorTests " DatedImageGeneratorExecutable.swift " Media.xcassets
116. // File structure of the Swift Lib ! DatedImageGenerator ! Plugins ! DatedImageGenerator " DatedImageGenerator.swift ! Sources ! DatedImage " DatedImage.swift " ExifDateFormatter.swift ! DatedImageGeneratorExecutable " DatedImageGeneratorExecutable.swift ! Tests ! DatedImageGeneratorTests " DatedImageGeneratorExecutable.swift " Media.xcassets
117. // Plugins/DatedImageGenerator.swift @main struct DatedImageGenerator: BuildToolPlugin { func createBuildCommands(context: PluginContext, target: Target) async throws -> [Command] { // TODO: Create info for build command return [ .buildCommand( displayName: "Generate Dated Images", executable: try context.tool(named: "DatedImageGeneratorExecutable").path, // more to come here arguments: "TODO", outputFiles: ["TODO"] ) ] } }
118. WHAT'S IN THE EXECUTABLE?
119. // Sources/DatedImageGeneratorExecutable/ // DatedImageGeneratorExecutable.swift @main struct DatedImageGeneratorExecutable { static func main() throws { // How do we get information from the Plugin? } }
120. // Sources/DatedImageGeneratorExecutable/ // DatedImageGeneratorExecutable.swift @main struct DatedImageGeneratorExecutable { static func main() throws { // Grab the 2nd argument as a string ProcessInfo.processInfo.arguments[1] } }
121. // Sources/DatedImageGeneratorExecutable/ // DatedImageGeneratorExecutable.swift @main struct DatedImageGeneratorExecutable { static func main() throws { // Turn it into Data Data(ProcessInfo.processInfo.arguments[1].utf8)) } }
122. // Sources/DatedImageGeneratorExecutable/ // DatedImageGeneratorExecutable.swift @main struct DatedImageGeneratorExecutable { static func main() throws { // Decode it into a codable object! let invocation = try JSONDecoder() .decode(PluginInvocation.self, from: Data(ProcessInfo.processInfo.arguments[1].utf8)) } }
123. "I should just put the codable object in the executable and import it into the plugin"
127. // One copy of this in the plugin, one in the executable struct PluginInvocation: Codable { let catalogPaths: [String] let outputPath: String func encodedString() throws -> String { let data = try JSONEncoder().encode(self) return String(decoding: data, as: UTF8.self) } }
128. // One copy of this in the plugin, one in the executable // (or share with a common lib) struct PluginInvocation: Codable { let catalogPaths: [String] let outputPath: String func encodedString() throws -> String { let data = try JSONEncoder().encode(self) return String(decoding: data, as: UTF8.self) } }
129. // Plugins/DatedImageGenerator.swift @main struct DatedImageGenerator: BuildToolPlugin { func createBuildCommands(context: PluginContext, target: Target) async throws -> [Command] { guard let target = target as? SourceModuleTarget else { throw ImageGenError.notASourceModule } let assetCatalogPaths = target.sourceFiles(withSuffix: "xcassets") .compactMap { assetCatalog in return assetCatalog.path.string } let outputPath = context.pluginWorkDirectory .appending(["DatedImages.swift"]) let invocation = PluginInvocation(catalogPaths: assetCatalogPaths, outputPath: outputPath.string) return [ .buildCommand( displayName: "Generate Dated Images", executable: try context.tool(named: "DatedImageGeneratorExecutable").path, arguments: [try invocation.encodedString()], outputFiles: [outputPath] ) ] } }
130. // Plugins/DatedImageGenerator.swift @main struct DatedImageGenerator: BuildToolPlugin { func createBuildCommands(context: PluginContext, target: Target) async throws -> [Command] { guard let target = target as? SourceModuleTarget else { throw ImageGenError.notASourceModule } let assetCatalogPaths = target.sourceFiles(withSuffix: "xcassets") .compactMap { assetCatalog in return assetCatalog.path.string } let outputPath = context.pluginWorkDirectory .appending(["DatedImages.swift"]) let invocation = PluginInvocation(catalogPaths: assetCatalogPaths, outputPath: outputPath.string) return [ .buildCommand( displayName: "Generate Dated Images", executable: try context.tool(named: "DatedImageGeneratorExecutable").path, arguments: [try invocation.encodedString()], outputFiles: [outputPath] ) ] } }
131. ! TARGET (TECHNICALLY PACKAGEPLUGIN.TARGET)
132. // Plugins/DatedImageGenerator.swift @main struct DatedImageGenerator: BuildToolPlugin { func createBuildCommands(context: PluginContext, target: Target) async throws -> [Command] { guard let target = target as? SourceModuleTarget else { throw ImageGenError.notASourceModule } let assetCatalogPaths = target.sourceFiles(withSuffix: "xcassets") .compactMap { assetCatalog in return assetCatalog.path.string } let outputPath = context.pluginWorkDirectory .appending(["DatedImages.swift"]) let invocation = PluginInvocation(catalogPaths: assetCatalogPaths, outputPath: outputPath.string) return [ .buildCommand( displayName: "Generate Dated Images", executable: try context.tool(named: "DatedImageGeneratorExecutable").path, arguments: [try invocation.encodedString()], outputFiles: [outputPath] ) ] } }
133. ! PLUGIN CONTEXT (TECHNICALLY PACKAGEPLUGIN.PLUGINCONTEXT)
134. PLUGINWORKDIRECTORY IS YOUR SANDBOX
135. // Plugins/DatedImageGenerator.swift @main struct DatedImageGenerator: BuildToolPlugin { func createBuildCommands(context: PluginContext, target: Target) async throws -> [Command] { guard let target = target as? SourceModuleTarget else { throw ImageGenError.notASourceModule } let assetCatalogPaths = target.sourceFiles(withSuffix: "xcassets") .compactMap { assetCatalog in return assetCatalog.path.string } let outputPath = context.pluginWorkDirectory .appending(["DatedImages.swift"]) let invocation = PluginInvocation(catalogPaths: assetCatalogPaths, outputPath: outputPath.string) return [ .buildCommand( displayName: "Generate Dated Images", executable: try context.tool(named: "DatedImageGeneratorExecutable").path, arguments: [try invocation.encodedString()], outputFiles: [outputPath] ) ] } }
136. // Plugins/DatedImageGenerator.swift @main struct DatedImageGenerator: BuildToolPlugin { func createBuildCommands(context: PluginContext, target: Target) async throws -> [Command] { guard let target = target as? SourceModuleTarget else { throw ImageGenError.notASourceModule } let assetCatalogPaths = target.sourceFiles(withSuffix: "xcassets") .compactMap { assetCatalog in return assetCatalog.path.string } let outputPath = context.pluginWorkDirectory .appending(["DatedImages.swift"]) let invocation = PluginInvocation(catalogPaths: assetCatalogPaths, outputPath: outputPath.string) return [ .buildCommand( displayName: "Generate Dated Images", executable: try context.tool(named: "DatedImageGeneratorExecutable").path, arguments: [try invocation.encodedString()], outputFiles: [outputPath] ) ] } }
138. QUICK RECAP OF PLUGIN CREATION
139. QUICK RECAP OF PLUGIN CREATION ▸ Create a plugin in Plugins
140. QUICK RECAP OF PLUGIN CREATION ▸ Create a plugin in Plugins ▸ Create an executable it will call in Sources
141. QUICK RECAP OF PLUGIN CREATION ▸ Create a plugin in Plugins ▸ Create an executable it will call in Sources ▸ Use a codable object to pass data between them
142. QUICK RECAP OF PLUGIN CREATION ▸ Create a plugin in Plugins ▸ Create an executable it will call in Sources ▸ Use a codable object to pass data between them ▸ Note that the plugin depends on but cannot import the executable
143. ! BACK TO THE KITTAHS!
144. // In Cats/Package.swift dependencies: [ .package(url: "https://github.com/designatednerd/ DatedImageGenerator.git", from: "0.0.1"), .package(url: "https://github.com/icanzilb/ DoNilDisturbPlugin.git", from: "0.0.5"), ],
145. // In Cats/Package.swift .target( name: "Cats", dependencies: [ .product(name: "DatedImage", package: "DatedImageGenerator"), ], plugins: [ .plugin(name: "DatedImageGenerator", package: "DatedImageGenerator"), .plugin(name: "DoNilDisturbPlugin", package: "DoNilDisturbPlugin") ] ),
146. // In Cats/Package.swift .target( name: "Cats", dependencies: [ .product(name: "DatedImage", package: "DatedImageGenerator"), ], plugins: [ .plugin(name: "DatedImageGenerator", package: "DatedImageGenerator"), .plugin(name: "DoNilDisturbPlugin", package: "DoNilDisturbPlugin") ] ),
147. // In Cats/Package.swift .target( name: "Cats", dependencies: [ .product(name: "DatedImage", package: "DatedImageGenerator"), ], plugins: [ .plugin(name: "DatedImageGenerator", package: "DatedImageGenerator"), .plugin(name: "DoNilDisturbPlugin", package: "DoNilDisturbPlugin") ] ),
154. WHAT ABOUT RUNNING PLUGINS ON AN APP TARGET?
155. XCODE PROJECT PLUGIN
156. XCODE PROJECT PLUGIN (A KIND OF BUILD TOOL PLUGIN)
157. THIS TALK, 360IDEV EDITION: HTTPS://WWW.YOUTUBE.COM/WATCH?V=1GCU70XZ-P8
159. ! TRUST
163. xcodebuild -skipPackagePluginValidation
165. !
166. https://forums.swift.org/t/how-do-i-get- xcode-to-programmatically-trust-a-specific- build-tool-plugin-for-ci/61331/2
169. ! ➡ ! LOCAL REPO CI
170. ! LOCALLY
171. ! LOCALLY ! CI MACHINE
175. ! LOCALLY ! CI MACHINE
176. ! FB11772479
177. SPM PLUGIN LIMITATIONS
178. SPM PLUGIN LIMITATIONS (that I've found so far)
179. SPM PLUGIN LIMITATIONS (that I've found so far) ▸ Still no UI for you
180. SPM PLUGIN LIMITATIONS (that I've found so far) ▸ Still no UI for you ▸ You can't even ask to use the network
181. SPM PLUGIN LIMITATIONS (that I've found so far) ▸ Still no UI for you ▸ You can't even ask to use the network ▸ You can't share data between plugins
182. SPM PLUGIN LIMITATIONS (that I've found so far) ▸ Still no UI for you ▸ You can't even ask to use the network ▸ You can't share data between plugins ▸ Can't do anything with build results
183. ! PLUGIN WISH LIST
184. PLUGIN WISH LIST ▸ File metadata
185. PLUGIN WISH LIST ▸ File metadata ▸ Ability to show UI in Xcode
186. OBLIGATORY SUMMARY SLIDE
187. OBLIGATORY SUMMARY SLIDE ▸ Alcatraz
188. OBLIGATORY SUMMARY SLIDE ▸ Alcatraz ! " Xcode 8 Extensions
189. OBLIGATORY SUMMARY SLIDE ▸ Alcatraz ! " Xcode 8 Extensions ▸ SPM Plugins to the rescue!
190. OBLIGATORY SUMMARY SLIDE ▸ Alcatraz ! " Xcode 8 Extensions ▸ SPM Plugins to the rescue! ▸ Command plugins: On demand, Build tool plugins: On every build
191. OBLIGATORY SUMMARY SLIDE ▸ Alcatraz ! " Xcode 8 Extensions ▸ SPM Plugins to the rescue! ▸ Command plugins: On demand, Build tool plugins: On every build ▸ Security: Be super-mindful of which plugins you use
192. OBLIGATORY SUMMARY SLIDE ▸ Alcatraz ! " Xcode 8 Extensions ▸ SPM Plugins to the rescue! ▸ Command plugins: On demand, Build tool plugins: On every build ▸ Security: Be super-mindful of which plugins you use ▸ Building plugins is a bit complicated, but also pretty fun
194. THANK YOU!
195. SWIFT-EVOLUTION LINKS! ▸ SE-0303: Extensible Build Tools github.com/apple/swift-evolution/ blob/main/proposals/0303-swiftpm-extensible-build-tools.md ▸ SE-0325: Additional Plugin APIs github.com/apple/swift-evolution/ blob/main/proposals/0325-swiftpm-additional-plugin-apis.md ▸ SE-0332: SPM Command Plugins github.com/apple/swift-evolution/ blob/main/proposals/0332-swiftpm-command-plugins.md
196. ! SLIGHTLY TERRIFYING LINKS! ▸ Xcode Ghost details: en.wikipedia.org/wiki/XcodeGhost ▸ Strawhorse leaked document: theintercept.com/document/ 2015/03/10/strawhorse-attacking-macos-ios-software- development-kit ▸ Strawhorse Context: theintercept.com/2015/03/10/ispy-cia- campaign-steal-apples-secrets/
197. ! PLUGIN LINKS! ▸ Do Nil Disturb: github.com/icanzilb/DoNilDisturbPlugin ▸ Swift SafeURL: github.com/baguio/SwiftSafeURL