Scala Bot in the World of Nuget Packages

Build Slack Bot on Scala can be very helpful, even if You are DevOps and your company is working on .Net framework stack.
This article describes how to build and maintain slack bot, which will help to publish Nuget packages, custom build on developers side, into the Artifactory.
This backdoor mechanism allowed us to save a lot of time and helped to find bottlenecks inside CI/CD process.

Transcript

1. 1 SCALA BOT IN THE WORLD OF NUGET PACKAGES
2. ABOUT THE AUTHOR SERGEY DZYUBAN, TECH LEAD • 10 years .NET development background • 4 year Cloud development experience (AWS) • 1 year of DevOps Tech Lead experience @SergeyDz SergeyDz
3. PART 1: IN THE WORLD OF NUGET PACKAGES
4. IN THE WORLD OF NUGET PACKAGES SBTech CI\CD infrastructure was born to produce NUGET packages • over 500 repositories • 70 Gb / 23 000 Nuget packages in artifactory • 100 package deployments per day • 2000 package deployments per month • Fully automated build and package process
5. IN THE WORLD OF NUGET PACKAGES commit hook
6. IN THE WORLD OF NUGET PACKAGES commit hook build
7. IN THE WORLD OF NUGET PACKAGES commit hook build
8. IN THE WORLD OF NUGET PACKAGES commit hook build upload
9. IN THE WORLD OF NUGET PACKAGES But sometimes (few time a day usually) something goes wrong • Need to create package without commit • Jenkins is broken • Package was removed and need to be redeployed • Need to deploy some 3d party component • etc. ... and urgent-asap-need-your-help
10. IN THE WORLD OF NUGET PACKAGES commit hook build
11. IN THE WORLD OF NUGET PACKAGES When regular Flow is broken, some additional work from PAAS is required to serve each upload request: PAAS Artifactory
12. IN THE WORLD OF NUGET PACKAGES The main issues to provide direct access for end users: • No NTLM support for Artifactory in Cloud • Artifactory doesn’t build package folders structure automatically • Artifactory audit is poor • Release and features packages should be put to different feeds
13. IN THE WORLD OF NUGET PACKAGES Package deployment logic was automated (forked from Jenkins CI scripts) and delegated to standalone Slack Bot Artifactory Slack BOT
14. IN THE WORLD OF NUGET PACKAGES Responsibility Segregation Slack Bot PAAS • UI/UX • Security/access • Audit • SLA • Backup storage • Information Slack Bot took responsibility to serve nuget package requests without DevOps to be involved.
15. IN THE WORLD OF NUGET PACKAGES Responsibility Segregation Slack Bot PAAS • UI/UX • Security/access • Audit • SLA • Backup storage • Information • Verify nuget • Upload to Artifactory • Package folder structure • Package Feed structure • Conflict resolving Slack Bot took responsibility to serve nuget package requests without DevOps to be involved.
16. IN THE WORLD OF NUGET PACKAGES Responsibility Segregation Slack Bot PAAS • UI/UX • Security/access • Audit • SLA • Backup storage • Information • Verify nuget • Upload to Artifactory • Package folder structure • Package Feed structure • Conflict resolving • Drink tequila Slack Bot took responsibility to serve nuget package requests without DevOps to be involved.
17. PART 2: CREATING SLACK BOT
18. CREATING SLACK BOT Prerequisites • Slack account (company or personal) • Artifactory account (with edit permissions to the feed) • Scala installation (v2.11.12) • Scala IDE (IntelliJ IDEA)
19. CREATING SLACK BOT Step 1: Creating Slack Bot API Key 1. Bot create page to generate API Key 2. setx slackkey xoxb-… to save key locally 3. Add newly created bot to target Slack channel
20. CREATING SLACK BOT Step 2: Creating Child Scala Bot 1. Create SBT Scala project 2. Import slack-scala-client 3. Add sample bot behavior DEMO (v1.0)
21. CREATING SLACK BOT Design for nuget upload Flow analysis Business Process Model User Attach Nuget Bot On Message Send is file? NUGET Download Is exists in AF Upload Artifactory Report Success/Failure
22. CREATING SLACK BOT General Project Overview Affected Components • Slack File API • Artifactory API • Slack Message API Authorization • Slack = API Key • Artifactory Task List 1. Detect file attached to Slack 2. Download File 3. Ensure File is Nuget 4. Detect version 5. Check if file not exists in AF 6. Calculate proper feed and path in Artifactrory 7. Upload file to Artifactory 8. Report user in Slack about success or failure
23. CREATING SLACK BOT See https://doc.akka.io/docs/akka/current/guide/introduction.html Welcome to Akka, a set of open-source libraries for designing scalable, resilient systems that span processor cores and networks. Akka allows you to focus on meeting business needs instead of writing low-level code to provide reliable behavior, fault tolerance, and high performance.
24. CREATING SLACK BOT See https://doc.akka.io/docs/akka/current/guide/introduction.html Actors interact each other by sending a messages A1 A2 A3 A4
25. CREATING SLACK BOT See https://doc.akka.io/docs/akka/current/guide/introduction.html Actors interact each other by sending a messages A1 A2 A3 A4 ??
26. CREATING SLACK BOT See https://doc.akka.io/docs/akka/current/guide/introduction.html Actors interact each other by sending a messages A1 A2 A3 A4 ?? But having access to actor system ….
27. CREATING SLACK BOT See https://doc.akka.io/docs/akka/current/guide/introduction.html Actors interact each other by sending a messages A1 A2 A3 A4 Always at most one message being processed per actor
28. CREATING SLACK BOT See https://doc.akka.io/docs/akka/current/guide/introduction.html Inside the Akka magic
29. CREATING SLACK BOT Don’t forget to commit your code GitHub is good enough for public repositories https://github.com/SergeyDz/slack-to-artifactory-bot Very easy is to add CI as a Service, like travis-ci and build each commit and pull request
30. PART 3: IT’S ALL ABOUT ACTORS
31. IT’S ALL ABOUT ACTORS Let’s design a simple Akka system, that will react on adding a file to the Slack. DEMO (v2.0) Task for the version : 1. Detect file attached to Slack 2. Create SlackFileActor for each file 3. Determine File Name and Path 4. Send Done message to Slack channel 5. Send Poison Pill to SlackFileActor sd Actor Model v2.0 SlackChannelActor SlackRTMClient SlackFileActor send(file, self) :attachment send(PoisonPill) send(UploadCompleted, self) new(api-key) send('Done') new(channel)
32. IT’S ALL ABOUT ACTORS The Final Sprint After making Slack to Bot integration, Actor system will proceed with the process of pushing attached nuget package to the Artifactrory. Tasks list: - file need to be downloaded - file need to be unzipped to get nuget version and package - check if this package exists in Artifactory - upload package - delete slack attachment - notify end user
33. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack file
34. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack file Slack File Actor channel
35. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack file Slack File Actor channel
36. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack file Slack File Actor download channel
37. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack file Slack File Actor channel, file
38. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack file Slack File Actor channel, file Nuget Actor
39. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack file Slack File Actor channel, file Nuget Actor
40. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack file Slack File Actor channel, file Nuget Actor
41. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack file Slack File Actor channel, file, nuget Nuget Actor
42. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack file Slack File Actor channel, file, nuget Nuget Actor
43. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack Slack File Actor channel, file, nuget Nuget Actor
44. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack Slack File Actor Nuget Actor Artifactory Actor channel, file, nuget
45. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack Slack File Actor Nuget Actor Artifactory Actor channel, file, nuget
46. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack Slack File Actor Nuget Actor Artifactory Actor channel, file, nuget Artifactory isExists?
47. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack Slack File Actor Nuget Actor Artifactory Actor channel, file, nuget Artifactory file upload
48. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack Slack File Actor Nuget Actor Artifactory Actor channel, file, nuget Artifactory file
49. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack Slack File Actor Nuget Actor Artifactory Actor channel, file, nuget Artifactory file
50. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack Slack File Actor Nuget Actor Artifactory Actor channel, file, nuget Artifactory file
51. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack Slack File Actor Nuget Actor Artifactory Actor channel, file, nuget Artifactory file Done
52. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack Slack File Actor Nuget Actor Artifactory Actor channel, file, nuget Artifactory file Done
53. IT’S ALL ABOUT ACTORS Slack Channel Actor Slack Artifactory file Done
54. IT’S ALL ABOUT ACTORS 1. SlackChannelActor • listen Slack • create SlackFileActor 2. SlackFileActor • download file • delete file 3. NugetActor • recognize nuget package • unzip nuget • get nuget id and version 4. ArtifactoryActor • check if package exists • do upload DEMO (v3.0)