GABC2018: Extend SharePoint Online into Azure by Konrad Brunner

I’ll show you an invoice processing system based on SharePoint Online. Users can send their invoices by email. The sent emails will be stored into SharePoint Online by an Azure Logic App followed by an Azure Function to prepare the metadata of the invoice. Users are guided through the invoice approval process by SharePoint workflows. The workflows communicate with Azure Functions to do different jobs. Azure Functions communicating through CSOM with SharePoint and storing files on Azure Blobs. Occurring errors in Azure Functions are reported by email to administrators An Azure GEO redundant storage is acting as the archive for approved and digitally signed invoices. Archived invoices are integrated back into SharePoint Online search to allow users finding actual and archived invoices.

Transcript

1. BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENF HAMBURG KOPENHAGEN LAUSANNE MÜNCHEN STUTTGART WIEN ZÜRICH Extend SharePoint Online into Azure Konrad Brunner https://github.com/TVDKoni
2. About me { “Name“: “Konrad Brunner“, “Short“: “Koni“, “Age“: “47“, “Childs“: [ { “Name“: “Andreas Brunner“, “Age“: “12“ } ], “Wives“: [ { “Name“: “Angela Brunner“, “Married“: “2001“ } ] } { “Ref“: “Konrad Brunner“, “Gaming“: “Minecraft, Rocket League“, “Email“: “konrad.brunner@trivadis.com“, “SharePoint“: “Since 2003“, “Azure“: “Since 2016“, “Trivadis“: “Since 2016“, “IT experience“: “More than 30 years“, “Bachelor“: “Mechanical Engineer“, “Photo“: }
3. Why this session? Online? ? Infra ? Dev Env Add-In
4. Why this session?
5. Agenda ▪ Why Azure ▪ Web App Service ▪ Logic Apps ▪ Azure Functions ▪ Full working example
6. Why Azure
7. Why Azure ▪ Learning curve ▪ Costs ➢ SharePoint Online: 205.- / TB / month ➢ Azure: 2.15 / TB / month ▪ Coding in Workflows and event receivers ➢ SharePoint Online Add-Ins: complex, lot of work, IaaS (PaaS) ➢ Azure Function: easy, fast, cheap and serverless ▪ Issues and limits ➢ List, request throttling ➢ Workflow data gets deleted after 30 days ➢ Max 30 emails per minute
8. Web App Service
9. Web App Service ▪ Multiple languages and frameworks ➢ ASP.NET, Node.js, Java, PHP, and Python, PowerShell and other scripts or executables ▪ Global scale with high availability ➢ Built-in autoscale and load balancing ➢ High availability with auto-patching ▪ DevOps optimization ➢ Continuous deployment with Git, Team Foundation Server, GitHub, and Visual Studio Team Services ▪ Security and compliance ➢ App Service is ISO, SOC, and PCI compliant ▪ Visual Studio integration ▪ Application templates ➢ WordPress, Joomla, and Drupal
10. Web App Service Choose at least basic for always on 9.- / mo 50 - 200.- / mo
11. Web App Service
12. Customer Example Push FTP On-Premises Azure DMZ Trigger ▪ Free WebApp plan ➢ CHF 0.0x Solution ➢ Max 1GB storage ➢ Outbound traffic charged if > 165 MB ▪ Free Automation plan ➢ Move from FTP to blob ➢ Max hourly schedule ➢ Max 500 run minutes a month ▪ Use blob triggers to continue ➢ Azure Function ➢ LogicApp ➢ … Azure Free plans do not have an SLA!!
13. Customer Example HTTP Web Request Azure DMZ Metadata Office Documents CSOM O365 Azure ▪ Free WebApp plan ➢ CHF 0.0x Solution ➢ You pay only outbound traffic ▪ ASP.NET MVC ➢ Async Requests for long running jobs ▪ Free Scheduler ➢ Max hourly schedule
14. Biggest tackles ▪ Webhook Timeout ➢ 1 minute
15. Async web request Use async web requests to prevent from timeout issues
16. Logic Apps
17. LogicApps ▪ Workflows, Triggers, Actions ▪ Easy to use design tools ▪ Connect APIs easily ▪ Get started quickly from templates ▪ Extensibility baked-in ▪ Real integration horsepower
18. LogicApps
19. Designer An email sent triggers the app For each attachment Create file in SharePoint Start an Azure Function to do stuff on the document with CSOM > 4 actions { }
20. LogicApps Starting every 5 minutes a logicApp with 4 actions: (60/5)*24*31*4 = CHF 25.80 / month
21. LogicApps Transparent logging
22. LogicApps ▪ Flow or LogicApps? ➢ Flow is built on top of Logic Apps ➢ They have the same workflow designer ➢ Connectors that work in one can also work in the other ➢ Flow empowers any office worker to perform simple integrations without going through developers or IT ➢ On the other hand, Logic Apps can enable advanced or mission-critical integrations where enterprise-level DevOps and security practices are required
23. Biggest tackles ▪ Webhook and Function Call Timeout ➢ 2 minutes ▪ Problems with encrypted or signed emails ▪ There is no List ID parameter ➢ Only identity is the path of the document
24. Customer Example OK Call async Error handling
25. Azure Functions
26. Azure Functions ▪ Choice of language ➢ C#, PowerShell, F#, Node.js, Java, PHP, batch, bash, or any executable ▪ Pay-per-use and AppService pricing model ▪ Bring your own dependencies (NUGet and NPM) ▪ Integrated security ▪ Flexible development (Visual Studio or in Portal) ▪ Triggers ➢ HTTPTrigger, TimerTrigger, GitHub webhook, Generic webhook, CosmosDBTrigger, BlobTrigger, QueueTrigger, EventHubTrigger, ServiceBusQueueTrigger, ServiceBusTopicTrigger
27. Azure Functions Used Memory * Seconds used * Number of runs + Total Execution rate Starting every 5 minutes a function which uses 40 MB memory and runs 10 seconds: (60/5)*24*31*(40/1024)*10 = 3500 GB-s = CHF 0.00 / month Pay-per-use plan For App Service plan prices, see Web App Service price slide
28. Azure Functions Transparent logging Monitoring problems with Pay-per-use plan!
29. Azure Functions ▪ Kudu PowerShell console ➢ Accessing host and function logs ➢ Transfering files
30. Azure Functions ▪ Report errors by email ➢ All function logs stored in storage tables ➢ Query the tables for errors once a day ➢ Send email over SMTP without authentication (only internal)
31. Azure Functions ▪ SharePoint update issues, HTTP status code 429 ➢ Refresh, Wait & Retry in Callee ➢ Refresh, Wait & Retry in Caller
32. Azure Functions ▪ Function Timeouts ➢ Consumption plan ❖ 5 minutes by default ❖ Extendable up to 10 minutes ➢ App Service plan ❖ No timeout
33. Customer Example Loop until finished is required to wait for cache update! Last action in function is an update on the field AzureFunctionState.
34. Customer Example On add O365 Azure Register or Update WebHook Process Add Background worker Message Queue LastQueryTime Trigger list.GetChanges(ChangeQuery)
35. Biggest tackles ▪ Monitoring ➢ Problems with Consumption plan ➢ Better in App Service plan ▪ Stability ➢ Problems with App Service plan ➢ Better in Consumption plan ➢ With PowerShell better than with C#
36. Full working example
37. Full working customer example Archive List Work List Approval SAP No Receipt No Approval SAP No Receipt No Storage URL On-Premises Office 365 Azure Electronic received receipt or scanned paper Trigger: New email Process new email Set state Set access rights Trinvoice Inbox PDF Document Metada maintenance Multi level approval SAP data maintenance Archiving Archive Metadata search in actual and archived documents Check for duplicate Archive Work List and Archive List indexed By email By upload Check Logs Errors to Admin by email Check storage and signature 6h 6h • Backup • Daily • Weekly • Monthly • Yearly 3 copies of each file • PDF Datei • XML Datei • Beleg Metadaten • Workflow Metadaten 3 copies of each file 3 copies of each file PDF and XML digitally signed Archive HSM protected certificate • SAP integration • PowerApps • Azure Search • ... Maintain meta data Workflow BCS workaround Backup
38. More?
39. More Azure and O365 stuff ▪ Sways https://sway.com/4awfdYgAeggrpWsG ➢ Useful O365 and Azure links ➢ Useful O365 and Azure tips and tricks ▪ Tools and scripts https://github.com/TVDKoni ➢ ARM-Base-Templates ➢ AzureStartStopVM ➢ AzureBlobStorageBackup ➢ SharePointFileShareMigrationAnalysis ➢ SharePointFileSync4Developers
40. Why this session?
41. Q&A
42. konrad.brunner@trivadis.com +41 79 960 61 49