Automation and Integration using Python

This presentation about "Automation and Integration using Python" was given at PyCon Uganda on September 22, 2023 in Kampala.

It summaries the common uses of Python in automation and integration and the libraries often used. It also contains examples and code snippets to how Python is used.

The article on this topic is here: https://kambale.dev/automation-and-integration-using-python

Transcript

1. Wesley Kambale @weskambale kambale.dev Automation and Integration using Python
2. • Machine Learning Engineer with 3 years of experience • Community Builder for 3 years • Explore ML Facilitator with Crowdsource by Google for 2 years • Consultant at The Innovation Village • Google Dev Library Contributor Profile Interests Experience • Research in TinyML, TTS & LLM
3. Automation & Integration… What about Automation & Integration? • Manual tasks • Merging systems or apps • Time is saved, errors are minimized, and productivity is significantly improved
4. Common Use What are the use cases? • Automating data backups and file synchronization • Automating repetitive data entry tasks • Scraping data from websites for analysis or monitoring • Integrating different systems or applications to exchange data
5. Common Use - Cont’d What are the use cases? • Automating report generation and data analysis • Interacting with APIs to fetch data from external services • Automating database operations and data processing • Monitoring and alerting systems based on predefined conditions
6. Libraries used What are the Python libraries? • os and shutil: These libraries provide functions for automating file operations like moving, copying, renaming, and deleting files. • selenium: This library is used for browser automation, allowing you to control web browsers programmatically and perform actions like filling forms, clicking buttons, and scraping dynamic web content.
7. Libraries used - cont’d What are the Python libraries? • pandas: This library is widely used for data manipulation and analysis. It can help automate tasks involving data processing and generating reports • pyodbc and sqlite3: These libraries enable database integration, allowing you to connect to databases, execute queries, and automate data operations
8. Automating File Operations How do you automate file operations? • The os and shutil libraries for automating file operations • The copytree() function copies the entire directories
9. import shutil
 
 # Define the source & destination paths
 source_directory = '/path/to/source'
 destination_directory = '/path/to/destination'
 
 shutil.copytree(source_directory, destination_directory)
 
 

10. Web Scraping How do you automate web scraping? • requests and beautifulsoup4 libraries • BS4 sends a HTTP GET requests
11. import requests
 from bs4 import BeautifulSoup
 
 url = 'https://example.com'
 
 response = requests.get(url)
 soup = BeautifulSoup(response.text, 'html.parser')
 
 title = soup.title.text
 paragraphs = soup.find_all('p')
 
 print('Title:', title)
 print('Paragraphs:')
 for p in paragraphs:
 print(p.text)
 

13. Interacting with APIs Automation in APIs • requests library interacts with web APIs • This library sends an HTTP GET request to a publicly available API, and acquire the response as JSON data.
14. import requests
 
 url = 'https://api.example.com/data'
 
 response = requests.get(url)
 data = response.json()
 
 # Process the data
 for item in data:
 print(item['name'], item['value'])
 

16. Database Integration Integrating databases in Python • pyodbc and sqlite3 are used for integrating with databases • connect() function, execute a SELECT query using the execute method, retrieve the results using the fetchall method, and then process the retrieved data
17. import sqlite3
 
 connection = sqlite3.connect('example.db')
 cursor = connection.cursor()
 
 cursor.execute('SELECT * FROM users')
 
 results = cursor.fetchall()
 
 for row in results:
 print(row)
 
 cursor.close()
 connection.close()

18. Creating Automated Reports Automated Reports in Python • pandas is used for data manipulation and analysis, making it helpful for creating automated reports • The read_csv() function reads data from a CSV file. Then use various functions and methods to perform calculations on the data and generate a report
19. import pandas as pd
 
 data = pd.read_csv('data.csv')
 
 total_sales = data['sales'].sum()
 average_sales = data['sales'].mean()
 
 report = f'Total sales: {total_sales}\nAverage sales: {average_sales}'
 print(report)

20. Automated Emails Sending Automated Emails in Python • smtplib, csv, ssl are some of the libraries that we use
21. import smtplib
 from email.mime.text import MIMEText 
 from email.mime.multipart import MIMEMultipart
 
 username = 'your_email_address'
 password = 'your_email_password'
 
 message = MIMEMultipart()
 message['From'] = username
 message['Subject'] = 'Subject line'
 
 body = 'Body of the email'
 message.attach(MIMEText(body, 'plain'))
 
 to_emails = ['recipient1@example.com', 'recipient2@example.com']

22. Read more… Blog: https://kambale.dev Article: https://kambale.dev/automation-and-integration-using-python
23. Wesley Kambale @weskambale kambale.dev Thank you! Any questions?