At the Crossroads of Agile and Automation - AWS MENA

Slide 1

Slide 1 text

COMMUNITY DAY MENA At the Crossroads of Agile and Automation

Slide 2

Slide 2 text

Introductions 2 Joseph Scholz Agile Delivery Lead Joe’s professional journey has been eclectic. He started as a Teacher and then moved to playing basketball in Spain. After that he worked at a Cafe and eventually settled in IT as an Agile Delivery Lead for the past 4 years. Currently he works with Cloud Engineering Teams to improve processes and delivery Enterprise solutions

Slide 3

Slide 3 text

Introductions 3 Eric Schultz Senior Software Engineer Eric is a Senior Software Engineer at Capital One. His role is focused on cloud engineering and compliance while improving the developer experience. Eric has experience with several cloud platforms and managing enterprise workloads.

Slide 4

Slide 4 text

Agenda 4 • Background • Agile Intent • Crossroads Solution

Slide 5

Slide 5 text

Background 5

Slide 6

Slide 6 text

Compliance Opportunities 6 ● Limit the amount of compliance notifications (email overload)

Slide 7

Slide 7 text

‘Radio Noise’ Effect 7

Slide 8

Slide 8 text

Compliance Opportunities 8 ● Limit the amount of compliance notifications (email overload) ● Track and Prioritize efforts with Agile principles ● Automated validation instead of verifying manually

Slide 9

Slide 9 text

What We Built 9 ● Integrated Solution that leverages Agile principles and single source-of-truth verification ● Automated workflow to map work items to Cloud resources

Slide 10

Slide 10 text

Agile Intent 10

Slide 11

Slide 11 text

Basic Agile and Business Intent ● Provide a consistent and organized format ● Adhere to established Agile patterns ● Give contextual information ● Control over the creation of work ● Tracking and Metrics 11

Slide 12

Slide 12 text

Agile Nomenclature and Hierarchy ● ‘Issue’ is a generic term for any work item. It doesn’t mean ‘bug’ or ‘defect’ even though it sounds that way, so we will try not to use that term ● ‘Epic’ is a defined effort with a result (e.g. Amazon Machine Image Rehydration ‘Epic’) ● ‘Story’ is a unit or part of an Epic such that all Stories must be complete for the overall Epic to be done (e.g. AMI Rehydration for a specific cloud resource) 12

Slide 13

Slide 13 text

Consistent Format 13

Slide 14

Slide 14 text

Visualize the Work and Involve the Team 14

Slide 15

Slide 15 text

Provide Criteria and Prioritization 15

Slide 16

Slide 16 text

Single Source of Truth ● Dedicated Agile ‘parent project’ 16 ● System Account with project access ○ reduce the noise ● Labels and Custom Fields ○ Sorting ○ Queries

Slide 17

Slide 17 text

Produce Metrics for Accountability 17

Slide 18

Slide 18 text

Crossroads Solution 18

Slide 19

Slide 19 text

Initial Implementation ● Architecture ○ Python package ○ Batch (AWS Lambda) ○ DynamoDB (NoSQL) ● Dependencies ○ jira-python ○ Jira API ○ Boto3 ○ ServiceNow API 19

Slide 20

Slide 20 text

Discovery Clients ● Discover non-compliant AWS resources ○ Across several AWS Accounts ● Discovery Client per AWS resource type ○ Example: S3 ● Return value ○ non-compliant resources grouped by Team - “Team” Tag 20

Slide 21

Slide 21 text

Jira Creator Client ● Input ○ Cloud control / Epic ○ Discovery Client output ● Stories are created per team ● Creates / Updates / Closes Stories ○ Create - no story exists or story is closed ○ Update - story exists and story is open ○ Close - story is open ● DynamoDB 21

Slide 22

Slide 22 text

The Life of a Story - Daily Cycle 1.Creation a.Boto3 i. Gather resources ii. Sort by Team iii.Create Story per Team Description includes list of resources 22

Slide 23

Slide 23 text

The Life of a Story - Accountability 2. Assignment a.Service Now i. Engineering Lead b.Additional Stakeholders i. Product Owner ii. Agile Delivery Lead 23

Slide 24

Slide 24 text

The Life of a Work Item - Compliance Assignment 3. Epic a.1 Epic per Cloud Control b.1 story per team c. Each story is assigned to the epic d.Track company’s overall completion status 24

Slide 25

Slide 25 text

The Life of a Story - Team Management 4. Workflow Progression a.Team can make manual transitions Backlog -> In-Progress 25

Slide 26

Slide 26 text

The Life of a Story - Automated Workflow 5. Closing a.Validation i. Resources’ action items are completed b.Completion i. Closed Status ii. Comment 26

Slide 27

Slide 27 text

Workflow ● Daily Trigger ● Multithreaded ○ 1 per Cloud Control ● Discovers resources ● Jira Actions ○ Create/Update/Close ● Stores results in DB 27

Slide 28

Slide 28 text

Results 28

Slide 29

Slide 29 text

Results ● Clear Expectations for the Teams ● Iteration and Roadmap Planning Capability ● Accountability and Actionability ● Ability to evaluate team efforts for improvement ● Easier tracking through a defined system of record ● Peace of mind 29

Slide 30

Slide 30 text

Questions and Conversations 30

Slide 31

Slide 31 text

That’s All Folks 31 Contact Joseph.Scholz@capitalone.com Eric.Schultz@capitalone.com Facebook – Agile Joe Instagram @AgileCupOfJoe LinkedIn - ericschultzd