Standardizing ICT for Development Towards the Definition of a Standard Process and Maturity Model for ICTD Projects Aaron Ciaghi April 11, 2014 

Outline • Defining ICTD • Project Management and SW Engineering in ICTD • The Problem • Research Methodology • An ICTD Maturity Model • Summary in 1 slide and Future Work 2 

ICTD Projects are… Multidisciplinary projects that aim at improving the socio-economic conditions of a marginalized community — often located in a rural area — using ICTs as the main tool to bridge some divide 3 “ ” 

Unique Characteristics • Beneficiaries living at the Base of the Pyramid ≠ End- Users • Many diverse stakeholders — different agendas • Social and (often) non-profit nature • Economic and social sustainability are as important as technical sustainability 4 

Challenges • Occur simultaneously • Increase risks • Unclear boundaries • Not only technical • Influence requirements • Sustainability 

Unclear Boundaries • Goals are not always well defined • Intangible/difficult to measure benefits • Impact can be observed years after the intervention • Blurred distinction between program and project (Golini and Landoni, 2013) 6 

The Issue of Sustainability • Favorite topic of ICTD scholars • Almost completely ignored by the PM discipline • Pade-Khene: Sustainability CSFs • Silvius et al.: Sustainability in PM for ICT projects • Inveneo: Guidelines for designing sustainable ICTs for low-resource environments 7 

PM in ICTD • Iterative processes • Comprehensive Evaluation • Change Management • Stakeholders management 8 

Development Processes in ICTD 9 

Development Processes in ICTD • Very limited documentation on process-oriented approaches • Increasing popularity of Agile techniques and UCD 9 

Development Processes in ICTD • Very limited documentation on process-oriented approaches • Increasing popularity of Agile techniques and UCD • ICTD as a Design Challenge (Tongia et al., 2006) 9 

Development Processes in ICTD • Very limited documentation on process-oriented approaches • Increasing popularity of Agile techniques and UCD • ICTD as a Design Challenge (Tongia et al., 2006) • Framework+Process for Inclusive Technology
 (Pitula et al., 2007) 9 

Development Processes in ICTD • Very limited documentation on process-oriented approaches • Increasing popularity of Agile techniques and UCD • ICTD as a Design Challenge (Tongia et al., 2006) • Framework+Process for Inclusive Technology
 (Pitula et al., 2007) • DRAMATICS
 (Dörflinger et al., 2013) 9 

The Problem 10 

The Problem High failure rate of ICTD projects 10 

The Problem High failure rate of ICTD projects Unsustainable interventions 10 

The Problem High failure rate of ICTD projects Unsustainable interventions Misaligned Perspectives and Focus 10 

The Problem High failure rate of ICTD projects Unsustainable interventions Misaligned Perspectives and Focus $4.2bn spent by The World Bank in ICT efforts evaluated as largely unsuccessful* * The World Bank Independent Evaluation Group, 2011 10 

Why do they fail? • Subjective concept of success and lack of clear metrics • Clashing agendas • Problems in capturing requirements • Risks deriving by inherent challenges not adequately addressed • Sustainability inadequately addressed • Inadequate documentation and evaluation • Rigid planning or “just do it” approach 11 

As an organization, I want to know how to carry out a successful ICT for Development project from start to finish 12 

Research Methodology Understanding the state of the practice Developing the vision of an ideal process Identification of process improvements • literature review (1999-2013) • semi-structured interviews with PMs • direct involvement and field visits • goal analysis • case studies A revised ICTD Value Chain An ICTD Maturity Model 13 A SCRUM-based process 

Highlights from Literature and Interviews • Understanding the “baseline” • Deal with multiple stakeholders • Flexibility and Informality* • Participation • Sustainability Planning • Evaluation 14 * Mileage may vary 

The Need for Evaluation (Heeks, 2009) 15 

Permanent Organization Temporary Organization (Project) Beneficiaries Potential for ICTD intervention Development Goals Values and Motivations Strategy External Stakeholders Support External Resources Organization Resources Portfolio / Program / Project Charter Readiness Availability iterative Uptake iterative Deployment Operation Usage Phase Out / Disposal Outputs (short term impact) Technical Deliverables Impact Development Impact Impact Assessment Uptake Assessment Strategic + Sustainability Planning External Factors Sustainability Plan Community Engagement Plan Build Relationship Sustainability Plan Other Projects Portfolio / Program / Project Assessment Viability + Trust Portfolio / Program / Project Planning Plan Execute Capabilities (Non-technical deliverables) Capability Development Technical Implementation Community Identification adoption A Revised Value Chain • 3 actors (columns) • performing organization • team • beneficiaries • 4 phases (rows) • from Strategy • to Evaluation/Impact 16 

Permanent Organization Temporary Organization (Project) Beneficiaries Potential for ICTD intervention Development Goals Values and Motivations Strategy External Stakeholders Support External Resources Organization Resources Portfolio / Program / Project Charter Readiness Availability iterative Uptake iterative Deployment Operation Usage Phase Out / Disposal Outputs (short term impact) Technical Deliverables Impact Development Impact Impact Assessment Uptake Assessment Strategic + Sustainability Planning External Factors Sustainability Plan Community Engagement Plan Build Relationship Sustainability Plan Other Projects Portfolio / Program / Project Assessment Viability + Trust Portfolio / Program / Project Planning Plan Execute Capabilities (Non-technical deliverables) Capability Development Technical Implementation Community Identification adoption 

- - - - - - - - - - - - - - ( - - - - - - - - - - - - - - - - - - - - - - - Mapping ICTD to the PMBOK • An ideal set of practices • SCRUM activities adapted to PMBOK Knowledge Areas and Phases* • Best practices added mostly in the Stakeholders Management area * adapted from Sutherland & Ahmad, 2011 17 

An ICTD Maturity Model 18 • A logical path to improve quality and produce comparable results • 4 areas x 3 levels • 6 Generic Goals • 27 Specific Goals satisfied by
 57 Specific Practices What • Guidance for all the phases of an intervention • Measure the effectiveness at conducting ICTD projects • Reduce failure • Produce comparable results Why 

Success Success Factor (CSF) Maturity Area Requirement Practice 1..* 1..* 1..* 1 1..* Maturity Model Generic Goal Generic Practice * Specific Goal Specific Practice * 1 1 4 satisfy contributes to contributes to Conceptual Model 19 

Success Success Factor (CSF) Maturity Area Requirement Practice 1..* 1..* 1..* 1 1..* Maturity Model Generic Goal Generic Practice * Specific Goal Specific Practice * 1 1 4 satisfy contributes to contributes to Conceptual Model Literature + Interviews 19 

Success Success Factor (CSF) Maturity Area Requirement Practice 1..* 1..* 1..* 1 1..* Maturity Model Generic Goal Generic Practice * Specific Goal Specific Practice * 1 1 4 satisfy contributes to contributes to Conceptual Model Literature + Interviews 19 Value Chain 

Success Success Factor (CSF) Maturity Area Requirement Practice 1..* 1..* 1..* 1 1..* Maturity Model Generic Goal Generic Practice * Specific Goal Specific Practice * 1 1 4 satisfy contributes to contributes to Conceptual Model Literature + Interviews Goal Analysis 19 Value Chain 

CMMI Levels • 5 levels • from high risk, low quality • to high quality & productivity • Staged and Continuous representation 20 

ICTDMM Levels • From ad-hoc and chaotic proceses • To defined organization standards • Based on PMMM levels • Continuous representation 1 Ad-hoc 2 Structured 3 Defined 21 

Process Areas Implementation Sustainability Evaluation and Impact Strategy • Value Chain phases • Assessed and improved simultaneously • CMMI institutionalization goals • Qualitative Generic Goals that apply to all areas • Specific Goals for each area matched with Specific Practices 22 

Strategy 23 

An Example from SIS-H • GPI contracted by Mozambican Ministry of Health • Completely “top-down” strategy: lack of contextual analysis and participation of the beneficiaries • Stakeholders influence not understood in time • Informal relationship established too late • Impossibility to impose organization’s own standards 

Implementation 25 

25 

25 

An Example from TW • Teleweaver: eBusiness/ Telecommunication platform for rural areas • Industry standards for processes and tools • Scrum process and team • Time/resource constraints • Support activities by SLL 

Sustainability • Technical • Social • Economic • Importance of
 support activities 27 

An Example from SLL • Long-term engagement of partners • Regular meetings and public functions • Continuous monitoring, training and champions involvement 

Evaluation and Impact • Topic of a large part of literature • …but often overlooked in practice • Not only at the end of a project 

An Example from SAMo • Social Accountability for Mozambique: pilot project to collect data from a rural population • Pilot evaluation in the intentions of the sponsor • Regular evaluation meetings with the assessors • Missing communication strategy for evaluation 30 

All in 1 Slide 31 

All in 1 Slide We want to improve the results of ICTD initiatives 31 

All in 1 Slide We want to improve the results of ICTD initiatives Various best practices and useful anecdotes 31 

All in 1 Slide We want to improve the results of ICTD initiatives Various best practices and useful anecdotes A revised value chain perspective 31 

All in 1 Slide We want to improve the results of ICTD initiatives Various best practices and useful anecdotes A revised value chain perspective An ICTD Maturity Model specification 31 

Future Work • Analysis of more projects (PMI grant application) • Refinement of the ICTDMM specification • Standard assessment procedure to certify ICTD organizations (think about this as a service for funding agencies) 32 

thank you 

Essential References • Pade et al. (2007), “An Exploration of the Critical Success Factors for the Sustainability of Rural ICT Projects - The Dwesa Case Study” • Heeks and Molla (2009), “Compendium on Impact Assessment of ICT-for- Development Projects” • Silvius and van den Brink (2010), “A Maturity Model for Integrating Sustainability Projects and Project Management” • Dodson et al. (2012), “Considering Failure: Eight Years of ITID Research” • Dörflinger and Dearden (2013), “Evolving a Software Development Methodology for Commercial ICTD Projects” • Golini and Landoni (2013), “International Development Projects: Peculiarities and Managerial Approaches”