Exploring social science research methods for engineers

Presented at the EWB-USA National Conference in Milwaukee, WI.

Transcript

1. EXPLORING SOCIAL SCIENCE RESEARCH METHODS FOR ENGINEERS Tina Lee Devin Berg Elizabeth Buchanan University of Wisconsin-Stout **This material is based upon work supported by the National Science Foundation under Grant No. 1540301. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation."
2. Introduction, Or, An Engineer, Anthropologist and Ethicist Walk into a Bar….
3. Motivation: A learning experience ▪ From a university’s perspective, service learning provides an opportunity for broader impact but generally there must also be a “learning” component. How do we strike a balance between the needs of the community and the needs of the students?
4. Background: Research Questions ▪ Does participation in service learning such as EWB-USA contribute to a culture of ethical STEM practice? ▪ Do participants from service learning projects experience their STEM education in a qualitatively different way than those who do not? ▪ How can we learn from the on-ground experiences of students and faculty to identify and promote best practices in humanitarian service learning for a more ethically aware STEM culture? ▪ Who is the primary client or beneficiary of SL? ▪ What is the balance between helping a community versus or contrasted to student experiences?
5. Work to Date ▪ A study methodology – https://www.ewb.org.au/jhe/index.php/jhe/article/view/47 ▪ Curriculum integration ▪ Learning assessment ▪ Report review ▪ Case study (Ecuador)
6. Why Social Science Research 101? ▪ Through constructing case studies and conducting interviews we’ve identified common problems and best practices ▪ Technical issues often not the difference between successful and not successful projects ▪ Assessment key and social science can help!
7. Avoiding Biases and Common Pitfalls ▪ What is “community” and how do you get all the diversity within it given short timelines? ▪ Potential biases in the design process ▪ Being told what people think you want to hear ▪ Getting wrong answers from asking questions inappropriately
8. What Methods Should You Choose? ▪ Really depends on the situation and what information you need ▪ Ideal situation: multiple methods + triangulation between them ▪ This can be accomplished in a short period of time with a team of the right people; divide and conquer ▪ Iterative process: data gained at one stage shapes next stage ▪ Possibilities: Interviews, Focus Groups, Surveys, Participant Observation, Observation, Social Network Mapping, Photovoice, Ethnographic Decision Modeling
9. Social Network Mapping ▪ Pros: helpful in learning community beyond your NGO/partners; you can see the factions, differences within the community, find key people to get you access to portions of a community ▪ Pitfalls: Can be hard to know if you have mapped the whole thing, people might not want to reveal splits or tell you things they think you don’t want to hear ▪ How to do it: Ask key people in the community to describe who talks to who, who works with who, who is related to who, maybe even who doesn’t get along, etc… Draw a diagram or use a program like Kumu; Think about having people give you a village tour to talk about networks ▪ Having a physical map might be helpful for continuity as your chapter changes over time
10. Surveys ▪ Pros: gets widest set of opinions and community wide patterns ▪ Cons: Takes more time to prep; writing good survey questions is hard; writing good surveys across cultural boundaries is even harder! ▪ Tips: avoid leading questions/questions with assumptions; avoid double or triple barreled questions; think carefully about the ordering of questions (broad to specific, usually); think through all the ways a question COULD be interpreted and be explicit about what you mean; test ALL questions ahead of time ▪ Keep in mind how language and culture affect how people will answer your questions
11. Interviews ▪ Pros: In-depth information, good for getting views of key individuals, helps get at issues/problems you might not have thought of in advance ▪ Cons: Can be time consuming, will not get a representative sample of a community ▪ Tips: Avoid leading questions, ask as many open ended-questions as possible, go from big picture to details, record and transcribe when possible, practice questions and follow-up questions/probes
12. Focus Groups ▪ Pros: can be more efficient time-wise; can give you a sense of community interactions ▪ Cons: can be easy to misinterpret as getting the opinions of all the people in the room ▪ Tips: NOT a group interview; be intentional about who you choose; use very few questions (2-4 for an hour); two people needed to facilitate (one to observe and take notes, one to get everyone to participate)
13. Ethnographic Assessment/Field Study ▪ Observation and Participant Observation ▪ People will tell you what you want to hear; you get a different sense of a problem by watching people in daily life ▪ Can be in the form of asking people to give you a tour of their community ▪ Take notes on informal interactions and conversations too! ▪ Pros: reliable information if used intentionally (what people say vs. what they really do); can help build trust with a community ▪ Cons: Can be time consuming, especially given time it can take to build rapport and help people be less self-conscious; biases can distort what you see
14. Ethnographic Assessment/Field Study ▪ "Rapid ethnographic assessment is an approach for rapidly characterizing the sociocultural landscape (Beebe 1995). ▪ Rapid ethnographic assessment provides insight into the socio-cultural nature of a region by identifying the current key actors, issues, sentiments, resources, activities and locations, and any recent changes. Rapid ethnographic assessment is particularly useful when product demands allow little time for detailed anthropological field work (Bauersfeld and Halgren 1996). ▪ It is also called for when groups are faced with operating in a region where they have little experience or working with an unfamiliar group (Bentley et al. 1988)."
15. Photovoice ▪ Ask people in the community to take pictures of ▪ things that are important to them, the things they most want changed, etc… ▪ They provide an explanation of why they took that picture and what it means to them ▪ Can be done between trips
16. Ethnographic Decision Modeling ▪ Ask people how what they to do in specific situations (and all the variations) ▪ This gets at criteria that might not be captured by a traditional alternatives analysis that might focus more on cost/benefit or objective criteria like distance (ex. a sick child might be taken farther to a traditional healer if there is a belief they are more effective for certain illnesses) ▪ Example: class attendance
17. Take-Aways (In General) ▪ Recent meeting at NIST, increasingly common for social scientists to join ”hard” science teams ▪ Ongoing emphasis of NSF on multi-disciplinary/cross-cutting research ▪ Need for coherent, systematic evaluation
18. Take-Aways (In Particular) ▪ Multiple sources of data + triangulation whenever possible ▪ Keep an open mind and avoid biases—let people guide you
19. Select References and Resources: ▪ Beebe, James (2014). Rapid Qualitative Inquiry: A Field Guide to Team-Based Assessment, 2nd ed. Lanham, MD: Rowman and Littlefield. ▪ Bernard, H. Russell. (2011). Research Methods in Anthropology: Qualitative and Quantitative Approaches, 5th ed. Lanham, MD: AltaMira Press. ▪ Carley, K. et al. (2012). Data-to-model: a mixed initiative approach for rapid ethnographic assessment. Computational Math Organ Theory, 18, 300–327. ▪ McNall, M. and Foster-Fishman, P. (2007). Methods of rapid evaluation, assessment, and appraisal. American Journal of Evaluation, Vol. 28 (2), 151-168. ▪ Van Holt et al (2013). Rapid ethnographic assessment for cultural mapping. Poetics Vol. 41 (4), 366-383. ▪ https://photovoice.org/matt-daw-participation-in-development/ ▪ Sage “Little Blue” Methods Books. http://methods.sagepub.com/ ▪ https://Kumu.io