OPPRESSION THROUGH BIG DATA

OBVIOUS THINGS ABOUT THIS • Frequently imposed top (of a given power/privilege hierarchy) down •… at least to start • Common pattern of implementation in less top-down-imposed situations: •Rationalization developed and publicized: “own the good, ignore the bad” •Surveillance instituted on the basis of that rationalization •Those who object on privacy grounds called FOMOngery names (“luddite,” “change resistant,” “barrier to progress”) •Data, once collected, repurposed for reasons nobody put forward at first •Oppression! • Alternately, data collected/offered by people for their own reasons gets repurposed to oppress them, with those reasons as rationalization. •One explanation for “wellness campaigns” and their use of e.g. fitness trackers!

COMMON SMELLS • Lack of transparency about data flows, and what they DO say is whiffy. • Inadequate, suboptimal, or no research into whether a given Big Data intervention actually works prior to hyping and selling it •2019: US cities that jumped onto the “predictive policing” bandwagon early are jumping off because it just doesn’t work. •A fair amount of ed-tech dies this way too. • Leveraging existing societal scorn, oppressions •Policing surveillance, overtly or co-, is largely driven by racism in the US. Yes, the peer-to-peer varieties as well. •A lot of health surveillance gets rationalized via hatred of fat people. •Isn’t it interesting, how many ed-tech companies talk teachers down?! (That teaching is feminized and ed-tech isn’t definitely matters here.) •“About them, without them” ethics smell very pungent here. • Surveillance by merger/acquisition •Want a ton of health data? Buy out a struggling wearable-tracker outfit. •(I really wish US bankruptcy/M&A law would address this one.)

NORMALIZATION OF SURVEILLANCE • Very not-new marketing strategy: grab ’em young •Apple absolutely ran on this in the 1980s. •Is this one reason for the push into K-12 surveillance? Of course. • More broadly, the strategy is “boil the frog.” Gradually get people used to surveillance and their context for it will shift (hi, Nissenbaum!) to accept more and more of it. Then you can oppress them with it! •“Convenience” and personalization are often Trojan horses here. Bluntly, though, this tends to work best via power (including design power) and privilege… especially since many oppressed populations are wise to this game. •The trick is that the surveillor can’t get into a situation where everybody’s all like “wait, what? they’re doing what with my data?” As Nissenbaum et al. would say, that’s a contextual-integrity violation and they’ll suffer for it. •So finding just the right rationalizations and amount of transparency—and keeping everything else secret—takes work, and can fail. (Has failed!) •It’s definitely a thing to watch out for.

FIGHTING NORMALIZATION •(I’m seeing a lot of good work done on this in the K-12 context.) • Insist on transparency about data collection, governance, etc. •Insisting on anything at least lets the surveillors know you’re (ahem) watching them! But transparency is always a reasonable ask. • Insist on assessment, so you can toss the thing when (!) it doesn't work. • Explain the harms publicly, in terms that resonate with audiences. •I think this may be part of what’s behind a spate of “Big Data means you can’t escape your childhood mistakes” pieces in the general and tech media lately. •Immediate harms resonate better than remote ones, documented already- existing harms better than theoretical ones, (sigh) individual harms better than broad societal harms. Communicate strategically. •I’m not fond of the “save the children!” argument generally—it gets used to justify censorship and lots of other harms—but I’m happier about it when children exercise their own voice. Help that happen?

No content

THANKS! This presentation copyright 2019 by Dorothea Salo. It is available under a Creative Commons Attribution 4.0 International license.

SO WHERE DOES THIS LEAVE US?

BIG DATA IS AN ETHICAL PROBLEM WE HAVEN’T SOLVED. • That doesn’t mean it’s insoluble! •(I mean, I tend to lean that way myself, but that doesn’t mean you have to.) • I’ve walked you through some of the most notable ethical dilemmas and problems it raises. (In a reasonably logical progression, I hope!) • I’ve also alluded to various avenues for improving the situation; I want to talk about those a tiny bit more systematically.

PRESENT LOCI OF RESISTANCE TO BIG DATA • Legislative and policy processes •Europe: General Data Protection Regulation •Several US states: working on it, though stoutly opposed by Big Tech •Not limited to government! Setting organizational policy counts too! • Worker resistance from within the industry •Individual: Edward Snowden, Eva Galperin, Alex Stamos •Collective: Google, Microsoft, EFF, others • Documenting and communicating the harms •Journalists and journalism: Kashmir Hill, ProPublica, others •Researchers: Zeynep Tufekci, Chris Gilliard, Ifeoma Ajunwa, others •Bloggers and speakers: Maciej Cegłowski, Alison Macrina •Citizens: see e.g. op-eds about K-12 surveillance, the death of InBloom • You? (I mean, definitely me, but maybe you too?)

HOW DO WE USE ETHICS FRAMEWORKS? • Probably not by naming them •I had you do that in this course to solidify your understanding of them, not because it’s necessarily rhetorically effective. • Perhaps by asking yourself “what will this crowd listen to?” •Virtue ethics’s “Is this who you want to be?” often resonates with people who are (re)forming their identity. If there is an applicable professional- ethics code, it is often a valuable tool here. •Are these idealists? Then a deontology-style appeal to higher values might be effective. •Are these people concerned for specific other people? Or themselves? Consequentialism framed around the objects of concern might do the job. •Identify strongly with society as a whole? With hierarchy? Take a look at Confucian-ethics arguments. •These aren’t mutually-exclusive options, of course. When I’m speaking on an ethical issue, I tend to combine several approaches.

EXAMPLE: VIRTUE ETHICS

EXAMPLE: DEONTOLOGY

EXAMPLE: CONSEQUENTIALISM

ALL OF THESE WERE FROM THE SAME PRESENTATION. “How do we not be Facebook?” https://speakerdeck.com/dsalo/how-do-we-not-be-facebook

WHAT ELSE HELPS? • Do policy work early and often. •Especially effective if you see harm coming… but no one else does. • Questions. Ask questions! • Humor—though deploy with care! • Horror stories, especially if you’re going consequentialist. •I unapologetically stockpile these in my Pinboard. Feel free to reuse anything (pinboard.in/u:dsalo/t:horrorstories) • Success stories! Especially when people feel overwhelmed, ground down. •Role models too, same reasons •“Solution journalism” is emerging as an antidote to relentless gloom- and-doom and the news avoidance it generates in many people. •“Solution data science” would be an interesting thing! (And is happening, e.g. research into differential privacy.)

BE CAREFUL OF… • Direct attacks on the people you’re speaking to •(she said ruefully, having had a keynote go really bad in 2018) •Can you find a proxy, or other target? • An endless litany of problems without suggested solutions •Or role models to follow, or organizations or bandwagons to join… •Acknowledge people’s contexts, even when they’re not acting in an ideal way; it helps them feel heard, less attacked. •The point is, let people leave feeling empowered. • Expecting too much of yourself! You can burn out that way. •Acknowledge your mistakes (as I did above), but don’t let them define you. Laugh at them if you can! •Be as realistic as you can about the work and sacrifice this can take.

GOATS, OXEN, AND PERSPECTIVE TAKING • For your final assignment, I asked you to reflect on why something that got your goat (or gored your ox), got your goat (or gored your ox). •(Both of these are English-language metaphors for “made you fighting mad.”) • I did this because passionate people often assume that everyone else is as passionate as they are about the same things for the same reasons they are. This is… often untrue. •(you all know by this time that I’m poking at myself here, right?) • Consciously inspecting your own perspective is the first step toward sincerely seeking to understand that of others… which, if you want to persuade them of anything, you’ll want to do.

THANKS! This presentation copyright 2019 by Dorothea Salo. It is available under a Creative Commons Attribution 4.0 International license.