AI, the double edged sword in dealing with disinformation

Talk given at the ThinkYoung Hackathon in Hamburg

https://www.thinkyounghackathons.org/

This talk explores how artificial intelligence both amplifies and combats election disinformation. We’ll examine why AI-generated falsehoods spread faster and deeper than ever before, the hidden biases and limitations in current AI tools, and practical strategies for nonprofits and civic innovators to use AI responsibly. Attendees will gain a critical, data-driven understanding of AI’s risks and opportunities, as well as actionable guidelines for building robust, human-centered systems to protect information integrity in the digital age.

Transcript

1. TEREZA IOFCIU Hamburg - Summer 2025 AI: THE DOUBLE EDGED SWORD IN DEALING WITH DISINFORMATION
2. - Data Leadership Coach - Freelance Data & AI Strategist - PSF Fellow - PyPodcats Host - PyLadies Hamburg - PSF D&I Work Group TEREZA IOFCIU SHE/HER
3. FALSE NEWS IS NOT NEW and in itself it hasn’t changed much …
4. SPEED & SCALE … according the a 2018 MIT study on social media (twitter) https://news.mit.edu/2018/study-twitter-false-news-travels-faster-true-stories-0308 FALSE NEWS SPREADS 70% FASTER THAN THE TRUTH Credit: Alina Kolyuka / Adobe Stock
5. SPEED & SCALE … according the a 2018 MIT study on social media (twitter) https://news.mit.edu/2018/study-twitter-false-news-travels-faster-true-stories-0308 FALSE NEWS SPREADS 70% FASTER THAN THE TRUTH Credit: Alina Kolyuka / Adobe Stock TRUE STORIES REACH 1K PEOPLE.. FALSE ONES 100K
6. SPEED & SCALE … according the a 2018 MIT study on social media (twitter) https://news.mit.edu/2018/study-twitter-false-news-travels-faster-true-stories-0308 FALSE NEWS SPREADS 70% FASTER THAN THE TRUTH Credit: Alina Kolyuka / Adobe Stock TRUE STORIES REACH 1K PEOPLE.. FALSE ONES 100K IT TAKES 6X LONGER FOR TRUTH TO REACH 1,500 PEOPLE
7. AI - AUTOMATION & AMPLIFICATION Generative AI tools allow anyone to create fake content https://akademie.dw.com/en/generative-ai-is-the-ultimate-disinformation-ampli fi er/a-68593890 Credit: Alina Kolyuka / Adobe Stock LLMS GENERATE INCORRECT INFORMATION WITH HIGH CONFIDENCE & PERSUASIVENESS TEXT VIDEO AUDIO
8. AI - AUTOMATION & AMPLIFICATION Generative AI tools allow anyone to create fake content https://akademie.dw.com/en/generative-ai-is-the-ultimate-disinformation-ampli fi er/a-68593890 Credit: Alina Kolyuka / Adobe Stock LLMS GENERATE INCORRECT INFORMATION WITH HIGH CONFIDENCE & PERSUASIVENESS MARGINAL COSTS OF PRODUCTION & DISSEMINATION ARE CLOSE TO ZERO TEXT VIDEO AUDIO
9. TRANSFORMATION OF THE PUBLIC ARENA Disinformation and the Structural Transformations of the Public Arena https://journals.sagepub.com/doi/full/10.1177/2056305121988928 Credit: Alina Kolyuka / Adobe Stock DIGITIZATION HAS BEEN TRANSFORMING THE PUBLIC SPHERE - TRUST IN DIGITAL TOOLS
10. TRANSFORMATION OF THE PUBLIC ARENA Disinformation and the Structural Transformations of the Public Arena https://journals.sagepub.com/doi/full/10.1177/2056305121988928 Credit: Alina Kolyuka / Adobe Stock DIGITIZATION HAS BEEN TRANSFORMING THE PUBLIC SPHERE - TRUST IN DIGITAL TOOLS AI-GENERATED CONTENT IS BECOMING DIFFICULT TO RECOGNIZE
11. PROMPT: I AM DOING A TALK ON DISINFORMATION AND I NEED SOME EXAMPLES TO ASK THE PARTICIPANTS TO GUESS IF THEY ARE FAKE OR NOT. Can LLM-Generated Misinformation Be Detected? https://arxiv.org/html/2309.13788v5#A5 TRUE OR FALSE? Hamburg to Become Germany’s First Fully Car ‑ Free City by 2035 By Anna Meyer, 18 June 2025 Hamburg has unveiled a comprehensive “2035 Mobility Master Plan” aiming to phase out private vehicle use across the city by 2035. The plan includes converting major thoroughfares into pedestrian boulevards, expanding public transit with autonomous electric shuttles, and offering substantial subsidies to residents who give up their cars. City of fi cials say emissions would drop by 60% and the plan could serve as a model for other European capitals. Yet, urban planners and business groups caution that such a sweeping transformation lacks legislative backing and realistic funding— raising doubts about its feasibility. German Customs Seize Over 500   kg of Cocaine at Hamburg Port By Jens Koch, 12 June 2025 Hamburg customs of fi cers have intercepted a shipment of 520   kg of cocaine concealed within refrigerated fruit containers imported from South America. With a street value estimated at over €20   million, the bust marks one of the largest in recent years. The operation involved coordinated efforts with EU partners, and several individuals have been detained for questioning. Interior Minister Andy Grote called the seizure “a major blow to organised smuggling rings.” Telemichel Reopening in Sight After 20+ Years By Claudia Becker, 14 June 2025 The Heinrich ‑ Hertz ‑ Fernsehturm— Hamburg’s famed “Telemichel”—is a major step closer to reopening after two decades of public closure. The Deutsche Funkturm GmbH recently submitted its renovation and entrance ‑ building permit, outlining plans for a rotating restaurant at 130   m, an event fl oor for up to 400 guests, and a panoramic observation deck. The reconstruction, budgeted at €37   million and co ‑ funded by federal and state governments, is slated to begin later this year. Of fi cials express optimism for a public reopening sometime in 2026.
12. LLM-GENERATED MISINFORMATION CAN BE HARDER TO DETECT FOR BOTH HUMANS AND DETECTORS THAN HUMAN-WRITTEN MISINFORMATION Can LLM-Generated Misinformation Be Detected? https://arxiv.org/html/2309.13788v5#A5 TRUE OR FALSE? Hamburg to Become Germany’s First Fully Car ‑ Free City by 2035 By Anna Meyer, 18 June 2025 Hamburg has unveiled a comprehensive “2035 Mobility Master Plan” aiming to phase out private vehicle use across the city by 2035. The plan includes converting major thoroughfares into pedestrian boulevards, expanding public transit with autonomous electric shuttles, and offering substantial subsidies to residents who give up their cars. City of fi cials say emissions would drop by 60% and the plan could serve as a model for other European capitals. Yet, urban planners and business groups caution that such a sweeping transformation lacks legislative backing and realistic funding— raising doubts about its feasibility. German Customs Seize Over 500   kg of Cocaine at Hamburg Port By Jens Koch, 12 June 2025 Hamburg customs of fi cers have intercepted a shipment of 520   kg of cocaine concealed within refrigerated fruit containers imported from South America. With a street value estimated at over €20   million, the bust marks one of the largest in recent years. The operation involved coordinated efforts with EU partners, and several individuals have been detained for questioning. Interior Minister Andy Grote called the seizure “a major blow to organised smuggling rings.” Telemichel Reopening in Sight After 20+ Years By Claudia Becker, 14 June 2025 The Heinrich ‑ Hertz ‑ Fernsehturm— Hamburg’s famed “Telemichel”—is a major step closer to reopening after two decades of public closure. The Deutsche Funkturm GmbH recently submitted its renovation and entrance ‑ building permit, outlining plans for a rotating restaurant at 130   m, an event fl oor for up to 400 guests, and a panoramic observation deck. The reconstruction, budgeted at €37   million and co ‑ funded by federal and state governments, is slated to begin later this year. Of fi cials express optimism for a public reopening sometime in 2026.
13. & DISINFORMATION AI
14. AI & DISINFORMATION AI models (LLMs, image generation, multimodal, .. ) are trained on large volumes of un fi ltered data on the internet / or on lightly fi ltered data BIAS
15. ECONOMIC BIAS 93% INTERNET USAGE IN HIGH-INCOME COUNTRIES VS. 27% IN LOW-INCOME https://www.itu.int/en/mediacentre/Pages/PR-2024-11-27-facts-and- fi gures.aspx
16. LANGUAGE BIAS LLMS PERFORM BETTER ON HIGH RESOURCE LANGUAGES - EN, DE, FR.. https://arxiv.org/pdf/2404.11553
17. OPTIMIZATION BIAS PRIORITIZING AGREEABLENESS OVER CRITICAL THINKING. WHAT IF IT WOULD PRIORITIZE ENGAGEMENT? https://openai.com/index/sycophancy-in-gpt-4o/
18. AI SLOP & FEEDBACK LOOPS CHATGPT HAS ALREADY POLLUTED THE INTERNET.. DATA CLEANING IS GOING TO BE PROHIBITIVELY EXPENSIVE https://futurism.com/chatgpt-polluted-ruined-ai-development https://magazine.rice.edu/winter-2025/ais-mad-loops Credit: Dan Bejar/ https://www.bejarprints.com
19. OUTPUT INTEGRITY THE HALLUCINATION PROBLEM IN LLMS IS INEVITABLE & CAN BE REDUCED VIA SYSTEM ARCHITECTURE DECISIONS https://www.kapa.ai/blog/ai-hallucination https://www.washingtonpost.com/technology/2023/12/15/microsoft-copilot-bing-ai-hallucinations-elections/
20. IS AI GOING TO PRODUCE MORE DISINFORMATION ? Credit: Dan Bejar/ https://www.bejarprints.com
21. Predicted numbers of electoral misinformation claims respondents believed over time https://academic.oup.com/poq/article/88/SI/681/7718066 “PROFESSIONALLY PRODUCED NEWS FROM LEGACY MEDIA ORGANIZATIONS MAY BE HELPFUL IN COUNTERING MISINFORMATION DURING A CONTENTIOUS ELECTION CYCLE” THE MORE RESPONDENTS ACCESSED SOURCES OF LEGACY NEWS MEDIA, THE FEWER BELIEFS THEY HELD IN ELECTORAL MISINFORMATION
22. USING DESPITE ITS BIASES AI
23. BIAS AWARENESS & STRATEGIC MITIGATION Credit: Dan Bejar/ https://www.bejarprints.com
24. DIVERSIFY YOUR SOURCES
25. DIVERSIFY YOUR SOURCES WHO BUILT IT MATTERS “YOU ARE AN AVERAGE HUMAN BEING BORN IN [COUNTRY/TERRITORY] AND LIVING IN [COUNTRY/TERRITORY] RESPONDING TO THE FOLLOWING SURVEY QUESTION Cultural Bias and Cultural Alignment of Large Language Models https://arxiv.org/html/2311.14096v2
26. DIVERSIFY YOUR SOURCES WHO BUILT IT MATTERS ASK DEEPSEEK R1 ABOUT TAIWAN OR TIANANMEN, AND THE MODEL IS UNLIKELY TO GIVE AN ANSWER. https://www.wired.com/story/deepseek-censorship/ IS IT CENSORED?
27. DIVERSIFY YOUR SOURCES WHO BUILT IT MATTERS GPT-4 MODELS CONSISTENTLY DEMONSTRATE SUPERIOR PERFORMANCE IN MISINFORMATION DETECTION TASKS A Comparative Study of O ff l ine Models and Online LLMs in Fake News Detection https://arxiv.org/pdf/2409.03067 IS IT CENSORED? CHOOSE THE RIGHT MODEL FOR YOUR USE CASE
28. DIVERSIFY YOUR SOURCES WHO BUILT IT MATTERS CLASSIC ML & DOMAIN SPECIFIC LLMS PROVIDE SUPERIOR ACCURACY FOR THE TASK/DOMAIN IT WAS BUILT FOR https://arya.ai/blog/generic-vs-domain-speci fi c-llms IS IT CENSORED? USE SPECIALIZED MODELS: DOMAIN SPECIFIC OR RAGS CHOOSE THE RIGHT MODEL FOR YOUR USE CASE
29. HOW MUCH AI & HOW MUCH HUMAN SHOULD WE USE?
30. REALITY IS NOT AN END-TO-END AI PREDICTION SYSTEM ITERATIVE APPROACH & HUMAN VALIDATION https://speakerdeck.com/inesmontani/reality-is-not-an-end-to-end-prediction-problem-applied-nlp-in-the-age-of-generative-ai
31. REALITY IS NOT AN END-TO-END AI PREDICTION SYSTEM ITERATIVE APPROACH & HUMAN VALIDATION FIND THE RIGHT TOOL FOR EACH TASK IN THE SYSTEM
32. REALITY IS NOT AN END-TO-END AI PREDICTION SYSTEM ITERATIVE APPROACH & HUMAN VALIDATION THE MORE COMPLEX THE SYSTEM … THE MORE ROOM FOR ERROR THERE IS FIND THE RIGHT TOOL FOR EACH TASK IN THE SYSTEM
33. REALITY IS NOT AN END-TO-END AI PREDICTION SYSTEM ITERATIVE APPROACH & HUMAN VALIDATION THE MORE COMPLEX THE SYSTEM … THE MORE ROOM FOR ERROR THERE IS EXPECT SURPRISES FROM DATA & ADJUST FIND THE RIGHT TOOL FOR EACH TASK IN THE SYSTEM
34. Whatever you build.. don’t forget about the humans who are supposed to use it https://www.linkedin.com/pulse/we-gaining-efficiency-llms-losing-ability-think-virginia-dignum-fjyjc
35. THANK YOU! LET’S CONNECT! ESPECIALLY IF YOU OR A FRIEND NEEDS DATA & CAREER HELP! LINKEDIN.COM/IN/TEREZA-IOFCIU/ Graphics Credit: Alina Kolyuka / Adobe Stock
36. - https://www.ic fj .org/sites/default/ fi les/2018-07/A Short Guide to History of Fake News and Disinformation_ICFJ Final.pdf - https://news.mit.edu/2018/study-twitter-false-news-travels-faster-true-stories-0308 - https://akademie.dw.com/en/generative-ai-is-the-ultimate-disinformation-ampli fi er/a-68593890 - https://journals.sagepub.com/doi/full/10.1177/2056305121988928 - https://arxiv.org/pdf/2404.11553 - https://arxiv.org/pdf/2401.11817 - https://www.kapa.ai/blog/ai-hallucination - https://futurism.com/chatgpt-polluted-ruined-ai-development - https://arxiv.org/html/2311.14096v2 - https://www.wired.com/story/deepseek-censorship/ - https://arxiv.org/pdf/2409.03067 - https://arxiv.org/html/2309.13788v5#A5.T15 - https://arxiv.org/html/2404.12065v2 - https://www.linkedin.com/pulse/we-gaining-e ffi ciency-llms-losing-ability-think-virginia-dignum- fj yjc - https://speakerdeck.com/inesmontani/reality-is-not-an-end-to-end-prediction-problem-applied-nlp-in-the-age-of-generative-ai REFERENCES