AI: The Double-Edged Sword in the Fight Against Disinformation V2

Presented at https://codetalks.com/ on Nov 5 2025

AI is everywhere, helping us spot fake news, but also making it easier to create. Ever wondered just how much AI is shaping the information you see, or what to do if you plan to use AI to fight back against disinformation?

In this talk, I'll break down the role of AI in the space of creating mis and disinformation, and what you can really do to build smarter, safer systems. We’ll look at real-world examples, surprising pitfalls, and practical ways to combine AI with human expertise.

Key Takeaways:
See how AI both fuels and fights disinformation Learn the hidden biases in AI tools, and how to spot them Get actionable tips for using AI responsibly and effectively in your own projects

If you care about truth, tech, or democracy, this session is for you. Whether you’re a nonprofit innovator, techie, or just AI-curious, you’ll leave with a clear-eyed view of the risks, the opportunities, and how to make AI work for you, not against you.

Transcript

1. TEREZA IOFCIU Hamburg - Nov 2025 AI: THE DOUBLE EDGED SWORD IN DEALING WITH DISINFORMATION
2. - Data Leadership Coach - Freelance Data & AI Strategist - PSF Fellow - PyPodcats Host - PyLadies Hamburg - PSF D&I Work Group TEREZA IOFCIU SHE/HER
3. DISINFORMATION MISINFORMATION AI HALLUCINATIONS Usually spread without intent Fabricated information that looks plausible Spread deliberately to mislead
4. FALSE NEWS IS NOT NEW and in itself it hasn’t changed much … ICFJ Short Guide to History of Fake News & Disinformation https://www.ic fj .org/sites/default/ fi les/2018-07/A Short Guide to History of Fake News and Disinformation_ICFJ Final.pdf
5. → false information published → spread using fake accounts → .. it circulates.. → original website is deleted → disinformation remains → .. shared online for years Credit: Dan Bejar/ https://www.bejarprints.com
6. SPEED & SCALE … according the a 2018 MIT study on social media (twitter) https://news.mit.edu/2018/study-twitter-false-news-travels-faster-true-stories-0308 FALSE NEWS SPREADS 70% FASTER THAN THE TRUTH Credit: Alina Kolyuka / Adobe Stock
7. SPEED & SCALE … according the a 2018 MIT study on social media (twitter) https://news.mit.edu/2018/study-twitter-false-news-travels-faster-true-stories-0308 FALSE NEWS SPREADS 70% FASTER THAN THE TRUTH Credit: Alina Kolyuka / Adobe Stock TRUE STORIES REACH 1K PEOPLE.. FALSE ONES 100K
8. SPEED & SCALE … according the a 2018 MIT study on social media (twitter) https://news.mit.edu/2018/study-twitter-false-news-travels-faster-true-stories-0308 FALSE NEWS SPREADS 70% FASTER THAN THE TRUTH Credit: Alina Kolyuka / Adobe Stock TRUE STORIES REACH 1K PEOPLE.. FALSE ONES 100K IT TAKES 6X LONGER FOR TRUTH TO REACH 1,500 PEOPLE
9. # countries with online disinformation campaigns 2017 2018 2019 70 48 28 … According to a 2019 analysis by researchers at the Oxford Internet Institute… https://allianceforscience.org/blog/2020/03/the-battle-against-disinformation-is-global/
10. AI - AUTOMATION & AMPLIFICATION Generative AI tools allow anyone to create fake content https://akademie.dw.com/en/generative-ai-is-the-ultimate-disinformation-ampli fi er/a-68593890 Credit: Alina Kolyuka / Adobe Stock LLMS GENERATE INCORRECT INFORMATION WITH HIGH CONFIDENCE & PERSUASIVENESS TEXT VIDEO AUDIO
11. AI - AUTOMATION & AMPLIFICATION Generative AI tools allow anyone to create fake content https://akademie.dw.com/en/generative-ai-is-the-ultimate-disinformation-ampli fi er/a-68593890 Credit: Alina Kolyuka / Adobe Stock LLMS GENERATE INCORRECT INFORMATION WITH HIGH CONFIDENCE & PERSUASIVENESS MARGINAL COSTS OF PRODUCTION & DISSEMINATION ARE CLOSE TO ZERO TEXT VIDEO AUDIO
12. TRANSFORMATION OF THE PUBLIC ARENA Disinformation and the Structural Transformations of the Public Arena https://journals.sagepub.com/doi/full/10.1177/2056305121988928 Credit: Alina Kolyuka / Adobe Stock DIGITIZATION HAS BEEN TRANSFORMING THE PUBLIC SPHERE - TRUST IN DIGITAL TOOLS
13. TRANSFORMATION OF THE PUBLIC ARENA Disinformation and the Structural Transformations of the Public Arena https://journals.sagepub.com/doi/full/10.1177/2056305121988928 Credit: Alina Kolyuka / Adobe Stock DIGITIZATION HAS BEEN TRANSFORMING THE PUBLIC SPHERE - TRUST IN DIGITAL TOOLS AI-GENERATED CONTENT IS BECOMING DIFFICULT TO RECOGNIZE
14. PROMPT: I AM DOING A TALK ON DISINFORMATION AND I NEED SOME EXAMPLES TO ASK THE PARTICIPANTS TO GUESS IF THEY ARE FAKE OR NOT TWO TRUTHS AND A LIE
15. PROMPT: I AM DOING A TALK ON DISINFORMATION AND I NEED SOME EXAMPLES TO ASK THE PARTICIPANTS TO GUESS IF THEY ARE FAKE OR NOT. Can LLM-Generated Misinformation Be Detected? https://arxiv.org/html/2309.13788v5#A5 TRUE OR FALSE? Hamburg to Become Germany’s First Fully Car ‑ Free City by 2035 By Anna Meyer, 18 June 2025 Hamburg has unveiled a comprehensive “2035 Mobility Master Plan” aiming to phase out private vehicle use across the city by 2035. The plan includes converting major thoroughfares into pedestrian boulevards, expanding public transit with autonomous electric shuttles, and offering substantial subsidies to residents who give up their cars. City of fi cials say emissions would drop by 60% and the plan could serve as a model for other European capitals. Yet, urban planners and business groups caution that such a sweeping transformation lacks legislative backing and realistic funding—raising doubts about its feasibility. German Customs Seize Over 500   kg of Cocaine at Hamburg Port By Jens Koch, 12 June 2025 Hamburg customs of fi cers have intercepted a shipment of 520   kg of cocaine concealed within refrigerated fruit containers imported from South America. With a street value estimated at over €20   million, the bust marks one of the largest in recent years. The operation involved coordinated efforts with EU partners, and several individuals have been detained for questioning. Interior Minister Andy Grote called the seizure “a major blow to organised smuggling rings.” Telemichel Reopening in Sight After 20+ Years By Claudia Becker, 14 June 2025 The Heinrich ‑ Hertz ‑ Fernsehturm— Hamburg’s famed “Telemichel”—is a major step closer to reopening after two decades of public closure. The Deutsche Funkturm GmbH recently submitted its renovation and entrance ‑ building permit, outlining plans for a rotating restaurant at 130   m, an event fl oor for up to 400 guests, and a panoramic observation deck. The reconstruction, budgeted at €37   million and co ‑ funded by federal and state governments, is slated to begin later this year. Of fi cials express optimism for a public reopening sometime in 2026.
16. PROMPT: I AM DOING A TALK ON DISINFORMATION AND I NEED SOME EXAMPLES TO ASK THE PARTICIPANTS TO GUESS IF THEY ARE FAKE OR NOT. Can LLM-Generated Misinformation Be Detected? https://arxiv.org/html/2309.13788v5#A5 TRUE OR FALSE? Hamburg to Become Germany’s First Fully Car ‑ Free City by 2035 By Anna Meyer, 18 June 2025 Hamburg has unveiled a comprehensive “2035 Mobility Master Plan” aiming to phase out private vehicle use across the city by 2035. The plan includes converting major thoroughfares into pedestrian boulevards, expanding public transit with autonomous electric shuttles, and offering substantial subsidies to residents who give up their cars. City of fi cials say emissions would drop by 60% and the plan could serve as a model for other European capitals. Yet, urban planners and business groups caution that such a sweeping transformation lacks legislative backing and realistic funding—raising doubts about its feasibility. German Customs Seize Over 500   kg of Cocaine at Hamburg Port By Jens Koch, 12 June 2025 Hamburg customs of fi cers have intercepted a shipment of 520   kg of cocaine concealed within refrigerated fruit containers imported from South America. With a street value estimated at over €20   million, the bust marks one of the largest in recent years. The operation involved coordinated efforts with EU partners, and several individuals have been detained for questioning. Interior Minister Andy Grote called the seizure “a major blow to organised smuggling rings.” Telemichel Reopening in Sight After 20+ Years By Claudia Becker, 14 June 2025 The Heinrich ‑ Hertz ‑ Fernsehturm— Hamburg’s famed “Telemichel”—is a major step closer to reopening after two decades of public closure. The Deutsche Funkturm GmbH recently submitted its renovation and entrance ‑ building permit, outlining plans for a rotating restaurant at 130   m, an event fl oor for up to 400 guests, and a panoramic observation deck. The reconstruction, budgeted at €37   million and co ‑ funded by federal and state governments, is slated to begin later this year. Of fi cials express optimism for a public reopening sometime in 2026.
17. CAN LLM-GENERATED MISINFORMATION BE DETECTED? Can LLM-Generated Misinformation Be Detected? https://arxiv.org/html/2309.13788v5#A5 “…LLM-generated misinformation can be harder to detect for humans and detectors compared to human-written misinformation with the same semantics, which suggests it can have more deceptive styles…”
18. & DISINFORMATION AI
19. AI & DISINFORMATION AI models (LLMs, image generation, multimodal, .. ) are trained on large volumes of un fi ltered data on the internet / or on lightly fi ltered data BIAS
20. ECONOMIC BIAS 93% INTERNET USAGE IN HIGH-INCOME COUNTRIES VS. 27% IN LOW-INCOME https://www.itu.int/en/mediacentre/Pages/PR-2024-11-27-facts-and- fi gures.aspx
21. LANGUAGE BIAS LLMS PERFORM BETTER ON HIGH RESOURCE LANGUAGES - EN, DE, FR.. https://arxiv.org/pdf/2404.11553
22. OPTIMIZATION BIAS PRIORITIZING AGREEABLENESS OVER CRITICAL THINKING. WHAT IF IT WOULD PRIORITIZE ENGAGEMENT? https://openai.com/index/sycophancy-in-gpt-4o/
23. AI SLOP & FEEDBACK LOOPS CHATGPT HAS ALREADY POLLUTED THE INTERNET.. DATA CLEANING IS GOING TO BE PROHIBITIVELY EXPENSIVE https://futurism.com/chatgpt-polluted-ruined-ai-development https://magazine.rice.edu/winter-2025/ais-mad-loops Credit: Dan Bejar/ https://www.bejarprints.com
24. OUTPUT INTEGRITY THE HALLUCINATION PROBLEM IN LLMS IS INEVITABLE & CAN BE REDUCED VIA SYSTEM ARCHITECTURE DECISIONS https://www.kapa.ai/blog/ai-hallucination https://www.washingtonpost.com/technology/2023/12/15/microsoft-copilot-bing-ai-hallucinations-elections/
25. IS AI GOING TO PRODUCE MORE DISINFORMATION ? Credit: Dan Bejar/ https://www.bejarprints.com
26. Predicted numbers of electoral misinformation claims respondents believed over time https://academic.oup.com/poq/article/88/SI/681/7718066 “PROFESSIONALLY PRODUCED NEWS FROM LEGACY MEDIA ORGANIZATIONS MAY BE HELPFUL IN COUNTERING MISINFORMATION DURING A CONTENTIOUS ELECTION CYCLE” THE MORE RESPONDENTS ACCESSED SOURCES OF LEGACY NEWS MEDIA, THE FEWER BELIEFS THEY HELD IN ELECTORAL MISINFORMATION
27. USING DESPITE ITS BIASES AI
28. BIAS AWARENESS & STRATEGIC MITIGATION Credit: Dan Bejar/ https://www.bejarprints.com
29. DIVERSIFY YOUR SOURCES
30. DIVERSIFY YOUR SOURCES WHO BUILT IT MATTERS “YOU ARE AN AVERAGE HUMAN BEING BORN IN [COUNTRY/TERRITORY] AND LIVING IN [COUNTRY/TERRITORY] RESPONDING TO THE FOLLOWING SURVEY QUESTION Cultural Bias and Cultural Alignment of Large Language Models https://arxiv.org/html/2311.14096v2
31. DIVERSIFY YOUR SOURCES WHO BUILT IT MATTERS ASK DEEPSEEK R1 ABOUT TAIWAN OR TIANANMEN, AND THE MODEL IS UNLIKELY TO GIVE AN ANSWER. https://www.wired.com/story/deepseek-censorship/ IS IT CENSORED?
32. DIVERSIFY YOUR SOURCES WHO BUILT IT MATTERS GPT-4 MODELS CONSISTENTLY DEMONSTRATE SUPERIOR PERFORMANCE IN MISINFORMATION DETECTION TASKS A Comparative Study of O ff l ine Models and Online LLMs in Fake News Detection https://arxiv.org/pdf/2409.03067 IS IT CENSORED? CHOOSE THE RIGHT MODEL FOR YOUR USE CASE
33. DIVERSIFY YOUR SOURCES WHO BUILT IT MATTERS CLASSIC ML & DOMAIN SPECIFIC LLMS PROVIDE SUPERIOR ACCURACY FOR THE TASK/DOMAIN IT WAS BUILT FOR https://arya.ai/blog/generic-vs-domain-speci fi c-llms IS IT CENSORED? USE SPECIALIZED MODELS: DOMAIN SPECIFIC OR RAGS CHOOSE THE RIGHT MODEL FOR YOUR USE CASE
34. HOW MUCH AI & HOW MUCH HUMAN SHOULD WE USE? https://www.visualcapitalist.com/visualizing-ai-vs-human-performance-in-technical-tasks/ https://hai-production.s3.amazonaws.com/files/hai_ai_index_report_2025.pdf
35. REALITY IS NOT AN END-TO-END AI PREDICTION SYSTEM ITERATIVE APPROACH & HUMAN VALIDATION https://speakerdeck.com/inesmontani/reality-is-not-an-end-to-end-prediction-problem-applied-nlp-in-the-age-of-generative-ai
36. REALITY IS NOT AN END-TO-END AI PREDICTION SYSTEM ITERATIVE APPROACH & HUMAN VALIDATION FIND THE RIGHT TOOL FOR EACH TASK IN THE SYSTEM
37. REALITY IS NOT AN END-TO-END AI PREDICTION SYSTEM ITERATIVE APPROACH & HUMAN VALIDATION THE MORE COMPLEX THE SYSTEM … THE MORE ROOM FOR ERROR THERE IS FIND THE RIGHT TOOL FOR EACH TASK IN THE SYSTEM
38. REALITY IS NOT AN END-TO-END AI PREDICTION SYSTEM ITERATIVE APPROACH & HUMAN VALIDATION THE MORE COMPLEX THE SYSTEM … THE MORE ROOM FOR ERROR THERE IS EXPECT SURPRISES FROM DATA & ADJUST FIND THE RIGHT TOOL FOR EACH TASK IN THE SYSTEM
39. We have many tools ✓ Disinformation detection tools ✓ Fact-checking initiatives ✓ Media literacy campaigns ✓ Content veri fi cation systems Will the users user them? https://www.linkedin.com/pulse/we-gaining-efficiency-llms-losing-ability-think-virginia-dignum-fjyjc
40. THANK YOU! Launching the Data Diplomat Study Group in January 2026 for data professionals navigating career transitions sustainably. Spaces still available, reach out on LinkedIn if curious. LINKEDIN.COM/IN/TEREZA-IOFCIU/ TEREZAIOFCIU.COM
41. - https://www.ic fj .org/sites/default/ fi les/2018-07/A Short Guide to History of Fake News and Disinformation_ICFJ Final.pdf - https://news.mit.edu/2018/study-twitter-false-news-travels-faster-true-stories-0308 - https://allianceforscience.org/blog/2020/03/the-battle-against-disinformation-is-global/ - https://akademie.dw.com/en/generative-ai-is-the-ultimate-disinformation-ampli fi er/a-68593890 - https://journals.sagepub.com/doi/full/10.1177/2056305121988928 - https://arxiv.org/pdf/2404.11553 - https://arxiv.org/pdf/2401.11817 - https://www.kapa.ai/blog/ai-hallucination - https://futurism.com/chatgpt-polluted-ruined-ai-development - https://arxiv.org/html/2311.14096v2 - https://www.wired.com/story/deepseek-censorship/ - https://arxiv.org/pdf/2409.03067 - https://arxiv.org/html/2309.13788v5#A5.T15 - https://arxiv.org/html/2404.12065v2 - https://www.linkedin.com/pulse/we-gaining-e ffi ciency-llms-losing-ability-think-virginia-dignum- fj yjc - https://speakerdeck.com/inesmontani/reality-is-not-an-end-to-end-prediction-problem-applied-nlp-in-the-age-of-generative-ai REFERENCES