Ruby Conf 2021 - Dishonest Software: Fighting Against Industry Norms

From daemons that conceal themselves, to apps which lie to us, every day you're impacted by software with dishonest intentions.

No one starts their career building dishonest tools, but over time, the norms & incentives in specific industries (ex: infosec, advertising) can compromise the ethics of even the most principled developer.

In this talk we...

* Define dishonest software using examples & counter-examples
* Arm you with compelling arguments to convince product leadership to build ethical software
* Explore how engineers can advocate for the data privacy rights of others

Transcript

1. Dishonest Software
   
   
   Fighting Back Against Industry Norms
   RubyConf 2021
   Jason Meller
   CEO & Founder of Kolide
   SOFTWARE ETHICS TRACK
   

   View Slide

2. Security app for devices. Instead of locking them down, it messages
   employees on Slack when their device has security/policy issues.
   INTRO Jason Meller
   • Building Rails apps for the cyber security
   industry since 2010.
   • Reformed Script Kiddie
   • CEO, Founder of Kolide
   

   View Slide

3. View Slide

4. MOST PEOPLE DO NOT INTEND TO BUILD OR
   BENEFIT FROM DISHONEST SOFTWARE
   YET MANY OF US WILL.
   

   View Slide

5. GE and the “Advanced Persistent Threat” (APT)
   The APT is a term that refers organized group of threat
   actors, sponsored speci
   fi
   cally by the Chinese
   Government that wage long-running and extensive cyber
   espionage campaigns against western corporations.
   THEIR GOAL:
   Ex
   fi
   ltrate valuable information that will advance Chinese military and
   economic interests (mostly proprietary IP)
   

   View Slide

6. View Slide

7. PLA UNIT 61398 CENTER BUILDING (MAIN GATE. SOLDIERS VISIBLE)
   

   View Slide

8. View Slide

9. The GE / Rolls-Royce F136
   Advanced Turbo Fan Engine
   proposed for the Joint Strike
   Fighter (JSF) Program
   

   View Slide

10. @echo off
    cd /d c:\windows\tasks
    rar.log a XXXXXXXX.rar -v200m “C:\Documents and Settings\Place\My
    Documents\XXXXXXXX” -hpsmy123!@#
    del *.vbs
    del %0
    FTP
    

    View Slide

11. GE’s Computer Incident Response Team (CIRT)
    DETECTION APPARATUS
    • Network taps Installed on all known o
    ff
    i
    ce / datacenter network egress ports
    

    and VPN concentrators
    
    
    • All Layer 3/4 Tra
    ff i
    c automatically analyzed using signatures matching known worrying
    behavior
    
    
    • Full packet captures (PCAP) recorded for all analyzed tra
    ffi
    c, and saved for at least 30 days.
    CIRCA 2010
    Remember: in 2010, nearly all sites did not use HTTPs. All tra
    ff i
    c was in the clear.
    

    View Slide

12. End Result: The GE-CIRT can essentially see everything each employee is doing.
    

    View Slide

13. In the United States, Yes.
    

    The Electronic Communications Privacy Act of 1986 (ECPA) allows employers to…
    “Wait, is this legal?”
    CONCERNS:
    • Open up physical mail addressed to you at the o
    ff
    i
    ce.
    
    
    • Track your location via GPS on company devices and vehicles.
    
    
    • Record keystrokes, take screenshots, & save network tra
    ffi
    c on company devices.
    Potentially Illegal: Remote activating the webcam or microphone
    without prior consent. (Robbins v. Lower Merion School District)
    

    View Slide

14. • Our mission is pure: we’re Americans
    fi
    ghting a foreign enemy.
    
    
    • We are looking for sophisticated heists, not petty crime. We have bigger
    fi
    sh to fry.
    
    
    • Each member of the security team is a good person and has been extensively vetted.
    
    
    • We audit each other’s activities.
    
    
    • The psychic costs (which cannot even be measured) of using dishonest software are
    worth it if they prevent true-harm (which is easily measured)
    Good Guys Can Do Good With Dishonest Software
    RATIONALE:
    

    View Slide

15. Then on one dark and stormy night…
    

    

    It happened.
    

    View Slide

16. • Contractor lost their job.
    
    
    • We destroyed the contractor’s personal photos forever.
    
    
    • Word spread fast throughout the company to other employees and contractor about
    the GE’s surveillance capabilities.
    
    
    • No tangible consequences for GE’s CIRT team members or analysis of our mission.
    
    
    • The lost credibility negatively impacted the security of the company.
    “Are We The Baddies?”
    OUTCOMES:
    

    View Slide

17. “Trust us. We are the good guys”
    DISHONEST:
    HONEST:
    “Trust us, because you can independently
    verify we are telling the truth.”
    

    View Slide

18. “You have the right to know what we can see”
    HONEST:
    

    View Slide

19. But do this right and it leads to so much more…
    

    View Slide

20. A Bad Test For
    Dishonest Software
    Does this software break the law?
    

    View Slide

21. A Good Test For
    Dishonest Software
    Would requiring informed consent break
    the software’s value proposition?
    

    View Slide

22. View Slide

23. View Slide

24. According to the complaint, the wiretaps embedded in the website’s code “are
    used by Defendants to secretly observe and record website visitors’ keystrokes,
    mouse clicks, and other electronic communications, including the entry of
    Personally Identi
    fi
    able Information (‘PII’), in real time.”
    

    View Slide

25. You should make sure the thing we made isn’t illegal.
    DISHONEST:
    

    View Slide

26. “Privacy means people know what they’re signing up for, in plain English, and
    repeatedly. That’s what it means. I’m an optimist, I believe people are smart. And
    some people want to share more data than other people do. Ask them. Ask them
    every time. Make them tell you to stop asking them if they get tired of your asking
    them. Let them know precisely what you’re going to do with their data.”
    Steve Jobs @ D8 Tech Conference (2010)
    

    View Slide

27. View Slide

28. A world full of Bug Bounty Programs
    Ask in plain language & require a response!
    The Anatomy of Informed Consent.
    

    View Slide

29. A world full of Bug Bounty Programs
    Let them see the data collected by default
    The Anatomy of Informed Consent.
    

    View Slide

30. A world full of Bug Bounty Programs
    Allow them to revoke consent at anytime, without
    talking to a person
    The Anatomy of Informed Consent.
    

    View Slide

31. A Good Test For
    Dishonest Software
    Would requiring informed consent break
    the software’s value proposition?
    

    View Slide

32. YOUR ROLE
    You are a developer, you have more power than you think,
    and you have the ability and responsibility to identify
    dishonest software and advocate for the privacy rights of
    your friends, family, and fellow co-workers.
    DO NOT ADVOCATE JUST FOR YOURSELF
    

    View Slide

33. ARGUMENTS
    • Building honest software is now a competitive advantage over incumbents.
    
    
    • Dishonest software is incompatible with ever-increasing privacy laws (ex: GDPR /
    California Consumer Privacy Act)
    
    
    • Device vendors (like Apple) will force you to be honest eventually, but then it will
    be on their terms.
    
    
    • People who make dishonest software
    fi
    nd it easier to be dishonest to the
    employees. Advocating for honesty will bene
    fi
    t everyone you work with.
    

    View Slide

34. Thank you!
    jason @ kolide.com
    

    github.com / terracatta
    

    Jason Meller @ Rails Link Slack
    

    twitter.com / jmeller
    

    View Slide