Ruby Conf 2021 - Dishonest Software: Fighting Against Industry Norms

From daemons that conceal themselves, to apps which lie to us, every day you're impacted by software with dishonest intentions.

No one starts their career building dishonest tools, but over time, the norms & incentives in specific industries (ex: infosec, advertising) can compromise the ethics of even the most principled developer.

In this talk we...

* Define dishonest software using examples & counter-examples
* Arm you with compelling arguments to convince product leadership to build ethical software
* Explore how engineers can advocate for the data privacy rights of others

Transcript

1. Dishonest Software Fighting Back Against Industry Norms RubyConf 2021 Jason Meller CEO & Founder of Kolide SOFTWARE ETHICS TRACK
2. Security app for devices. Instead of locking them down, it messages employees on Slack when their device has security/policy issues. INTRO Jason Meller • Building Rails apps for the cyber security industry since 2010. • Reformed Script Kiddie • CEO, Founder of Kolide
4. MOST PEOPLE DO NOT INTEND TO BUILD OR BENEFIT FROM DISHONEST SOFTWARE YET MANY OF US WILL.
5. GE and the “Advanced Persistent Threat” (APT) The APT is a term that refers organized group of threat actors, sponsored speci fi cally by the Chinese Government that wage long-running and extensive cyber espionage campaigns against western corporations. THEIR GOAL: Ex fi ltrate valuable information that will advance Chinese military and economic interests (mostly proprietary IP)
7. PLA UNIT 61398 CENTER BUILDING (MAIN GATE. SOLDIERS VISIBLE)
9. The GE / Rolls-Royce F136 Advanced Turbo Fan Engine proposed for the Joint Strike Fighter (JSF) Program
10. @echo off cd /d c:\windows\tasks rar.log a XXXXXXXX.rar -v200m “C:\Documents and Settings\Place\My Documents\XXXXXXXX” -hpsmy123!@# del *.vbs del %0 FTP
11. GE’s Computer Incident Response Team (CIRT) DETECTION APPARATUS • Network taps Installed on all known o ff i ce / datacenter network egress ports 
 and VPN concentrators • All Layer 3/4 Tra ff i c automatically analyzed using signatures matching known worrying behavior • Full packet captures (PCAP) recorded for all analyzed tra ffi c, and saved for at least 30 days. CIRCA 2010 Remember: in 2010, nearly all sites did not use HTTPs. All tra ff i c was in the clear.
12. End Result: The GE-CIRT can essentially see everything each employee is doing.
13. In the United States, Yes. 
 The Electronic Communications Privacy Act of 1986 (ECPA) allows employers to… “Wait, is this legal?” CONCERNS: • Open up physical mail addressed to you at the o ff i ce. • Track your location via GPS on company devices and vehicles. • Record keystrokes, take screenshots, & save network tra ffi c on company devices. Potentially Illegal: Remote activating the webcam or microphone without prior consent. (Robbins v. Lower Merion School District)
14. • Our mission is pure: we’re Americans fi ghting a foreign enemy. • We are looking for sophisticated heists, not petty crime. We have bigger fi sh to fry. • Each member of the security team is a good person and has been extensively vetted. • We audit each other’s activities. • The psychic costs (which cannot even be measured) of using dishonest software are worth it if they prevent true-harm (which is easily measured) Good Guys Can Do Good With Dishonest Software RATIONALE:
15. Then on one dark and stormy night… 
 
 It happened.
16. • Contractor lost their job. • We destroyed the contractor’s personal photos forever. • Word spread fast throughout the company to other employees and contractor about the GE’s surveillance capabilities. • No tangible consequences for GE’s CIRT team members or analysis of our mission. • The lost credibility negatively impacted the security of the company. “Are We The Baddies?” OUTCOMES:
17. “Trust us. We are the good guys” DISHONEST: HONEST: “Trust us, because you can independently verify we are telling the truth.”
18. “You have the right to know what we can see” HONEST:
19. But do this right and it leads to so much more…
20. A Bad Test For Dishonest Software Does this software break the law?
21. A Good Test For Dishonest Software Would requiring informed consent break the software’s value proposition?
24. According to the complaint, the wiretaps embedded in the website’s code “are used by Defendants to secretly observe and record website visitors’ keystrokes, mouse clicks, and other electronic communications, including the entry of Personally Identi fi able Information (‘PII’), in real time.”
25. You should make sure the thing we made isn’t illegal. DISHONEST:
26. “Privacy means people know what they’re signing up for, in plain English, and repeatedly. That’s what it means. I’m an optimist, I believe people are smart. And some people want to share more data than other people do. Ask them. Ask them every time. Make them tell you to stop asking them if they get tired of your asking them. Let them know precisely what you’re going to do with their data.” Steve Jobs @ D8 Tech Conference (2010)
28. A world full of Bug Bounty Programs Ask in plain language & require a response! The Anatomy of Informed Consent.
29. A world full of Bug Bounty Programs Let them see the data collected by default The Anatomy of Informed Consent.
30. A world full of Bug Bounty Programs Allow them to revoke consent at anytime, without talking to a person The Anatomy of Informed Consent.
31. A Good Test For Dishonest Software Would requiring informed consent break the software’s value proposition?
32. YOUR ROLE You are a developer, you have more power than you think, and you have the ability and responsibility to identify dishonest software and advocate for the privacy rights of your friends, family, and fellow co-workers. DO NOT ADVOCATE JUST FOR YOURSELF
33. ARGUMENTS • Building honest software is now a competitive advantage over incumbents. • Dishonest software is incompatible with ever-increasing privacy laws (ex: GDPR / California Consumer Privacy Act) • Device vendors (like Apple) will force you to be honest eventually, but then it will be on their terms. • People who make dishonest software fi nd it easier to be dishonest to the employees. Advocating for honesty will bene fi t everyone you work with.
34. Thank you! jason @ kolide.com 
 github.com / terracatta 
 Jason Meller @ Rails Link Slack 
 twitter.com / jmeller