Death of the Trusted Internet

In 2007 the White House issued a policy instructing all government agencies to enhance their perimeter security by routing connections to the public internet through a Trusted Internet Connection (TIC). Part reverse proxy, part logging system, TICs have been single handedly responsible for some of the greatest scalability challenges facing government IT and have locked billions of dollars in taxpayer money into aging and archaic technology. By 2017 the problem had grown so extreme that the United States Digital Service assembled a team of engineers to figure out how to kill the TIC and move Agencies into the cloud without opening up a wild west of irresponsible security configurations. This is the story of their challenges, complications, own-goals, and engineering factions.

Transcript

1. DEATH OF THE TRUSTED INTERNET DEPARTMENT OF COMMERCE
2. @bellmar | @usds DEATH OF THE TRUSTED INTERNET UNITED STATES DIGITAL SERVICE ๏ Helping civil servants provide better service to American citizens with technology ๏ Big government -vs- small government is false dichotomy ๏ Why is government technology hard: ๏ government has lots of talent and plenty of money ๏ “private sector experience” pushes the boundaries of privacy, security and free markets ๏ Blockers come from legitimate questions about what is appropriate for government ๏ The men and women of USDS are not geniuses, just people empowered to roam
3. @bellmar | @usds DEATH OF THE TRUSTED INTERNET ONCE UPON A TIME…
4. @bellmar | @usds DEATH OF THE TRUSTED INTERNET ONCE UPON A TIME… Data Center AWS Public Internet
5. @bellmar | @usds DEATH OF THE TRUSTED INTERNET ONCE UPON A TIME… Data Center AWS Public Internet Dynamo
6. @bellmar | @usds DEATH OF THE TRUSTED INTERNET ONCE UPON A TIME… Has anyone at USDS dealt with this TIC thing
7. @bellmar | @usds DEATH OF THE TRUSTED INTERNET ONCE UPON A TIME… Has anyone at USDS dealt with this TIC thing
8. @bellmar | @usds DEATH OF THE TRUSTED INTERNET ONCE UPON A TIME… Has anyone at USDS dealt with this TIC thing
9. @bellmar | @usds DEATH OF THE TRUSTED INTERNET ONCE UPON A TIME… Has anyone at USDS dealt with this TIC thing WE’VE BE TRYING TO FIX THIS FOR THREE YEARS, HELP!!!!
10. @bellmar | @usds DEATH OF THE TRUSTED INTERNET “TRUSTED INTERNET CONNECTION” ๏ Office of Management and Budget policy written in 2007 ๏ Perimeter defense tactic, connections between the private network and the public internet go through specific access points ๏ Each agency gets two access points (sort of) ๏ Net flow and packet inspection tools connected to access points ๏ Signatures (classified and unclassified) match to known threats
11. @bellmar | @usds DEATH OF THE TRUSTED INTERNET BACK IN 2007 ๏ First generation iPhone has just come out. There is no App Store ๏ Amazon only just launched EC2 and S3. Microsoft and Google won’t enter the cloud market for another year ๏ “You shouldn’t get SaaS for any application where your entire company is depending on that application running successfully all the time” Network World Magazine
12. @bellmar | @usds DEATH OF THE TRUSTED INTERNET WHERE IS THE PERIMETER? ๏ …when someone else runs an agency’s email servers (Gmail, Microsoft 365) ๏ …when applications for travel (Concur) or case management (Salesforce) are in the cloud ๏ …when employees expect to be able to work remotely with laptops, tablets, and cellphones ๏ …when on-prem is going out of style in favor of the cost benefits of IaaS
13. @bellmar | @usds DEATH OF THE TRUSTED INTERNET TIC IN 2017 ๏ While agencies wait for a new TIC policy, their only option is to manipulate how they define “internal” and “external” ๏ In theory, external applications are not supposed to have sensitive government data on them ๏ FISMA - Federal Information Security Management Act ๏ defines seriousness based on how critical the data is to the agency’s function ๏ certification is hard, and harder the higher the level ๏ Limits hosting options for custom built applications ๏ FedRAMP - Federal Risk and Authorization Management Program ๏ Limits products/services that can be used ๏ Agency specific laws (Census Title 13, IRS Title 26)
14. @bellmar | @usds DEATH OF THE TRUSTED INTERNET BACK TO THIS Data Center AWS Public Internet Dynamo
15. @bellmar | @usds DEATH OF THE TRUSTED INTERNET BACK TO THIS Data Center AWS Public Internet Dynamo
16. @bellmar | @usds DEATH OF THE TRUSTED INTERNET BACK TO THIS Data Center AWS Public Internet Dynamo
17. @bellmar | @usds DEATH OF THE TRUSTED INTERNET BACK TO THIS Data Center AWS Public Internet Dynamo
18. @bellmar | @usds DEATH OF THE TRUSTED INTERNET OR THIS?
19. @bellmar | @usds DEATH OF THE TRUSTED INTERNET OR THIS? Data Center AWS MDM
20. @bellmar | @usds DEATH OF THE TRUSTED INTERNET OR THIS? Data Center AWS
21. @bellmar | @usds DEATH OF THE TRUSTED INTERNET FURTHER COMPLICATIONS ๏ GovCloud is on the West Coast, most government data centers are on the East Coast ๏ Agencies also required to use DHS’s special classified intrusion detection software ๏ TIC creates a false sense of security + single point of failure ๏ TIC -vs- Encryption
22. @bellmar | @usds DEATH OF THE TRUSTED INTERNET CAN WE WORK AROUND IT? ๏ Gmail and Outlook allow outbound/inbound traffic to be forwarded to a custom SMTP server ๏ If the 2 TIC limit was gotten rid of, TICs could be used as a government CDN ๏ Use Virtual Private Clouds to manipulate internal -vs- external ๏ Use peering and direct connect
23. @bellmar | @usds DEATH OF THE TRUSTED INTERNET HOW TO MODERNIZE? VIRTUALIZE Move the TIC into the cloud INNOVATE Doing something completely new
24. LET’S DO BEYONDCORP ZERO TRUST NETWORK. IT WORKS FOR GOOGLE! Well Intentioned Engineer DEATH OF THE TRUSTED INTERNET
25. @bellmar | @usds DEATH OF THE TRUSTED INTERNET ZERO TRUST NETWORK ๏ Mutual TLS ๏ Multi factor Authentication ๏ Strict Access Control ๏ Device registration
26. @bellmar | @usds DEATH OF THE TRUSTED INTERNET ZERO TRUST NETWORK ๏ Relies heavily on PKI, a place where government has struggled ๏ 2004 Homeland Security Presidential Directive 12: secure electronic ids ๏ 2011 GAO: “progress but have not fully implemented” ๏ Engineering groups are frequently siloed from each other: Dev/Ops, Dev/Dev ๏ Former DHS secretary Michael Chertoff: “Federal agencies themselves are uneven in the way they protect their own assets” ๏ User design matters and government is the user
27. WE CAN’T PUT TIC IN THE CLOUD BECAUSE AMAZON WON’T BUILD US A SCIF Good Natured Govie DEATH OF THE TRUSTED INTERNET
28. @bellmar | @usds DEATH OF THE TRUSTED INTERNET TIC MANAGEMENT LOCATION Data Center TIC
29. @bellmar | @usds DEATH OF THE TRUSTED INTERNET TIC MANAGEMENT LOCATION Data Center TIC ๏ Must be in a government exclusive space ๏ Must be staffed 24/7 ๏ Must have 30 minutes from a SCIF ๏ Ops people have remote access
30. @bellmar | @usds ๏ Must be in a government exclusive space ๏ Must be staffed 24/7 ๏ Must have 30 minutes from a SCIF ๏ Ops people have remote access DEATH OF THE TRUSTED INTERNET TIC MANAGEMENT LOCATION Data Center TIC APIs
31. WE CAN’T PUT CLASSIFIED SIGNATURES IN THE CLOUD Good Natured Govie DEATH OF THE TRUSTED INTERNET
32. BUT WHY WOULD WE NEED TO? Me…. many, many times DEATH OF THE TRUSTED INTERNET
33. @bellmar | @usds DEATH OF THE TRUSTED INTERNET EINSTEIN ๏ Intrusion Detection Software built and maintained by DHS ๏ GAO: As of 2016 does not yet block malicious traffic from the web ๏ DHS: EINSTEIN is about situational awareness and automating data collection
34. @bellmar | @usds DEATH OF THE TRUSTED INTERNET EINSTEIN EINSTEIN TODAY EINSTEIN TOMORROW
35. @bellmar | @usds DEATH OF THE TRUSTED INTERNET EINSTEIN EINSTEIN TODAY EINSTEIN TOMORROW
36. @bellmar | @usds DEATH OF THE TRUSTED INTERNET EINSTEIN ๏ Requirements for the EINSTEIN of today: ๏ Visibility ๏ Alerting to agency ๏ Data feeds for DHS government wide intelligence ๏ Things USDS is concerned about: ๏ Email security ๏ Government Network Security ๏ Public facing government websites
37. SO… CAN WE JUST MIRROR THE PACKETS FROM THE CLOUD TO EINSTEIN? Engineers DEATH OF THE TRUSTED INTERNET
38. @bellmar | @usds DEATH OF THE TRUSTED INTERNET CAN WE VIRTUALIZE TIC? ๏ Mirror from… ๏ Switches - Not cloud friendly, depending on SPAN/TAP may drop packets ๏ Router - Dependent on specific cloud offerings ๏ VM - possible with iptables but adds overhead to instance
39. @bellmar | @usds DEATH OF THE TRUSTED INTERNET CAN WE VIRTUALIZE TIC? Load Balancer
40. @bellmar | @usds DEATH OF THE TRUSTED INTERNET CAN WE VIRTUALIZE TIC? VTIC DHS
41. @bellmar | @usds DEATH OF THE TRUSTED INTERNET THINGS LEFT TO SORT OUT ๏ Implementation: ๏ Resource load on the VMs ๏ Amount of bandwidth needed ๏ Probably best this traffic is encrypted right? ๏ Value add: ๏ GovCloud ELB are inferior to AWS ELB ๏ Compliance -vs- Security ๏ Incremental improvement: Can the government ever get to zero-trust?
42. @bellmar | @usds DEATH OF THE TRUSTED INTERNET WHY IS THIS SO HARD? ๏ SaaS, cloud hosting and mobile are different issues that require different security approaches ๏ Getting people to separate current needs, from future needs, from wishlist takes a lot of careful communication ๏ People who are good at writing policy and people who have technical expertise are rarely the same people and have trouble identifying each other ๏ If the best solution can’t be understood by its users, it's not a solution
43. @bellmar | @usds DEATH OF THE TRUSTED INTERNET CURRENT STATUS ๏ TIC Working Group’s output contributed to Office of American Innovation’s IT Modernization Report ๏ OAI report: Let’s do some pilots of new more modern ways to do TIC ๏ Acting Federal CIO: first things we’re tackling is our TIC policy
44. @bellmar | @usds DEATH OF THE TRUSTED INTERNET WHY IS THIS SO HARD? Draft Policy Agency Comments Legal Clearance Public Comments
45. @bellmar | @usds DEATH OF THE TRUSTED INTERNET WHY IS THIS SO HARD? Draft Policy Agency Comments Legal Clearance Public Comments
46. @bellmar | @usds DEATH OF THE TRUSTED INTERNET WHY IS THIS SO HARD? Draft Policy Agency Comments Legal Clearance Public Comments 100~
47. @bellmar | @usds DEATH OF THE TRUSTED INTERNET WHY IS THIS SO HARD? Draft Policy Agency Comments Legal Clearance Public Comments 100~ Do we CC Fannie Mae?
48. @bellmar | @usds DEATH OF THE TRUSTED INTERNET WHY IS THIS SO HARD? Draft Policy Agency Comments Legal Clearance Public Comments 100~ Individuals + Lobbyists + Corporations
49. @bellmar | @usds DEATH OF THE TRUSTED INTERNET PUBLIC COMMENT AS CORPORATE COMMENT ๏ Public comment period is open to everyone and run through Github (https://policy.cio.gov/) ๏ Every single comment is read and adjudicated, even the ones that are about aliens ๏ Corporations often use the public comment period to submit marketing materials to the government under the guise of “comments”: ๏ Comments on IT Modernization: (https://github.com/GSA/modernization/issues) ๏ Writing policy is expensive (time and energy), changing policy even more so ๏ Despite this there is a movement inside the government to make policy more agile and easier to iterate.
50. @bellmar | @usds DEATH OF THE TRUSTED INTERNET GET INVOLVED ๏ USDS is always hiring: usds.gov/join ๏ 18F offers remote opportunities: 18f.gsa.gov/join/ ๏ Want to try your hand at tech policy? OFCIO regularly recruits: cio.gov/hiringevent/ ๏ Interested in my Technical SWAT team work? Feel free to reach out ๏ Twitter: bellmar ๏ keybase.io: bellmar
51. THANK YOU!