How to spy with Python: So easy, the NSA can do it!

This particular version was given at Kiwi PyCon, 2017.

A brief history of signal intelligence around the world, where we are today, and how you can do it with Python.

Transcript

1. How to Spy with Python
2. How to Spy with Python So easy, the NSA can do it! Lynn Root | @roguelynn | roguelynn.com
3. $ whoami • Site Reliability Engineer at Spotify in NYC • Spotify’s FOSS evangelist • Global PyLadies Leader
4. Today’s Talk
5. Today’s Talk • Who’s doing the spying • What they’re doing: Historical context • How its being done • Why it matters • How you can do it, too!
6. Today’s Talk • Who’s doing the spying • What they’re doing: Historical context • How its being done • Why it matters • How you can do it, too!
7. Today’s Talk • Who’s doing the spying • What they’re doing: Historical context • How its being done • Why it matters • How you can do it, too!
8. Today’s Talk • Who’s doing the spying • What they’re doing: Historical context • How its being done • Why it matters • How you can do it, too!
9. Today’s Talk • Who’s doing the spying • What they’re doing: Historical context • How its being done • Why it matters • How you can do it, too!
10. What This Talk Is Not
11. What This Talk Is Not • Condoning what the NSA, CSE, GCHQ, etc. are doing.
12. What This Talk Is Not • Condoning what the NSA, CSE, GCHQ, etc. are doing. • How to avoid being tracked and spied upon.
13. What This Talk Is Not • Condoning what the NSA, CSE, GCHQ, etc. are doing. • How to avoid being tracked and spied upon. • Encouraging you to spy on friends, family, patrons of cafes with free wifi, etc.
14. Obligatory Disclaimer
15. Obligatory Disclaimer • I am not affiliated with the NSA, FBI, CIA, etc.
16. Obligatory Disclaimer • I am not affiliated with the NSA, FBI, CIA, etc. • I am not a lawyer.
17. Obligatory Disclaimer • I am not affiliated with the NSA, FBI, CIA, etc. • I am not a lawyer. • I am not a (black|white) hat.
18. Who?
19. Five Eyes
20. Nine Eyes
21. Fourteen Eyes
22. Forty One Eyes
23. Historical Context
24. Historical Context TL;DR: we should have seen it coming.
25. The Five Eyes group includes the USA (NSA), UK (GCHQ), Australia (ASD), Canada (CSE), and New Zealand (SIS) for the purpose of sharing signal intelligence. Five Eyes 1941 1941
26. Collection of all telegraphic data entering into or exiting from the United States, including daily microfilm copies of all incoming, outgoing, and transiting telegrams. SHAMROCK 1945 1975 1945 – 1975 1941
27. New Zealand’s first permanent SIGINT site, NR1 was built just south of Waiouru, disguised by a nearby naval receiving station. NR1 Station Built 1947 1947 1945 1941 1975
28. The National Security Agency has origins from within the US Army in 1917 during WWI. It separated into the Armed Forces Security Agency after WWII, then regrouped as the National Security Agency in 1952 NSA Established 1952 1952 1945 1941 1975 1947
29. Intercepted communications of targeted US Citizens, and passed it onto relevant authorities (FBI, CIA, Secret Service, etc.). MINARET 1967 1973 1967 – 1973 1945 1941 1952 1947
30. Started in late 1960s, formally established in 1971, ECHELON is a surveillance program ran by the Five Eyes to monitor the military and diplomatic communications of the Soviet Union and its Eastern Bloc allies during the Cold War. ECHELON 1971 1971 1967 1945 1941 1975 1952 1947
31. 1971 Only after over 20 years from when the NSA was established, and over 50 years after the start of the US’s surveillance, did the Supreme Court require warrants for domestic surveillance. The Keith Case 1972 1972 1967 1945 1941 1952 1947
32. 1971 Various committees within the US government investigated the NSA and the CIA’s illegal activity, exposing surveillance programs like MINARET, SHAMROCK, and other abuses of power like MKUltra and government-sponsored assassination of foreign leaders. The (1st) Big Reveal 1974 – 1976 1967 1945 1941 1974 1973 1952 1947
33. New Zealand’s Government Communications Security Bureau to consolidate responsibilities over the nation’s communications security and signal intelligence. GCSB Established 1977 1971 1967 1973 1975 1945 1941 1952 1977 1947
34. 1971 The Foreign Intelligence Surveillance Act (FISA) was signed to protect Americans. A “secret” sourt, the Foreign Intelligence Surveillance Court, was created for the purpose of hearing requests for warrants. FISA Signed into Law 1978 1978 1967 1973 1975 1945 1941 1952 1947
35. Operating under FISA – and expanded after September 11th, 2001 – BLARNEY collects data directly from top-level telecommunication facilities within the United States. BLARNEY 1978, Revealed 2013 1978 1967 1973 1975 1971 1945 1941 1952 1947
36. A federal database containing personal and financial data of millions of United States citizens suspected as threads to national security. Main Core 1982, Revealed 2008 1982 1967 1973 1975 1971 1977 1945 1941 1952 1947
37. In 1982, the GCSB opened an interception station at Tangimoana, taking over much of NR1’s work. Seven years later, another station was opened in Waihopai. GCSB Opened Two SIGINT Sites 1982 & 1989 1982 1967 1973 1975 1971 1977 1945 1941 1952 1947 1989
38. An FBI developed customizable packet sniffer that can monitor all internet traffic of a particular person. Carnivore 1997 – 2005 1997 2005 1967 1973 1975 1971 1982 1945 1941 1952 1977 1947 1989
39. After the 9/11 World Trade Center attacks, the culture against spying begins to shift within the NSA. Culture Shift 2001 2001 1967 1973 1975 1971 1982 1997 1945 1941 1952 1977 1947 1989
40. Telecoms voluntarily provide data to the NSA. Domestic Spying & Telecom Involvement 2002 – 2003 1967 1973 1975 1971 1982 1997 2001 2002 1945 1941 1952 1977 1947 1989
41. a.k.a the Manhattan Project of counter-terrorism Total Terrorist Information Awareness 2003 1967 1973 1975 1971 1982 1997 2001 2003 1945 1941 1952 1977 1947 1989
42. Collection of nine programs, including those focused on decrypting communications, malware defence infrastructure, and database of the metadata of particular surveillance targets. Turbulance 2003 1967 1973 1975 1971 1982 1997 2001 2003 1945 1941 1952 1977 1947 1989
43. In December 2005, the New York Times reveals that the NSA has been spying on Americans without warrants. Soon after, President Bush confirms the NSA’s warrantless eavesdropping. The New York Times also reveals some of the NSA’s spying is purely domestic with some telecoms giving backdoor access to communication streams. NSA Exposed 2005 2005 1967 1973 1975 1971 1982 1945 1941 2001 2003 1997 1952 1977 1947 1989
44. Database containing metadata for hundreds of billions of telephone calls made through the four largest telephone carriers in the US: AT&T, SBC, BellSouth, and Verizon. MAINWAY Revealed 2006 2006 1967 1973 1975 1971 1982 1945 1941 2005 2001 2003 1997 1952 1977 1947 1989
45. Surveillance program aimed at foreign embassies and diplomatic staff, including NATO allies. Dropmire 2007, Revealed 2013 2007 1967 1973 1975 1971 1982 1945 1941 2005 2001 2003 1997 1952 1977 1947 1989
46. The Protect America Act was passed, allowing the NSA to not need warrants for collecting communications. Protect America Act 2007 2007 1967 1973 1975 1971 1982 1945 1941 2005 2001 2003 1997 1952 1977 1947 1989
47. UK’s system that buffers internet traffic directly via fiber-optic cables to be processed and searched later. Tempora Tested 2008, Released 2011 2008 1967 1973 1975 1971 1982 1945 1941 2005 2001 2003 2007 1997 1952 1977 1947 1989
48. A GCHQ program that intercepts millions of Yahoo users’ webcams. Optic Nerve 2008 – 2012, Revealed 2014 2008 2012 1967 1973 1975 1971 1982 1945 1941 2005 2001 2003 2007 1997 1952 1977 1947 1989
49. FISA Amendments 2008 2008 1967 1973 1975 1971 1982 1945 1941 2005 2001 2003 2007 1997 President Bush signs into law amendemnds to FISA that gives telecoms retroactive immunity to complying in warrantless surveillance. 1952 1977 1947 1989
50. 1971 1967 1973 1975 1982 2005 2001 2003 2007 1945 1941 2009 1997 1952 NZSIS seeks help from unis 2009 SIS asks university staff to watch for “terror science” among students and colleagues. 1977 1947 1989
51. 1971 GCHQ’s data reconnaissance tool to port scan entire countries, particularly looking for vulnerable services, and shared with the Five Eyes group to launch exploits or steal data. HACIENDA Started 2009, Revealed 2014 1967 1973 1975 1982 2005 2001 2003 2007 1945 1941 2009 1997 1952 1977 1947 1989
52. Canadian Sells SIGINT Data 2012 1967 1973 1975 1971 1982 1945 1941 2005 2001 2003 2007 1997 October 2012, a Royal Canadian Navy intelligence officer pleads guilty to sharing SIGINT data collected from a program called STONEGHOST. 2012 2009 1952 1977 1947 1989
53. Kitterage Report 2012 1967 1973 1975 1971 1982 1945 1941 2005 2001 2003 2007 1997 GCSB had been illegally intercepting Kim Dotcom’s communications, leading to indictments from US on copyright infringement and money laundering. 2012 2009 1952 1977 1947 1989
54. GCSB taps directly into the main undersea cable link, collecting ~95% of New Zealand’s internet traffic. SPEARGUN Started 2012, Revealed 2014 1967 1973 1975 1971 1982 1945 1941 2005 2001 2003 2007 1997 2012 2009 1952 1977 1947 1989
55. 2013 1967 1973 1975 1971 1982 2009 2012 1945 1941 2005 2001 2003 2007 1997 2015 The (2nd) Big Reveal 2013 – 2015 From whistleblower Edward Snowden, we now know of the following: 
 FAIRVIEW, STORMBREW: Upstream collection with voluntary cooperation with AT&T and Verizon. MUSCULAR: allowed warrantless data syphoning from Yahoo & Google. 1952 1977 1947 1989
56. 2013 1967 1973 1975 1971 1982 2009 2012 1945 1941 2005 2001 2003 2007 1997 2015 The (2nd) Big Reveal 2013 – 2015 BULLRUN: NSA cracking encryption & storing encrypted data for future breakthroughs. Royal Concierge: GCHQ tracking bookings made at targeted hotels. P.R.I.S.M.: NSA’s collecting & mining (meta)data. XKeyScore: NSA’s query system for PRISM-collected data. 1952 1977 1947 1989
57. 2013 1967 1973 1975 1971 1982 2009 2012 1945 1941 2005 2001 2003 2007 1997 2015 The (2nd) Big Reveal 2013 – 2015 DISHFIRE: Global collection & storage of text messages, ran by the NSA & GCHQ. MYSTIC: Collects phone call metadata and content from several entire countries. BADASS: Joint CSE and GCHQ program that tracks users via privacy leaks in mobile apps (including Angry Birds). 1952 1977 1947 1989
58. How they’re doing it
59. How? TL;DR: drinking directly from the hose
60. How? TL;DR: drinking directly from the hose • Tier 1 network – backbone of the internet – provided by companies like Level 3, AT&T, Verizon, Deutsche Telecom, and ~15 others.
61. How? TL;DR: drinking directly from the hose • Tier 1 network – backbone of the internet – provided by companies like Level 3, AT&T, Verizon, Deutsche Telecom, and ~15 others. • Facebook, Microsoft, Google, etc. tap directly into Tier 1 via edges, or own their own transatlantic cables.
62. How? TL;DR: drinking directly from the hose • Tier 1 network – backbone of the internet – provided by companies like Level 3, AT&T, Verizon, Deutsche Telecom, and ~15 others. • Facebook, Microsoft, Google, etc. tap directly into Tier 1 via edges, or own their own transatlantic cables. • NSA “covertly” does the same thing: tapping edges
63. How?
64. Why it matters
65. Unanswered questions
66. Unanswered questions • How do companies not notice being backdoored? Or are they lying when denying cooperation?
67. Unanswered questions • How do companies not notice being backdoored? Or are they lying when denying cooperation? • How is “foreignness” determined? Am I roped in because I interact daily with non-US citizens?
68. Unanswered questions • How do companies not notice being backdoored? Or are they lying when denying cooperation? • How is “foreignness” determined? Am I roped in because I interact daily with non-US citizens? • What is done with data that’s “accidentally” collected on Americans?
69. Unanswered questions • How do companies not notice being backdoored? Or are they lying when denying cooperation? • How is “foreignness” determined? Am I roped in because I interact daily with non-US citizens? • What is done with data that’s “accidentally” collected on Americans? • How secure is the stored information?
70. So What?
71. So What? • Do you have curtains in your home?
72. So What? • Do you have curtains in your home? • Can I see your credit card bills?
73. So What? • Do you have curtains in your home? • Can I see your credit card bills? • How about your text messages?
74. Metadata Matters
75. Metadata Matters Taken from the EFF presentation at 30C3 in December 2013 • They know you rang a phone sex service at 2:24am and spoke for 18 minutes. But they don’t know what you talked about. • They know you called the suicide prevention hotline from the Golden Gate Bridge. But the topic of the call remains secret. • They know you spoke with an HIV testing service, then your doctor, then your health insurance company in the same hour. But they don’t know what was discussed.
76. Metadata Matters Taken from the EFF presentation at 30C3 in December 2013 • They know you rang a phone sex service at 2:24am and spoke for 18 minutes. But they don’t know what you talked about. • They know you called the suicide prevention hotline from the Golden Gate Bridge. But the topic of the call remains secret. • They know you spoke with an HIV testing service, then your doctor, then your health insurance company in the same hour. But they don’t know what was discussed.
77. Metadata Matters Taken from the EFF presentation at 30C3 in December 2013 • They know you rang a phone sex service at 2:24am and spoke for 18 minutes. But they don’t know what you talked about. • They know you called the suicide prevention hotline from the Golden Gate Bridge. But the topic of the call remains secret. • They know you spoke with an HIV testing service, then your doctor, then your health insurance company in the same hour. But they don’t know what was discussed.
78. How you can do it, too!
79. Let’s first look the part…
81. Tools used • scapy – packet sniffing & manipulation • pygeoip – API for GeoIP databases • python-geojson – bindings & utilities for GeoJSON • python-nmap – wrapper around nmap port scanner
82. Obligatory Statement • Proof of concept! No warranty. • I’m not condoning the use of traffic sniffing, spying, etc.
83. Quick intro to scapy
84. >>> from scapy.all import *
85. >>> from scapy.all import * >>> a = sniff(iface="en0", filter="tcp and port 80", count=10)
86. >>> from scapy.all import * >>> a = sniff(iface="en0", filter="tcp and port 80", count=10) >>> a
87. >>> from scapy.all import * >>> a = sniff(iface="en0", filter="tcp and port 80", count=10) >>> a <Sniffed: TCP:10 UDP:0 ICMP:0 Other:0>
88. >>> from scapy.all import * >>> a = sniff(iface="en0", filter="tcp and port 80", count=10) >>> a <Sniffed: TCP:10 UDP:0 ICMP:0 Other:0> >>> a.res
89. >>> from scapy.all import * >>> a = sniff(iface="en0", filter="tcp and port 80", count=10) >>> a <Sniffed: TCP:10 UDP:0 ICMP:0 Other:0> >>> a.res [<Ether dst=00:1d:70:df:2d:11 src=14:10:9f:e1:54:9b type=0x800 |<IP version=4L ihl=5L tos=0x0 len=64 id=650 flags=DF frag=0L ttl=64 proto=tcp chksum=0x9f88 src=10.25.3.61 dst=184.73.211.6 options=[] |<TCP sport=53491 dport=http seq=3474155615 ack=0 dataofs=11L reserved=0L flags=S window=65535 chksum=0xecd6 urgptr=0 options=[('MSS', 1460), ('NOP', None), ('WScale', 4), ('NOP', None), ('NOP', None), ('Timestamp', (1224433615, 0)), ('SAckOK', ''), ('EOL', None)] |>>>, <Ether dst=00:1d:70:df:2d:11 src=14:10:9f:e1:54:9b type=0x800 |<IP version=4L ihl=5L tos=0x0 len=64 id=41196 flags=DF frag=0L ttl=64 proto=tcp chksum=0xb59a src=10.25.3.61 dst=50.31.164.188 options=[] |<TCP sport=53492 dport=http seq=3315328916 ack=0 dataofs=11L reserved=0L flags=S window=65535 chksum=0x2b8d urgptr=0 options=[('MSS', 1460), ('NOP', None), ('WScale', 4), ('NOP', None), ('NOP', None), ('Timestamp', (1224433615, 0)), ('SAckOK', ''), ('EOL', None)] |>>>, # <--snip-->
90. >>> a.res[0] # first packet
91. >>> a.res[0] # first packet <Ether dst=00:1d:70:df:2d:11 src=14:10:9f:e1:54:9b type=0x800 |<IP version=4L ihl=5L tos=0x0 len=64 id=650 flags=DF frag=0L ttl=64 proto=tcp chksum=0x9f88 src=10.25.3.61 dst=184.73.211.6 options=[] |<TCP sport=53491 dport=http seq=3474155615 ack=0 dataofs=11L reserved=0L flags=S window=65535 chksum=0xecd6 urgptr=0 options=[('MSS', 1460), ('NOP', None), ('WScale', 4), ('NOP', None), ('NOP', None), ('Timestamp', (1224433615, 0)), ('SAckOK', ''), ('EOL', None)] |>>>
92. >>> a.res[0].show()
93. >>> a.res[0].show() ###[ Ethernet ]### dst = 00:1d:70:df:2d:11 src = 14:10:9f:e1:54:9b type = 0x800 ###[ IP ]### # <--snip--> flags = DF frag = 0L ttl = 64 proto = tcp chksum = 0x9f88 src = 10.25.3.61 dst = 184.73.211.6 ###[ TCP ]### # <--snip--> sport = 53491 dport = http options = [('MSS', 1460), ('NOP', None), ('WScale', 4), ('NOP', None), ('NOP', None), ('Timestamp', (1224433615, 0)), ('SAckOK', ''), ('EOL', None)]
94. Snippet 1
95. Snippet 1 Tempura Goal: Show me everyone that has searched for a certain term
96. >>> # importing pcap file >>> sample_http = 'data/search.cap'
97. >>> # importing pcap file >>> sample_http = 'data/search.cap' >>> pkts = sniff(offline=sample_http)
98. >>> # importing pcap file >>> sample_http = 'data/search.cap' >>> pkts = sniff(offline=sample_http) >>> # online sniffing >>> pkts = sniff(filter="tcp and host search.yahoo.com", count=300)
99. >>> # importing pcap file >>> sample_http = 'data/search.cap' >>> pkts = sniff(offline=sample_http) >>> # online sniffing >>> pkts = sniff(filter="tcp and host search.yahoo.com", count=300) >>> # saving for later >>> wrpcap("data/search.cap", pkts)
100. >>> # importing pcap file >>> sample_http = 'data/search.cap' >>> pkts = sniff(offline=sample_http) >>> # online sniffing >>> pkts = sniff(filter="tcp and host search.yahoo.com", count=300) >>> # saving for later >>> wrpcap("data/search.cap", pkts) >>> pkts
101. >>> # importing pcap file >>> sample_http = 'data/search.cap' >>> pkts = sniff(offline=sample_http) >>> # online sniffing >>> pkts = sniff(filter="tcp and host search.yahoo.com", count=300) >>> # saving for later >>> wrpcap("data/search.cap", pkts) >>> pkts <Sniffed: TCP:300 UDP:0 ICMP:0 Other:0>
102. >>> pkts.nsummary()
103. >>> pkts.nsummary() 0000 Ether / IP / TCP 10.25.3.61:53261 > 74.6.239.58:http S 0001 Ether / IP / TCP 10.25.3.61:53262 > 74.6.239.58:http S 0002 Ether / IP / TCP 10.25.3.61:53263 > 74.6.239.58:http S 0003 Ether / IP / TCP 74.6.239.58:http > 10.25.3.61:53261 SA 0004 Ether / IP / TCP 10.25.3.61:53261 > 74.6.239.58:http A 0005 Ether / IP / TCP 74.6.239.58:http > 10.25.3.61:53263 SA 0006 Ether / IP / TCP 10.25.3.61:53263 > 74.6.239.58:http A 0007 Ether / IP / TCP 74.6.239.58:http > 10.25.3.61:53262 SA 0008 Ether / IP / TCP 10.25.3.61:53262 > 74.6.239.58:http A 0009 Ether / IP / TCP 10.25.3.61:53261 > 74.6.239.58:http PA / Raw 0010 Ether / IP / TCP 74.6.239.58:http > 10.25.3.61:53261 A 0011 Ether / IP / TCP 74.6.239.58:http > 10.25.3.61:53261 PA / Raw 0012 Ether / IP / TCP 10.25.3.61:53261 > 74.6.239.58:http A 0013 Ether / IP / TCP 10.25.3.61:53261 > 74.6.239.58:http PA / Raw 0014 Ether / IP / TCP 74.6.239.58:http > 10.25.3.61:53261 A 0015 Ether / IP / TCP 74.6.239.58:http > 10.25.3.61:53261 A / Raw / Padding 0016 Ether / IP / TCP 74.6.239.58:http > 10.25.3.61:53261 A / Raw / Padding 0017 Ether / IP / TCP 10.25.3.61:53261 > 74.6.239.58:http A 0018 Ether / IP / TCP 74.6.239.58:http > 10.25.3.61:53261 A / Raw / Padding # <--snip-->
104. >>> pkts[79].show()
105. >>> pkts[79].show() ###[ Ethernet ]### # <--snip--> ###[ IP ]### # <--snip--> ###[ TCP ]### # <--snip--> ###[ Raw ]### load = 'GET / search;_ylt=A0oG7mGUD49SBxcA3WpXNyoA;_ylc=X1MDMjc2NjY3OQRfcgMyBGJjawNmbWVsb2 s1OTRqZ3UyJTI2YiUzRDQlMjZkJTNEOU15M2RnMXBZRUtpdVJyeG9BWlNlRGxLcjJFLSUyNnMlM0 Q4ciUyNmklM0RTSjdlY2Y4ZURZakZnbS5DRWRucgRjc3JjcHZpZANHcC5VRjBnZXVyRDdPcmloVW tuRHdnWUFYWjUwR1ZLUEQ1UUFCc3hpBGZyA3lmcC10LTE0MARmcjIDc2ItdG9wBGdwcmlkA2NlVH N4WXhzUWIuOW51aGNlWG9TTUEEbXRlc3RpZANBRDAxJTNEU01FMzQxJTI2QURTUlAlM0RBRFNSUE MxJTI2QVNTVCUzRFZJUDI4OSUyNk1TRlQlM0RNU1kwMTAlMjZVSTAxJTNEVUlDMSUyNlVOSSUzRF JDRjA0NARuX3JzbHQDMTAEbl9zdWdnAzgEb3JpZ2luA3NlYXJjaC55YWhvby5jb20EcG9zAzAEcH FzdHIDBHBxc3RybAMEcXN0cmwDNgRxdWVyeQNNYWRyaWQEdF9zdG1wAzEzODUxMDczNTU4MzEEdn Rlc3RpZANVSUMx?p=Madrid&fr2=sb-top&fr=yfp-t-140 HTTP/1.1\r\nHost: search.yahoo.com\r\nConnection: keep-alive\r\nAccept: text/html,application/ xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\nUser-Agent: Mozilla/ 5.0 (Macintosh; Intel Mac OS X 10_9_0) AppleWebKit/537.36 # <--snip-->
106. >>> pkts[79].getLayer(Raw)
107. >>> pkts[79].getLayer(Raw) <Raw load='GET / search;_ylt=A0oG7mGUD49SBxcA3WpXNyoA;_ylc=X1MDMjc2NjY3OQRfcgMyBGJjawNmbWVsb2 s1OTRqZ3UyJTI2YiUzRDQlMjZkJTNEOU15M2RnMXBZRUtpdVJyeG9BWlNlRGxLcjJFLSUyNnMlM0 Q4ciUyNmklM0RTSjdlY2Y4ZURZakZnbS5DRWRucgRjc3JjcHZpZANHcC5VRjBnZXVyRDdPcmloVW tuRHdnWUFYWjUwR1ZLUEQ1UUFCc3hpBGZyA3lmcC10LTE0MARmcjIDc2ItdG9wBGdwcmlkA2NlVH N4WXhzUWIuOW51aGNlWG9TTUEEbXRlc3RpZANBRDAxJTNEU01FMzQxJTI2QURTUlAlM0RBRFNSUE MxJTI2QVNTVCUzRFZJUDI4OSUyNk1TRlQlM0RNU1kwMTAlMjZVSTAxJTNEVUlDMSUyNlVOSSUzRF JDRjA0NARuX3JzbHQDMTAEbl9zdWdnAzgEb3JpZ2luA3NlYXJjaC55YWhvby5jb20EcG9zAzAEcH FzdHIDBHBxc3RybAMEcXN0cmwDNgRxdWVyeQNNYWRyaWQEdF9zdG1wAzEzODUxMDczNTU4MzEEdn Rlc3RpZANVSUMx?p=Madrid&fr2=sb-top&fr=yfp-t-140 HTTP/1.1\r\nHost:search. yahoo.com\r\nConnection: keep-alive\r\nAccept: text/html,application/ xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\nUser-Agent: Mozilla/ 5.0 (Macintosh; Intel Mac OS X 10_9_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36\r\nReferer: http://search.yahoo.com /search;_ylt=ApD.LW7jivmrlmZzNKxChqqbvZx4?p=Python&toggle=1&cop=mss&ei=UTF-8 &fr=yfp-t-140\r\nAccept-Encoding: gzip,deflate,sdch\r\nAccept-Language: en- US,en;q=0.8\r\nCookie: B=fmelok594jgu2&b=4&d=9My3dg1pYEKiuRrxoAZSeDlKr2E- &s=8r&i=SJ7ecf8eDYjFgm.CEdnr; AO=o=0;YLS=v=1&p=1&n=1;F=a=I.qqZFgMvSp1S MQ7oNaJGIBu5DAJGO25SeRxXSKxg6_KZLWHQMHEkeFQrEOxAH9BOvMhwKs-&b=.hBp; Y=v=1&n=fr6nunkr11qks&l=he6k4bodd/o&p=f2m0' |>
108. >>> first_query = pkts[79].getlayer(Raw)
109. >>> first_query = pkts[79].getlayer(Raw) >>> print first_query.fields.get('load').split('?p=')[1].split('&')[0]
110. >>> first_query = pkts[79].getlayer(Raw) >>> print first_query.fields.get('load').split('?p=')[1].split('&')[0] Montreal
111. >>> first_query = pkts[79].getlayer(Raw) >>> print first_query.fields.get('load').split('?p=')[1].split('&')[0] Montreal >>> second_query = pkts[148].getlayer(Raw)
112. >>> first_query = pkts[79].getlayer(Raw) >>> print first_query.fields.get('load').split('?p=')[1].split('&')[0] Montreal >>> second_query = pkts[148].getlayer(Raw) >>> print second_query.fields.get('load').split('?p=')[1].split('&')[0]
113. >>> first_query = pkts[79].getlayer(Raw) >>> print first_query.fields.get('load').split('?p=')[1].split('&')[0] Montreal >>> second_query = pkts[148].getlayer(Raw) >>> print second_query.fields.get('load').split('?p=')[1].split('&')[0] Best+Chocolate+in+Montreal
114. >>> first_query = pkts[79].getlayer(Raw) >>> print first_query.fields.get('load').split('?p=')[1].split('&')[0] Montreal >>> second_query = pkts[148].getlayer(Raw) >>> print second_query.fields.get('load').split('?p=')[1].split('&')[0] Best+Chocolate+in+Montreal >>> third_query = pkts[227].getlayer(Raw)
115. >>> first_query = pkts[79].getlayer(Raw) >>> print first_query.fields.get('load').split('?p=')[1].split('&')[0] Montreal >>> second_query = pkts[148].getlayer(Raw) >>> print second_query.fields.get('load').split('?p=')[1].split('&')[0] Best+Chocolate+in+Montreal >>> third_query = pkts[227].getlayer(Raw) >>> print third_query.fields.get('load').split('?p=')[1].split('&')[0]
116. >>> first_query = pkts[79].getlayer(Raw) >>> print first_query.fields.get('load').split('?p=')[1].split('&')[0] Montreal >>> second_query = pkts[148].getlayer(Raw) >>> print second_query.fields.get('load').split('?p=')[1].split('&')[0] Best+Chocolate+in+Montreal >>> third_query = pkts[227].getlayer(Raw) >>> print third_query.fields.get('load').split('?p=')[1].split('&')[0] Best+Coffee+in+Montreal
117. Snippet 2
118. Snippet 2 Omnivore Goal: Show me everyone from a certain country that has visited a particular extremist forum
119. >>> pkts = sniff(filter="tcp and host $SOME_EXTREMIST_FORUM")
120. >>> # pkts = sniff(filter="tcp and host $SOME_EXTREMIST_FORUM") >>> sample_http = 'data/http.cap' >>> pkts = sniff(offline=sample_http)
121. >>> # pkts = sniff(filter="tcp and host $SOME_EXTREMIST_FORUM") >>> sample_http = 'data/http.cap' >>> pkts = sniff(offline=sample_http) >>> pkts
122. >>> # pkts = sniff(filter="tcp and host $SOME_EXTREMIST_FORUM") >>> sample_http = 'data/http.cap' >>> pkts = sniff(offline=sample_http) >>> pkts <Sniffed: TCP:258 UDP:0 ICMP:0 Other:0>
123. >>> pkts[7].show()
124. >>> pkts[7].show() ###[ IP ]### proto = tcp src = 10.77.19.246 dst = 81.28.232.58 ###[ TCP ]### sport = 52281 dport = http flags = PA options = [('NOP', None), ('NOP', None), ('Timestamp', (1046978977, 817411654))] ###[ Raw ]### load = 'GET / HTTP/1.1\r\nHost: www.pyladies.com\r\nConnection: keep-alive\r\nPragma: no-cache\r\nCache- Control: no-cache\r\nUpgrade-Insecure-Requests: 1\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36\r\nAccept: text/html,application/ xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\nAccept-Encoding: gzip, deflate, sdch\r\nAccept-Language: en-US,en;q=0.8\r\nCookie: __utma=226990108.1846941016.1400135873.1440434789.1440440902.71; __utmc=226990108\r\n’
125. >>> raw = pkts[7].getlayer(Raw)
126. >>> raw = pkts[7].getlayer(Raw) >>> load = raw.fields.get('load')
127. >>> raw = pkts[7].getlayer(Raw) >>> load = raw.fields.get('load') >>> print load
128. >>> raw = pkts[7].getlayer(Raw) >>> load = raw.fields.get('load') >>> print load GET / HTTP/1.1 Host: www.pyladies.com Connection: keep-alive Pragma: no-cache Cache-Control: no-cache Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/ 537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/ *;q=0.8 Accept-Encoding: gzip, deflate, sdch Accept-Language: en-US,en;q=0.8 Cookie: __utma=226990108.1846941016.1400135873.1440434789.1440440902.71; __utmc=226990108
129. >>> raw = pkts[7].getlayer(Raw) >>> load = raw.fields.get('load') >>> print load >>> '$EXTREMIST_TERM' in load
130. >>> raw = pkts[7].getlayer(Raw) >>> load = raw.fields.get('load') >>> print load >>> '$EXTREMIST_TERM' in load False
131. def trace_route(target): res, unans = traceroute(target) # scapy's traceroute traces = res.res hops = [] for trace in traces: if trace[1].src not in hops: hops.append(trace[1].src) return hops
132. >>> tr = trace_route("www.spotify.com")
133. >>> tr = trace_route("www.spotify.com") >>> tr
134. >>> tr = trace_route("www.spotify.com") >>> tr ['10.76.0.1', '216.46.28.65', '78.152.34.202', '216.46.0.20', '129.250. 202.89', '129.250.2.7', '78.152.34.10', '129.250.2.182', '129.250.3.124', '129.250.6.238', '129.250.2.133', '129.250.199.234', '10.224.160. 125', '193.235.203.132']
135. import pygeoip def map_ip(hops): gip = pygeoip.GeoIP('data/GeoLiteCity.dat') coordinates = [] for hop in hops: geo_data = gip.record_by_addr(hop) if geo_data: lat = geo_data['latitude'] lon = geo_data['longitude'] coordinates.append((lon, lat)) return coordinates
136. >>> coordinates = map_ip(tr)
137. >>> coordinates = map_ip(tr) >>> coordinates
138. >>> coordinates = map_ip(tr) >>> coordinates [(-73.5739, 45.51480000000001), (5.75, 52.5), (-104.8738, 39.623700000000014), (15.0, 62.0)]
139. import geojson def create_geojson(coordinates): geo_list = [] j = 1 data = {} data["type"] = "Feature" data["id"] = j data["properties"] = {"title": "hop %i" % j} data["geometry"] = {"type": "LineString", "coordinates": [list(c) for c in coordinates]} geo_list.append(data) d = {"type": "FeatureCollection"} for item in geo_list: d.setdefault("features", []).append(item) return geojson.dumps(d)
140. >>> search_route = create_geojson(coordinates)
141. >>> search_route = create_geojson(coordinates) >>> search_route
142. >>> search_route = create_geojson(coordinates) >>> search_route {"type": "FeatureCollection","features": [{"geometry": {"type": "LineString","coordinates": [[-73.5739, 45.51480000000001],[5.75, 52.5]]},"type": "Feature","id": 1,"properties": {"title": "hop 1"}}, {"geometry": {"type": "LineString","coordinates": [[5.75, 52.5],[-104.8738, 39.623700000000014]]},"type": "Feature","id": 2,"properties": {"title": "hop 2"}},{"geometry": {"type": "LineString","coordinates": [[-104.8738, 39.623700000000014], [15.0, 62.0]]},"type": "Feature","id": 3,"properties": {"title": "hop 3"}}]}
144. Snippet 3
145. Snippet 3 TEFLON Goal: Give me all emails with a particular phrase in the body of an email
146. >>> pkts = sniff(filter="tcp and port 25")
147. >>> # pkts = sniff(filter="tcp and port 25") >>> sample_smtp = 'data/smtp.cap' >>> pkts = sniff(offline=sample_smtp)
148. >>> pkts.nsummary()
149. >>> pkts.nsummary() 0000 Ether / IP / TCP 10.10.1.4:uaiact > 74.53.140.153:smtp S 0001 Ether / IP / TCP 74.53.140.153:smtp > 10.10.1.4:uaiact SA 0002 Ether / IP / TCP 10.10.1.4:uaiact > 74.53.140.153:smtp A 0003 Ether / IP / TCP 74.53.140.153:smtp > 10.10.1.4:uaiact PA / Raw 0004 Ether / IP / TCP 10.10.1.4:uaiact > 74.53.140.153:smtp PA / Raw 0005 Ether / IP / TCP 74.53.140.153:smtp > 10.10.1.4:uaiact A / Padding 0006 Ether / IP / TCP 74.53.140.153:smtp > 10.10.1.4:uaiact PA / Raw 0007 Ether / IP / TCP 10.10.1.4:uaiact > 74.53.140.153:smtp PA / Raw 0008 Ether / IP / TCP 74.53.140.153:smtp > 10.10.1.4:uaiact PA / Raw 0009 Ether / IP / TCP 10.10.1.4:uaiact > 74.53.140.153:smtp PA / Raw 0010 Ether / IP / TCP 74.53.140.153:smtp > 10.10.1.4:uaiact PA / Raw 0011 Ether / IP / TCP 10.10.1.4:uaiact > 74.53.140.153:smtp PA / Raw 0012 Ether / IP / TCP 74.53.140.153:smtp > 10.10.1.4:uaiact PA / Raw 0013 Ether / IP / TCP 10.10.1.4:uaiact > 74.53.140.153:smtp PA / Raw 0014 Ether / IP / TCP 74.53.140.153:smtp > 10.10.1.4:uaiact PA / Raw 0015 Ether / IP / TCP 10.10.1.4:uaiact > 74.53.140.153:smtp PA / Raw 0016 Ether / IP / TCP 74.53.140.153:smtp > 10.10.1.4:uaiact PA / Raw # <--snip-->
150. >>> pkts[11]
151. >>> pkts[11] <Ether dst=00:1f:33:d9:81:60 src=00:e0:1c:3c:17:c2 type=0x800 |<IP version=4L ihl=5L tos=0x0 len=70 id=9513 flags=DF frag=0L ttl=128 proto=tcp chksum=0xf3ac src=10.10.1.4 dst=74.53.140.153 options=[] |<TCP sport=uaiact dport=smtp seq=2126795718 ack=2934727424 dataofs=5L reserved=0L flags=PA window=65199 chksum=0x22a4 urgptr=0 options=[] |<Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |>>>>
152. >>> pkts[11] <Ether dst=00:1f:33:d9:81:60 src=00:e0:1c:3c:17:c2 type=0x800 |<IP version=4L ihl=5L tos=0x0 len=70 id=9513 flags=DF frag=0L ttl=128 proto=tcp chksum=0xf3ac src=10.10.1.4 dst=74.53.140.153 options=[] |<TCP sport=uaiact dport=smtp seq=2126795718 ack=2934727424 dataofs=5L reserved=0L flags=PA window=65199 chksum=0x22a4 urgptr=0 options=[] |<Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |>>>> >>> raw = pkts[11].getlayer(Raw)
153. >>> pkts[11] <Ether dst=00:1f:33:d9:81:60 src=00:e0:1c:3c:17:c2 type=0x800 |<IP version=4L ihl=5L tos=0x0 len=70 id=9513 flags=DF frag=0L ttl=128 proto=tcp chksum=0xf3ac src=10.10.1.4 dst=74.53.140.153 options=[] |<TCP sport=uaiact dport=smtp seq=2126795718 ack=2934727424 dataofs=5L reserved=0L flags=PA window=65199 chksum=0x22a4 urgptr=0 options=[] |<Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |>>>> >>> raw = pkts[11].getlayer(Raw) >>> raw
154. >>> pkts[11] <Ether dst=00:1f:33:d9:81:60 src=00:e0:1c:3c:17:c2 type=0x800 |<IP version=4L ihl=5L tos=0x0 len=70 id=9513 flags=DF frag=0L ttl=128 proto=tcp chksum=0xf3ac src=10.10.1.4 dst=74.53.140.153 options=[] |<TCP sport=uaiact dport=smtp seq=2126795718 ack=2934727424 dataofs=5L reserved=0L flags=PA window=65199 chksum=0x22a4 urgptr=0 options=[] |<Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |>>>> >>> raw = pkts[11].getlayer(Raw) >>> raw <Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |>
155. >>> pkts[11] <Ether dst=00:1f:33:d9:81:60 src=00:e0:1c:3c:17:c2 type=0x800 |<IP version=4L ihl=5L tos=0x0 len=70 id=9513 flags=DF frag=0L ttl=128 proto=tcp chksum=0xf3ac src=10.10.1.4 dst=74.53.140.153 options=[] |<TCP sport=uaiact dport=smtp seq=2126795718 ack=2934727424 dataofs=5L reserved=0L flags=PA window=65199 chksum=0x22a4 urgptr=0 options=[] |<Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |>>>> >>> raw = pkts[11].getlayer(Raw) >>> raw <Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |> >>> load = raw.fields.get('load').split()[0]
156. >>> pkts[11] <Ether dst=00:1f:33:d9:81:60 src=00:e0:1c:3c:17:c2 type=0x800 |<IP version=4L ihl=5L tos=0x0 len=70 id=9513 flags=DF frag=0L ttl=128 proto=tcp chksum=0xf3ac src=10.10.1.4 dst=74.53.140.153 options=[] |<TCP sport=uaiact dport=smtp seq=2126795718 ack=2934727424 dataofs=5L reserved=0L flags=PA window=65199 chksum=0x22a4 urgptr=0 options=[] |<Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |>>>> >>> raw = pkts[11].getlayer(Raw) >>> raw <Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |> >>> load = raw.fields.get('load').split()[0] >>> load
157. >>> pkts[11] <Ether dst=00:1f:33:d9:81:60 src=00:e0:1c:3c:17:c2 type=0x800 |<IP version=4L ihl=5L tos=0x0 len=70 id=9513 flags=DF frag=0L ttl=128 proto=tcp chksum=0xf3ac src=10.10.1.4 dst=74.53.140.153 options=[] |<TCP sport=uaiact dport=smtp seq=2126795718 ack=2934727424 dataofs=5L reserved=0L flags=PA window=65199 chksum=0x22a4 urgptr=0 options=[] |<Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |>>>> >>> raw = pkts[11].getlayer(Raw) >>> raw <Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |> >>> load = raw.fields.get('load').split()[0] >>> load 'Z3VycGFydGFwQHBhdHJpb3RzLmlu'
158. >>> pkts[11] <Ether dst=00:1f:33:d9:81:60 src=00:e0:1c:3c:17:c2 type=0x800 |<IP version=4L ihl=5L tos=0x0 len=70 id=9513 flags=DF frag=0L ttl=128 proto=tcp chksum=0xf3ac src=10.10.1.4 dst=74.53.140.153 options=[] |<TCP sport=uaiact dport=smtp seq=2126795718 ack=2934727424 dataofs=5L reserved=0L flags=PA window=65199 chksum=0x22a4 urgptr=0 options=[] |<Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |>>>> >>> raw = pkts[11].getlayer(Raw) >>> raw <Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |> >>> load = raw.fields.get('load').split()[0] >>> load 'Z3VycGFydGFwQHBhdHJpb3RzLmlu' >>> import base64
159. >>> pkts[11] <Ether dst=00:1f:33:d9:81:60 src=00:e0:1c:3c:17:c2 type=0x800 |<IP version=4L ihl=5L tos=0x0 len=70 id=9513 flags=DF frag=0L ttl=128 proto=tcp chksum=0xf3ac src=10.10.1.4 dst=74.53.140.153 options=[] |<TCP sport=uaiact dport=smtp seq=2126795718 ack=2934727424 dataofs=5L reserved=0L flags=PA window=65199 chksum=0x22a4 urgptr=0 options=[] |<Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |>>>> >>> raw = pkts[11].getlayer(Raw) >>> raw <Raw load='Z3VycGFydGFwQHBhdHJpb3RzLmlu\r\n' |> >>> load = raw.fields.get('load').split()[0] >>> load 'Z3VycGFydGFwQHBhdHJpb3RzLmlu' >>> import base64 >>> base64.b64decode(load) 'gurpartap@patriots.in'
160. >>> pkts[12]
161. >>> pkts[12] >>> raw = pkts[12].getlayer(Raw)
162. >>> pkts[12] >>> raw = pkts[12].getlayer(Raw) >>> load = raw.fields.get('load')
163. >>> pkts[12] >>> raw = pkts[12].getlayer(Raw) >>> load = raw.fields.get('load') >>> some_encoded_string = load.split(' ')[1]
164. >>> pkts[12] >>> raw = pkts[12].getlayer(Raw) >>> load = raw.fields.get('load') >>> some_encoded_string = load.split(' ')[1] >>> some_encoded_string
165. >>> pkts[12] >>> raw = pkts[12].getlayer(Raw) >>> load = raw.fields.get('load') >>> some_encoded_string = load.split(' ')[1] >>> some_encoded_string 'UGFzc3dvcmQ6'
166. >>> pkts[12] >>> raw = pkts[12].getlayer(Raw) >>> load = raw.fields.get('load') >>> some_encoded_string = load.split(' ')[1] >>> some_encoded_string 'UGFzc3dvcmQ6' >>> base64.b64decode(some_encoded_string) 'Password:'
167. >>> pkts[12] >>> raw = pkts[12].getlayer(Raw) >>> load = raw.fields.get('load') >>> some_encoded_string = load.split(' ')[1] >>> some_encoded_string 'UGFzc3dvcmQ6' >>> base64.b64decode(some_encoded_string) 'Password:' >>> raw = pkts[13].getlayer(Raw)
168. >>> pkts[12] >>> raw = pkts[12].getlayer(Raw) >>> load = raw.fields.get('load') >>> some_encoded_string = load.split(' ')[1] >>> some_encoded_string 'UGFzc3dvcmQ6' >>> base64.b64decode(some_encoded_string) 'Password:' >>> raw = pkts[13].getlayer(Raw) >>> load = raw.fields.get('load').split()[0]
169. >>> pkts[12] >>> raw = pkts[12].getlayer(Raw) >>> load = raw.fields.get('load') >>> some_encoded_string = load.split(' ')[1] >>> some_encoded_string 'UGFzc3dvcmQ6' >>> base64.b64decode(some_encoded_string) 'Password:' >>> raw = pkts[13].getlayer(Raw) >>> load = raw.fields.get('load').split()[0] >>> load
170. >>> pkts[12] >>> raw = pkts[12].getlayer(Raw) >>> load = raw.fields.get('load') >>> some_encoded_string = load.split(' ')[1] >>> some_encoded_string 'UGFzc3dvcmQ6' >>> base64.b64decode(some_encoded_string) 'Password:' >>> raw = pkts[13].getlayer(Raw) >>> load = raw.fields.get(‘load').split()[0] >>> load 'aHVudGVyMg=='
171. >>> pkts[12] >>> raw = pkts[12].getlayer(Raw) >>> load = raw.fields.get('load') >>> some_encoded_string = load.split(' ')[1] >>> some_encoded_string 'UGFzc3dvcmQ6' >>> base64.b64decode(some_encoded_string) 'Password:' >>> raw = pkts[13].getlayer(Raw) >>> load = raw.fields.get('load').split()[0] >>> load 'aHVudGVyMg==' >>> # what could this be?!? >>> base64.b64decode(load)
172. >>> pkts[12] >>> raw = pkts[12].getlayer(Raw) >>> load = raw.fields.get('load') >>> some_encoded_string = load.split(' ')[1] >>> some_encoded_string 'UGFzc3dvcmQ6' >>> base64.b64decode(some_encoded_string) ‘Password:' >>> raw = pkts[13].getlayer(Raw) >>> load = raw.fields.get('load').split()[0] >>> load 'aHVudGVyMg==' >>> # what could this be?!? >>> base64.b64decode(load) 'hunter2'
173. def filter_packet_by_string(pkt, string): if pkt.haslayer(Raw): raw_load = pkt.getlayer(Raw).fields.get('load') if string in raw_load: print pkt.sprintf("\n**QUERY FOUND:**\n" "From {IP:%IP.src% -> %IP.dst%\n}") print raw_load
174. >>> for pkt in pkts: ... filter_packet_by_string(pkt, 'attachment')
175. >>> for pkt in pkts: ... filter_packet_by_string(pkt, 'attachment') **QUERY FOUND:** From 10.10.1.4 -> 74.53.140.153 From: "Gurpartap Singh" <gurpartap@patriots.in> To: <raj_deol2002in@yahoo.co.in> Subject: SMTP Date: Mon, 5 Oct 2009 11:36:07 +0530 Message-ID: <000301ca4581$ef9e57f0$cedb07d0$@in> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0004_01CA45B0.095693F0" X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcpFgem9BvjjZEDeR1Kh8i+hUyVo0A== Content-Language: en-us x-cr-hashedpuzzle: SeA= AAR2 ADaH BpiO C4G1 D1gW FNB1 FPkR Fn+W HFCP HnYJ JO7s Kum6 KytW LFcI LjUt;
176. Snippet 4
177. Snippet 4 COLDBREW Goal: Find chats for this user during a particular timeframe
178. >>> pkts = sniff(filter="tcp and port 6667")
179. >>> # pkts = sniff(filter="tcp and port 6667”) >>> sample_irc = 'data/irc.cap' >>> pkts = sniff(offline=sample_irc)
180. >>> pkts.show()
181. >>> pkts.show() 0000 Ether / IP / TCP 192.168.1.2:amt_blc_port > 212.204.214.114:6667 PA / Raw 0001 Ether / IP / TCP 212.204.214.114:6667 > 192.168.1.2:amt_blc_port A 0002 Ether / IP / TCP 212.204.214.114:6667 > 192.168.1.2:amt_blc_port PA / Raw 0003 Ether / IP / TCP 192.168.1.2:amt_blc_port > 212.204.214.114:6667 A 0004 Ether / IP / UDP / DNS Ans "sterling.freenode.net." 0005 Ether / IP / UDP / DNS Qry "sterling.freenode.net." 0006 Ether / IP / UDP / DNS Ans "212.204.214.114" 0007 Ether / IP / UDP / DNS Qry "1.1.168.192.in-addr.arpa." 0008 Ether / IP / TCP 212.204.214.114:6667 > 192.168.1.2:amt_blc_port PA / Raw 0009 Ether / IP / TCP 192.168.1.2:amt_blc_port > 212.204.214.114:6667 A 0010 Ether / IP / TCP 71.10.179.129:14232 > 192.168.1.2:as_debug A 0011 Ether / IP / TCP 212.204.214.114:6667 > 192.168.1.2:amt_blc_port PA / Raw 0012 Ether / IP / TCP 192.168.1.2:amt_blc_port > 212.204.214.114:6667 A 0013 Ether / IP / TCP 212.204.214.114:6667 > 192.168.1.2:amt_blc_port PA / Raw
182. >>> pkts.summary(prn=lambda x:x.sprintf("{IP:%IP.src% -> %IP.dst%\n}" ... "{Raw:%Raw.load%\n}"))
183. >>> pkts.summary(prn=lambda x:x.sprintf("{IP:%IP.src% -> %IP.dst%\n}" ... "{Raw:%Raw.load%\n}")) 192.168.1.2 -> 212.204.214.114 'ISON Thunfisch Smiley SmileyG\n' 212.204.214.114 -> 192.168.1.2 ':sterling.freenode.net 303 vmlemon :SmileyG \r\n' 71.10.179.129 -> 192.168.1.2 '\x88\xd3\xcdC\xf5\xa0\x0c\xc7\x0e\xc7\x94Wj\x97\xcb\xa7a\x84! @\x00\x00\x00\x00\xff\xff\x02' 212.204.214.114 -> 192.168.1.2 ':yaloki!n=yaloki@156.185-64-87.adsl-dyn.isp.belgacom.be PRIVMSG #amarok : +how do you mean\r\n' 212.204.214.114 -> 192.168.1.2 ':jefferai!n=jefferai@amarok/developer/mitchell PRIVMSG #amarok :+eh?\r\n' 212.204.214.114 -> 192.168.1.2 ':jefferai!n=jefferai@amarok/developer/mitchell PRIVMSG #amarok :+what, what\r\n' 212.204.214.114 -> 192.168.1.2 ':hurra!n=lolo@p54921341.dip0.t-ipconnect.de PRIVMSG #amarok :+;p\r\n' 192.168.1.2 -> 212.204.214.114 'WHO #rokymotion\n'
184. def filter_packet_by_string(pkt, string): try: raw_load = pkt.getlayer(Raw).fields.get('load') if string in raw_load: print pkt.sprintf("QUERY FOUND:\nFrom " "{IP:%IP.src% -> %IP.dst%\n}") print raw_load except Exception: pass
185. >>> for pkt in pkts: ... filter_packet_by_string(pkt, 'amarok')
186. >>> for pkt in pkts: ... filter_packet_by_string(pkt, 'amarok') QUERY FOUND: From 212.204.214.114 -> 192.168.1.2 :yaloki!n=yaloki@156.185-64-87.adsl-dyn.isp.belgacom.be PRIVMSG #amarok : +how do you mean QUERY FOUND: From 212.204.214.114 -> 192.168.1.2 :jefferai!n=jefferai@amarok/developer/mitchell PRIVMSG #amarok :+eh? QUERY FOUND: From 212.204.214.114 -> 192.168.1.2 :jefferai!n=jefferai@amarok/developer/mitchell PRIVMSG #amarok :+what, what QUERY FOUND: From 212.204.214.114 -> 192.168.1.2 :hurra!n=lolo@p54921341.dip0.t-ipconnect.de PRIVMSG #amarok :+;p
187. Snippet 5
188. Snippet 5 LuckyCharms Goal: Find all machines within a given country that has a particular vulnerability
189. >>> import nmap >>> nm = nmap.PortScanner() >>> result = nm.scan('209.238.99.227')
190. >>> result.get('scan')
191. >>> result.get('scan') {'209.238.99.227': {'addresses': {'ipv4': '209.238.99.227'}, 'hostnames': [{'name': 'giulianisecurity.com', 'type': 'PTR'}], 'status': {'reason': 'syn-ack', 'state': 'up'}, 'tcp': {21: {'conf': '10', 'cpe': 'cpe:/a:proftpd:proftpd', 'extrainfo': '', 'name': 'ftp', 'product': 'ProFTPD', 'reason': 'syn-ack', 'state': 'open', 'version': ''}, 22: {'conf': '10', 'cpe': 'cpe:/a:openbsd:openssh:4.7', 'extrainfo': 'protocol 2.0', 'name': 'ssh', 'product': 'OpenSSH', 'reason': 'syn-ack', 'state': 'open', 'version': '4.7'}, 25: {'conf': '10', ...
192. import pygeoip def map_ip(ip): # locally saved dat file gip = pygeoip.GeoIP('data/GeoLiteCity.dat') geo_data = gip.record_by_addr(ip) if geo_data: lat = geo_data['latitude'] lon = geo_data['longitude'] return lon, lat
193. import geojson def create_geojson(coordinates): geo_list = [] data = {} data["type"] = "Feature" data["id"] = 1 data["properties"] = {"title": "hop %i" % 1} data["geometry"] = {"type": "Point", "coordinates": coordinates} geo_list.append(data) d = {"type": "FeatureCollection"} for item in geo_list: d.setdefault("features", []).append(item) return geojson.dumps(d)
194. >>> coordinates = map_ip('209.238.99.227')
195. >>> coordinates = map_ip('209.238.99.227') >>> geojson_data = create_geojson(coordinates)
196. >>> coordinates = map_ip('209.238.99.227') >>> geojson_data = create_geojson(coordinates) >>> geojson_data {"type": "FeatureCollection", "features": [{"geometry": {"type": "Point", "coordinates": [-104.8738, 39.623700000000014]}, "type": "Feature", "id": 1, "properties": {"title": "hop 1"}}]}
198. Wrap up
199. Thank you! Lynn Root | @roguelynn | roguelynn.com