$30 off During Our Annual Pro Sale. View Details »

SF-TAP Tutorial Flow Abstractor ver.

SF-TAP Tutorial Flow Abstractor ver.

A tutorial of SF-TAP flow abstractor.

ytakano

May 31, 2016
Tweet

More Decks by ytakano

Other Decks in Programming

Transcript

  1. SF-TAP Tutorial Flow Abstractor ver. National Institute of Information and

    Communications Technology Yuuki Takano
  2. 8IBU'MPX"CTUSBDUPS%P 2 /8*' )551*' 5-4*' 'MPX"CTUSBDUPS 'MPX $MBTTJpFS 5-4"OBMZ[FS )551"OBMZ[FS

    )5511SPYZ 5$1BOE6%1 )BOEMFS pMUFSBOE DMBTTJpFS SVMF --PPQCBDL*' %# 'PSFOTJD *%4*14 FUD "QQMJDBUJPO 1SPUPDPM"OBMZ[FS FUD 5$1%FGBVMU*' 6%1%FGBVMU*' "OBMZ[FS1MBOF "CTUSBDUPS1MBOF 'MPX *EFOUJpFS 4'5"1$FMM *11BDLFU %FGSBHNFOUFS 'MPX$MBTTJGJDBUJPO 5$13FBTTFNCMF *1%FGSBHNFOUBUJPO :PVDBOFBTJMZEFWFMPQ BQQMJDBUJPOMFWFMBOBMZ[FST
  3. 0QFSBUJOH4ZTUFNT  POXIJDI'MPX"CTUSBDUPSDBO3VO -JOVY #4% .BD049 3

  4. 3FRVJSFE-JCSBSJFT %FQFOEFODJFT #PPTU$ -JCSBSZ MJCQDBQ MJCFWFOU PSMBUFS  3& ZBNMDQQ

    0UIFS5PPMT DNBLF HJU $ $PNQJMFS HDDPSDMBOH 4
  5. *OTUBMM3FRVJSFE-JCSBSJFT "TTVNJOH6CVOUV 5 $ sudo apt-get install build-essential cmake \

    git libevent-dev libboost-all-dev libpcap-dev \ libre2-dev libyaml-cpp-dev
  6. %PXOMPBE4PVSDF$PEF BOE$PNQJMF*U 6 $ git clone https://github.com/SF-TAP/flow- abstractor.git $ cd

    flow-abstractor $ cmake -DCMAKE_BUILD_TYPE=Release CMakeLists.txt $ make
  7. $POGJHVSBUJPO'JMF DPOU 7 # global configuration global: home: /tmp/sf-tap #

    directory, on which UNIX domain files are placed timeout: 600 # close long-lived (over 600[s]) but do-nothing connections lru: yes # bring the least recently used pattern to front of list cache: yes # use cache for regex # loopback interface for injecting L7 traffic to the flow abstractor loopback7: if: loopback7 format: text tcp_default: if: default # for every flow that wasn't matched by any rules proto: TCP format: text body: yes udp_default: if: default # for every flow that wasn't matched by any rules proto: UDP format: text body: yes
  8. $POGJHVSBUJPO'JMF 8 http: up: '^[-a-zA-Z]+ .+ HTTP/1\.(0\r?\n|1\r?\n([-a-zA-Z]+: .+\r?\n)+)' down: '^HTTP/1\.[01]

    [1-9][0-9]{2} .+\r?\n' proto: TCP # TCP or UDP if: http # file name of UNIX domain socket format: text # text or binary body: yes # if specified 'no', only header is output nice: 100 # the smaller a value is, the higher a priority is # balance = 2 # flows are balanced by 2 interfaces dns_udp: proto: UDP if: dns port: 53 # port number format: text nice: 200
  9. 3VO'MPX"CTUSBDUPS 9 $ sudo ./src/sftap_fabs -i en1 -c ./examples/fabs.yaml SVOUIFGPXBCTUSBDUPS

    $ ls -R /tmp/sf-tap loopback7= tcp/ udp/ /tmp/sf-tap/tcp: default= http= smtp= torrent_tracker= dns= http_proxy= ssh= websocket= ftp= irc= ssl= /tmp/sf-tap/udp: default= dns= torrent_dht= DPOGJSNUIBUGMPXBCTUSBDUJPOJOUFSGBDFTXFSFDSFBUFE
  10. 4OJGG)551'MPXT 10 $ sudo nc -U /tmp/sf-tap/tcp/http $ curl http://www.google.com/

    SFBEUIFBCTUSBDUJPOJOUFSGBDFPG)551 BDDFTTTPNFXFCTJUFT
  11. 1SPUPDPM'PSNBUPG'MPX "CTUSBDUJPO*OUFSGBDFT 11 $ sudo nc -U /tmp/sf-tap/tcp/http ip1=192.168.24.54,ip2=216.58.221.196,port1=59547,port2=80,hop=0,l3=ipv4, l4=tcp,event=CREATED

    ip1=192.168.24.54,ip2=216.58.221.196,port1=59547,port2=80,hop=0,l3=ipv4, l4=tcp,event=DATA,from=2,match=down,len=494 HTTP/1.1 302 Found Cache-Control: private Content-Type: text/html; charset=UTF-8 Location: http://www.google.co.jp/?gfe_rd=cr&ei=oVcLVvL7JsHD8AfZnYHQAQ (omitted) ip1=192.168.24.54,ip2=216.58.221.196,port1=59547,port2=80,hop=0,l3=ipv4, l4=tcp,event=DATA,from=1,match=up,len=78 GET / HTTP/1.1 Host: www.google.com User-Agent: curl/7.43.0 Accept: */* ip1=192.168.24.54,ip2=216.58.221.196,port1=59547,port2=80,hop=0,l3=ipv4, l4=tcp,event=DESTROYED IFBEFS IFBEFS EBUB IFBEFS EBUB IFBEFS
  12. )FBEFS'PSNBU $47MJLFLFZWBMVFQBJST $POTJTUJOHPGPOFMJOF FOEFEXJUIaO 12 ip1=192.168.24.54,ip2=216.58.221.196,port1=59547,port2=80,hop=0, l3=ipv4,l4=tcp,event=CREATED { “ip1”: “192.168.24.54”,

    “ip2”: “216.58.221.196”, “port1”: 59547, “port2”: 80, “hop”: 0, “l3”: “ipv4”, “l4”: “tcp”, “event”: “CREATED” } FRVJWBMFOUTGPS
  13. -JGF$ZDMFPGB'MPX 13 $3&"5&% %&4530:&% %"5" 8IFO5$1DPOOFDUJPOJTFTUBCMJTIFE QFSGPSNFEXBZIBOETIBLF  $3&"5&%FWFOUJTJOWPLFE 8IFO5$1DPOOFDUJPOJTEFTUSPZFE

    SFDFJWFE'*/345 PSUJNFPVU  %&4530:&%FWFOUJTJOWPLFE 8IFOBSSJWJOHEBUB %"5"FWFOUJTJOWPLFE
  14. 1SPUPDPMTPG6%1 6%1JTOPUDPOOFDUJPOPSJFOUFE 5IFSFGPSF POMZ%"5"FWFOUJTJOWPLFE 14

  15. 'MPX*EFOUJGJDBUJPO &BDIGMPXJTJEFOUJGJFECZ*1BEESFTTFT  1PSUOVNCFSTBOEIPQDPVOU 'MPXTBSF*EFOUJGJFECZUVQMFPG
  JQ QPSU JQ QPSU

    IPQ  )PQGJMFEJOEJDBUFTUIBUIPXNBOZUJNFT UIFGMPXJTSFJOKFDUFEUPUIF-MPPQCBDL JOUFSGBDF 15 ip1=192.168.24.54,ip2=216.58.221.196,port1=59547,port2=80, hop=0,l3=ipv4,l4=tcp,event=CREATED
  16. 0SJHJOPG%"5" 5$1JTDPOOFDUJPOPSJFOUFE 5IFSFGPSF EBUBJTDPNJOHGSPNPSJHJOT 16 JQ QPSU JQ QPSU EBUBGSPNIPTU

    EBUBGSPNIPTU IPTU IPTU ip1=192.168.24.54,ip2=216.58.221.196,po rt1=59547,port2=80,hop=0,l3=ipv4,l4=tcp ,event=DATA,from=2,match=down,len=494 GSPNGJFMEJOEJDBUFTUIFPSJHJOPGEBUB
  17. -FOHUIPG%"5" -FOGJMFEJOEJDBUFTUIFMFOHUIPGEBUB 17 ip1=192.168.24.54,ip2=216.58.221.196,port1=59547,port2=80,hop=0, l3=ipv4,l4=tcp,event=DATA,from=2,match=down,len=494 IFBEFS FWFOU%"5" MFO EBUB CZUFT

  18. 6QTUSFBNBOE%PXOTUSFBN .BUDIGJMFEJOEJDBUFTUIBUXIJDIQBUUFSOJT VTFEGPSNBUDIJOH 18 ip1=192.168.24.54,ip2=216.58.221.196,port1=59547,port2=80,hop=0,l3=ipv4, l4=tcp,event=DATA,from=2,match=down,len=494 http: up: '^[-a-zA-Z]+ .+

    HTTP/1\.(0\r?\n|1\r?\n([-a-zA-Z]+: .+\r?\n)+)' down: '^HTTP/1\.[01] [1-9][0-9]{2} .+\r?\n' proto: TCP # TCP or UDP if: http # file name of UNIX domain socket format: text # text or binary body: yes # if specified 'no', only header is output nice: 100 # the smaller a value is, the higher a priority is # balance = 2 # flows are balanced by 2 interfaces $POGJHVSBUJPO .BUDIFEXJUIUIFQBUUFSOPGEPXOTUSFBN .BUDIFEXJUIUIFQBUUFSOPGVQTUSFBN ip1=192.168.24.54,ip2=216.58.221.196,port1=59547,port2=80,hop=0,l3=ipv4, l4=tcp,event=DATA,from=1,match=up,len=78
  19. 8SJUF:PVS0XO"OBMZ[FST 4LFMUPOJO1TFVEP$PEF 19 // connect to socket s = socket();

    connect(s, “/tmp/sf-tap/tcp/http”); for (;;) { // read header readline(s, line); h = parse_header(line); // generate session ID sid = new sessionID(h[“ip1”], h[“ip2”], h[“port1”], h[“port2”], h[“hop”]); if (h[“event”] == “DATA”) { read(s, buf, h[“len”]); } }
  20. 4LFMUPOJO1ZUIPO 20 IUUQTHJTUHJUIVCDPNZUBLBOPGDCEGDDD

  21. &YBNQMFT 1SPUPDPM1BSTFST 21 $ git clone https://github.com/SF-TAP/protocol- parser.git $ cd

    protocol-parser/http $ sudo python3 sftap_http.py NPSFJOGPSNBUJPOJTBWBJMBCMFPO IUUQTHJUIVCDPN4'5"1EPDVNFOUTCMPCNBTUFS UVUPSJBM@GBCT@VCVOUVNE