Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Reverse Engineering on Xunlei

A6a197ee96a2939f320fa7ac33bdb144?s=47 Hao Guan
December 20, 2010

Reverse Engineering on Xunlei

A6a197ee96a2939f320fa7ac33bdb144?s=128

Hao Guan

December 20, 2010
Tweet

Transcript

  1. Analysis of the Xunlei P2SP Content Distribution Protocol Wan Yujie

    & Guan Hao
  2. Xunlei  A download manager  It is very fast

     Large user population in China  Proprietary Protocol
  3. Reverse Engineering on Xunlei

  4. Challenges  Proprietary Protocol  Encrypted Packet  Binary Protocol

  5. Reverse Engineering  Techniques Used  Analysis on captured traffic

     Debugging  Analysis on disassembled/decompiled code  Programming  Tools Used  Wireshark  Hash & Crypto Detector  OllyDbg – ring 3 debugger  IDA Pro – disassembler  Detours – hooking library from Microsoft
  6. Encrypted Packet

  7. Identify the Encryption Algorithm

  8. Find the Key Generation Procedure

  9. Find the Key Generation Procedure AES Key 128 bits Content

    n Bytes Packet Type 4 Bytes Sequence No. 4 Bytes Packet Length 4 Bytes MD5 Hash
  10. Packet Encryption  Padding  Append padding byte until the

    length is a multiple of 16 (required for AES block cipher)  Key Generation  Take the MD5 hash of the first 8 bytes of the packet header as the key, 128 bit  Encryption  Electronic Codebook Mode (Simple)
  11. Packet Decryption  The algorithm is known  Need a

    program to decrypt the packets  A wireshark plugin maybe a good choice
  12. Wireshark Plugin

  13. Wireshark Plugin

  14. Wireshark Plugin

  15. API Hooking with DLL injection Function Caller Real Function Invoke,

    Passing Parameters Return result Hook Function (Do whatever we like) Real Function Function Caller Invode Return Return Invoke
  16. A Hooking Example  Get the input buffer  Write

    the content of buffer to a log file  Call the original sha1_update() function  Return the result sha1_update (original) sha1_update (hook function) Function Caller Invode Return Return Invoke
  17. A Hooking Example

  18. The Peer-to-Server-and-Peer Protocol

  19. Http Download  File: QQ2010SP3.exe  Web Site: http://im.qq.com/qq/all.shtml 

    File URL: http://softdl1.tech.qq.com/soft/21/QQ2010SP3.exe
  20.  File URL: http://softdl1.tech.qq.com/soft/21/QQ2010SP3.exe  HTTP Header: Http Download --

    HTTP Request -- GET /soft/21/QQ2010SP3.exe HTTP/1.1 Host: softdl1.tech.qq.com -- HTTP Response - HTTP/1.0 200 OK Content-Type: application/octet-stream Content-Length: 35657080 … Message body(Content)… Server Name request-URL
  21. Resume Download  A Simple Case:  Total Length: 1000

    Bytes  Stop at: 500 Bytes  Add Range to HTTP Header  Other Range format by Xunlei  Range: XXXX-XXXX (starting – ending) -- HTTP Request -- GET /soft/21/QQ2010SP3.exe HTTP/1.1 Host: softdl1.tech.qq.com Range: bytes=501-
  22. Multi-Flow Acceleration  Key Ideas:  Multiple HTTP flows 

    Different range in each HTTP request
  23. Multi-Flow Acceleration  Simple Case:  Total Length: 5000 Bytes

     Two flows  Message Flows Xunlei Client HTTP Server HTTP Request Local file
  24. Multi-Flow Acceleration  Simple Case:  Total Length: 5000 Bytes

     Two flows  Message Flows  First HTTP flow: get file size & first part of data Xunlei Client HTTP Server HTTP Response HTTP Request File size first part data Local file 1000 5000
  25. Multi-Flow Acceleration  Simple Case:  Total Length: 5000 Bytes

     Two flows  Message Flows  First HTTP flow: get file size & first part of data  Two parallel HTTP flows Xunlei Client HTTP Server HTTP Response HTTP Request HTTP Request HTTP Request First connection is aborted Start two parallel connections Range: 1001-3000 Range: 3001- File size first part data Local file 1001-3000 3001-
  26. Multi-Flow Acceleration  Simple Case:  Total Length: 5000 Bytes

     Two flows  Message Flows  First HTTP flow: get file size & first part of data  Two parallel HTTP flows Xunlei Client HTTP Server HTTP Request HTTP Request HTTP Request HTTP Request First connection is aborted Start two parallel connections Range: 1001-3000 Range: 3001- File size first part data Local file HTTP Response HTTP Response
  27. Mirror Acceleration – Why Fact: HOT content is available at

    multiple servers (web sites)  Key Ideas  Multiple HTTP connections with different servers  Different ranges
  28. Mirror Acceleration – How  Steps  Get file information

    from original server, e. g size
  29. Mirror Acceleration – How  Steps  Get file information

    from original server, e. g size  HTTP Request (Post) to Xunlei server -- HTTP Request –- POST / HTTP/1.1 Host: 123.129.242.168:80 …message body(Encrypted )… -URL of file: http://softdl1.tech.qq.com/soft/21/QQ2010SP3.exe -File length -Xunlei version -MAC address -IP address
  30. Mirror Acceleration – How  Steps  Get file information

    from original server, e. g size  HTTP Request (Post) to Xunlei server  HTTP Response from Xunlei server -- HTTP Response –- HTTP/1.1 200 OK …message body(Encrypted)… -List of URLs http://dl_dir.qq.com/qqfile/qq/QQ2010/QQ2010SP3.exe http://ydsoft1.greendown.cn:8880/201011/QQ2010SP3_1125.exe ………
  31. Mirror Acceleration – How  Steps  Get file information

    from original server, e. g size  HTTP Request (Post) to Xunlei server  HTTP Response from Xunlei server  Download from mirror servers
  32. Mirror Acceleration – Challenges  Content provider (owner of mirror

    servers) tries to ban direct linking  Server responds HTTP request: get only if it comes from the content providers’ page  Xunlei’s Solutions: provide both URLs and corresponding referrers direct linking : It usually occurs when a web page use of a linked object that belonging to a second site -- HTTP Request–- GET /soft/21/QQ2010SP3.exe HTTP/1.1 Referrer: http://softdl1.tech.qq.com/soft/21 -- HTTP Response –- HTTP/1.1 200 OK …message body(Encrypted)… http://dl_dir.qq.com/qqfile/qq/QQ2010/QQ20 10SP3.exe (referrer) http://ydsoft1.greendown.cn:8880/201011/QQ 2010SP3_1125.exe (referrer) ……
  33. Mirror Acceleration – Challenges  How to collect URLs and

    Referrers of so many mirror servers?  Key Ideas: make use of large user base  Xunlei browser plug-in collects referrer (address of content providers’ page) -- HTTP Request –- POST / HTTP/1.1 Host: 123.129.242.169:80 …message body(Encrypted )… -URL of file: http://softdl1.tech.qq.com/soft/21/QQ2010S P3.exe -Referrer: http://softdl1.tech.qq.com/soft/21
  34. Mirror Acceleration – Challenges  How to indentify whether these

    URLs refer to the same content?  Key Ideas: make use of large user base  Xunlei clients update summary reports to Xunlei server -- HTTP Request –- POST / HTTP/1.1 Host: 123.129.242.179:80 …message body(Encrypted)… -Partial hash -size -file name -url -referrer -etc. 20480 Bytes 20480 Bytes 20480 Bytes length/3 SHA-1 partial hash
  35. Mirror Database Conjecture 1  conjecture1 (fake data)  Query

    mirror list Find mirrors that have the same partial hash Partial_Hash Mirror hash(QQ2010PS3.exe) Tencent.com/QQ2010PS3.exe hash(QQ2010PS3.exe) huanJun.com/QQ2010PS3.exe hash(QQ2010PS3.1.exe) Tencent.com/QQ2010PS3.1.exe hash(QQ2010PS3.1.exe) AOL.com/QQ2010PS3.1.exe
  36. Test Conjecture 1  Experiment Setting:  URL http://softdl1.tech.qq.com/soft/21/QQ2010SP3.exe 

    Change local hosts file, and redirect softdl1.tech.qq.com to localhost  In folder /soft/21/ we put different test files each time
  37. • Analysis of Results -Query mirror list twice: (1) by

    length and URL (2) by partial hash and length -There is a validation mechanism, chunk hash? Test Conjecture 1 Content File Name Length Result real content QQ2010SP3.exe 30.2MB get mirror list fake content QQ2010SP3.exe 15.0MB Fail to get mirror list real content QQ2010SP3_2.exe 30.2MB Fail at the first try, obtain mirror list after 10s fake content (change a few bytes) QQ2010SP3.exe 30.2MB (1)get mirror list, but abort download from mirrors later (2)get mirror list, continue download from mirrors until finish
  38. Mirror Database Conjecture 2  Conjecture 2 Partial_Hash Mirror Partial_Hash

    Length Chunk Hash
  39. Test Conjecture 2  Test 1: download CV.pdf at Yujie’s

    home page  First Query - by length and URL
  40. Test Conjecture 2  Test 1: download CV.pdf at Yujie’s

    home page  Second Query - by partial hash and length
  41. Test Conjecture 2  Test 1: download CV.pdf at Yujie’s

    home page  Update summery report -- HTTP Request –- POST / HTTP/1.1 Host: 123.129.242.179:80 …message body(Encrypted)… -Partial hash -Chunk hash -length -file name -url -referrer -etc. Partial_Hash Mirror Hash(CV.pdf) http://personal.ie.cuhk.edu.hk/~wyj009/home/file/CV.pdf Partial_Hash Length Chunk Hash Hash(CV.pdf) 34.7MB xxxx-xxxxx-xxxxx….
  42. Test Conjecture 2  Test 2: download CV2.rar (same as

    CV.pdf) at Hao’s home page  First Query - by length and URL
  43. Test Conjecture 2  Test 2: download CV2.rar (same as

    CV.pdf) at Hao’s home page  Second Query - by partial hash and length
  44. • Composite primary key (Hash_ChunkHash, Length) • Index (Length, Partial

    _Hash) • Composite foreign key (Hash_ChunkHash, Length) • Index (Length, Mirror) Mirror Database Conjecture – Final Hash_ChunkHash Length Partial_Hash Chunk_Hash Hash_ChunkHash Length Mirror
  45. Peer-to-Peer Acceleration  Comparison with Bit Torrent  Similarity 

    Xunlei server performs as a tracker  Xunlei server provides torrent file (name, size, chunk hash, etc.)  UDP between peers
  46. Peer-to-Peer Acceleration  Comparison with Bit Torrent  Similarity 

    Xunlei server performs as a tracker  Xunlei server provides torrent file (name, size, chunk hash, etc.)  UDP between peers  Difference  Aware of peers in the same LAN (TCP)  Both push and pull  Optimized peer selection (no snubbing)
  47. Summary of Xunlei P2SP Design  Content provider can be

    regarded seeder  HTTP, BT, eDonkey can be treated uniformly  Role of Xunlei Server  Indexing  Search (search by URL, Partial Content, Torrent File )  Coordination (more than a tracker)
  48. Summary of Xunlei P2SP Design  Content provider can be

    regarded seeder  HTTP, BT, eDonkey can be treated uniformly  Role of Xunlei Server  Indexing  Search (search by URL, Partial Content, Torrent File )  Coordination (more than a tracker)  Trend of Xunlei  To be a seeder for paid users  Cross protocol sharing (we observe a BT task downloads though HTTP !)
  49. Thank You