Reverse Engineering on Xunlei

Transcript

1. Analysis of the Xunlei P2SP Content Distribution Protocol Wan Yujie & Guan Hao
2. Xunlei  A download manager  It is very fast  Large user population in China  Proprietary Protocol
3. Reverse Engineering on Xunlei
4. Challenges  Proprietary Protocol  Encrypted Packet  Binary Protocol
5. Reverse Engineering  Techniques Used  Analysis on captured traffic  Debugging  Analysis on disassembled/decompiled code  Programming  Tools Used  Wireshark  Hash & Crypto Detector  OllyDbg – ring 3 debugger  IDA Pro – disassembler  Detours – hooking library from Microsoft
6. Encrypted Packet
7. Identify the Encryption Algorithm
8. Find the Key Generation Procedure
9. Find the Key Generation Procedure AES Key 128 bits Content n Bytes Packet Type 4 Bytes Sequence No. 4 Bytes Packet Length 4 Bytes MD5 Hash
10. Packet Encryption  Padding  Append padding byte until the length is a multiple of 16 (required for AES block cipher)  Key Generation  Take the MD5 hash of the first 8 bytes of the packet header as the key, 128 bit  Encryption  Electronic Codebook Mode (Simple)
11. Packet Decryption  The algorithm is known  Need a program to decrypt the packets  A wireshark plugin maybe a good choice
12. Wireshark Plugin
13. Wireshark Plugin
14. Wireshark Plugin
15. API Hooking with DLL injection Function Caller Real Function Invoke, Passing Parameters Return result Hook Function (Do whatever we like) Real Function Function Caller Invode Return Return Invoke
16. A Hooking Example  Get the input buffer  Write the content of buffer to a log file  Call the original sha1_update() function  Return the result sha1_update (original) sha1_update (hook function) Function Caller Invode Return Return Invoke
17. A Hooking Example
18. The Peer-to-Server-and-Peer Protocol
19. Http Download  File: QQ2010SP3.exe  Web Site: http://im.qq.com/qq/all.shtml  File URL: http://softdl1.tech.qq.com/soft/21/QQ2010SP3.exe
20.  File URL: http://softdl1.tech.qq.com/soft/21/QQ2010SP3.exe  HTTP Header: Http Download -- HTTP Request -- GET /soft/21/QQ2010SP3.exe HTTP/1.1 Host: softdl1.tech.qq.com -- HTTP Response - HTTP/1.0 200 OK Content-Type: application/octet-stream Content-Length: 35657080 … Message body(Content)… Server Name request-URL
21. Resume Download  A Simple Case:  Total Length: 1000 Bytes  Stop at: 500 Bytes  Add Range to HTTP Header  Other Range format by Xunlei  Range: XXXX-XXXX (starting – ending) -- HTTP Request -- GET /soft/21/QQ2010SP3.exe HTTP/1.1 Host: softdl1.tech.qq.com Range: bytes=501-
22. Multi-Flow Acceleration  Key Ideas:  Multiple HTTP flows  Different range in each HTTP request
23. Multi-Flow Acceleration  Simple Case:  Total Length: 5000 Bytes  Two flows  Message Flows Xunlei Client HTTP Server HTTP Request Local file
24. Multi-Flow Acceleration  Simple Case:  Total Length: 5000 Bytes  Two flows  Message Flows  First HTTP flow: get file size & first part of data Xunlei Client HTTP Server HTTP Response HTTP Request File size first part data Local file 1000 5000
25. Multi-Flow Acceleration  Simple Case:  Total Length: 5000 Bytes  Two flows  Message Flows  First HTTP flow: get file size & first part of data  Two parallel HTTP flows Xunlei Client HTTP Server HTTP Response HTTP Request HTTP Request HTTP Request First connection is aborted Start two parallel connections Range: 1001-3000 Range: 3001- File size first part data Local file 1001-3000 3001-
26. Multi-Flow Acceleration  Simple Case:  Total Length: 5000 Bytes  Two flows  Message Flows  First HTTP flow: get file size & first part of data  Two parallel HTTP flows Xunlei Client HTTP Server HTTP Request HTTP Request HTTP Request HTTP Request First connection is aborted Start two parallel connections Range: 1001-3000 Range: 3001- File size first part data Local file HTTP Response HTTP Response
27. Mirror Acceleration – Why Fact: HOT content is available at multiple servers (web sites)  Key Ideas  Multiple HTTP connections with different servers  Different ranges
28. Mirror Acceleration – How  Steps  Get file information from original server, e. g size
29. Mirror Acceleration – How  Steps  Get file information from original server, e. g size  HTTP Request (Post) to Xunlei server -- HTTP Request –- POST / HTTP/1.1 Host: 123.129.242.168:80 …message body(Encrypted )… -URL of file: http://softdl1.tech.qq.com/soft/21/QQ2010SP3.exe -File length -Xunlei version -MAC address -IP address
30. Mirror Acceleration – How  Steps  Get file information from original server, e. g size  HTTP Request (Post) to Xunlei server  HTTP Response from Xunlei server -- HTTP Response –- HTTP/1.1 200 OK …message body(Encrypted)… -List of URLs http://dl_dir.qq.com/qqfile/qq/QQ2010/QQ2010SP3.exe http://ydsoft1.greendown.cn:8880/201011/QQ2010SP3_1125.exe ………
31. Mirror Acceleration – How  Steps  Get file information from original server, e. g size  HTTP Request (Post) to Xunlei server  HTTP Response from Xunlei server  Download from mirror servers
32. Mirror Acceleration – Challenges  Content provider (owner of mirror servers) tries to ban direct linking  Server responds HTTP request: get only if it comes from the content providers’ page  Xunlei’s Solutions: provide both URLs and corresponding referrers direct linking : It usually occurs when a web page use of a linked object that belonging to a second site -- HTTP Request–- GET /soft/21/QQ2010SP3.exe HTTP/1.1 Referrer: http://softdl1.tech.qq.com/soft/21 -- HTTP Response –- HTTP/1.1 200 OK …message body(Encrypted)… http://dl_dir.qq.com/qqfile/qq/QQ2010/QQ20 10SP3.exe (referrer) http://ydsoft1.greendown.cn:8880/201011/QQ 2010SP3_1125.exe (referrer) ……
33. Mirror Acceleration – Challenges  How to collect URLs and Referrers of so many mirror servers?  Key Ideas: make use of large user base  Xunlei browser plug-in collects referrer (address of content providers’ page) -- HTTP Request –- POST / HTTP/1.1 Host: 123.129.242.169:80 …message body(Encrypted )… -URL of file: http://softdl1.tech.qq.com/soft/21/QQ2010S P3.exe -Referrer: http://softdl1.tech.qq.com/soft/21
34. Mirror Acceleration – Challenges  How to indentify whether these URLs refer to the same content?  Key Ideas: make use of large user base  Xunlei clients update summary reports to Xunlei server -- HTTP Request –- POST / HTTP/1.1 Host: 123.129.242.179:80 …message body(Encrypted)… -Partial hash -size -file name -url -referrer -etc. 20480 Bytes 20480 Bytes 20480 Bytes length/3 SHA-1 partial hash
35. Mirror Database Conjecture 1  conjecture1 (fake data)  Query mirror list Find mirrors that have the same partial hash Partial_Hash Mirror hash(QQ2010PS3.exe) Tencent.com/QQ2010PS3.exe hash(QQ2010PS3.exe) huanJun.com/QQ2010PS3.exe hash(QQ2010PS3.1.exe) Tencent.com/QQ2010PS3.1.exe hash(QQ2010PS3.1.exe) AOL.com/QQ2010PS3.1.exe
36. Test Conjecture 1  Experiment Setting:  URL http://softdl1.tech.qq.com/soft/21/QQ2010SP3.exe  Change local hosts file, and redirect softdl1.tech.qq.com to localhost  In folder /soft/21/ we put different test files each time
37. • Analysis of Results -Query mirror list twice: (1) by length and URL (2) by partial hash and length -There is a validation mechanism, chunk hash? Test Conjecture 1 Content File Name Length Result real content QQ2010SP3.exe 30.2MB get mirror list fake content QQ2010SP3.exe 15.0MB Fail to get mirror list real content QQ2010SP3_2.exe 30.2MB Fail at the first try, obtain mirror list after 10s fake content (change a few bytes) QQ2010SP3.exe 30.2MB (1)get mirror list, but abort download from mirrors later (2)get mirror list, continue download from mirrors until finish
38. Mirror Database Conjecture 2  Conjecture 2 Partial_Hash Mirror Partial_Hash Length Chunk Hash
39. Test Conjecture 2  Test 1: download CV.pdf at Yujie’s home page  First Query - by length and URL
40. Test Conjecture 2  Test 1: download CV.pdf at Yujie’s home page  Second Query - by partial hash and length
41. Test Conjecture 2  Test 1: download CV.pdf at Yujie’s home page  Update summery report -- HTTP Request –- POST / HTTP/1.1 Host: 123.129.242.179:80 …message body(Encrypted)… -Partial hash -Chunk hash -length -file name -url -referrer -etc. Partial_Hash Mirror Hash(CV.pdf) http://personal.ie.cuhk.edu.hk/~wyj009/home/file/CV.pdf Partial_Hash Length Chunk Hash Hash(CV.pdf) 34.7MB xxxx-xxxxx-xxxxx….
42. Test Conjecture 2  Test 2: download CV2.rar (same as CV.pdf) at Hao’s home page  First Query - by length and URL
43. Test Conjecture 2  Test 2: download CV2.rar (same as CV.pdf) at Hao’s home page  Second Query - by partial hash and length
44. • Composite primary key (Hash_ChunkHash, Length) • Index (Length, Partial _Hash) • Composite foreign key (Hash_ChunkHash, Length) • Index (Length, Mirror) Mirror Database Conjecture – Final Hash_ChunkHash Length Partial_Hash Chunk_Hash Hash_ChunkHash Length Mirror
45. Peer-to-Peer Acceleration  Comparison with Bit Torrent  Similarity  Xunlei server performs as a tracker  Xunlei server provides torrent file (name, size, chunk hash, etc.)  UDP between peers
46. Peer-to-Peer Acceleration  Comparison with Bit Torrent  Similarity  Xunlei server performs as a tracker  Xunlei server provides torrent file (name, size, chunk hash, etc.)  UDP between peers  Difference  Aware of peers in the same LAN (TCP)  Both push and pull  Optimized peer selection (no snubbing)
47. Summary of Xunlei P2SP Design  Content provider can be regarded seeder  HTTP, BT, eDonkey can be treated uniformly  Role of Xunlei Server  Indexing  Search (search by URL, Partial Content, Torrent File )  Coordination (more than a tracker)
48. Summary of Xunlei P2SP Design  Content provider can be regarded seeder  HTTP, BT, eDonkey can be treated uniformly  Role of Xunlei Server  Indexing  Search (search by URL, Partial Content, Torrent File )  Coordination (more than a tracker)  Trend of Xunlei  To be a seeder for paid users  Cross protocol sharing (we observe a BT task downloads though HTTP !)
49. Thank You