ExPo2: High Speed Bot Interfaces

A presentation given that summarizes the augmentations made to a stock exchange simulator as part of my final year project.

Transcript

1. 2: High Speed Bot Interfaces …in other words, enabling programs to talk to each other in the most direct way possible, in their native language Bryan Kok Spring 2017
2. Pocketing the profit: Arbitrage Buyer Exchange Willing to pay $100 New sell order for $50 Market Maker 1ms latency 10ms latency 1) MM receives the update before the buyer 2) MM buys the stock for $50 and immediately enters a sell order for $100; happens with very high frequency! 3) Buyer buys at $100, MM pockets $50 profit. (Automated trading program, also referred to as a bot)
3. Exchange Portal 2’s messaging system CommServer SimpleExchange OrderBook WebSocket Endpoint AbstractBaseBot BotClient OrderManagementSystem Human clients via webpage Assigner (Experiment Coordinator) Before…
4. Exchange Portal 2’s messaging system CommServer SimpleExchange OrderBook WebSocket Transport AbstractBaseBot BotClient OrderManagementSystem Human clients via webpage Assigner (Experiment Coordinator) ØMQ Transport BotClientAPI After… BotClient ZMQ or exchange_zmq.c Bots in C
5. Where do I start? 1. A benchmark is essentially an experiment • Any original systems are used as controls 2. Generate hypotheses based on observation 3. Implement hypotheses 4. Go to 2 5. Generalize “We should forget about small efficiencies, say about 97% of the time: Premature optimization is the root of all evil, yet we should not pass up our opportunities in that critical 3%.” (Knuth, 1974)
6. Exchange Portal 2’s messaging system CommServer SimpleExchange OrderBook WebSocket Endpoint AbstractBaseBot BotClient OrderManagementSystem Human clients via webpage Assigner (Experiment Coordinator) ØMQ Endpoint BotClientAPI After… BotClient ZMQ or exchange_zmq.c Bots in C
7. Making Observations: Inside the Bot (AbstractBaseBot) Thread.sleep(500) OrderManagementSystem Thread which receives assignments from the Assigner Exchange communication thread Thread which accesses the OMS, executes some trading logic, and decides whether to send a response or not .interrupt() .interrupt() .sendNewMessage()
8. CommServer “A 67 231870732 B L MSFT 10 296 720000” 110101010101…. Between the Bot and the Exchange Type: LimitShoutMessage shoutMessageFields { assignID: 67 shoutID: 231870732 isBuy: false stockName: "MSFT" Size savings up to 40% with very large order book updates (artificially constructed up to 20 levels of depth), while nice on paper won’t be very significant with the message sizes encountered
9. Source: https://martin.kleppmann.com/2012/12/05/schem a-evolution-in-avro-protocol-buffers-thrift.html
10. Inside the Exchange’s CommServer SimpleExchange WS CommServer isBuy : True stockName : “MSFT” quantity : 20 price : 100 clientID : 1 shoutID: 1234 ØMQ Queue broadcast decoding Type: OwnShoutFulfilled shoutID: 1234 price: 100 buyBook: [45 10 40 5…] sellBook: [50 10 75 5…]
11. 0 1000 2000 3000 4000 5000 6000 7000 8000 0 100 200 300 400 500 600 700 800 1GbE Raw TCP 1GbE 0MQ TCP 1GbE WS TCP bytes sent us One way latency between two Linux machines
12. Profiling Two main approaches Instrumentation: Inserts instructions in order to obtain fine-grained timing information; disadvantage: will skew methods which are very quick Sampling: Collects information at fixed intervals And now for two concrete improvements made.. https://visualvm.java.net/
13. Yes, HashMaps are constant time in theory, but constant time cannot be helped if we are spending 13ms per invocation
15. Incoming messages. In ØMQ and raw sockets a basic way to know if a message is ready is to call poll() in a loop. OwnShoutUnfulfilledMessage (Order still on the books) callback OwnShoutMessage (Trade Executed) callback Thread ... other callbacks… What is the problem?
16. We’re broadcasting to every single client, and in the process unnecessarily serializing calling getUtf8Bytes() on the string to be sent over and over again?
17. Microbenchmarking JMH Benchmark Mode Cnt Score Error Units ExPo2Benchmarks.serializing.ProtobufBenchmark.decodeO wnFulfilledMessage ,! avgt 25 2010.144 ± 216.946 ns/op ExPo2Benchmarks.serializing.StringBenchmark.decodeOwn FulfilledMessage ,! avgt 25 5128.016 ± 567.425 ns/op ExPo2Benchmarks.exchange.OrderBookBenchmarks.addOrder avgt 100 348069.133 ± 19039.716 ns/op
18. Quantifying performance Coarse-grained benchmarking Client 1 shoutID : 2345 isBuy : True … Unique ID System.nano time() isSend 1234 1493800886 881057 True -2146933580 … False … … … CommServer Unique ID System.nano time() isSend cID 1234 1493800889 999999 False 1 … … … .. -2146933580 … True 1 buyBook: […] sellBook: […] isBuy: True …
19. WebSocket Endpoint BotClient 0 0.2 0.4 0.6 0.8 1 1.2 Bot To Server Inside the Exchange Server To Bot Packets in transit Breakdown of Communication Time Building Shout Message String Blocking sendString() Decoding Shout Message Order Matching Return sendString Decode received order book On the wire Total: ~2.4ms 4x Before…
20. ØMQ Endpoint BotClientZMQ 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Bot To Server Inside the Exchange Server To Bot Packets in transit Breakdown of Communication Time Encoding Shout Message Nonblocking ZMQ.Socket.send() Decoding Shout Message Order Matching Encoding protobuf return message from POJO Nonblocking return send Decoding Orderbook On the wire Total: ~1.594ms 4x After…
21. Talk is cheap, show me the code! Very quick demo: Write and run a ØMQ bot in C which attempts to make $1 profit given the assignment Core deliverables of this project: • ØMQ library in Java • ØMQ library in C • Library which allows bots to be run inside the CommServer using reflection • Sample bots (here’s one in front of you now) • Benchmarking stuff
22. Live benchmarking stuff…? Soak test Test some aspect of a system over a certain amount of time, in this case: Throughput Number of requests fulfilled per second.
23. Key takeaways of today: • Measure before going headlong into optimization. • Measuring at different granularity levels can be helpful, lots of tools at our disposal. • Measure after finished with fixing a code hotspot. • Programming to interfaces instead of implementations makes things maintainable and easy to use.
24. Key takeaways of today: • Measure before going headlong into optimization. • Measuring at different granularity levels can be helpful, lots of tools at our disposal. • Measure after finished with fixing a code hotspot. • Programming to interfaces instead of implementations makes things maintainable and easy to use. Question Time
25. Setting the scene… If information is valuable and information takes a certain amount of time to travel, then time is valuable too.
26. Assigner Exchange CommServer WS øMQ API Human Browser Client Instantiated bot inside the server Modified Bots Generic Transport, TransportSession, Client Protobuf- encoded String Messages WS/øMQ POJO Message New Comms Library øMQ Handler AssignerWebSocketV1Handler Existing Bots SimpleExchange.java Assigner.java WS Handler Bot Upload System