Milan Dvorak Director Netcope Technologies Achieving High Quality and Performance of FPGA-based Trade Solutions

Milan Dvorak Achieving High Performance and Quality of FPGA-based Trading Solutions [email protected]

• Company introduction • Testing & verification • Black box vs. White box approach • Unit tests • Functional models • Automated testing (Nightly builds) • Performance tests Overview

• European vendor of FPGA-based network solutions • Formerly FPGA department of Invea-Tech • First to introduce 100GE NIC (PCI-E form factor)  Vast experience with FPGA technology since 2002  Xilinx Alliance program partner  PCI-SIG® member company • Primary focus  Low-latency electronic trading  High-speed packet capture  Smart traffic filtering  FPGA firmware development kit Company introduction

• FPGA based low latency trading solution  Pre-built blocks for communication with an exchange  FIX/FAST and binary market data protocols  Full order book, aggregated level book  Order sessions management (FIX, ArcaDirect, OUCH, …)  User trading strategy in C/C++ (optional)  Latency optimized API Tradecope

• Requirements for HW solutions  Reliability  Time-to-market • Design vs. Verification (testing) = 30% vs. 70% • Black box approach  Setting inputs, checking outputs  Error gets propagated to outputs • White box approach  Mechanisms for checking internal state  Automatic check and reporting of defined properties Testing of HW systems

• Simplest white box approach in HW design • Statements, that are always true  Reading from empty FIFO  Multiple 1 bits in one-hot-encoding  “Things that won’t ever happen” • Asserts get triggered during simulations • Helps localizing the bug • Does not affect hardware performance  Assert is only checked in simulation  In HW – interrupts (report & log the problem) Assertions

• Testbenches for each unit • UVM/OVM methodology • SystemVerilog verification • Pseudo-random generator of inputs • Checking of outputs  Functional model in SystemVerilog  Object based programming, high-level description • Simulation can run for days  Basic test has 10,000 transactions • Monitoring of coverage  Did we cover all input combinations? Unit tests

• HW pipeline implemented in SW (C++)  Same functionality  Verification or SW simulation • Separate output for each stage • Same data capture passed to both HW and SW • Outputs compared for each stage • Over 20 Mbps testing speed Functional model

• How to test a change didn’t break anything?  Too many configurations to test (exchanges, protocols) • Automated testing using Cdash  Nightly builds, SW model, live exchange access • GIT branching model (master vs. devel) Automated testing

• Ensure performance parameters of the design  Low tick-to-trade latency  High throughput, zero packet loss • Online latency reporting  Internal latency of each order reported to SW  Online analysis of results, statistics over time • Black box measurements  Optical TAPs on input and output wire  Precise timestamps for market data & outgoing order  Independent latency measurement Performance testing

• Most of the time-to-market spent in verification  Automate the process as much as possible  Functional models for your cores • Simplify localizing and fixing bugs  Assertions and checks in the logic  Outputs from each stage • Don’t forget about performance!  Automatic latency measurements and reporting  Sustainable throughput without packet loss Conclusions

Netcope Technologies a.s. U Vodárny 2965/2 616 00 Brno, Czech Republic www.netcope.com Contacts Milan Dvorak [email protected] +420 511 205 378