AAT-IP Suite for ITCH-OUCH Demo Instruction
4.2 Alveo Host System Initialization
4.4 Simple Moving Average Crossover Strategy
5.2 MCS File Programming via PCIe
1 Overview
The Accelerated Algorithmic Trading IP Suite (AAT-IP Suite) provides an ultra-low-latency FPGA-based platform for accelerating market-data processing and order execution. This demo uses the AAT-IP Suite to minimize protocol-processing and trading-decision latency while providing deterministic FPGA processing. It implements an ITCH/OUCH-based trading workflow and demonstrates end-to-end FPGA processing, including ITCH market-data reception, order-book maintenance, pricing-decision logic, and OUCH order transmission.
A key component of the AAT-IP Suite is the Pricing Engine, which is implemented using High-Level Synthesis (HLS). The trading strategy can be developed in C/C++ and synthesized into FPGA logic, allowing users to modify and evaluate custom strategies without redesigning the underlying RTL-based system architecture. This approach reduces development effort and enables rapid iteration from strategy development to FPGA implementation and evaluation.
This demo implements the AAT-IP Suite for the ASX (Australian Securities Exchange) Trade ITCH/OUCH specifications. To support for other market standards, please contact our sales team.
The demo runs on the AMD Alveo X3522PV accelerator card and demonstrates system operation using 10GbE connections. The X3522PV provides two DSFP28 network connections, each supporting up to two 10GbE ports.
In this demo, two 10GbE channels are used for UDP- and TCP-based communication with the following roles:
1) Market Data Channel (UDP): Receives sample ASX Trade ITCH market data transported using MoldUDP64 over UDP multicast.
2) Trading Control Channel (TCP):
· Receives the initial order-book snapshot using ITCH message formats over TCP.
· Transmits ASX Trade OUCH order messages over a TCP connection.
To set up the demo system, the host system containing the Alveo X3522PV card runs the ‘aat-itch-ouch’ application to control and operate the AAT-IP Suite demo. The other ends of the two 10GbE connections are connected to a target system (PC equipped with two 10GbE ports). The target system acts as the ASX market emulator by running the ‘asx_market_emu.py’ application. The ‘asx_market_emu.py’ application provides the order-book snapshot, replays ITCH market data to the Alveo card, and monitors OUCH orders received from the Alveo card.
The hardware and software environment used for the demo described in this document is listed below.
Alveo Host System
1) Supported Alveo accelerator card: X3522PV
2) Host system configuration:
· Operating System: Ubuntu Server 22.04 LTS
· DPDK: 20.11 LTS
· AMD QDMA DPDK Poll Mode Driver (PMD) for DPDK 20.11
https://xilinx.github.io/dma_ip_drivers/2020.2/QDMA/DPDK/html/index.html
· AMD Cable Driver (required for programming the Alveo X3522PV through the Alveo Debug Kit)
· Application: aat-itch-ouch
3) Vivado Design Suite installed on the host system to program the Alveo card
4) Programming cable: Alveo Debug Kit (ADK2)
5) 10GbE connectivity for X3522PV:
· Two 10GbE channels using 2 x SFP+ Active Optical Cables (AOCs):
https://www.10gtek.com/10gsfp+aoc
· Four 10GbE channels using 2 x DSFP-to-2xSFP28 breakout DAC cables:
https://ascentoptics.com/product/50g-dsfp-to-2x-25g-sfp28-breakout-dac-1m.html
Figure 1 AAT-IP Suite for ITCH-OUCH Demo Using Alveo X3522PV Card
Target System (PC as ASX Market Emulator)
1) Target system network interface: Two 10GbE ports provided by a 10GbE NIC
2) Target system configuration:
· Operating System: Ubuntu Server 22.04 LTS
· Market Emulator: asx_market_emu.py
· Market Data: Sample market data file (itch-market-data.pcap)
2 Target System Setup
This section provides step-by-step instructions for preparing the target system (PC), which is equipped with two 10G Ethernet ports, to exchange market data and order messages with the Alveo accelerator card. The target system runs Ubuntu Server 22.04 LTS. The setup involves identifying the correct network interfaces and assigning IP addresses to the two 10G Ethernet ports connected to the Alveo card.
First, identify the logical interface names of the two 10G Ethernet ports connected to SFP+#1 and SFP+#2. These logical names may vary depending on the system configuration. Therefore, ensure that the correct IP address is configured for the SFP+#1 and SFP+#2 connections.
1) Open a Linux terminal and run the following command to list the logical names of the 10G Ethernet ports:
>> sudo lshw -C network
Figure 2 Display Logical Name of 10G Ethernet Ports
The command output displays detailed information about each network interface. For example, Figure 2 shows logical interface names such as ‘enp6s0f0np0’ for SFP+#1 and ‘enp6s0f1np1’ for SFP+#2.
2) Configure the IP address for each Ethernet port using the ‘ifconfig’ command.
· Set SFP+#1 (enp6s0f0np0) to “192.168.10.100”.
· Set SFP+#2 (enp6s0f1np1) to “192.168.20.100”.
Configure the network mask as 255.255.255.0 (i.e., /24 subnet) for both interfaces, as shown in Figure 3.
Figure 3 Configure IP Address and Netmask
3) After assigning the IP addresses and netmask, use the “ifconfig” command to confirm that both Ethernet ports are correctly configured.
Figure 4 Verify IP Address and Netmask Setting
Ensure that both Ethernet ports are correctly assigned with their respective IP addresses and netmask values.
3 Alveo Host System Setup
This section outlines the steps for preparing the host system equipped with an Alveo accelerator card to run the AAT-IP Suite for ITCH-OUCH demo.
1) The AAT-IP Suite for ITCH-OUCH demo uses QDMA over PCIe for control and data transfer between the host system and the Alveo card. Therefore, DPDK 20.11 and the AMD QDMA DPDK Poll Mode Driver (PMD) must be installed on the host system. For the environment used in this demo, QDMA DPDK Driver 2020.2.1 is used with DPDK 20.11.
The installation guide is available on the AMD website under “Building QDMA DPDK Software”:
https://xilinx.github.io/dma_ip_drivers/2020.2/QDMA/DPDK/html/build.html
Note: Because the demo uses the ‘vfio-pci’ driver for PCIe device access, ensure that IOMMU is enabled on the host system before proceeding.
2) Connect the 10GbE cables between the Alveo card and the target system. Also connect the programming cable (Alveo Debug Kit - ADK2 for Alveo X3522PV) to the Alveo card and host system for FPGA programming.
i) Insert each SFP+ AOC connector into each of the two network connectors on the Alveo card.
ii) Connect Alveo SFP+#1 to the target-system Ethernet port configured with IP address ‘192.168.10.100’, and connect Alveo SFP+#2 to the target-system Ethernet port configured with IP address ‘192.168.20.100’.
iii) For Alveo X3522PV, connect the flex cable between the Alveo accelerator card and the Alveo Debug Kit (ADK2). Then, connect the ADK2 to the host system using the USB cable. Ensure that all connections are secure.
Figure 5 SFP+ AOC and ADK2 Flex Cable Connection on the Alveo X3522PV Card
3) Use Vivado Hardware Manager to program the Alveo card with the required bitstream, as illustrated in Figure 6.
Figure 6 Programming the Alveo Card Using Vivado
4) After programming the FPGA, perform a warm reboot of the host system and verify that the Alveo card is correctly enumerated by the PCIe subsystem using the ‘lspci’ command.
The console should display “Network controller: Xilinx Corporation Device 9048”, as shown in Figure 7.
Figure 7 Alveo PCIe Device Detected by the ‘lspci’ Command
5) Bind the Alveo PCIe function to the ‘vfio-pci’ driver so that the QDMA DPDK PMD can access the device.
i) Load the ‘vfio-pci’ kernel module.
>> sudo modprobe vfio-pci
ii) Bind the Alveo PCIe function to ‘vfio-pci’, replacing <BDF> with the PCIe BDF identified in step (4).
>> sudo dpdk-devbind.py -b vfio-pci <BDF>
For example,
>> sudo dpdk-devbind.py -b vfio-pci 01:00.0
iii) Verify the binding using the following command:
>> sudo dpdk-devbind.py -s
Confirm that the Alveo PCIe function is listed with: ‘drv=vfio-pci’.
Note: Further details about using ‘vfio-pci’ with the AMD QDMA DPDK driver are available at:
https://xilinx.github.io/dma_ip_drivers/2020.2/QDMA/DPDK/html/userguide.html
6) Boot the AAT-IP Suite for ITCH-OUCH demo by executing the ‘aat-itch-ouch’ application on the Alveo host system.
i) Navigate to the demo package directory (download):
>> cd <demo_package_directory>
ii) Execute the application:
>> sudo ./software/aat-itch-ouch
After execution, the application initializes the DPDK EAL, QDMA PMD, and AAT-IP Suite, as shown in Figure 8.
Figure 8 AAT-IP Suite Initialization Status on the Alveo Host System
4 Run AAT ITCH/OUCH Demo
In a price-driven trading environment, the trading workflow uses three communication paths: Glimpse, Live Feed, and Order Entry. Each path serves a distinct role in the overall trading process.
· The Glimpse session retrieves the initial order-book snapshot, allowing the AAT-IP Suite on the Alveo host system to establish the current market state before live trading begins.
· The Live Feed continuously delivers real-time ITCH market data over UDP multicast, allowing the Client to monitor market changes and trigger trading decisions.
· The Order Entry session is used to submit OUCH order messages (buy and sell orders) to the ASX Market Emulator based on trading decisions generated by the Pricing Engine (the trading logic) on the Client.
The demo establishes these communication paths in the sequence illustrated in Figure 9.
Figure 9 Trading Process Diagram
1) The Client first connects to the ASX Market Emulator through the Glimpse session to obtain the initial order-book snapshot.
2) Next, the Client establishes the Order Entry session with the ASX Market Emulator to prepare for OUCH order submission.
3) Trading begins when the Client starts receiving the Live Feed by joining the configured multicast group. As real-time ITCH market data is received, the trading logic evaluates the predefined trading conditions and automatically submits OUCH orders through the Order Entry session when the conditions are met.
The following sections describe how to configure the ASX Market Emulator using the provided test software, configure the AAT-IP Suite to communicate with the ASX Market Emulator through the required communication paths, and enable order submission. They also describe how to configure the simple trading algorithm implemented in the FPGA-based Pricing Engine.
4.1 ASX Market Emulator Setup
In this demo, the ‘asx_market_emu.py’ application is used to set up the Market Emulator on the target system. The application provides a GUI for serving the order-book snapshot, replaying market data to the Alveo card, and monitoring OUCH orders received from the Alveo card.
Before the Alveo host system connects to the ASX Market Emulator, the emulator must be started and ready to accept the Glimpse and Order Entry TCP connections. Follow the steps below to prepare and start the application.
1) Before running the ASX Market Emulator for the first time, install the required Python packages on the target system using the following commands:
>> cd <demo_package_directory>/marketEmu
>> pip3 install -r requirements.txt
This step is required only during the initial setup or when the package requirements are updated.
2) On the target system console, execute the following command to start the ASX Market Emulator:
>> python3 asx_market_emu.py -c config/config.json
The -c option specifies the configuration file. For this demo, config/config.json contains the market-data input files, GUI display settings, Glimpse and Order Entry TCP server parameters, multicast Live Feed settings, and replay configuration.
Figure 10 shows an example of the ASX Market Emulator GUI. The IP address and TCP port number assigned for the Glimpse and Order Entry services are displayed.
Figure 10 ASX Market Emulator GUI
3) On the Alveo host system, open the ‘aat-itch-ouch’ console and execute the following command to configure the network paths and establish the Glimpse and Order Entry TCP connections to the ASX Market Emulator:
>> run support/network.cfg
After executing the command, verify the connection status on the ASX Market Emulator. Both the Glimpse and Order Entry sections should indicate ‘Client: Connected’, as shown in Figure 11.
Figure 11 Glimpse and Order Entry Connection Status on the Market Emulator
The Alveo host system can also verify the TCP connection status from the ‘aat-itch-ouch’ console using the following command:
>> tcphandler get-status
Confirm that the Connection Status for both Glimpse and Order Entry is displayed as ‘CONNECTED’, as shown in Figure 12.
Figure 12 TCP Handler Connection Status
4.2 Alveo Host System Initialization
After the Glimpse and Order Entry TCP connections are successfully established, the ‘aat-itch-ouch’ application on the Alveo host system is used to configure and control the AAT-IP Suite to retrieve the initial order-book snapshot and perform the Order Entry login sequence, corresponding to steps (1) and (2) in Figure 9. After these initialization steps are completed, the system is ready to submit trading orders.
The following steps describe the Alveo host system initialization sequence.
1) On the ‘aat-itch-ouch’ console, execute the following command to configure the Glimpse login credentials, perform the Glimpse login, and retrieve the initial order-book snapshot from the ASX Market Emulator:
>> run support/snapshot.cfg
After the script is executed, the ASX Market Emulator indicates that the Glimpse login has been accepted and the snapshot has been transmitted, as shown in Figure 13.
Figure 13 ASX Market Emulator After Glimpse Snapshot Transmission
2) Verify that the Glimpse login has completed successfully and that the snapshot data has been received from the ASX Market Emulator.
i) First, execute the following command in the ‘aat-itch-ouch’ console to check the Glimpse login status:
>> glimpsehandler get-login-info
Confirm that Login Status shows LOGGED IN, indicating that the Glimpse login has completed successfully, as shown in Figure 14.
Figure 14 Glimpse Handler Status After Snapshot Completion
ii) Next, execute the following command to check the received snapshot packets:
>> glimpsehandler get-system-info
Confirm that ‘Num Sequenced Data Pkt’ shows the number of snapshot packets received and that ‘Num End Of Snapshot Msg’ is incremented, indicating that the snapshot transmission has completed. In the example shown in Figure 15, 14 snapshot packets are received.
Figure 15 Number of Snapshot Packets
3) On the ‘aat-itch-ouch’ console, execute the following script to configure the FPGA-based Pricing Engine:
>> run support/trade_on_card.cfg
The ‘trade_on_card.cfg’ script configures the Pricing Engine implemented on the Alveo card. The Pricing Engine processes received market data and generates trading order requests directly using FPGA logic, minimizing the processing latency from market data reception to order generation.
The Pricing Engine implements the Simple Moving Average (SMA) Crossover trading strategy described in Section 4.4.
The script also configures the Order Entry login parameters and initiates the Order Entry login with the ASX Market Emulator.
4) After the Order Entry login is completed, verify the login status on the ASX Market Emulator, as shown in Figure 16.
Figure 16 ASX Market Emulator After Order Entry Login
The Order Entry login status can also be verified from the ‘aat-itch-ouch’ console on the Alveo host system using the following command:
>> orderentry get-login-info
Figure 17 Order Entry Login Status
4.3 Market Data Transmission
This section describes how to start the ITCH market-data replay and monitor the test results using the ASX Market Emulator. During the test, the Replay Progress, Trade Ticker with order plot, ITCH trade details, OUCH order details, and final order-book state can be monitored.
Click ‘Start’ on the ASX Market Emulator to begin replaying market data to the Alveo card. The Replay Progress bar shows the number of packets transmitted, as shown in Figure 18.
Figure 18 Start Market-Data Replay
During market-data replay, the ‘Trade Ticker’ updates continuously. OUCH orders received from the Alveo card are plotted on the same chart. A ‘green triangle’ represents a Buy order, while a ‘red triangle’ represents a Sell order, as shown in Figure 19.
Figure 19 Trade Ticker and Order Plot
The ‘ITCH Trades’ tab displays the details of the ITCH trade messages observed during market-data replay, as shown in Figure 20.
Figure 20 ITCH Trade Details
The ‘OUCH Orders’ tab displays the details of OUCH orders received from the Alveo card as the market-data replay proceeds, as shown in Figure 21.
Figure 21 OUCH Orders Received from the Alveo Card
After the market-data replay is complete, the ‘Order Book’ tab displays the final order-book state maintained by the ASX Market Emulator, as shown in Figure 22.
Figure 22 Final Order-Book State on the ASX Market Emulator
After the market data replay is complete, execute the following command in the ‘aat-itch-ouch’ console to check the Order Entry transmission status
>> orderentry get-system-info
Check the ‘Num Tx Meta’ value to confirm the number of order messages transmitted by the Order Entry module. In the example shown in Figure 23, 31 order messages have been transmitted.
Figure 23 Order Entry Transmission Status
4.4 Simple Moving Average Crossover Strategy
In this demo, the FPGA-based Pricing Engine implements a ‘Simple Moving Average (SMA) Crossover’ trading strategy. The strategy monitors valid trade prices received from the ITCH market data and calculates two SMAs:
· Fast SMA (SMA(5)): Average of the five most recent valid trade prices.
· Slow SMA (SMA(20)): Average of the twenty most recent valid trade prices.
The FPGA-based Pricing Engine continuously calculates the Fast SMA and Slow SMA from the trade prices decoded from the market data and compares the two SMAs. An order is generated only when their relationship changes, indicating that a crossover has occurred.
Both SMAs becomes valid after 20 valid trade prices have been received. After both SMAs become valid, the first non-equal relationship between the Fast SMA and Slow SMA is stored as the initial relationship. No order is generated while establishing this initial relationship. Subsequent valid trade prices are processed to detect a crossover.
The order-trigger conditions are as follows:
· Bullish crossover: The Fast SMA changes from below the Slow SMA to above the Slow SMA. This indicates that recent trade prices are increasing relative to the longer-term price trend. In this demo, the crossover is interpreted as an upward market signal, and the FPGA-based Pricing Engine generates a ‘BUY’ order request.
· Bearish crossover: The Fast SMA changes from above the Slow SMA to below the Slow SMA. This indicates that the recent trade prices are decreasing relative to the longer-term price trend. In this demo, the crossover is interpreted as a downward market signal, and the FPGA-based Pricing Engine generates a ‘SELL’ order request.
For each generated order, the order quantity is fixed at 1000, and the order price is set to the trade price that causes the crossover. Therefore, when a valid crossover is detected, the current trade price is used as the order price.
If the Fast SMA and Slow SMA are equal, no order is generated. The strategy retains the previous relationship between the two moving averages and waits until a valid crossover occurs.
Figure 24 shows the Trade Ticker together with the Fast SMA (labelled ‘SMA(5)’), the Slow SMA (labelled ‘SMA(20)’), and the orders generated by the FPGA-based Pricing Engine. There are four crossover points that trigger trading events, two bullish crossovers and two bearish crossovers.
Figure 24 Simple Moving Average Crossover Strategy
As shown in Figure 24, the orange SMA(5) line crosses the purple SMA(20) line from below to above at event (1) and (3). These bullish crossovers trigger ‘BUY’ order request using the current trade price, indicated by the blue data point, as the order price. At event (2) and (4), the SMA(5) line crosses the SMA(20) line from above to below. These bearish crossovers trigger ‘SELL’ order requests.
5 Update Hardware via PCIe
In certain deployment environments, it is useful to update the FPGA configuration without connecting a physical programming cable. The AAT-IP Suite for ITCH-OUCH includes a dedicated flash programming module that enables the FPGA configuration image stored in the Alveo X3522PV QSPI flash memory to be updated through PCIe.
This section describes the two main steps involved in this process: MCS file creation and MCS file programming via PCIe.
The PCIe update procedure uses the AMD ‘xbflash2’ utility. ‘xbflash2’ accesses the flash programming controller in the FPGA design through the PCIe BAR; therefore, the currently running AAT-IP Suite FPGA image must include the required flash-programming hardware. The ‘xbflash2’ utility can be downloaded from the following link:
https://www.amd.com/en/support/downloads/alveo-downloads.html/accelerators/alveo/u250.html
5.1 MCS File Creation
Figure 25 MCS File Creation
1) Initialize the Vivado environment and start Vivado in Tcl mode:
>> source <Vivado_install_path>/2022.1/settings64.sh
>> vivado -mode tcl
2) Execute the following command to generate the MCS image from the FPGA bitstream:
>> write_cfgmem -force -format mcs -interface spix4 -size 256 -loadbit "up 0x01002000 <input_file.bit>" -file "output_file.mcs"
Note: For Alveo X3522PC, use configuration interface - ‘spix4’, configuration memory size – ‘256’, and user configuration region offset – ‘0x01002000’.
3) Upon successful completion, the Vivado console displays “write_cfgmem completed successfully”, confirming that the MCS file has been created successfully.
5.2 MCS File Programming via PCIe
1) Before programming the MCS image via PCIe, verify that the Alveo PCIe function is not bound to the ‘vfio-pci’ driver. Use the following command to check the current device binding status:
>> sudo dpdk-devbind.py -s
Figure 26 PCIe Device Binding Status
2) Locate the Alveo PCIe function using its BDF and verify its current driver-binding status.
Figure 27 Alveo PCIe Function Binding Status
· If the Alveo PCIe function is already unbound, it is displayed without a ‘drv=’ entry, as shown on the left side of Figure 27.
The ‘unused=vfio-pci’ field indicates that ‘vfio-pci’ is an available compatible driver, but the PCIe function is not currently bound to it.
· If the Alveo PCIe function is bound to ‘vfio-pci’, it is displayed with ‘drv=vfio-pci’, as shown on the right side of Figure 27.
In this case, unbind the PCIe function using the following command:
>> sudo dpdk-devbind.py -u <BDF>
After unbinding the function, return to Step (1) and confirm that the ‘drv=vfio-pci’ entry is no longer displayed.
Figure 28 ‘xbflash2’ Programming
3) Execute the ‘xbflash2’ utility with root permissions to program the MCS image to the flash memory via PCIe using the following command:
>> sudo xbflash2 program --spi --image <target_mcs_file>.mcs --bar 2 --bar-offset 0x80000 -d <BDF>
Note: The ‘--bar 2’ and ‘--bar-offset 0x80000’ parameters identify the PCIe BAR and base address of the QSPI flash controller implemented in the currently running default AAT-IP Suite for ITCH-OUCH demo design. If a different FPGA design is currently loaded on the card, these values must be changed to match the PCIe mapping of its flash controller.
4) When prompted, enter ‘Y’ to confirm the programming operation.
5) Wait until the following message is displayed, indicating that flash programming has completed: “Cold reboot machine to load the new image on device.”
6) Perform a cold reboot of the Alveo host system by fully powering off the system and then powering it on again. After power-up, the Alveo card loads the newly programmed image from the flash memory.
6 Revision History
|
Revision |
Date (D-M-Y) |
Description |
|
2.00 |
18-Sep-26 |
Updated Test Application |
|
1.00 |
26-Jan-26 |
Initial version release |