Development of a data acquisition and analytics platform

A unique project to create a cloud-based solution consolidating historical data from different exchanges around the world with the ability to implement their algorithms by means of a scripting language

A financial corporation

Deadlines

2020 – 2021

Team

6 persons

Technologies

• Deephaven Platform
• AWS
• Java 11
• Reactor
• Spring Boot
• Spring Cloud
• Python
• Groovy

Goal

Creation of a unique consolidated storage of market data from 20 exchanges in Europe and America

The specifics of the developed product

• Cloud platform for storing, processing and analyzing historical market data, such as prices, tools, transactions. Provides access to data in a normalized form, has the ability to embed its own algorithms for calculation, modeling of trading strategies, identification of suspicious transactions, etc.
• The platform follows the philosophy of data storage with differentiation of access by exchanges, types of tools, sectors, etc.
• The platform is designed to become a source of high quality historical and within-the-day market data aggregated from external sources (exchanges) for the following purposes: technical analysis, quantum analysis, trading activity control, position management, regulatory reporting, etc.

Results

• A technical architecture for the product was developed to include the functionality to collect market data from external sources worldwide
• The ability to upload and test Python and Groovy-based strategies was implemented
• A module was created with the functionality of converting messages from sources into a readable format, data standardization, saving market data in time-series database, calculation and storage of additional entities, such as an order book and candlesticks.
• The solution was implemented completely, from the concept, through architecture, down to prototyping and product launch
• The solution, which consolidates data of 20 stock exchanges in Europe and America, was implemented and put into commercial operation