Filtered, Consolidated, Cryptocurrency Best Bid and Offer (FCCBBO) data feed and historical data server

Abstract

The FCCBBO provides investors, speculators and traders in cryptocurrency products with a unique, customized data feed that solves for several problems unique to the cryptocurrency markets. The ability to filter based on order size allows for efficient scanning of available order books as well as to understand the recent historical trading patterns. This, along with the unique ability to select which exchanges should be included based on the unique account status of software users at each exchange allows for a much more contextual view of the data. While other financial markets have examples of consolidated data feeds, none of them allow for multiple size filters and no commercial feeds support customizable exchange selection based on user account status. Those features are uniquely important in the cryptocurrency markets, which is what is the driver behind this invention.

Description

BACKGROUND

The Crypto-Currency¹ market is served by multiple exchanges, many of which are inaccessible to investors in various jurisdictions or, at a particular moment invalid for a client to utilize due to lack of funding or available cryptocurrencies in the appropriate wallet. Each exchange distributes their order books to their clients by a combination of user interface (UI), direct data feed or data request mechanism via an application program interface (API). The individual order books of the exchanges can include orders for extremely small size at price levels due to nature of Cryptocurrencies, which allows bids or offers for as little as 0.00000001 of a product. As a result, it is extremely difficult for investors to determine the prevailing price for products they are looking to trade, at the order sizes they are interested in, particularly when they have valid accounts and the ability to trade on only one or more, but not necessarily all, exchanges. ¹Crypto-Currency is defined herein as BitCoin, Ethereum, LiteCoin or alternative coins that are based on blockchain technology.

BRIEF SUMMARY OF THE INVENTION

The invention is a consolidated data feed and database for cryptocurrencies that processes all quotes for each product pair covered, but allows clients to access data filtered for multiple, selectable order sizes as well as for a customized subset of exchanges that specific clients are eligible to trade on at a particular point in time for the product being queried. The invention includes a user interface which shows the consolidated best bid and offer filtered by multiple sizes over time, both in real time and for selectable historical periods as well as an application program interface (API) to provide the same information to clients systems.

FIG. 1 shows the system components and flowchart for the acquisition of the order book data, calculation of the raw and filtered book, and storage of data in order to be accessible by user interfaces and client applications.

FIG. 2 illustrates the User Interface for displaying the CBBO for Bitcoin/USD in realtime, on an unfiltered basis. Note that the invention covers a wide range of products, including all tradeable cryptocurrencies. In this example, the market is considered “crossed” as the offer on the GDAX exchange is below the best bid on the Gemini exchange.

FIG. 3 illustrates the User Interface for displaying the CBBO for Bitcoin/USD in realtime, filtered for orders of 20 Bitcoin or more. Note that, despite being captured at virtually the same time as the market in FIG. 2, the market does not appear as “crossed” due to the fact that when each exchange was filtered for orders over 20, the spreads were wider.

FIG. 4 illustrates the User Interface for displaying the Historical CBBO for Bitcoin/USD for a 1 day period, on an unfiltered basis. Notice that the market is considered “crossed” and that the software categorizes the magnitude of the crossed quotes as well as the % of time each of the three exchanges, in this example, are at the best bid and best offer.

FIG. 5 illustrates the User Interface for displaying the Historical CBBO for Bitcoin/USD for a 1 day period, filtered for orders of 20 Bitcoin or more. Notice that the filtered CBBO shows that the market is now “normal” 46% of the time and only “crossed” 54% of the time.

FIG. 6 illustrates the User Interface for displaying the Historical CBBO for Litecoin/USD for a 1 day period, on an unfiltered basis. Notice that the market is considered “crossed,” except for 16.3% of the time and that the software categorizes the magnitude of the crossed quotes as well as the % of time each of the two exchanges, in this example, are at the best bid and best offer.

FIG. 7 illustrates the User Interface for displaying the Historical CBBO for Litecoin/USD for a 1 day period, filtered for orders of 20 Litecoin or more. Notice that the filtered CBBO shows that the market is now “normal” 29.8% of the time.

FIG. 8 illustrates the User Interface for displaying the Historical CBBO for Ethereum/USD for a 1 day period, on an unfiltered basis. Notice that the market is considered “crossed,” except for 0.61% of the time and that the software categorizes the magnitude of the crossed quotes as well as the % of time each of the three exchanges, in this example, are at the best bid and best offer.

FIG. 9 illustrates the User Interface for displaying the Historical CBBO for Ethereum/USD for a 1 day period, filtered for orders of 20 Ethereum or more. Notice that the filtered CBBO shows that the market is now “normal” 12% of the time.

DETAILED DESCRIPTION AND BEST MODE OF IMPLEMENTATION

The system is built upon a few basic design goals: First, each exchange can have different APIs or methods of acquiring their book data, but the FCCBBO system will normalize all data into one format after acquisition. Therefore, whether an exchange supports an API that requires users to request quotes or the exchange has an interface that publishes all quote updates via an API in a continuous stream, the FCCBBO system will incorporate the data. Second, all of the supported filter levels will be applied to processed quote data from each exchange as it is received and will be stored in real time and historically for every supported exchange. Third, each of our clients will have their own configuration that determines which exchanges will be included in their version of the Filtered Cryptocurrency Consolidated Best Bid and Offer, which necessitates a separate layer for determining which quotes to serve via the UI or API. In order to satisfy these goals, the system has the following components: Individual feed handlers with local in-memory order-book storage, a book size and BBO processing component in the feed handler layer to create the filtered book data, a raw storage solution for the data acquired, a CBBO aggregation layer that calculates both the filtered and unfiltered best bid and offer incorporating each change of size or BBO in the individual feeds being acquired, a network-accessible, in-memory datastore to store book size and BBO metrics used by the CBBO aggregator in real time, a Real Time messaging layer to communicate between the processes, a historical database and both a UI and API which serves the data to clients.

Feed Handlers: The feed handlers receive events and can also make requests to exchanges in the case that no feed-based API is available. Initially and over the course of periodic intervals, book data will be read in snapshots from exchanges. When an exchange does not have streaming support, only full book refresh will be supported. In this mode, the exchange will be polled at periodic intervals. Book size and BBO and CBBO metrics will be calculated in the same manner as with feed-based APIs, however instead of the feed handler maintaining a real-time order-book, the in-memory orderbook is simply refreshed at intervals.

Redundancy: Multiple feed handlers will be subscribed to a single exchange/product feed at the same time for redundancy purposes. This is because in order for a real-time order book to be maintained, messages must be received roughly in order and without dropped messages.

Recovery: If messages become too disordered due to packet loss or exchange server issues, then the feed handlers order book must be discarded, and the feed handler restarted. This duration would cause inaccuracies in the CBBO and routing logic without multiple feed handlers and order books per exchange.

Update processing: When order book updates are received by the feed handler, the local in-memory orderbook is updated. Book size and BBO metrics are then calculated on the updated order-book, and written to an instance of the network accessible in-memory datastore. If any of the order book or BBO metrics have changed, a book change event is sent to the messaging layer.

Network Accessible, In-Memory Datastore: The network accessible in-memory data structure stores book size and BBO metrics for a given exchange and product. Multiple products and/or exchanges can be stored within the same instance, and metadata is stored as to what instances of the network accessible in-memory datastore has the data for a given product and exchange. Each process of the network-accessible in-memory Datastore can store book size and BBO metrics for one or more products on one or more exchanges. A metadata component runs which maps a given product for a given exchange to a set of datastore instances that store the associated book size and BBO metrics. Each feed handler for a product/exchange combination stores data in one or more datastore instances. This allows the scaling out of the system by adding additional instances of the datastore.

CBBO Aggregator: A CBBO Aggregator process runs for each product, which listens for book update messages and compiles a composite order book feed based on each exchange's book size and BBO metrics. For a given product, it fetches the book size and BBO metrics from all network-accessible in-memory datastores that maintain the order book of that product for every exchange. It maintains a cache of the metadata that maps products and exchanges to instances of the in-memory datastore. The CBBO aggregator will include the most up-to-date copy of the order book for each (Exchange, Product) in the CBBO based on last updated timestamp. If a book size or BBO metric changes after a book update, the feed handler will send a message to the distributed messaging layer that is listened to by the CBBO Aggregator. On receipt of these messages, the CBBO Aggregator will calculate the full CBBO for each size threshold by aggregating the book size and BBO metrics for all applicable exchanges from the Network Accessible in-memory Datastore. The CBBO values for each filter level are written to historical storage, as well as sent to the messaging layer for consumption by the UI and API.

Modes of operation: The CBBO supports multiple modes of operation. Tick locked mode executes the CBBO calculation inside the feed handler on a tick-by-tick basis. High-throughput mode instead sends a message to the messaging layer on every book update at which point a process listening on that messaging channels calculates the updated CBBO.

Real Time Data Store Architecture: As the system supports multiple product pairs, the architecture is such that each product pair occupies a messaging channel flow to notify the CBBO aggregator of book updates, as well as a set of datastore instances to store book size and BBO metrics for aggregation. Partitioning on the book size and BBO metrics is on the (Exchange, Product) level, meaning that if a product pair is traded on 7 exchanges, that product pair can have up to 7 datastore instances associated with it. This means the architecture scales to many products and exchanges. The implementation of the Real Time Data Store utilizes an in-memory data structure store, used as a database, cache and message broker, that has built-in replication.

Messaging Layer: The system uses standard publish/subscribe inter-process communications.

API/UI functionality: The system supports user configurable exchange selections and filters. It acquires these settings before initiating a query or real-time session and passes them to the API. The API then passes them to the historical or real-time data store to acquire the data requested as such data is stored with those selections in the keys. This allows the system to build the query on the fly, rather than having to calculate the results upon request.

Other considerations: The system is built to provide the CBBO and depth of book for each exchange as a central service leveraging standard cloud based architecture. As this service is designed to work directly with a Distributed Cryptocurrency Smart Order Router (DCCSOR), it can be accessed over peering connections to client instances of the DCCSOR.

Claims

1. A method for processing quotes for product pairs, the method comprising:

providing access to data filtered for a plurality of selectable order sizes; and

generating, for display, a user interface that includes a consolidated best bid and offer that is filtered by a plurality of order sizes over time.