Market Data Processing Session

Previously, the Handler offered a predefined set of market data processing patterns. The given processing patterns covered the most frequently used cases of market data processing like late join with an accurate order book maintenance or order book natural refresh using only real-time feeds.

However, if the users wanted to have more customization for a processing session like using a TCP Recovery facility in case of a natural refresh, there was no solution for the case.

Starting with version 4, the SDK offers improved and enhanced market data processing session. Now the Handler allows to use a TCP recovery while books are naturally refreshed or to combine join-time recovery with natural refresh and so forth.

Defining Processing Behavior

A set of parameters, gathered into the the OnixS::CME::MDH::SessionSettings class, now defines behavior of a processing session. The OnixS::CME::MDH::Handler::start member requires an instance of the OnixS::CME::MDH::SessionSettings as a parameter.

Settings allow defining whether market data recovery must be done at the beginning of market data processing or in case of data loss or other processing error, and so on. Also, settings allow defining which type of market (by price or by order or both) to recover.

The table below describes primary parameters affecting market data processing:

Parameter	Description	Behavior Visualization
OnixS::CME::MDH::SessionSettings::joinRecovery	Defines whether the Handler must recover either instruments or the entire market state, or nothing when it joins a trading session. For a complete list of available options see OnixS::CME::MDH::JoinRecoveryOptions::Enum items. Note Recovery at join-time is a single-time procedure executed by the Handler. Performing recovery, for example, due to book update failure or data loss at a later time is defined by other parameters. See OnixS::CME::MDH::SessionSettings::gapAndErrorHandling for more information.
OnixS::CME::MDH::SessionSettings::gapAndErrorHandling	When real-time market data is processed, a failure may take place due to errors in transmitted data or due to data loss. The Handler allows controlling its behavior in such cases. With the help of the given parameter, it's possible to specify whether processing real-time market data should continue or the Handler must recover market state to have accurate order books. Also, the Handler may recover all instruments in addition to the market state to have an accurate list of traded securities. See the OnixS::CME::MDH::GapAndErrorHandlingOptions::Enum for a complete list of available options. Note The given parameter does not affect join-time behavior and thus if recovery is desired at join time, the OnixS::CME::MDH::SessionSettings::joinRecovery member must be used to set a correspondent value.
OnixS::CME::MDH::SessionSettings::marketRecovery	The given parameter controls aspects of the Handler behavior in case of market recovery. If processing session has to perform market state recovery due to either late join or processing failure, the given parameter allows defining a type of market to recover. Starting with Q4 of 2016, CME MDP offers market by order (MBO) data in addition to market by price (MBP) one. Although the data for both MBO and MBP is combined on the real-time feed, recovery is separated for each type of market. As a result, the Handler has to be configured with the types of markets which must be recovered. For a complete list of options, see OnixS::CME::MDH::MarketRecoveryOptions::Enum. Note Value of the given parameter must be adjusted with order books maintaining settings. For example, if a processing session is configured to recover market at the join time or due to data loss and recovery must be done for market by price only, then the accurate maintenance of MBO books is not possible. Therefore, the if the Handler is configured to maintain MBO books in such case, an exception is thrown.

Configuring Handler For Accurate Book Maintenance While Joining Lately

The following code snippet depicts how to configure market data processing session to maintain order books accurately starting from the middle of a trading session (a.k.a. late join).

// Processing session settings. 
SessionSettings settings;

// In case of late join, instruments and market state must be 
// recovered before switching to process real-time (incremental) data.
settings.joinRecovery(JoinRecoveryOptions::InstrumentsAndMarketState);

// Accurate order book maintenance assumes performing market recovery 
// each time data is lost or other processing failure occurs. 
settings.gapAndErrorHandling(GapAndErrorHandlingOptions::RecoverMarketState);

// Now, spawns live market data processing. 
handler.start(settings);

Configuring Handler for Accurate MBP Book Maintenance while Joining in Preopening

The following code snippet depicts how to configure a market data processing session to maintain MBP books accurately starting from the beginning of a trading session when market is at a blank state, and no initial recovery is needed to catch-up an up-to-date market state.

// Processing session settings. 
SessionSettings settings;

// In case of preopening, instruments are transmitted via real-time feed 
// and market state is blank. Therefore, join recovery is not needed at all.
settings.joinRecovery(JoinRecoveryOptions::InstrumentsAndMarketState);

// Accurate order book maintenance assumes performing market recovery 
// each time data is lost or other processing failure occurs. 
settings.gapAndErrorHandling(GapAndErrorHandlingOptions::RecoverMarketState);

// Limits market recovery to MBP only.
settings.marketRecovery(MarketRecoveryOptions::MbpOnly);

// Now, spawns live market data processing. 
handler.start(settings);

Configuring Handler to Refresh Books Naturally with Initial Recovery

The following code snippet depicts how to configure market data processing to maintain order books in the natural refresh mode and to perform a full state recovery at a join stage. The given configuration is frequently used in cases, when a server doing market data processing is collocated in the CME network and a possibility of data loss is minimal. It gives almost same benefits as an accurate order book maintenance, but in case of data loss, it does not let real-time processing to freeze due to necessity in performing market recovery.

// Processing session settings. 
SessionSettings settings;

// This forces Handler to perform instrument and market recovery at join time.
settings.joinRecovery(JoinRecoveryOptions::InstrumentsAndMarketState);

// Natural refresh assumes continuation of real-time data 
// processing even if some of transmitted data is lost. 
settings.gapAndErrorHandling(GapAndErrorHandlingOptions::ContinueProcessing);

// Now, spawns live market data processing. 
handler.start(settings);

Configuring Handler To Refresh Books Naturally But Use TCP Recovery In Case Of Data Losses

The following code snippet helps to configure market data processing to maintain order books in natural refresh mode. However, in contrast to a regular case, when real-time processing is resumed immediately, in the given case the Handler tries to recover missing data using TCP Recovery Feed.

// Processing session settings.
SessionSettings settings;

// Natural refresh assumes continuation of real-time data 
// processing even if some of transmitted data is lost. 
settings.gapAndErrorHandling(GapAndErrorHandlingOptions::ContinueProcessing);

// Settings for TCP recovery facility.
TcpRecoverySetting tcpRecoverySettings;

// Credentials are to be set as there's no defaults for
// these parameters. Other parameters have default values.
tcpRecoverySettings.username("CME");
tcpRecoverySettings.password("CME");

// Instance of TCP recovery service to be bound to the Handler.
TcpRecoveryService tcpRecovery(tcpRecoverySettings);
handler.bindTcpRecovery(tcpRecovery);

// Now, spawns live market data processing. 
handler.start(settings);

See also

OnixS::CME::MDH::AccurateLateJoinSession, OnixS::CME::MDH::AccurateLateJoinMboSession, OnixS::CME::MDH::AccurateLateJoinMbpSession, OnixS::CME::MDH::AccurateLateJoinSession, OnixS::CME::MDH::AccuratePreopeningSession, OnixS::CME::MDH::AccuratePreopeningMboSession, OnixS::CME::MDH::AccuratePreopeningMbpSession, OnixS::CME::MDH::NaturalRefreshSession.