1 \addtogroup feed-engine
3 \brief Uncovers Feed Engine concept and its internals.
5 The Feed Engine concept encapsulates network-related aspects to let the
6 Handler focus on market data processing. It also provides advanced facilities
7 on utilizing system resources used by the Handler to achieve best performance
8 and latency characteristics.
10 Previously, there were no shared services between the Handler instances.
11 Thus, system resources were allocated exactly for and by a particular
12 instance. That caused certain inefficiency regarding resources utilization.
13 For example, each Handler instance used from one to two threads to manage
14 incoming market data processing. As a result, the number of threads used by
15 multiple Handler instances was multiplied by the number of those instances
16 and significantly exceeded the effective amount of resources required to
17 solve the market data processing task.
19 The concept of the Feed Engine was involved to make things more
20 efficient. The Feed Engine machinery is used by the Handler to
21 receive and process market data starting from the recent major
22 release. Multiple Handler instances can share a single instance
25 \section using-feed-engine Using Feed Engine
27 The OnixS::CME::MDH::FeedEngine class represents the Feed
28 Engine concept. Feed Engine events are reported through an
29 instance of the OnixS::CME::MDH::FeedEngineListener class,
30 associated with the Feed Engine instance at constructing
33 The following code sample depicts primary aspects of using
34 the Feed Engine with two Handler instances:
38 // STEP 1: is to set configuration parameters.
39 FeedEngineSettings feedEngineSettings;
41 feedEngineSettings.threadsCount(NUMBER_OF_WORKING_THREADS);
43 // STEP 2: is to construct shared feed engine.
44 FeedEngine feedEngine(feedEngineSettings);
46 // STEP 3: is to bind feed engine to Handler instances.
47 handlerForChannel310.bindFeedEngine(feedEngine);
48 handlerForChannel311.bindFeedEngine(feedEngine);
52 \note Market data processing is done inside of working threads
53 managed by the Feed Engine. A size of the internal thread pool
54 is defined by the OnixS::CME::MDH::FeedEngineSettings::threadCount
57 \section feed-engine-events Feed Engine Events
59 All Feed engine events are encapsulated into OnixS::CME::MDH::FeedEngineListener
60 class interface. OnixS::CME::MDH::FeedEngine class accepts an instance of listener
63 The following table uncovers Feed Engine events:
71 <td>OnixS::CME::MDH::FeedEngineListener::onFeedEngineThreadBegin</td>
73 Invoked by the Feed Engine working thread before entering a master
76 Event callback is invoked in the context of working thread allowing
77 the subscriber to perform thread-specific turning like setting affinity
78 or priority for the thread.
82 <td>OnixS::CME::MDH::FeedEngineListener::onFeedEngineThreadEnd</td>
84 Invoked by Feed Engine when working thread is about to end.
86 The given event callback is invoked in the context of the working thread
87 allowing the subscriber to perform thread-specific cleanup like deleting
88 data stored in thread local storage.
92 <td>OnixS::CME::MDH::FeedEngineListener::onFeedEngineThreadIdle</td>
94 Invoked by the Feed Engine when the working thread is idle.
96 At an active phase, the Feed Engine working thread is running
97 a market data processing loop. The working thread may wait for
98 incoming market data using an appropriate I/O operation.
100 A working thread raises the given event if there was no
101 incoming data detected for a certain time interval (defined
102 by the OnixS::CME::MDH::FeedEngineSettings::dataWaitTime
103 parameter). The OnixS::CME::MDH::FeedEngineThreadIdleReasons::DataWaitTimeout
104 is passed as a reason of the idle state.
106 It may happen that a thread misses entering waiting for incoming data
107 because other threads have already been doing that for all active feeds,
108 and there are no other tasks like the processing already received market
109 data. In such case, the working threads spins for a certain time (defined
110 by the OnixS::CME::MDH::FeedEngineSettings::redundancySpinTime parameter
111 value) while waiting for any pending tasks to be executed. If no pending
112 tasks were found, the working thread enters an idle state with the
113 OnixS::CME::MDH::FeedEngineThreadIdleReasons::Redundant reason.
115 The given event callback also exposes a parameter-variable which value
116 represents the time interval the working thread is suggested to spend
117 sleeping to reduce races between working threads for executing pending
118 tasks (data reception, data processing, etc.) and thus to reduce a load
119 onto CPU. The parameter value can be modified inside the callback body.
121 \note It's not recommended to change the sleep time interval when
122 the working thread entered idle state after waiting for incoming data
123 to avoid overflow of a buffer used for incoming data.
125 \warning There's no predictability in firing the given event by the
126 Feed Engine. Each working thread may wait for incoming data or receive
127 incoming data or process data previously received by that or any other
128 threads. Availability of active tasks, which are executed by the working
129 threads, depends on many factors including some bound Handler instances,
130 system capacity, and a number of allocated working threads,
131 clock resolution and other factors.
