externals/archive/development.zip

\addtogroup benchmarking-results

\brief Latest achievements in market data processing performance.

\section benchmarking-preface Preface

Benchmarking was performed using CME MDP emulator which
transmits data previously recorded on channels 360 and
384 of real CME production environment.

Simulated environment makes pauses (_Sending Delay_) between
two packets sent to determine the influence of network activity
onto market data processing performance.

Benchmarking measures _Market Data Processing Latency_ - the
time used by C++ CME Market Data Handler to process market data
starting with the moment when the Handler receives a packet from the
network till the moment when the Handler passes the results of a
packet processing to the users through correspondent callbacks.

Benchmarking results were obtained by the _Benchmark_ sample supplied
in a distribution package, and include latency for **Messaging events**,
**Direct Book Updates** and **MBO Book Updates**.

Benchmarking was done on a machine with Intel
Core i7-7700K CPU running Ubuntu 17.04 Linux.

\section benchmarking-market-data Market Data Traits

At the moment of benchmarking, 99% of messages, sent by MDP, contained
a single atomic operation over the book for a particular instrument.

The following table summarizes characteristics of market data on the channels:

<center>

 Trait                                                | 360     | 384
------------------------------------------------------|---------:|---------:
 Instruments on the channel                           | 4069    | 1905
 Average packet size in bytes                         | 429     | 178
 Average/maximal number of bids/offers in direct book | 10/10   | 7/10
 Average/maximal number of bids/offers in MBO book    | 794/1569 | 270/690

</center>

\section benchmarking-messaging Messaging Latency

The given major release introduced a redesigned messaging subsystem.
Messaging the latency defines time since the moment of data reception
till the moment when received data is validated by a processing session
and passed to the users through the new OnixS::CME::MDH::MarketDataListener
callbacks.

<center>
  <table>
    <tr>
      <th rowspan=3>Sending<br>Delay<br> (ms)</th>
      <th colspan=12>Latency (&mu;s)</th>
    </tr>
    <tr>
      <th colspan=6>360</th>
      <th colspan=6>384</th>
    </tr>
    <tr>
      <th>Minimal</th>
      <th>Median</th>
      <th>Mean</th>
      <th>95%</th>
      <th>99%</th>
      <th>Maximal</th>
      <th>Minimal</th>
      <th>Median</th>
      <th>Mean</th>
      <th>95%</th>
      <th>99%</th>
      <th>Maximal</th>
    </tr>
    <tr>
      <td align=right>0</td>
      <td align=right>0.047</td>
      <td align=right>0.049</td>
      <td align=right>0.052</td>
      <td align=right>0.066</td>
      <td align=right>0.099</td>
      <td align=right>0.378</td>
     <td align=right>0.047</td>
      <td align=right>0.048</td>
      <td align=right>0.051</td>
      <td align=right>0.060</td>
      <td align=right>0.081</td>
      <td align=right>0.435</td>
    </tr>
    <tr>
      <td align=right>10</td>
      <td align=right>0.079</td>
      <td align=right>0.095</td>
      <td align=right>0.096</td>
      <td align=right>0.117</td>
      <td align=right>0.135</td>
      <td align=right>0.386</td>
     <td align=right>0.047</td>
      <td align=right>0.097</td>
      <td align=right>0.087</td>
      <td align=right>0.120</td>
      <td align=right>0.153</td>
      <td align=right>0.975</td>
    </tr>
    <tr>
      <td align=right>50</td>
      <td align=right>0.047</td>
      <td align=right>0.071</td>
      <td align=right>0.075</td>
      <td align=right>0.110</td>
      <td align=right>0.129</td>
      <td align=right>0.592</td>
     <td align=right>0.052</td>
      <td align=right>0.082</td>
      <td align=right>0.084</td>
      <td align=right>0.120</td>
      <td align=right>0.187</td>
      <td align=right>0.571</td>
   </tr>
  </table>
</center>

The visual presentation of latency depending on the interval
between two transmitted packets depicted on the following chart:

![Messaging Latency](@ref Latency-Messaging.jpg)

\section benchmarking-book Book Maintenance Latency

Book maintenance latency represents the time spent by the Handler
starting from the moment data is received till the moment when
an updated order book is supplied through a correspondent callback.
Depending on the type of book, different callbacks are invoked by the
Handler.

\warning The Handler publishes book updates at the end
of a certain market event like the _end of real quotes_
(atomic updates into direct and MBO books) or the _end of
implied quotes_ (atomic updates into implied books). Also,
in general, an event may consist of multiple messages
distributed across multiple packets. Therefore, the latency
for different types of books may differ significantly.

\subsection benchmarking-direct-book Market By Price, Direct Book

Measurements for direct book updates are gathered into the following table:

<center>
  <table>
    <tr>
      <th rowspan=3>Sending<br>Delay<br> (ms)</th>
      <th colspan=12>Latency (&mu;s)</th>
    </tr>
    <tr>
      <th colspan=6>360</th>
      <th colspan=6>384</th>
    </tr>
    <tr>
      <th>Minimal</th>
      <th>Median</th>
      <th>Mean</th>
      <th>95%</th>
      <th>99%</th>
      <th>Maximal</th>
      <th>Minimal</th>
      <th>Median</th>
      <th>Mean</th>
      <th>95%</th>
      <th>99%</th>
      <th>Maximal</th>
    </tr>
    <tr>
      <td align=right>0</td>
      <td align=right>0.104</td>
      <td align=right>0.169</td>
      <td align=right>0.183</td>
      <td align=right>0.289</td>
      <td align=right>0.379</td>
      <td align=right>0.780</td>
     <td align=right>0.107</td>
      <td align=right>0.202</td>
      <td align=right>0.214</td>
      <td align=right>0.307</td>
      <td align=right>0.378</td>
      <td align=right>0.855</td>
    </tr>
    <tr>
      <td align=right>10</td>
      <td align=right>0.115</td>
      <td align=right>0.217</td>
      <td align=right>0.239</td>
      <td align=right>0.390</td>
      <td align=right>0.512</td>
      <td align=right>1.420</td>
      <td align=right>0.109</td>
      <td align=right>0.225</td>
      <td align=right>0.236</td>
      <td align=right>0.344</td>
      <td align=right>0.450</td>
      <td align=right>1.678</td>
    </tr>
    <tr>
      <td align=right>50</td>
      <td align=right>0.126</td>
      <td align=right>0.233</td>
      <td align=right>0.250</td>
      <td align=right>0.386</td>
      <td align=right>0.469</td>
      <td align=right>1.102</td>
     <td align=right>0.128</td>
      <td align=right>0.284</td>
      <td align=right>0.301</td>
      <td align=right>0.480</td>
      <td align=right>0.700</td>
      <td align=right>1.706</td>
    </tr>
  </table>
</center>

Following diagram depicts table values visually:

![Direct Book Update Latency](@ref Latency-DirectBookUpdate.jpg)

\subsection benchmarking-mbo-book Market By Order

The following table presents benchmarking results for MBO book updates:

<center>
  <table>
    <tr>
      <th rowspan=3>Sending<br>Delay<br> (ms)</th>
      <th colspan=12>Latency (&mu;s)</th>
    </tr>
    <tr>
      <th colspan=6>360</th>
      <th colspan=6>384</th>
    </tr>
    <tr>
      <th>Minimal</th>
      <th>Median</th>
      <th>Mean</th>
      <th>95%</th>
      <th>99%</th>
      <th>Maximal</th>
      <th>Minimal</th>
      <th>Median</th>
      <th>Mean</th>
      <th>95%</th>
      <th>99%</th>
      <th>Maximal</th>
    </tr>
    <tr>
      <td align=right>0</td>
      <td align=right>0.108</td>
      <td align=right>0.252</td>
      <td align=right>0.462</td>
      <td align=right>1.591</td>
      <td align=right>3.372</td>
      <td align=right>8.223</td>
     <td align=right>0.151</td>
      <td align=right>0.300</td>
      <td align=right>0.362</td>
      <td align=right>0.704</td>
      <td align=right>0.905</td>
      <td align=right>3.158</td>
    </tr>
    <tr>
      <td align=right>10</td>
      <td align=right>0.181</td>
      <td align=right>0.391</td>
      <td align=right>0.653</td>
      <td align=right>1.960</td>
      <td align=right>4.022</td>
      <td align=right>12.457</td>
     <td align=right>0.125</td>
      <td align=right>0.312</td>
      <td align=right>0.394</td>
      <td align=right>0.848</td>
      <td align=right>1.056</td>
      <td align=right>3.776</td>
    </tr>
    <tr>
      <td align=right>50</td>
      <td align=right>0.154</td>
      <td align=right>0.395</td>
      <td align=right>0.642</td>
      <td align=right>1.933</td>
      <td align=right>3.866</td>
      <td align=right>14.535</td>
     <td align=right>0.145</td>
      <td align=right>0.370</td>
      <td align=right>0.478</td>
      <td align=right>1.015</td>
      <td align=right>1.267</td>
      <td align=right>4.188</td>
      </tr>
  </table>
</center>

The diagram below displays the results visually:

![MBO Book Update Latency](@ref Latency-MboBookUpdate.jpg)

\subsection benchmarking-book-conclusions Conclusions and Important Notes

**Minimal latency of a book maintenance machinery is almost 10
times less in comparison to the book maintenance latency measured
for previous major release**. Such performance boost is achieved
by reducing processing time at session (messaging) level.

Also, in spite of both previous and latest major releases use
almost the same book maintenance machinery, **improvements implemented
in latest release provide significant (up to 25 times) reduction of
median latency compare to previous major releases** due to switching
using new memory management strategies and more effective data structures.

\note An important aspect of measuring an order book maintenance
latency is that order books are passed to the user at the end of
market event which in general may consist of multiple packets and
messages. Therefore, amount of data proportionally increases the
latency. One of the possible ways to have better latency is to use
a per-security book update notification strategy which instructs
the Handler to expose book as soon as a chain of atomic updates
is ended for a particular security without waiting for the end
of the entire event. This feature is supported by the Handler,
but it's not officially approved by CME. Therefore, using noted
book update notification strategy is on customer's risk.