work in progress

SPARK-2179 PIQ

.vscode/launch.json

@@ -215,7 +215,7 @@
             "type": "cppdbg",
             "request": "launch",
             "program": "/home/bts/spark/build/ninja_debug/server/sparkd",
-            "args": ["test", "[md]"],
+            "args": ["test", "[mbo]"],
             "stopAtEntry": false,
             "cwd": "/home/bts/spark/build/ninja_debug/server/",
             "environment": [],

doc/user/content/bts_spark/Algos/_index.md

@@ -0,0 +1,46 @@
++++
+title = "BTS Basic Algos"
+weight = 10
+sort_by = "weight"
+insert_anchor_links = "right"
++++
+
+
+## Lean
+
+The goal of the lean order is to watch the opposite-inside market size and cross when that size goes below the threshold (this also implies a cross if that size goes to zero and the market trades away from you) If the market trades toward you, you can opt into several different behaviors
+
+##### Parameters
+  * Cross Thresh - size thresh in absolute or ratio terms
+  * Max Chase Ticks - max levels to pay up, in ticks
+
+
+## Join
+The goal of the join order is to watch the opposite-inside market and join when the market flips (that is, when the best offer becomes the best bid) and has at least threshold size.
+
+##### Parameters
+  * Join Thresh - size thresh in absolute or ratio terms
+
+
+## Top
+
+The goal of the top order is to not trade by yourself, but the intent at the time of order entry is to get filled.If the order is on the market (say current bid price) AND the quantity on that bid price is below top thresh then cancel the order else remain working.
+
+##### Parameters
+  * Top Thresh - size thresh in absolute or ratio terms
+
+
+## Cover
+A Cover order’s purpose is to fire a hedge order (of any type) in response to fills from it’s child order on the opposite side of the market N ticks away
+
+##### Parameters
+  * Profit Ticks - ticks above or below fill price to place profit / cover order
+
+
+## OCO
+An OCO order creates a relationship between a child order and a 'stop' or 'other' order such that when one of these is cancelled, both cancel, or when one is filled, the other order's size is debited in the amount of the fill.
+
+Composing this order with Cover simulates scalping functionality
+
+##### Parameters
+  * Stop Ticks - ticks below or above main order to place stop / other order

doc/user/content/bts_spark_sdk/Client/_index.md

@@ -1,6 +0,0 @@
-+++
-title = "Client"
-weight = 1
-sort_by = "weight"
-redirect_to = "bts_spark_sdk/Client/overview/"
-+++

doc/user/content/bts_spark_sdk/Server/_index.md

@@ -1,6 +0,0 @@
-+++
-title = "Server"
-weight = 2
-sort_by = "weight"
-insert_anchor_links = "right"
-+++

doc/user/content/bts_spark_sdk/_index.md

@@ -1,6 +0,0 @@
-+++
-title = "BTS Spark SDK"
-weight = 2
-sort_by = "weight"
-insert_anchor_links = "right"
-+++

doc/user/content/release_notes/_index.md

@@ -4,6 +4,23 @@ weight = 3
 sort_by = "weight"
 insert_anchor_links = "right"
 +++
+## 1.6.0
+### User Visible Changes
+#### New Features
+  - You can now view your order's position in queue in the ladder
+#### Bug Fixes
+  - Fix a bug in the order popup in the ladder which could lead to a crash.
+
+## 1.5.9
+### User Visible Changes
+  - Fix for stunnel connections on Windows 7
+
+## 1.5.8
+### User Visible Changes
+  - Fixed a bug where it was possible to crash the UI by entering an order through the order ticket if Enter was used to dismiss the dialog.
+  - Fixed a spreader reporting bug which could cause the server to crash if an algo was used to control certain types of synthetics
+  - Improved user message when the UI fails to login
+  - Integrated stunnel into UI to make VPN setup unnecessary in unsecured environments.
 ## 1.5.7
 ### User Visible Changes
   - New videos have been added to the manual describing the [Algo Studio](./bts_spark/Trading/index.md#algo-studio) and [Spreader](./bts_spark/Spreader/index.md)

src/client/spark/App.xaml.cs

@@ -1,4 +1,6 @@
 using System;
+using System.IO;
+using System.Diagnostics;
 using System.Collections.Generic;
 using System.ComponentModel;
 using System.Linq;
@@ -72,13 +74,71 @@ namespace mbts.ui.shell
          base.OnExit(e);
          Environment.Exit(0);
       }
+      
+      void start_stunnel(string path, uint job_error)
+      {
+         string config_error; 
+         if (!mbts.config.Configuration.try_initialize(out config_error))
+         {
+            _stunnel_process = Process.Start(path + "bin\\stunnel.exe");         
+            if (_stunnel_process != null)
+            {
+               for (int i = 0; i < 10; i++)
+               {
+                  if (mbts.config.Configuration.try_initialize(out config_error))
+                     break;
+                  System.Threading.Thread.Sleep(500);
+               }
+            }
+            else
+               stunnel_job_error = "Stunnel Job Error code: " + job_error + 
+                                   " could not start stunnel " + (config_error ?? string.Empty);
+         }
+      }
+      
+      void kill_stunnel()
+      {
+         try
+         {
+            Process[] ps = Process.GetProcessesByName("stunnel");
+            ps.ForEach(p => 
+            {  
+               try { p.Kill(); }
+               catch {}
+            });
+         }
+         catch { }         
+      }
+      
+      Process _stunnel_process; 
+      static internal string stunnel_job_error { get; private set;}
+      void init_stunnel(string[] argv)
+      {
+         try
+         {
+            var path = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData) + "\\BTS\\stunnel\\";
+            if (File.Exists(path + "config\\stunnel.conf"))
+            {
+               uint job_error = BTSUINativeMethods.setup_process_job_object();   
+               //if we can't put this process and stunnel into a job, try to stop stunnel then restart
+               if(job_error != 0)
+                  kill_stunnel();
+               start_stunnel(path, job_error);
+            }
+         }
+         catch(Exception ex)
+         {
+            stunnel_job_error = "Error starting stunnel: " + ex.Message;
+         } 
+      }
 
       public new int Run() {
          var argv = Environment.GetCommandLineArgs();
-         try {
-            if (argv.Contains("-multiple") ||
-               _mutex == null ||
-               _mutex.WaitOne(TimeSpan.Zero, true)) {
+         try {            
+            if (argv.Contains("-multiple") || _mutex == null || _mutex.WaitOne(TimeSpan.Zero, true)) 
+            {
+               if(_mutex == null || _mutex.WaitOne(TimeSpan.Zero, true))
+                  init_stunnel(argv);
                return base.Run();
             }
             else {

src/client/spark/Daemons/ShellServices.cs

@@ -6,6 +6,7 @@ using bts.api.persistence;
 using bts.api.position;
 using bts.api.synthetic;
 using bts.co2;
+using bts.md;
 using bts.metrics;
 using bts.support;
 using System.ComponentModel.Composition;
@@ -50,6 +51,9 @@ namespace Spark {
       AlgoServiceWrapper _algo_service_wrapper;
       [Export]
       public AlgoServiceWrapper algo_service_wrapper => _algo_service_wrapper ?? (_algo_service_wrapper = new AlgoServiceWrapper(context));
+
+      [Export]
+      public MBOService mbo_service => context.locate<MBOService>();
    }
 
    class AlgoServiceWrapper {

src/client/spark/Modules/Trading/OrderBook/OrderBookVM.cs

@@ -64,7 +64,7 @@ namespace mbts.spark {
             }),
             _columns,
             _current_row.SubscribeHookLatest(
-               r => r.newval.subscribe_data().Subscribe( 
+               r => r.newval?.subscribe_data().Subscribe(
                   data => selected_order_changed?.Invoke(data.order)
                )
             )

src/client/spark/Notifications/INotificationService.cs

@@ -55,7 +55,7 @@ namespace bts.shell {
       void _on_activity(Activity activity) =>
          _activity_notifications.process_item(activity);
 
-      void _on_orders(TableUpdate<(SecurityImmutable?,OrderRecordImmutable),uint> update) {
+      void _on_orders(TableUpdate<(SecurityImmutable? sec, OrderRecordImmutable order), uint> update) {
          // must update notification_service_data_cache first, some of the predicate/triggers (esp. ToastNotifications) will use the notification_service_data_cache to lookup order-history
          _data_cache.update(update.as_iincremental());
          foreach (var o in update.added.Concat(update.changed))

src/client/spark/Orders/OrderActivitySummarizer.cs

@@ -27,7 +27,7 @@ namespace bts.spark {
       static readonly IEqualityComparer<OrderRecordImmutable> _id_comparer =
          bts.utils.EqualityComparer.Create<OrderRecordImmutable, uint>(a => a.Key);
 
-      public OrderActivitySummarizer(IBus bus, IObservable<TableUpdate<(SecurityImmutable? sec,OrderRecordImmutable order), uint>> orders) {
+      public OrderActivitySummarizer(IBus bus, IObservable<TableUpdate<(SecurityImmutable? sec, OrderRecordImmutable order), uint>> orders) {
          _bus = bus;
          _dispose = Disposable.Create(
             bus.subscribe(SparkTags.SecuritiesTag).Subscribe(sec => _security = sec),

src/client/spark/Orders/OrderSecurity.cs

@@ -7,18 +7,31 @@ namespace bts.api.orderman {
       public OrderSecurity(SecurityImmutable? sec, OrderRecordImmutable order) {
          this.security = sec;
          this.order = order;
+         piq = null;
       }
       public OrderSecurity((SecurityImmutable? sec, OrderRecordImmutable order) t) {
          security = t.sec;
          order = t.order;
+         piq = null;
       }
+      public OrderSecurity(SecurityImmutable? sec, OrderRecordImmutable order, uint? piq) {
+         this.security = sec;
+         this.order = order;
+         this.piq = piq;
+      }
+      //public OrderSecurity(OrderSecurity t) {
+      //   security = t.security;
+      //   order = t.order;
+      //   piq = t.piq;
+      //}
       public static implicit operator OrderSecurity(ValueTuple<SecurityImmutable?, OrderRecordImmutable> x) => new OrderSecurity(x);
       public SecurityImmutable? security { get; }
       public OrderRecordImmutable order { get; }
+      public uint? piq { get; }
 
       public override bool Equals(object obj) => (obj is OrderSecurity os) && Equals(os);
-      public bool Equals(OrderSecurity other) => order.Equals(other.order) && security == other.security;
-      public override int GetHashCode() => (security, order).GetHashCode();
+      public bool Equals(OrderSecurity other) => order.Equals(other.order) && security == other.security && piq == other.piq;
+      public override int GetHashCode() => (security, order, piq).GetHashCode();
 
       public static bool operator == (OrderSecurity left, OrderSecurity right) => left.Equals(right);
       public static bool operator != (OrderSecurity left, OrderSecurity right) => !left.Equals(right);

src/client/spark/Orders/OrdersModel.cs

@@ -3,9 +3,11 @@ using bts.api;
 using bts.api.core;
 using bts.api.orderman;
 using bts.api.position;
+using bts.md;
 using bts.spark;
 using bts.utils;
 using mbts.collections;
+using mbts.config;
 using mbts.spark.utilities;
 using mbts.utils;
 using System;
@@ -16,8 +18,10 @@ using System.ComponentModel.Composition;
 using System.Linq;
 using System.Reactive.Concurrency;
 using System.Reactive.Linq;
+using System.Reactive.Subjects;
 using System.Threading.Tasks;
 using OrderUpdate = bts.api.core.TableUpdate<(bts.api.SecurityImmutable? sec, bts.api.orderman.OrderRecordImmutable order), uint>;
+using OrderUpdatePiq = bts.api.core.TableUpdate<bts.api.orderman.OrderSecurity, uint>;
 
 namespace mbts.spark {
    [Export]
@@ -26,21 +30,26 @@ namespace mbts.spark {
       readonly OrderFactory _factory;
       readonly OrderService _service;
       readonly PositionService _position_service;
+      readonly MBOService _mbo_service;
       readonly SpreaderModel _spreader;
       readonly ErrorReporter _error;
       readonly DynamicOrderHelper _dynamic_helper;
       readonly OrderActivitySummarizer _activity;
       readonly TableUpdateDemuxer<OrderRecordImmutable, uint, uint> _security_orders_demuxer;
+      // we need a new one of these for the time being since the ladder is subscribing to orders with piq
+      readonly TableUpdateDemuxer<OrderRecordImmutable, uint, uint> _security_orders_piq_demuxer;
       readonly TableUpdateDemuxer<PositionUnitImmutable, int, uint> _security_positions_demuxer;
       IDisposable _dispose;
+      static readonly bool _piq_enabled = Configuration.get<bts.config.toggles.PlacementInQueue>().enabled;
 
       [ImportingConstructor]
-      OrdersModel(IBus bus, OrderFactory factory, OrderService service, PositionService position_service, 
+      OrdersModel(IBus bus, OrderFactory factory, OrderService service, PositionService position_service, MBOService mbo_service,
          SpreaderModel spreader, DynamicOrderHelper dynamic_helper, AdminModel admin_model, IAlgoStudioModel algo_model, SecurityStore sstore, ErrorReporter error) {
          _bus = bus;
          _factory = factory;
          _service = service;
          _position_service = position_service;
+         _mbo_service = mbo_service;
          _spreader = spreader;
          _error = error;
          _dynamic_helper = dynamic_helper;
@@ -48,13 +57,34 @@ namespace mbts.spark {
          this.algo_model = algo_model;
          var scheduler = ThreadPoolScheduler.Instance;
          var interval = TimeSpan.FromMilliseconds(100);
-         on_orders = 
+         on_orders =
             _service
             .subscribei_orders()
             .Naggle(interval, scheduler)
             .PermissionFilter(admin_model.on_admin_data)
             .join(sstore, order => order.security_key())
-            .Table().Cache();
+            .Table()
+            .Cache();
+
+         if (_piq_enabled) {
+            on_orders_with_piq = on_orders
+            .LeftJoin<
+               (SecurityImmutable?, OrderRecordImmutable),
+               OrderPIQImmutable,
+               OrderSecurity,
+               uint
+               >(mbo_service.subscribei_piq().Table(),
+                 si_ort => si_ort.Item2.Key,
+                 p => p.OrderId,
+                 (si_ort, r, p) => new OrderSecurity(si_ort.Item1, si_ort.Item2, p.Position),
+                 (si_ort, r) => new OrderSecurity(si_ort.Item1, si_ort.Item2, null),
+                 osp => osp.order.Key);
+         } else {
+            on_orders_with_piq = on_orders
+               .Select((ft) => new OrderSecurity(ft.Item1, ft.Item2, null), os => os.order.Key)
+               .Replay(1);
+         }
+
          on_positions = _position_service.subscribei_position_data().Naggle(interval, scheduler).PermissionFilter(admin_model.on_admin_data).Table().Cache();
          working_orders = 
             on_orders.Scan(
@@ -62,6 +92,13 @@ namespace mbts.spark {
                (workers, table) => _update_working_orders(workers, table))
                .Replay(1).RefCount();
 
+         working_orders_with_piq = 
+            on_orders_with_piq.Scan(
+               ImmutableDictionary<uint, OrderSecurity>.Empty,
+               (workers, table) => _update_working_orders(workers, table))
+               .Replay(1).RefCount();
+
+
          on_order_securities = 
             on_orders
             .as_iincremental()
@@ -70,6 +107,7 @@ namespace mbts.spark {
 
          _activity = new OrderActivitySummarizer(bus, on_orders);
          _security_orders_demuxer = new TableUpdateDemuxer<OrderRecordImmutable, uint, uint>(o => o.Key, o => o.security_key());
+         _security_orders_piq_demuxer = new TableUpdateDemuxer<OrderRecordImmutable, uint, uint>(o => o.Key, o => o.security_key());
          _security_positions_demuxer = new TableUpdateDemuxer<PositionUnitImmutable, int, uint>(o => o.RecordId, p => p.SecurityId);
          var algo_meta_join = algo_model.definitions.LeftJoin<AlgoData, AlgoDefinitionMetadata, (AlgoData algo, AlgoDefinitionMetadata? meta), AlgoKey>(
             algo_model.metadata_service.subscribe_metadata().Table(m => m.id),
@@ -104,6 +142,7 @@ namespace mbts.spark {
          on_fill_tuples = fill_order_tuples_join;
          _dispose = Disposable.Create(
             on_orders.Select(t => t.order, o => o.Key).Subscribe(_security_orders_demuxer),
+            on_orders_with_piq.RefCount().Select(t => t.order, o => o.Key).Subscribe(_security_orders_piq_demuxer),
             on_positions.Subscribe(_security_positions_demuxer),
             order_secs_algo_meta_join.Connect(),
             algo_meta_join.Connect(),
@@ -121,11 +160,14 @@ namespace mbts.spark {
       public SpreaderModel spreader => _spreader;
       public DynamicOrderHelper dynamic_helper => _dynamic_helper;
       public IObservable<OrderUpdate> on_orders { get; }
+      public IConnectableObservable<OrderUpdatePiq> on_orders_with_piq { get; }
       public IObservable<TableUpdate<OrderSecurityAlgoMeta, uint>> on_order_tuples { get; }
       public IObservable<IIncrementalUpdate<SecurityImmutable, uint>> on_order_securities { get; }
       public IObservable<TableUpdate<FillTuple, string>> on_fill_tuples { get; }
       public IObservable<IReadOnlyDictionary<uint,OrderSecurity>> working_orders { get; }
+      public IObservable<IReadOnlyDictionary<uint, OrderSecurity>> working_orders_with_piq { get; }
       public IObservable<TableUpdate<PositionUnitImmutable, int>> on_positions { get; }
+
       public AdminModel admin_model { get; }
       public IAlgoStudioModel algo_model { get; }
 
@@ -197,8 +239,30 @@ namespace mbts.spark {
       public void Dispose() =>
          Disposable.dispose(ref _dispose);
 
-      static ImmutableDictionary<uint, OrderSecurity> 
-         _update_working_orders(ImmutableDictionary<uint, OrderSecurity> target, OrderUpdate table) {
+      static ImmutableDictionary<uint, OrderSecurity> _update_working_orders(ImmutableDictionary<uint, OrderSecurity> target, OrderUpdate table) {
+         var clone = target.ToBuilder();
+         foreach (var o in table.removed) {
+            if (o.order.is_active()) {
+               clone.Remove(o.order.Key);
+            }
+         }
+         foreach (var o in table.changes) {
+            if (o.Old.order.is_active()) {
+               clone.Remove(o.Old.order.Key);
+            }
+            if (o.New.order.is_active()) {
+               clone.Add(o.New.order.Key, o.New);
+            }
+         }
+         foreach (var o in table.added) {
+            if (o.order.is_active()) {
+               clone.Add(o.order.Key, o);
+            }
+         }
+         return clone.ToImmutableDictionary();
+      }
+
+      static ImmutableDictionary<uint, OrderSecurity> _update_working_orders(ImmutableDictionary<uint, OrderSecurity> target, OrderUpdatePiq table) {
          var clone = target.ToBuilder();
          foreach (var o in table.removed) {
             if (o.order.is_active()) {
@@ -222,6 +286,7 @@ namespace mbts.spark {
       }
    }
 
+
    // usinig new OrdersModel extensions to reduce the amount of code change needed to switch uis over to the new OrdersModel,
    // without changing any code in the actual new OrdersModel class.
    static class OrdersModelExtensions {

src/client/spark/Predicates/Data/Events/OrderFilledEventPredicate.cs

@@ -82,7 +82,7 @@ namespace mbts {
                   // old order will be a copy of new order, except with filledQuantity set to 0
                   var old_state = ort.order.State.CloneToMutable();
                   old_state.FilledQuantity = 0;
-                  return (new OrderSecurity(ort.security, ort.order.with(state: old_state)), ort);
+                  return (new OrderSecurity(ort.security, ort.order.with(state: old_state), ort.piq), ort);
                }
             }
             return null;

src/client/spark/Spark.csproj

@@ -4,7 +4,8 @@
     <Authors>Blue Trading Systems</Authors>
     <Company>Blue Trading Systems</Company>
     <Product>Spark</Product>
-    <Description>The game has changed. Make your Move.</Description>
+    <Description>Advanced Derivatives Trading</Description>
+    <ApplicationManifest>spark.manifest</ApplicationManifest>
     <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
     <Configurations>Debug;Release</Configurations>
     <Platforms>x64</Platforms>

src/client/spark/core/NativeMethods/BTSNativeMethods.cs

@@ -11,6 +11,9 @@ namespace mbts.ui.native
       [DllImport("libspark")]
       public static extern void enable_limits();
 
+      [DllImport("libspark")]
+      public static extern uint setup_process_job_object();      
+
       [DllImport("libspark", CharSet = CharSet.Unicode)]
       public static extern void set_window_app_id(IntPtr hwnd, string stack_id);
    }

src/client/spark/view/Controls/DIYGrid/Columns/ColumnBase.cs

@@ -97,13 +97,13 @@ namespace mbts.ui.controls {
             TotalPnL = "Total PnL",
             PnLPrice = "PnL Price",
             Cancel = "Cancel",
-
             MarketDescription = "Market Description",
             ClearedAlias = "Cleared Alias",
             ProductDescription = "Product Description",
             HubAlias = "Hub Alias",
             StripName = "Strip Name",
-            MicCode = "Mic Code"
+            MicCode = "Mic Code",          
+            PositionInQueue = "PIQ"
             ;
       }
 

src/client/spark/view/Extensions/TextElement/TextBoxBehavior.cs

@@ -103,10 +103,8 @@ namespace mbts.ui {
       }
 
       static void tb_keydown(object sender, KeyEventArgs e) {
-         if (e.Key == Key.Enter) {
-            var binding = BindingOperations.GetBindingExpression(sender as DependencyObject, TextBox.TextProperty);
-            binding.UpdateSource();
-         }
+         if (e.Key == Key.Enter)
+            BindingOperations.GetBindingExpression(sender as DependencyObject, TextBox.TextProperty)?.UpdateSource();
       }
 
       public static bool GetCommitOnEnter(DependencyObject obj) => (bool)obj.GetValue(CommitOnEnterProperty);

src/client/spark/view/Shell/Login/LoginModel.cs

@@ -94,7 +94,7 @@ namespace mbts.ui.shell {
             Configuration.initialize();
          }
          catch (ApplicationException e) {
-            ErrorMessage = e.Message;
+            ErrorMessage = (e.Message ?? string.Empty) + " " + (App.stunnel_job_error ?? string.Empty);
             HasFatalError = true;
             return;
          }
@@ -106,7 +106,13 @@ namespace mbts.ui.shell {
          var auth = _ctx.locate<AuthService>();
          var response = await auth.get_users_async(new GetUsersRequest());
          if (response.success) {
-            Users = response.value.Users.Select(x => new User(x.Login,x.Id)).OrderBy(u => u.name).ToArray();
+            if (!string.IsNullOrEmpty(response.value.Error)) {
+               ErrorMessage = response.value.Error;
+               HasFatalError = true;
+            }
+            else {
+               Users = response.value.Users.Select(x => new User(x.Login,x.Id)).OrderBy(u => u.name).ToArray();
+            }
          }
          else {
             ErrorMessage = "Logon: " + response.error_message;

src/client/spark/view/Spark/Ladder/LadderGridHost.cs

@@ -42,7 +42,8 @@ namespace mbts.spark {
          _ask_vis_tup = new VisualGlyphTuple(),
          _price_vis_tup = new VisualGlyphTuple(),
          _last_vis_tup = new VisualGlyphTuple(),
-         _volume_vis_tup = new VisualGlyphTuple();
+         _volume_vis_tup = new VisualGlyphTuple(),
+         _piq_vis_tup = new VisualGlyphTuple();
 
       readonly RefValueContainer<IEnumerable<VolumeAtPriceLevelImmutable>> _volume_container =
          new RefValueContainer<IEnumerable<VolumeAtPriceLevelImmutable>>();
@@ -92,7 +93,7 @@ namespace mbts.spark {
          };
       }
 
-      public enum ColumnType { Order, Bid, Price, Ask, Last, Volume, None }
+      public enum ColumnType { Order, Bid, Price, Ask, Last, Volume, None, PIQ }
 
       public event Action visible_columns_changed;
 
@@ -192,6 +193,18 @@ namespace mbts.spark {
          set => SetValue(LastColumnWidthProperty, value);
       }
 
+      public static readonly DependencyProperty PIQColumnWidthProperty =
+         WPFHelper.create_dp<uint, LadderGridHost>(
+            nameof(PIQColumnWidth),
+            FrameworkPropertyMetadataOptions.BindsTwoWayByDefault,
+            (o, oldval, newval) => o._adjust_column_width(o.columns.FirstOrDefault(c => c.column_type == ColumnType.PIQ), (double)newval - (double)oldval),
+            50u);
+
+      public uint PIQColumnWidth {
+         get => (uint)GetValue(PIQColumnWidthProperty);
+         set => SetValue(PIQColumnWidthProperty, value);
+      }
+
       #region Command
       public ICommand Command {
          get => (ICommand)GetValue(CommandProperty);
@@ -404,7 +417,8 @@ namespace mbts.spark {
          double x = 0;
          var col_list = new List<Column> {
             new Column(_settings.orders_column, ColumnType.Order, null, x),
-            new Column(_settings.bid_column, ColumnType.Bid, _bid_vis_tup, (x += _settings.orders_column.show ? _settings.orders_column.width : 0)),
+            new Column(_settings.piq_column, ColumnType.PIQ, _piq_vis_tup, (x +=  _settings.orders_column.show ? _settings.orders_column.width : 0)),
+            new Column(_settings.bid_column, ColumnType.Bid, _bid_vis_tup, (x += _settings.piq_column.show ? _settings.piq_column.width : 0)),
             (_price_column = new Column(_settings.price_column, ColumnType.Price, _price_vis_tup, (x += _settings.bid_column.show ? _settings.bid_column.width : 0))),
             new Column(_settings.ask_column, ColumnType.Ask, _ask_vis_tup, (x += _settings.price_column.show ? _settings.price_column.width : 0))
          };
@@ -509,6 +523,8 @@ namespace mbts.spark {
                if (depth_changed)
                   _refresh_sizes();
             }
+            // TODO - detect changes in PIQ to selectively redraw
+            _refresh_piq();
             _refresh_volume_at_price();
             _last_origin_price = _mkt_break_lines.origin;
             _price_dirty = false;
@@ -545,6 +561,14 @@ namespace mbts.spark {
             LadderRender.render_last(_last_vis_tup.visual, _last_vis_tup.glyph_context, _last_trade, last_index, _settings.row_height, _settings.last_column);
       }
 
+      void _refresh_piq() {
+         if (!_settings.piq_column.show)
+            return;
+         int origin_index = LadderRender.index_of(_price_list, _mkt_break_lines.origin, _price_comparer);
+         var piq_list = _vm.price_vs_piq;
+         LadderRender.render_piqs(_piq_vis_tup.visual, _last_vis_tup.glyph_context, origin_index, _price_list, piq_list.ToArray(), _settings.row_height, security.ContractSpec, _settings.piq_column);
+      }
+
       void _refresh_sizes() {
          Log.assert_d(_price_list != null);
          int origin_index = LadderRender.index_of(_price_list, _mkt_break_lines.origin, _price_comparer);
@@ -693,6 +717,7 @@ namespace mbts.spark {
          _ask_vis_tup.glyph_context = glyph_context(s.ask_column);
          _price_vis_tup.glyph_context = glyph_context(s.price_column);
          _last_vis_tup.glyph_context = glyph_context(s.last_column);
+         _piq_vis_tup.glyph_context = glyph_context(s.piq_column);
          _volume_vis_tup.glyph_context = glyph_context(s.volume_column.general);
 
          GlyphContext glyph_context(LadderColumnSettings cs) =>
@@ -720,6 +745,7 @@ namespace mbts.spark {
                      case ColumnType.Price: SetCurrentValue(price_column_widthProperty, col.settings.width); break;
                      case ColumnType.Ask: SetCurrentValue(ask_column_widthProperty, col.settings.width); break;
                      case ColumnType.Last: SetCurrentValue(LastColumnWidthProperty, col.settings.width); break;
+                     case ColumnType.PIQ: SetCurrentValue(PIQColumnWidthProperty, col.settings.width); break;
                      case ColumnType.Volume:
                         SetCurrentValue(VolumeColumnWidthProperty, col.settings.width);
                         _volume_container.mark_dirty();

src/client/spark/view/Spark/Ladder/LadderOrderVM.cs

@@ -46,9 +46,9 @@ namespace mbts.spark {
       int working_quantity;
 
       LadderModel model => _parent.model;
-      Dictionary<uint, OrderRecordImmutable> _orders = new Dictionary<uint, OrderRecordImmutable>();
+      Dictionary<uint, (OrderRecordImmutable order, uint? p)> _orders = new Dictionary<uint, (OrderRecordImmutable order, uint? p)>();
 
-      public LadderOrderVM(OrderRecordImmutable o, double real_price, LadderVM parent, IBus bus, IObservable<LadderSettings.BadgeStyle> bs) {
+      public LadderOrderVM((OrderRecordImmutable order, uint? p) op, double real_price, LadderVM parent, IBus bus, IObservable<LadderSettings.BadgeStyle> bs) {
          _parent = parent;
          _bus = bus;
          _bs = bs;
@@ -73,7 +73,7 @@ namespace mbts.spark {
          double get_font_size(LadderSettings s) => s.orders_column.font_size;
          Brush get_foreground(LadderSettings s) => s.orders_column.foreground.rgb_brush();
          //
-         add_order(o);
+         add_order(op);
 
          _dispose = Disposable.Create(
             _bs.Subscribe(badge => {
@@ -109,19 +109,19 @@ namespace mbts.spark {
          // if this badge only represents a single order, then is_exchange_order and the border (= algo color)
          // should be set appropriately
          if (_orders.Count == 1) {
-            var single_order = _orders.Values.First();
+            var single_order = _orders.Values.First().order;
             var single_order_metadata = _parent.model.order_gateway.factory.get_metadata(single_order);
             border = single_order_metadata.Color.rgb_brush();
-            is_exchange_order = OrderUtils.is_exchange_order(_orders.Values.First());
+            is_exchange_order = OrderUtils.is_exchange_order(single_order);
          } else {
             border = new SolidColorBrush(color);
             is_exchange_order = false;
          }
          has_multiple_orders_at_same_level = _orders.Count > 1;
-         is_hedge_order = _orders.Any(o => o.Value.sub_type() == OrderSubType.HedgingLeg);
+         is_hedge_order = _orders.Any(op => op.Value.order.sub_type() == OrderSubType.HedgingLeg);
 
          if (_orders.Any()) {
-            summary = _orders.Values.Select(order => order.short_description(_parent.security.ContractSpec)).Aggregate((s1, s2) => $"{s1}{Environment.NewLine}{Environment.NewLine}{s2}");
+            summary = _orders.Values.Select(op => op.order.short_description(_parent.security.ContractSpec)).Aggregate((s1, s2) => $"{s1}{Environment.NewLine}{Environment.NewLine}{s2}");
          }
          _set_badge_text();
 
@@ -132,8 +132,8 @@ namespace mbts.spark {
          RaisePropertyChanged(nameof(text_lines));
       }
 
-      public void add_order(OrderRecordImmutable new_order) {
-         _orders[new_order.Key] = new_order;
+      public void add_order((OrderRecordImmutable order, uint? p) new_op) {
+         _orders[new_op.order.Key] = new_op;
       }
 
       public void remove_order(uint order_key) {
@@ -149,10 +149,11 @@ namespace mbts.spark {
 
       public uint security_key => _parent.security.Id;
       public double price { get => double.IsNaN(_phantom_price) ? _real_price : _phantom_price; }
+      public uint? min_piq => _orders.Select(op => op.Value.p).Where(piq => piq.HasValue).Min();
       public double real_price => _real_price;
       public Side side {
          get {
-            var distinct_sides = _orders.Values.Select(o => o.side()).Distinct();
+            var distinct_sides = _orders.Values.Select(op => op.order.side()).Distinct();
             if (distinct_sides.Count() > 1) {
                return Side.Undefined;
             }
@@ -160,11 +161,11 @@ namespace mbts.spark {
          }
       }
 
-      IEnumerable<OrderRecordImmutable> _parent_and_child_not_sharing_slot() => _orders.Values.Where(o => !_orders.ContainsKey(o.ParentKey));
+      IEnumerable<OrderRecordImmutable> _parent_and_child_not_sharing_slot() => _orders.Values.Select(op => op.order).Where(o => !_orders.ContainsKey(o.ParentKey));
 
       public IReadOnlyList<string> text_lines { get; private set; }
 
-      public bool has_parent => _orders.Values.Any( order => order.has_parent());
+      public bool has_parent => _orders.Values.Any( op => op.order.has_parent());
       public bool has_phantom_price => !double.IsNaN(_phantom_price);
 
       public ICommand detach_command => new DelegateCommand(() => _execute_command(_crack_order), () => !has_parent);
@@ -191,16 +192,16 @@ namespace mbts.spark {
       // the setter is public so that the LadderVM can set this field
       public CustomOrderDefinition.CustomOneofCase parent_algo_type { get => _parent_algo_type.Value; set => _parent_algo_type.Value = value; }
 
-      public IEnumerable<OrderRecordImmutable> orders => _orders.Values;
+      public IEnumerable<OrderRecordImmutable> orders => _orders.Values.Select(o => o.order);
 
       void _cancel_order() {
-         foreach (var order in _orders.Values) {
-            model.order_gateway.cancel_order(order).ignore();
+         foreach (var op in _orders.Values) {
+            model.order_gateway.cancel_order(op.order).ignore();
          }
       }
 
       void _crack_order() {
-         var distinct_parent_keys = _orders.Values.Where(o => o.has_parent()).Select(x => x.ParentKey).Distinct();
+         var distinct_parent_keys = _orders.Values.Where(op => op.order.has_parent()).Select(x => x.order.ParentKey).Distinct();
          model.order_gateway.crack_orders(distinct_parent_keys).ignore();
       }
 
@@ -279,7 +280,7 @@ namespace mbts.spark {
       }
 
       public override string ToString() {
-         return $"Price: {this.price} Order Count: {this.orders.Count()}";
+         return $"Price: {this.price} Order Count: {this.orders.Count()} PIQ: {(this.min_piq.HasValue ? this.min_piq.Value.ToString() : "")}";
       }
    }
 }

src/client/spark/view/Spark/Ladder/LadderSettings.cs

@@ -249,7 +249,8 @@ namespace mbts.spark {
                bid_column = LadderColumnSettings.Default,
                ask_column = LadderColumnSettings.Default,
                price_column = LadderColumnSettings.Default,
-               last_column = LadderColumnSettings.Default;
+               last_column = LadderColumnSettings.Default,
+               piq_column = LadderColumnSettings.Default;
             orders_column.background = BTSColors.LadderSpecial.default_column_background;
             bid_column.foreground = 0xFFFFFF;
             bid_column.background = BTSColors.MarketData.bid;
@@ -257,6 +258,7 @@ namespace mbts.spark {
             ask_column.background = BTSColors.MarketData.ask;
             price_column.background = BTSColors.LadderSpecial.default_column_background;
             last_column.background = 0xFFFFFF;
+            piq_column.background = BTSColors.LadderSpecial.default_column_background;
             return new LadderSettings {
                row_height = 20,
                grid_line_color = 0x535353,
@@ -265,6 +267,7 @@ namespace mbts.spark {
                ask_column = ask_column,
                price_column = price_column,
                last_column = last_column,
+               piq_column = piq_column,
                volume_column = VolumeColumnSettings.Default,
                calculator_buttons = new[] { 1, 5, 10, 25, 50, 100 },
                scroll_bar_visible = false,
@@ -398,7 +401,7 @@ namespace mbts.spark {
       public bool clear_visible { get; set; }
       //
       [ProtoMember(43), DefaultValue(_default_order_entry_panel_width)]
-      public double order_entry_panel_initial_width { get; set; }  
+      public double order_entry_panel_initial_width { get; set; }
 
       [ProtoMember(44), DefaultValue(true)]
       public bool use_defaults { get; set; }
@@ -417,6 +420,13 @@ namespace mbts.spark {
       [ProtoMember(48)]
       public NetChangeTerms net_change { get; set; }
 
+      [ProtoMember(49)]
+      LadderColumnSettings _piq_column;
+      public LadderColumnSettings piq_column {
+         get => _piq_column ?? (_piq_column = LadderSettings.Default.piq_column);
+         set => _piq_column = value;
+      }
+
       public bool Equals(LadderSettings other) =>
          other != null &&
          row_height == other.row_height &&
@@ -426,6 +436,7 @@ namespace mbts.spark {
          ask_column.safe_equals(other.ask_column) &&
          price_column.safe_equals(other.price_column) &&
          last_column.safe_equals(other.last_column) &&
+         piq_column.safe_equals(other.piq_column) &&
          calculator_buttons.safe_equals(other.calculator_buttons, (a, b) => a.SequenceEqual(b)) &&
          scroll_bar_visible == other.scroll_bar_visible &&
          default_order_type == other.default_order_type &&
@@ -476,6 +487,7 @@ namespace mbts.spark {
             EqualityComparer.safe_get_hash_code(ask_column),
             EqualityComparer.safe_get_hash_code(price_column),
             EqualityComparer.safe_get_hash_code(last_column),
+            EqualityComparer.safe_get_hash_code(piq_column),
             EqualityComparer.safe_get_hash_code(calculator_buttons),
             scroll_bar_visible.GetHashCode(),
             default_order_type.GetHashCode(),
@@ -517,12 +529,12 @@ namespace mbts.spark {
 
       public struct WindowSpecifics {
          public readonly double order_entry_panel_width, zoom_factor;
-         public readonly uint orders_column_width, bid_column_width, price_column_width, ask_column_width, last_column_width, volume_column_width;
+         public readonly uint orders_column_width, bid_column_width, price_column_width, ask_column_width, last_column_width, volume_column_width, piq_column_width;
 
          public WindowSpecifics(LadderSettings s)
-            : this(s.order_entry_panel_initial_width, s.orders_column.width, s.bid_column.width, s.price_column.width, s.ask_column.width, s.last_column.width, s.volume_column.general.width, s.zoom_factor) { }
+            : this(s.order_entry_panel_initial_width, s.orders_column.width, s.bid_column.width, s.price_column.width, s.ask_column.width, s.last_column.width, s.volume_column.general.width, s.zoom_factor, s.piq_column.width) { }
 
-         public WindowSpecifics(double order_entry_panel_width, uint orders_column_width, uint bid_column_width, uint price_column_width, uint ask_column_width, uint last_column_width, uint volume_column_width, double zoom_factor) {
+         public WindowSpecifics(double order_entry_panel_width, uint orders_column_width, uint bid_column_width, uint price_column_width, uint ask_column_width, uint last_column_width, uint volume_column_width, double zoom_factor, uint piq_column_width) {
             this.order_entry_panel_width = order_entry_panel_width;
             this.orders_column_width = orders_column_width;
             this.bid_column_width = bid_column_width;
@@ -531,6 +543,7 @@ namespace mbts.spark {
             this.last_column_width = last_column_width;
             this.volume_column_width = volume_column_width;
             this.zoom_factor = zoom_factor;
+            this.piq_column_width = piq_column_width;
          }
       }
    }
@@ -543,6 +556,7 @@ namespace mbts.spark {
          yield return settings.ask_column;
          yield return settings.volume_column.general;
          yield return settings.last_column;
+         yield return settings.piq_column;
       }
 
       public static IEnumerable<LadderColumnSettings> visible_columns(this LadderSettings settings) =>
@@ -560,6 +574,7 @@ namespace mbts.spark {
          source.price_column.width = specifics.price_column_width;
          source.ask_column.width = specifics.ask_column_width;
          source.last_column.width = specifics.last_column_width;
+         source.piq_column.width = specifics.piq_column_width;
          source.volume_column.general.width = specifics.volume_column_width;
          source.zoom_factor = specifics.zoom_factor;
          return source;

src/client/spark/view/Spark/Ladder/LadderSettingsVM.cs

@@ -29,7 +29,8 @@ namespace mbts.spark {
          _bid = new LadderColumnSettingsVM("Bid"),
          _price = new LadderColumnSettingsVM("Price"),
          _ask = new LadderColumnSettingsVM("Ask"),
-         _last = new LadderColumnSettingsVM("Last", true);
+         _last = new LadderColumnSettingsVM("Last", true),
+         _piq = new LadderColumnSettingsVM("PIQ", true);
       VolumeColumnVM _volume = new VolumeColumnVM();
       LadderMarketBreakLineVM
          _bid_mkt_line = new LadderMarketBreakLineVM("Bid"),
@@ -82,6 +83,7 @@ namespace mbts.spark {
          _price.font_size_changed += on_font_size_changed;
          _ask.font_size_changed += on_font_size_changed;
          _last.font_size_changed += on_font_size_changed;
+         _piq.font_size_changed += on_font_size_changed;
 
          void on_font_size_changed(double oldval, double newval) => _row_height.refresh();
       }
@@ -120,6 +122,7 @@ namespace mbts.spark {
             if (!is_synthetic) {
                yield return _volume.general;
                yield return _last;
+               yield return _piq;
             }
          }
       }
@@ -130,6 +133,7 @@ namespace mbts.spark {
       public LadderColumnSettingsVM ask_column => _ask;
       public LadderColumnSettingsVM volume_column => _volume.general;
       public LadderColumnSettingsVM last_column => _last;
+      public LadderColumnSettingsVM piq_column => _piq;
       public VolumeColumnVM volume => _volume;
       public LadderMarketBreakLineVM session_range_fill => _session_range_fill;
       public ObservableEditableCollection<IMutableValueWrapperVM<uint>> size_presets { get; }
@@ -209,6 +213,7 @@ namespace mbts.spark {
          _price.assign_from(_current.price_column);
          _volume.assign_from(_current.volume_column);
          _last.assign_from(_current.last_column);
+         _piq.assign_from(_current.piq_column);
          _bid_mkt_line.assign_from(_current.bid_break_line);
          _ask_mkt_line.assign_from(_current.ask_break_line);
          _mid_mkt_line.assign_from(_current.mid_break_line);
@@ -254,6 +259,7 @@ namespace mbts.spark {
          clone.ask_column = _ask.get_settings();
          clone.volume_column = _volume.get_settings();
          clone.last_column = _last.get_settings();
+         clone.piq_column = _piq.get_settings();
          clone.bid_break_line = _bid_mkt_line.get_settings();
          clone.ask_break_line = _ask_mkt_line.get_settings();
          clone.mid_break_line = _mid_mkt_line.get_settings();

src/client/spark/view/Spark/Ladder/LadderVM.cs

@@ -26,7 +26,7 @@ namespace mbts.spark {
       readonly LadderModel _model;
       readonly Property<double> _high, _low, _change, _open, _settlement, _position, _left_panel_width, _zoom_factor;
       //readonly Property<double> _window_width, _window_height;
-      readonly Property<uint> _orders_column_width, _bid_column_width, _price_column_width, _ask_column_width, _last_column_width, _volume_column_width;
+      readonly Property<uint> _orders_column_width, _bid_column_width, _price_column_width, _ask_column_width, _last_column_width, _volume_column_width, _piq_column_width;
       readonly Property<ulong> _volume;
       readonly Property<bool> _auto_clear, _show_day, _show_high, _show_low, _show_open, _show_settle, _show_volume, _show_account,
          _show_tif, _show_order_type, _show_account_position, _show_size_preset, _show_auto_clear, _show_clear, _show_cancel_buys,
@@ -58,6 +58,7 @@ namespace mbts.spark {
          _price_column_width = backing(nameof(price_column_width), ws.price_column_width);
          _ask_column_width = backing(nameof(ask_column_width), ws.ask_column_width);
          _last_column_width = backing(nameof(last_column_width), ws.last_column_width);
+         _piq_column_width = backing(nameof(piq_column_width), ws.piq_column_width);
          _volume_column_width = backing(nameof(volume_column_width), ws.volume_column_width);
          _left_panel_width = backing(nameof(left_panel_width), ws.order_entry_panel_width);
          _zoom_factor = backing(nameof(zoom_factor), ws.zoom_factor);
@@ -132,7 +133,7 @@ namespace mbts.spark {
             account_positions,
             size_calculator = new CalculatorLogic(0, _settings_manager.subscribe_current().Select(s => s.calculator_buttons)),
             bus.subscribe(SparkTags.LogActivity).OnDispatcher().Subscribe(status_update),
-            model.order_gateway.working_orders.OnDispatcher().Subscribe(working => {
+            model.order_gateway.working_orders_with_piq.OnDispatcher().Subscribe(working => {
                // Here we manage N orders to 1 price slot in the ladder
 
                Dictionary<double, HashSet<uint>> orders_used_in_this_update = new Dictionary<double, HashSet<uint>>(_equality);
@@ -157,9 +158,9 @@ namespace mbts.spark {
                   }
 
                   if (_slot_price_vs_orders.TryGetValue(slot_price_target, out LadderOrderVM slot_vm)) {
-                     slot_vm.add_order(wo);
+                     slot_vm.add_order((wo, osec.piq));
                   } else {
-                     slot_vm = new LadderOrderVM(wo, slot_price_target, this, _bus, _settings_manager.subscribe_current().Select(s => s.badge_style));
+                     slot_vm = new LadderOrderVM((wo, osec.piq), slot_price_target, this, _bus, _settings_manager.subscribe_current().Select(s => s.badge_style));
                      _slot_price_vs_orders[slot_price_target] = slot_vm;
                   }
 
@@ -232,6 +233,10 @@ namespace mbts.spark {
       public event Action on_tick;
 
       public MDConsumerHandle md => _model.md;
+      // we return only entries who have piqs and whose piqs are not uint.Max
+      // uint.Max happens when the server is restarted but the client is still up
+      // maybe long term we don't show this data while the exchange connection is down
+      public IEnumerable<(double, uint)> price_vs_piq => _slot_price_vs_orders.Values.Where(result => result.min_piq.HasValue && result.min_piq.Value != uint.MaxValue).Select(o => (o.price, o.min_piq.Value));  
       public bool is_synthetic => _model.security.ContractCase == Security.ContractOneofCase.Synthetic;
 
       Property<string> _status;
@@ -306,13 +311,14 @@ namespace mbts.spark {
       public uint price_column_width { get => _price_column_width.Value; set => _price_column_width.set(value); }
       public uint ask_column_width { get => _ask_column_width.Value; set => _ask_column_width.set(value); }
       public uint last_column_width { get => _last_column_width.Value; set => _last_column_width.set(value); }
+      public uint piq_column_width { get => _piq_column_width.Value; set => _piq_column_width.set(value); }
       public uint volume_column_width { get => _volume_column_width.Value; set => _volume_column_width.set(value); }
       internal LadderModel model => _model;
       internal IObservable<LadderSettings> settings => _settings_manager.subscribe_current();
 
       public LadderSettings get_save_state() {
          var state = _settings_manager.current_settings();
-         state = state.with_window_specifics(new LadderSettings.WindowSpecifics(left_panel_width, orders_column_width, bid_column_width, price_column_width, ask_column_width, last_column_width, volume_column_width, zoom_factor));
+         state = state.with_window_specifics(new LadderSettings.WindowSpecifics(left_panel_width, orders_column_width, bid_column_width, price_column_width, ask_column_width, last_column_width, volume_column_width, zoom_factor, piq_column_width));
          state.use_defaults = _settings_manager.instance_settings == null;
          return state;
       }

src/client/spark/view/Spark/Ladder/LadderView.xaml

@@ -541,6 +541,7 @@
         price_column_width="{Binding price_column_width, Mode=TwoWay}"
         ask_column_width="{Binding ask_column_width, Mode=TwoWay}"
         LastColumnWidth="{Binding last_column_width, Mode=TwoWay}"
+        PIQColumnWidth="{Binding piq_column_width, Mode=TwoWay}"
         VolumeColumnWidth="{Binding volume_column_width, Mode=TwoWay}"
       >
         <spark:LadderGridHost.Resources>

src/client/spark/view/Spark/Ladder/LadderView.xaml.cs

@@ -141,6 +141,10 @@ namespace mbts.spark {
                   header = "Last";
                   width_prop = nameof(LadderVM.last_column_width);
                   break;
+               case LadderGridHost.ColumnType.PIQ:
+                  header = "PIQ";
+                  width_prop = nameof(LadderVM.piq_column_width);
+                  break;
                default: throw ExceptionBuilder.Argument.unexpected_case(type, nameof(type));
             }
             return _create(host, header, $"DataContext.{width_prop}");

src/client/spark/view/Spark/Ladder/LaderRender.cs

@@ -277,6 +277,30 @@ namespace mbts.spark {
          }
       }
 
+      // this is basically a copy of redner_sizes up above
+      internal static void render_piqs(DrawingVisual dv, GlyphContext glyph_context, int origin_index, double[] prices, (double price, uint piq)[] piqs, double row_height, ContractSpecificationImmutable spec, LadderColumnSettings settings) {
+         using (var dc = dv.RenderOpen()) {
+            using (_draw_with_clip(dc, settings, row_height, prices.Length)) {
+               if (!prices.within_range(origin_index) || piqs.is_null_or_empty())
+                  return;
+               var foreground = settings.foreground.rgb_brush();
+               int curr_index = origin_index;
+               double curr_price = prices[curr_index];
+               for (int i = 0; i < piqs.Length; i++) {
+                  double price = piqs[i].price;
+                  // prices are in descending order, so get diff from curr_price - price
+                  int tick_diff = spec.tickdiff_unchecked(curr_price, price);
+                  curr_index += tick_diff;
+                  curr_price = price;
+                  if (curr_index >= 0 && curr_index < prices.Length) {
+                     var gr = format_text(piqs[i].piq.ToString(), curr_index, row_height, settings, glyph_context);
+                     dc.DrawGlyphRun(foreground, gr);
+                  } else break;
+               }
+            }
+         }
+      }
+
       public static void render_mkt_break(DrawingVisual dv, int index, double row_height, uint color) =>
          render_single_grid_line(dv, index, row_height, color, 0, SystemParameters.FullPrimaryScreenWidth);
 

src/client/spark_api/api/Channels/QuickBusClientChannel.cs

@@ -110,19 +110,21 @@ namespace bts.api.core {
             the_update.Clear();
             try {
                if (event_type == ApiNativeMethods.EventType.TABLE_UPDATED) {
-                  foreach (var change in changes) {
-                     switch (change.type) {
-                        case ApiNativeMethods.ChangeType.REMOVE:
-                           the_update.removed_keys.Add(change.key);
-                           break;
-                        case ApiNativeMethods.ChangeType.REPLACE:
-                           the_update.changed_items.Add(converter(parser.ParseFrom(new UnmanagedMemoryStream(change.buffer, change.buffer_len))));
-                           the_update.changed_keys.Add(change.key);
-                           break;
-                        case ApiNativeMethods.ChangeType.CREATE:
-                           the_update.added_items.Add(converter(parser.ParseFrom(new UnmanagedMemoryStream(change.buffer, change.buffer_len))));
-                           the_update.added_keys.Add(change.key);
-                           break;
+                  if (changes != null) {
+                     foreach (var change in changes) {
+                        switch (change.type) {
+                           case ApiNativeMethods.ChangeType.REMOVE:
+                              the_update.removed_keys.Add(change.key);
+                              break;
+                           case ApiNativeMethods.ChangeType.REPLACE:
+                              the_update.changed_items.Add(converter(parser.ParseFrom(new UnmanagedMemoryStream(change.buffer, change.buffer_len))));
+                              the_update.changed_keys.Add(change.key);
+                              break;
+                           case ApiNativeMethods.ChangeType.CREATE:
+                              the_update.added_items.Add(converter(parser.ParseFrom(new UnmanagedMemoryStream(change.buffer, change.buffer_len))));
+                              the_update.added_keys.Add(change.key);
+                              break;
+                        }
                      }
                   }
                   observer.OnNext(the_update);

src/client/spark_core/config/config.cs

@@ -51,6 +51,12 @@ namespace mbts.config
          reset();
          initialize();
       }
+      
+      public static bool try_initialize(out string error)
+      {
+         reset();
+         return initialize(out error);
+      }
 
       ////////////
       /// private

src/client/spark_core/utils/OrderUtilities.cs

@@ -0,0 +1,19 @@
+namespace bts.api.orderman {
+
+   public static class OrderUtilities {
+      public static NativeOrderStateImmutable? native_order_state(this OrderStateImmutable order) {
+         switch(order.CustomCase) {
+            case OrderState.CustomOneofCase.Market:
+               return order.Market.NativeState;
+            case OrderState.CustomOneofCase.Limit:
+               return order.Limit.NativeState;
+            case OrderState.CustomOneofCase.Stop:
+               return order.Stop.NativeState;
+            case OrderState.CustomOneofCase.StopLimit:
+               return order.StopLimit.NativeState;
+            default:
+            return null;
+         }
+      }
+   }
+}

src/client/sparkcmd/Cmd/ConfigCheckCommand.cs

@@ -1,14 +1,19 @@
-using ManyConsole;
 using System.Linq;
+using System;
+using System.Collections.Generic;
+using CommandLine;
 
 namespace bts {
-   public class Configcheck : ConsoleCommand {
-      public Configcheck() {
-         IsCommand("configcheck", "Ye olde Configcheck");
-         HasAdditionalArguments(null, " [Keys to check]");
-      }
-      public override int Run(string[] args) {
-         mbts.config.Configuration.configcheck(args.Length, args.ToArray());
+
+   [Verb("configcheck", HelpText="Ye olde Configcheck")]
+   public class ConfigcheckOptions {
+      [Value(0, MetaName="keys", HelpText="Keys to print")]
+      public IEnumerable<string> keys { get; set; }
+   }
+   public class Configcheck {
+      public static int Run(ConfigcheckOptions options) {
+         var keys = options.keys.ToArray();
+         mbts.config.Configuration.configcheck(keys.Length, keys);
          return 0;
       }
    }

src/client/sparkcmd/Cmd/ConsoleCommandEx.cs

@@ -1,20 +0,0 @@
-using System;
-using System.Collections.Generic;
-using System.Linq;
-using System.Text;
-using System.Threading.Tasks;
-
-namespace ManyConsole {
-   public static class ConsoleCommandEx {
-      public static string get_argument(this ConsoleCommand _, string arg_name) {
-         var arg = Environment.GetCommandLineArgs().FirstOrDefault(str => str.StartsWith("--"+arg_name));
-         if (arg != null) {
-            var parts = arg.Split(new[] { '=' }, 2);
-            if (parts.Length > 1) {
-               return parts[1];
-            }
-         }
-         return "";
-      }
-   }
-}

src/client/sparkcmd/Cmd/ScriptClientCommand.cs

@@ -6,32 +6,33 @@ using System.Reflection;
 using System.Threading.Tasks;
 using System.Reactive.Linq;
 using System.Reactive;
-using ManyConsole;
 using System.Linq;
+using CommandLine;
 
 namespace sparkcmd {
-   public class ScriptClientCommand : ConsoleCommand {
-      bool _server = false;
-      string _host;
-      string _logfile;
+   [Verb("script", HelpText="Remote control for shell diagnostic host")]
+   public class ScriptClientOptions {
+      [Option('s', "server", HelpText="Run a script server for testing.")]
+      public bool server { get; set; }
 
-      public ScriptClientCommand() {
-         IsCommand("script", "Remote control for shell diagnostic host");
-         HasOption("s|server", "Run a script server for testing.", x => _server = true);
-         HasOption("h|host", "Connect to {host}", h => _host = h);
-         HasOption("f|fromfile", "Read commands from file instead of stdin", f => _logfile = f);
-      }
-      public override int Run(string[] args) {
-         if (_server) {
+      [Option('h', "host", HelpText="Connect to {host}")]
+      public string host { get; set; }
+
+      [Option('f', "fromfile", HelpText="Read commands from file instead of stdin")]
+      public string file { get; set; }
+
+   }
+   public class ScriptClientCommand {
+      public static int Run(ScriptClientOptions o) {
+         if (o.server) {
             ScriptClient.runserver();
          }
          else {
-            var host = string.IsNullOrEmpty(_host) ? args.First() : _host;
-            if (string.IsNullOrEmpty(_logfile) && File.Exists(_logfile)) {
-               ScriptClient.start(host, File.OpenRead(_logfile));
+            if (string.IsNullOrEmpty(o.file) && File.Exists(o.file)) {
+               ScriptClient.start(o.host, File.OpenRead(o.file));
             }
             else {
-               ScriptClient.start(host, Console.OpenStandardInput());
+               ScriptClient.start(o.host, Console.OpenStandardInput());
             }
          }
          return 0;

src/client/sparkcmd/Cmd/ServiceTail.cs

@@ -1,6 +1,6 @@
 using bts.api.core;
 using bts.test;
-using ManyConsole;
+using CommandLine;
 using System;
 using System.Collections.Generic;
 using System.Linq;
@@ -8,26 +8,44 @@ using System.Text;
 using System.Threading.Tasks;
 
 namespace bts {
-   class TailService : ConsoleCommand {
+
+   enum PrintFormat {
+      ToString,
+      Json
+   }
+   [Verb("tail", HelpText="'console dsqa'")]
+   class TailServiceOptions {
+      [Option('m', "method", Required=true, HelpText = "'[ServiceName].[Method]' to call")]
+      public string method { get;set; }
+
+      [Option('t', "time", HelpText="Include timestamp")]
+      public bool time { get; set; }
+
+      [Option('p', "path", HelpText="Path inside the row to use for display. Ex: 'Id' or 'Name' ")]
+      public string path { get; set; }
+
+      [Option('a', "all", HelpText="Include 'all' rows on every print")]
+      public bool all { get; set; }
+
+      [Option('j', "json", HelpText="Format rows as json")]
+      public bool json { get; set; }
+   }
+   class TailService {
       static bool _all = false;
+      static PrintFormat _format = PrintFormat.ToString;
+
       static bool _timestamp = false;
-      public TailService() {
-         void ignore(string _) { }
-         IsCommand("tailservice", "Tail updates to named service api. Ex: tail --method=SyntheticSecurityService.metadata");
-         HasRequiredOption("method", "Service.method to tail", ignore);
-         HasOption("time", "Include timestamp", _ => _timestamp = true);
-         // The following are not implemented. If you need them get to work.
-         HasOption("path", "Path inside the row to use for display. Ex: 'Id' or 'Name' ", ignore);
-         HasOption("all", "Invlude 'all' rows on every print", _ => _all = true);
-         HasAdditionalArguments(null, " [nothing]");
-      }
-      public override int Run(string[] remainingArguments) {
-         var arg = this.get_argument("method");
-         var split = arg.Split('.');
+      public static int Run(TailServiceOptions o) {
+         var split = o.method.Split('.');
          if (split.Length != 2) {
-            Console.WriteLine("Try --method=[ServiceName].[Method]");
+            Console.WriteLine("Try --method [ServiceName].[Method]");
             return -1;
          }
+
+         if (o.json) {
+            _format = PrintFormat.Json;
+         }
+
          tail(split[0], split[1]);
          return 0;
       }
@@ -46,7 +64,17 @@ namespace bts {
                Console.Write(" ");
             }
             if (_all) { }
-            Console.WriteLine(value);
+
+            switch(_format) {
+               case PrintFormat.Json:
+                  dynamic dv = value;
+                  var json = Newtonsoft.Json.JsonConvert.SerializeObject(new { dv.added, dv.removed, dv.changed });
+                  Console.WriteLine(json);
+               break;
+               case PrintFormat.ToString:
+                  Console.WriteLine(value);
+               break;
+            }
          }
       }
       private static void tail(string service_name, string method_name) {

src/client/sparkcmd/Cmd/ServiceTestCommand.cs

@@ -2,35 +2,74 @@ using bts.api;
 using bts.api.core;
 using bts.api.synthetic;
 using bts.test;
-using ManyConsole;
+using bts.md;
+using CommandLine;
 using System;
 using System.Threading;
 
 namespace bts {
-   public class ServiceTest : ConsoleCommand {
-      bool _synthetic;
-      bool _user;
-      bool _admin;
 
-      public ServiceTest() {
-         IsCommand("service", "Put code here for testing during service development");
-         HasOption("synthetic", "Run synthetic test", _ => _synthetic = true);
-         HasOption("user", "user test", _ => _user = true);
-         HasOption("admin", "admin test", _ => _admin = true);
-      }
-      public override int Run(string[] remainingArguments) {
-         if (_synthetic) {
+   public class ServiceTestOptions {
+      [Option(nameof(synthetic))]
+      public bool synthetic { get; set; }
+      [Option(nameof(user))]
+      public bool user { get; set; }
+      [Option(nameof(admin))]
+      public bool admin { get; set; }
+
+      [Option(nameof(mbo))]
+      public bool mbo { get; set; }
+
+      [Option("key")]
+      public UInt32 key { get; set; }
+
+      [Option("side")]
+      public api.Side side { get; set; }
+   }
+
+   public class ServiceTest {
+      public static int Run(ServiceTestOptions o) {
+         if (o.synthetic) {
             synthetic_test();
          }
-         if (_user) {
+         if (o.user) {
             user_test();
          }
-         if (_admin) {
+         if (o.admin) {
             admin_test();
          }
+         if (o.mbo) {
+            mbo_test(o);
+         }
          return 0;
       }
 
+      static string try_get(string[] args, int index) {
+         if (args.Length < index) {
+            return args[index];
+         }
+         return "";
+      }
+
+      private static void mbo_test(ServiceTestOptions o) {
+         var key = o.key;
+         var side = o.side;
+         using (var ctx = new ServiceContext()) {
+            var mbo = ctx.locate<MBOService>();
+            var data = mbo.get_orders_for_security(new MBOSecurityQuery { SecurityId = key });
+
+            Console.WriteLine($"Order data for {data.SecurityId}");
+            var vector = side == api.Side.Buy ? data.Bid : data.Ask;
+            var price = 0.0;
+            foreach(var e in vector) {
+               if (e.Price != price) {
+                  price = e.Price;
+                  Console.WriteLine("================");
+               }
+               Console.WriteLine(e);
+            }
+         }
+      }
 
       private static void synthetic_test() {
          using (var ctx = new ServiceContext()) {

src/client/sparkcmd/Cmd/SettingsManagerCommand.cs

@@ -1,14 +1,13 @@
-using ManyConsole;
 using System.Reflection;
+using CommandLine;
 
 namespace bts {
 
-   public class SettinsManagerCommand : ConsoleCommand {
-      public SettinsManagerCommand() {
-         IsCommand("SettingsManager", "Copy workspaces around");
-      }
+   [Verb("settingsmanager", HelpText="Copy workspaces around")]
+   public class SettingsManagerOptions { }
 
-      public override int Run(string[] remainingArguments) {
+   public class SettingsManagerCommand {
+      public static int Run(SettingsManagerOptions _) {
          var asm = Assembly.Load("spark");
          var type = asm.GetType("bts.settings.SettingsManagerTool");
          var run = type.GetMethod("run", BindingFlags.Static | BindingFlags.Public);

src/client/sparkcmd/Cmd/TestCommand.cs

@@ -1,6 +1,5 @@
 
 using bts.test;
-using ManyConsole;
 using System;
 using mbts;
 using System.Collections.Generic;
@@ -8,64 +7,73 @@ using bts.api.test;
 using System.Threading.Tasks;
 using mbts.config;
 using System.Linq;
+using CommandLine;
+using Newtonsoft.Json;
 
 namespace sparkcmd {
-   public class TestCommand : ConsoleCommand {
-      uint _timeout = 0;
-      uint _parallel = 1;
-      bool _list;
 
-      TestType test_type = TestType.Unit;
+   [Verb("test", HelpText="Run Tests")]
+   public class TestCommandOptions {
 
-      bool self_contained = false;
+      [Option('t', "timeout", HelpText="Timeout unit tets", Default=0u)]
+      public uint timeout { get; set; }
+      [Option('l', "list", HelpText="List tests")]
+      public bool list { get; set; }
+      [Option('i', "insular", HelpText="Tests run sparkd and use their own config (if specified)")]
+      public bool self_contained { get; set; }
 
-      public TestCommand() {
-         IsCommand("test", "Run managed unit tests");
-         // i comes thorugh as "t" which seems daft.
-         HasOption("t|timeout", "Timeout unit tests", i => _timeout = 10);
-         HasOption<uint>("n|parallel", "How many tests to run in parallel", i => _parallel = i);
-         HasOption("l|list", "List all tests", _ => _list = true);
-         HasOption("unit","Run unit tests", _ => test_type = TestType.Unit);
-         HasOption("system","Run system tests", _ => test_type = TestType.System);
-         HasOption("shell","Run shell tests", _ => test_type = TestType.Shell);
-         HasOption("i|insular|selfcontained", "Tests run sparkd and use their own config (if specified)", _ => self_contained = true);
-         HasAdditionalArguments(null, " [Test List]");
+      [Option("type", HelpText ="Run all tests of type [Unit|System|Shell]")]
+      public TestType type { get; set; }
+
+
+      [Value(1, MetaName="Tests", HelpText="Test list")]
+      public IEnumerable<string> tests { get; set; }
+
+   }
+   public static class TestCommand {
+      static TestType test_type(string t) {
+         switch(t.ToLower()) {
+            case "unit": return TestType.Unit;
+            case "system": return TestType.System;
+            case "shell": return TestType.Shell;
+         }
+         return TestType.Unit;
       }
 
-      public override int Run(string[] tests) {
-         if (_list) {
-            list_tests();
+      public static int Run(TestCommandOptions o) {
+         Console.WriteLine(JsonConvert.SerializeObject(o));
+
+         if (o.list) {
+            list_tests((o.type));
             return 0;
          }
 
-         //Configuration.initialize_for_test();
-
-         if (_timeout == 0 && tests.Length == 1 && !self_contained) {
+         if (o.timeout == 0 && o.tests.Count() == 1 && !o.self_contained) {
             //System.Threading.Thread.Sleep(5000);
             using (var trr = new TestResultReceiver(true)) {
-               InProcessTestRunner.run_test_synchronously(tests[0], Enumerator, trr.get_reporter());
+               InProcessTestRunner.run_test_synchronously(o.tests.First(), Enumerator, trr.get_reporter());
                trr.print_summary();
                return trr.ErrorCount;
             }
          }
-         else if (tests.Length > 0) {
-            return run_tests(Enumerator.match(tests));
+         else if (o.tests.Count() > 0) {
+            return run_tests(o, Enumerator.match(o.tests));
          }
          else {
-            return run_tests(Enumerator.get_of_type(test_type));
+            return run_tests(o, Enumerator.get_of_type(o.type));
          }
       }
 
-      Lazy<TestEnumerator> _enumerator = new Lazy<TestEnumerator>(() => new TestEnumerator("spark_test"));
-      TestEnumerator Enumerator => _enumerator.Value;
+      static Lazy<TestEnumerator> _enumerator = new Lazy<TestEnumerator>(() => new TestEnumerator("spark_test"));
+      static TestEnumerator Enumerator => _enumerator.Value;
 
-      int run_tests(IEnumerable<TestMethod> tests) {
+      static int run_tests(TestCommandOptions o, IEnumerable<TestMethod> tests) {
          var runner = new TestRunner();
          // waits for all child processes to exit or time out
-         return runner.run_tests(tests, _timeout, self_contained);
+         return runner.run_tests(tests, o.timeout, o.self_contained);
       }
 
-      void list_tests() {
+      static void list_tests(TestType test_type) {
          Console.WriteLine("All tests: ");
          foreach (var t in Enumerator.get_groups_of_type(test_type)) {
             ConsoleEx.WriteLine(t.name, ConsoleColor.Green);

src/client/sparkcmd/Program.cs

@@ -2,16 +2,58 @@
 using bts.api.core;
 using bts.api.synthetic;
 using bts.api.test;
-using ManyConsole;
 using System;
 using System.Linq;
+using CommandLine;
+using bts;
+using System.Collections.Generic;
+using System.Linq.Expressions;
+using System.Reflection;
 
 namespace sparkcmd {
    class Program {
+
+      static object option<T>(Func<T, int> run) { return run; }
+      static Type argType(object arg) {
+         return arg.GetType().GetGenericArguments().First();
+      }
+      static int Run(Func<IEnumerable<Error>, int> on_error, params Type[] runners) {
+         var run_methods =
+            runners
+            .Select(r => r.GetMethod("Run", BindingFlags.Static | BindingFlags.Public))
+            .ToList();
+         var args =
+            run_methods
+            .Select(m => m.GetParameters().First().GetType());
+
+         var parser = Parser.Default.GetType().GetMethod("ParseArguments", new[] { typeof(IEnumerable<string>) });
+
+         //var types = args.Select(argType).ToArray();
+         //var parse = CommandLine.Parser.Default.GetType().GetMethod("ParseArguments", BindingFlags.Public).MakeGenericMethod(types);
+         return 3;
+      }
       static int Main(string[] args) {
-         mbts.config.Configuration.initialize();
-         var commands = ConsoleCommandDispatcher.FindCommandsInSameAssemblyAs(typeof(Program));
-         return ConsoleCommandDispatcher.DispatchCommand(commands, args, Console.Out);
+         mbts.utils.BTSNativeMethods.provide_command_line(args, args.Length);
+
+         //return Parser.Default
+         //.ParseArguments<ConfigcheckOptions,ScriptClientOptions,TailServiceOptions,ServiceTestOptions,SettingsManagerOptions,TestCommandOptions>(args)
+         //.MapResult<ConfigcheckOptions,ScriptClientOptions,TailServiceOptions,ServiceTestOptions,SettingsManagerOptions,TestCommandOptions,int>(
+         //   Configcheck.Run,
+         //   ScriptClientCommand.Run,
+         //   TailService.Run,
+         //   ServiceTest.Run,
+         //   SettingsManagerCommand.Run,
+         //   TestCommand.Run,
+         //   errors => 1);
+
+         return Run(
+            errors => 1,
+            typeof(Configcheck),
+            typeof(ScriptClientCommand),
+            typeof(TailService),
+            typeof(ServiceTest),
+            typeof(SettingsManagerCommand),
+            typeof(TestCommand));
       }
    }
 }

src/client/sparkcmd/sparkcmd.csproj

@@ -1,27 +1,30 @@
-<Project Sdk="Microsoft.NET.Sdk">
-
-  <PropertyGroup>
-    <OutputType>Exe</OutputType>
-    <TargetFrameworks>net471;netcoreapp2.1</TargetFrameworks>
-    <RuntimeIdentifiers>win7-x64;ubuntu.16.04-x64</RuntimeIdentifiers>
-    <Authors>Blue Trading Systems</Authors>
-    <Company>Blue Trading Systems</Company>
-    <Product>Spark</Product>
-    <Platform>x64</Platform>
-    <PlatformTarget>x64</PlatformTarget>
-    <Version>1.0.0</Version>
-    <LangVersion>7.2</LangVersion>
-  </PropertyGroup>
-
-  <ItemGroup>
-    <ProjectReference Include="../spark_api/spark_api.csproj" />
-    <!--<ProjectReference Include="..\spark\Spark.csproj" />-->
-    <ProjectReference Include="..\spark_core\spark_core.csproj" />
-    <ProjectReference Include="..\spark_test\spark_test.csproj" />
-    <!--<Proto3 Include="**/*.proto" />-->
-  </ItemGroup>
-
-  <ItemGroup>
-    <PackageReference Include="ManyConsole" Version="1.0.0.4" />
-  </ItemGroup>  
-</Project>
+<Project Sdk="Microsoft.NET.Sdk">
+
+  <PropertyGroup>
+    <OutputType>Exe</OutputType>
+    <TargetFrameworks>net471;netcoreapp2.1</TargetFrameworks>
+    <RuntimeIdentifiers>win7-x64;ubuntu.16.04-x64</RuntimeIdentifiers>
+    <Authors>Blue Trading Systems</Authors>
+    <Company>Blue Trading Systems</Company>
+    <Product>Spark</Product>
+    <Platform>x64</Platform>
+    <PlatformTarget>x64</PlatformTarget>
+    <Version>1.0.0</Version>
+    <LangVersion>7.2</LangVersion>
+  </PropertyGroup>
+
+  <ItemGroup>
+    <ProjectReference Include="../spark_api/spark_api.csproj" />
+    <!--<ProjectReference Include="..\spark\Spark.csproj" />-->
+    <ProjectReference Include="..\spark_core\spark_core.csproj" />
+    <ProjectReference Include="..\spark_test\spark_test.csproj" />
+    <!--<Proto3 Include="**/*.proto" />-->
+  </ItemGroup>
+
+  <ItemGroup>
+    <PackageReference Include="CommandLineParser" Version="2.4.3" />
+    <PackageReference Include="Microsoft.CSharp" Version="4.5.0" />
+    <PackageReference Include="Newtonsoft.Json" Version="10.0.3" />
+  </ItemGroup>
+
+</Project>

src/lib/CMakeDeps_win.cmake

@@ -1 +1 @@
-set(sources api/apicore.proto;api/apigateway.proto;api/apioptions.proto;api/core/ApiClientInterop.cpp;api/core/ApiServerInterop.cpp;api/public/ApiServer.cpp;api/public/ClientChannel.cpp;api/public/QuickbusClientChannel.cpp;api/public/ServerChannel.cpp;api/public/ServiceContext.cpp;api/service_helpers/AdminServiceUtils.cpp;api/service_helpers/AlgoServiceUtils.cpp;api/services/AdminService.proto;api/services/AlgoDefinitionService.proto;api/services/AlgoService.proto;api/services/ApiUnitTestService.proto;api/services/AuthService.proto;api/services/ErrorReportService.proto;api/services/HelloService.proto;api/services/MarketDataAuxiliaryService.proto;api/services/MarketDataRelay.proto;api/services/OrderService.proto;api/services/OutOfProcessURLListener.proto;api/services/PersistenceService.proto;api/services/PositionService.proto;api/services/ShellDiagnosticService.proto;api/services/ShellLatencyService.proto;api/services/SyntheticSecurityService.proto;api/test/ApiTests.cpp;api/test/HelloServiceServer.cpp;config/ConfigCheck.cpp;config/ConfigDownloader.cpp;config/ConfigErrorCollector.cpp;config/ConfigLog.cpp;config/ConfigPropertyEvaluator.cpp;config/ConfigValueQueue.cpp;config/ConfigurationException.cpp;config/ConfigurationState.cpp;config/PreprocessInputStream.cpp;config/config.cpp;config/defaults/MarketData.conf;config/defaults/Ports.conf;config/defaults/Symbols.conf;config/defaults/cme_reference_data.conf;config/defaults/edge.conf;config/defaults/jetfix.conf;config/defaults/logging.conf;config/defaults/orderman_cfg.conf;config/defaults/orders_fills_cleanup.conf;config/defaults/spreader.conf;config/defaults/testkeys.conf;config/defaults/threads.conf;config/defaults/toggles.conf;config/extensions/extensions.proto;config/invalid/invalid.proto;config/protos/ExchangeSimCfg.proto;config/protos/MarketData.proto;config/protos/Ports.proto;config/protos/Symbols.proto;config/protos/UI.proto;config/protos/cme_reference_data.proto;config/protos/jetfix.proto;config/protos/logging.proto;config/protos/mercury_cfg.proto;config/protos/orderman_cfg.proto;config/protos/orders_fills_cleanup.proto;config/protos/perflog_cfg.proto;config/protos/quickbus_cfg.proto;config/protos/spreader.proto;config/protos/system.proto;config/protos/testkeys.proto;config/protos/threads.proto;config/protos/toggles.proto;config/tests/HTTPServer.cpp;config/tests/config_ref_tests.cpp;config/tests/config_tests.cpp;config/text_format/text_format.cpp;domain_model/Security.cpp;domain_model/Security.proto;domain_model/SecurityTests.cpp;domain_model/tests/domain_tests.cpp;expression/BooleanGrammar.cpp;expression/ExpressionEvaluator.cpp;expression/ParserExceptions.cpp;expression/Value.cpp;expression/tests/expression_parser_tests.cpp;logging/LoggerHub.cpp;logging/Sink.cpp;logging/logging.cpp;logging/tests/logging_tests.cpp;md/Diag/MDPeriodicDumper.cpp;md/Diag/ShmDumper.cpp;md/MDClient.cpp;md/MDConsumer.cpp;md/MDConsumerHandle.cpp;md/MDConsumerInterop.cpp;md/MDProducer.cpp;md/MDProducerHandle.cpp;md/MemorySegmentUtils.cpp;md/PlatformLoader.cpp;md/PriceSize.cpp;md/SecurityCache.cpp;md/SecurityDB.cpp;md/SecurityIDRecorder.cpp;md/SecurityOracle.cpp;md/SettlementRecorder.cpp;md/Shm/FixedSecurity.cpp;md/SyntheticSecurityService.cpp;md/cme/MDP3Utils.cpp;md/common.cpp;md/ice/ICEUtils.cpp;md/protos/co2.proto;md/relay/QueryServer.cpp;md/relay/RelayClient.cpp;md/relay/RelayRecorder.cpp;md/test/Co2ManagedTestHelper.cpp;md/test/Co2TestInstrument.cpp;md/test/md2_tests.cpp;md/test/synthetic_book_merge_tests.cpp;os/os.cpp;os/tests/os_tests.cpp;quickbus/core/LocalNode.cpp;quickbus/core/Quickbus.cpp;quickbus/core/SubscriptionID.cpp;quickbus/core/TableID.cpp;quickbus/message_model/Update.cpp;quickbus/message_model/merge_updates.cpp;quickbus/network/QBConnection.cpp;quickbus/network/QBConnectionManager.cpp;quickbus/network/endpoint_description.cpp;quickbus/network/quickbus_msgs.proto;quickbus/pubsub/TablePublisher.cpp;quickbus/pubsub/TableSubscriber.cpp;quickbus/pubsub/tests/qb_pubsub_tests.cpp;quickbus/routing/INode.cpp;quickbus/routing/NodeID.cpp;quickbus/routing/Router.cpp;quickbus/rpc/RPCClient.cpp;quickbus/rpc/RPCServer.cpp;quickbus/rpc/RPCSessionID.cpp;quickbus/rpc/rpc_utils.cpp;quickbus/rpc/tests/qb_rpc_tests.cpp;quickbus/tests/quickbus_tests.cpp;sparkproto/SparkProtos.proto;testing/catch_test_runner.cpp;utils/AuthToken.cpp;utils/Dispatcher.cpp;utils/GrowableSPSCQueue.cpp;utils/HTTPStream.cpp;utils/PBUtils.cpp;utils/Puid.cpp;utils/SecureLogon.cpp;utils/SharedMemory.cpp;utils/Sqlite_utils.cpp;utils/Uuid.cpp;utils/datetime.cpp;utils/download.cpp;utils/file_utils.cpp;utils/init_spark_lib.cpp;utils/is_unittest_environment.cpp;utils/stats.cpp;utils/tests/AuthTokenTests.cpp;utils/thread_manager.cpp)
+set(sources api/apicore.proto;api/apigateway.proto;api/apioptions.proto;api/core/ApiClientInterop.cpp;api/core/ApiServerInterop.cpp;api/public/ApiServer.cpp;api/public/ClientChannel.cpp;api/public/QuickbusClientChannel.cpp;api/public/ServerChannel.cpp;api/public/ServiceContext.cpp;api/service_helpers/AdminServiceUtils.cpp;api/service_helpers/AlgoServiceUtils.cpp;api/services/AdminService.proto;api/services/AlgoDefinitionService.proto;api/services/AlgoService.proto;api/services/ApiUnitTestService.proto;api/services/AuthService.proto;api/services/ErrorReportService.proto;api/services/HelloService.proto;api/services/MBOService.proto;api/services/MarketDataAuxiliaryService.proto;api/services/MarketDataRelay.proto;api/services/OrderService.proto;api/services/OutOfProcessURLListener.proto;api/services/PersistenceService.proto;api/services/PositionService.proto;api/services/ShellDiagnosticService.proto;api/services/ShellLatencyService.proto;api/services/SyntheticSecurityService.proto;api/test/ApiTests.cpp;api/test/HelloServiceServer.cpp;config/ConfigCheck.cpp;config/ConfigDownloader.cpp;config/ConfigErrorCollector.cpp;config/ConfigLog.cpp;config/ConfigPropertyEvaluator.cpp;config/ConfigValueQueue.cpp;config/ConfigurationException.cpp;config/ConfigurationState.cpp;config/PreprocessInputStream.cpp;config/config.cpp;config/defaults/MarketData.conf;config/defaults/Ports.conf;config/defaults/Symbols.conf;config/defaults/cme_reference_data.conf;config/defaults/edge.conf;config/defaults/jetfix.conf;config/defaults/logging.conf;config/defaults/orderman_cfg.conf;config/defaults/orders_fills_cleanup.conf;config/defaults/spreader.conf;config/defaults/testkeys.conf;config/defaults/threads.conf;config/defaults/toggles.conf;config/extensions/extensions.proto;config/invalid/invalid.proto;config/protos/ExchangeSimCfg.proto;config/protos/MarketData.proto;config/protos/Ports.proto;config/protos/Symbols.proto;config/protos/UI.proto;config/protos/cme_reference_data.proto;config/protos/jetfix.proto;config/protos/logging.proto;config/protos/mercury_cfg.proto;config/protos/orderman_cfg.proto;config/protos/orders_fills_cleanup.proto;config/protos/perflog_cfg.proto;config/protos/quickbus_cfg.proto;config/protos/spreader.proto;config/protos/system.proto;config/protos/testkeys.proto;config/protos/threads.proto;config/protos/toggles.proto;config/tests/HTTPServer.cpp;config/tests/config_ref_tests.cpp;config/tests/config_tests.cpp;config/text_format/text_format.cpp;domain_model/Security.cpp;domain_model/Security.proto;domain_model/SecurityTests.cpp;domain_model/tests/domain_tests.cpp;expression/BooleanGrammar.cpp;expression/ExpressionEvaluator.cpp;expression/ParserExceptions.cpp;expression/Value.cpp;expression/tests/expression_parser_tests.cpp;logging/LoggerHub.cpp;logging/Sink.cpp;logging/logging.cpp;logging/tests/logging_tests.cpp;md/Diag/MDPeriodicDumper.cpp;md/Diag/ShmDumper.cpp;md/MBOProcessor.cpp;md/MBOServiceServer.cpp;md/MDClient.cpp;md/MDConsumer.cpp;md/MDConsumerHandle.cpp;md/MDConsumerInterop.cpp;md/MDProducer.cpp;md/MDProducerHandle.cpp;md/MemorySegmentUtils.cpp;md/PlatformLoader.cpp;md/PriceSize.cpp;md/SecurityCache.cpp;md/SecurityDB.cpp;md/SecurityIDRecorder.cpp;md/SecurityOracle.cpp;md/SettlementRecorder.cpp;md/Shm/FixedSecurity.cpp;md/SyntheticSecurityService.cpp;md/cme/MDP3Utils.cpp;md/common.cpp;md/ice/ICEUtils.cpp;md/protos/co2.proto;md/relay/QueryServer.cpp;md/relay/RelayClient.cpp;md/relay/RelayRecorder.cpp;md/test/Co2ManagedTestHelper.cpp;md/test/Co2TestInstrument.cpp;md/test/mbo_tests.cpp;md/test/md2_tests.cpp;md/test/md_test_utils.cpp;md/test/synthetic_book_merge_tests.cpp;os/os.cpp;os/tests/os_tests.cpp;quickbus/core/LocalNode.cpp;quickbus/core/Quickbus.cpp;quickbus/core/SubscriptionID.cpp;quickbus/core/TableID.cpp;quickbus/message_model/Update.cpp;quickbus/message_model/merge_updates.cpp;quickbus/network/QBConnection.cpp;quickbus/network/QBConnectionManager.cpp;quickbus/network/endpoint_description.cpp;quickbus/network/quickbus_msgs.proto;quickbus/pubsub/TablePublisher.cpp;quickbus/pubsub/TableSubscriber.cpp;quickbus/pubsub/tests/qb_pubsub_tests.cpp;quickbus/routing/INode.cpp;quickbus/routing/NodeID.cpp;quickbus/routing/Router.cpp;quickbus/rpc/RPCClient.cpp;quickbus/rpc/RPCServer.cpp;quickbus/rpc/RPCSessionID.cpp;quickbus/rpc/rpc_utils.cpp;quickbus/rpc/tests/qb_rpc_tests.cpp;quickbus/tests/quickbus_tests.cpp;sparkproto/SparkProtos.proto;testing/catch_test_runner.cpp;utils/AuthToken.cpp;utils/Dispatcher.cpp;utils/GrowableSPSCQueue.cpp;utils/HTTPStream.cpp;utils/PBUtils.cpp;utils/Puid.cpp;utils/SecureLogon.cpp;utils/SharedMemory.cpp;utils/Sqlite_utils.cpp;utils/Uuid.cpp;utils/datetime.cpp;utils/download.cpp;utils/file_utils.cpp;utils/init_spark_lib.cpp;utils/is_unittest_environment.cpp;utils/stats.cpp;utils/tests/AuthTokenTests.cpp;utils/thread_manager.cpp)

src/lib/api/core/ApiClientInterop.cpp

@@ -176,7 +176,7 @@ void safe_call_subscription_callback(raw_subscription_callback cb, EventType typ
       if (updates.size() > 0) 
          cb(type, &updates[0], updates.size(), extra);
       else
-         cb(type, nullptr, updates.size(), extra);
+         cb(type, nullptr, 0, extra);
    });
 }
 

src/lib/api/public/ClientChannel.h

@@ -5,6 +5,8 @@
 #include "ApiException.h"
 #include "RawUpdate.h"
 #include "ServiceContext.h"
+#include "lib/logging/logging.h"
+
 #include <rxcpp/rx.hpp>
 #include <functional>
 #include <google/protobuf/io/zero_copy_stream_impl_lite.h>
@@ -12,6 +14,17 @@
 namespace bts {
 namespace api {
 
+template <typename T>
+std::shared_ptr<T> deserialize(const char* buffer, size_t length) {
+   google::protobuf::io::ArrayInputStream stream(buffer, length);
+   auto item = std::make_shared<T>();
+   if (item->ParseFromZeroCopyStream(&stream)) {
+      return item;
+   }
+   LOG(Api,Error,"Failed to deserialize proto from api stream. This is likely fatal.");
+   return nullptr;
+}
+
 class ClientChannel
 {
    int _id;
@@ -80,29 +93,29 @@ public:
             }
             else {
                bool first_update = true;
-               auto cb = [observer, first_update](EventType event_type, const StreamUpdate* changes, int change_count) mutable {
+               auto cb = [observer, first_update, method, qualifier](EventType event_type, const StreamUpdate* changes, int change_count) mutable {
                   if (event_type == EventType::TABLE_UPDATED) {
-                     std::vector<std::shared_ptr<const T>> added_items;
+                     std::vector<std::shared_ptr<const T>> added_items, changed_items;
                      std::vector<int> added_keys,
-                                    removed_keys;
+                                      changed_keys,
+                                      removed_keys;
                      for (size_t x = 0; x < change_count; ++x) {
                         const StreamUpdate& change = changes[x];
-                        if (change.type != StreamUpdate::REMOVE) {
-                           added_keys.reserve(change_count);
-                           added_items.reserve(change_count);
-                           added_keys.push_back(change.key);
-                           google::protobuf::io::ArrayInputStream stream(change.buffer, change.buffer_len);
-                           auto item = std::make_shared<T>();
-                           if (item->ParseFromZeroCopyStream(&stream)) {
-                              added_items.push_back(move(item));
-                           }
-                        }
-                        if (change.type != StreamUpdate::CREATE) {
-                           removed_keys.reserve(change_count);
-                           removed_keys.push_back(change.key);
+                        switch(change.type) {
+                           case StreamUpdate::REMOVE:
+                              removed_keys.push_back(change.key);
+                              break;
+                           case StreamUpdate::REPLACE:
+                              changed_items.push_back(deserialize<T>(change.buffer, change.buffer_len));
+                              changed_keys.push_back(change.key);
+                              break;
+                           case StreamUpdate::CREATE:
+                              added_items.push_back(deserialize<T>(change.buffer, change.buffer_len));
+                              added_keys.push_back(change.key);
+                              break;
                         }
                      }
-                     observer.on_next(RawUpdate<std::shared_ptr<const T>>(added_items, added_keys, removed_keys));
+                     observer.on_next(RawUpdate<std::shared_ptr<const T>>(added_items, added_keys, changed_items, changed_keys, removed_keys));
                   }
                   else if (event_type == EventType::TABLE_CREATED) {
                      if (!first_update) {

src/lib/api/public/RawUpdate.h

@@ -6,10 +6,16 @@ namespace api {
 
    template <typename T>
    struct RawUpdate {
-      RawUpdate(std::vector<T>& added_items, std::vector<int>& added_keys, std::vector<int>& removed_keys) :
+      RawUpdate(std::vector<T>& added_items, 
+                std::vector<int>& added_keys, 
+                std::vector<T>& changed_items,
+                std::vector<int>& changed_keys,
+                std::vector<int>& removed_keys) :
          is_reset(false),
          added_items(std::move(added_items)),
          added_keys(std::move(added_keys)),
+         changed_items(std::move(changed_items)),
+         changed_keys(std::move(changed_keys)),
          removed_keys(std::move(removed_keys))
       {}
 
@@ -20,12 +26,16 @@ namespace api {
       bool is_reset;
       std::vector<T> added_items;
       std::vector<int> added_keys;
+
+      std::vector<T> changed_items;
+      std::vector<int> changed_keys;
+
       std::vector<int> removed_keys;
 
    private:
       RawUpdate() :
          is_reset(true),
-         added_items(), added_keys(), removed_keys()
+         added_items(), added_keys(), changed_items(), changed_keys(), removed_keys()
       {}
    };
 

src/lib/api/public/Update.h

@@ -9,12 +9,21 @@
 namespace bts {
 namespace api {
 
+   template <typename T>
+   class Change {
+      T _old, _new;
+      public:
+      Change(const T& old_value, const T& new_value) : _old(old_value), _new(new_value) { }
+      const T& previous() const { return _old; }
+      const T& current() const { return _new; }
+   };
 
    template <typename T, typename TKey, typename ServiceType>
    struct Update {
-      Update(std::vector<T>& all, std::vector<T>& added, std::vector<T>& removed) :
+      Update(std::vector<T>& all, std::vector<T>& added, std::vector<Change<T>> changed, std::vector<T>& removed) :
          _all_items(std::move(all)),
          _added_items(std::move(added)),
+         _changed_items(std::move(changed)),
          _removed_items(std::move(removed))
       {}
 
@@ -22,17 +31,20 @@ namespace api {
       Update(Update&& other) :
          _all_items(std::move(other._all_items)),
          _added_items(std::move(other._added_items)),
+         _changed_items(std::move(other._changed_items)),
          _removed_items(std::move(other._removed_items))
       {}
 
       TKey key(const T& t) const { return ServiceType::key(t); }
       const std::vector<T>& all() const { return _all_items; }
       const std::vector<T>& added() const { return _added_items; }
+      const std::vector<Change<T>>& changed() const { return _changed_items; }
       const std::vector<T>& removed() const { return _removed_items; }
 
    private:
       std::vector<T> _all_items;
       std::vector<T> _added_items;
+      std::vector<Change<T>> _changed_items;
       std::vector<T> _removed_items;
    };
 
@@ -49,9 +61,12 @@ namespace api {
                }
                state->clear();
                std::vector<T> all, added;
-               return Update<T, TKey, ServiceType>(all, added, removed);
+               std::vector<Change<T>> changed;
+               return Update<T, TKey, ServiceType>(all, added, changed, removed);
             }
             else {
+               //
+               // Remove
                std::vector<T> removed;
                removed.reserve(raw.removed_keys.size());
                for (int key : raw.removed_keys) {
@@ -61,6 +76,8 @@ namespace api {
                      state->erase(iter);
                   }
                }
+               //
+               // Add
                for (size_t x = 0; x < raw.added_keys.size(); ++x) {
                   (*state)[raw.added_keys[x]] = raw.added_items[x];
                }
@@ -69,7 +86,16 @@ namespace api {
                for (const auto& item : *state) {
                   all.push_back(item.second);
                }
-               return Update<T, TKey, ServiceType>(all, raw.added_items, removed);
+               //
+               // Change
+               std::vector<Change<T>> changed;
+               changed.reserve(raw.changed_keys.size());
+               for (size_t x = 0; x < raw.changed_keys.size(); ++x) {
+                  changed.push_back(Change<T>{(*state)[raw.changed_keys[x]], raw.changed_items[x]});
+                  (*state)[raw.changed_keys[x]] = raw.changed_items[x];
+               }
+               
+               return Update<T, TKey, ServiceType>(all, raw.added_items, changed, removed);
             }
          });
       };

src/lib/api/services/MBOService.proto

@@ -0,0 +1,32 @@
+syntax="proto3";
+package bts.md;
+option csharp_namespace="bts.md";
+option cc_enable_arenas=true;
+import "lib/api/apioptions.proto";
+import "lib/api/apicore.proto";
+
+message OrderPIQ {
+  uint32 order_id = 1 [(bts.api.key)=true];
+  uint32 position = 2;
+}
+
+message OrderPosition {
+  uint64 exchange_id = 1;
+  uint32 order_id = 2;
+  float price = 10;
+  int32 size = 11;
+}
+message PositionData {
+  uint32 security_id = 1;
+  repeated OrderPosition bid = 10;
+  repeated OrderPosition ask = 20;
+}
+message MBOSecurityQuery {
+  uint32 security_id = 1;
+}
+
+service MBOService {
+  rpc piq (bts.api.VoidMessage) returns (stream OrderPIQ); 
+  rpc get_orders_for_security (MBOSecurityQuery) returns (PositionData);
+
+}

src/lib/api/services/MarketDataRelay.proto

@@ -17,7 +17,6 @@ message ExpirationNameList {
 message ExpirationNameListReply {
     repeated ExpirationNameList securities = 2;
 }
-
 message SecurityKeyList {
     repeated uint32 keys = 1;
 }
@@ -28,6 +27,5 @@ service MarketDataRelay {
 
     rpc request_products(bts.api.VoidMessage) returns (ProductListReply);
     rpc request_names(bts.api.VoidMessage) returns (ExpirationNameListReply);
-
     rpc get_securities(SecurityKeyList) returns (bts.api.SecurityList);
 }

src/lib/config/defaults/Symbols.conf

@@ -1,11 +1,71 @@
-SymbolLists 
-{    
-   value 
+
+SymbolList {
+    value {
+        # This key is here for backwards compatability ONLY.
+        # If you need to update symbols you should update SymbolLists (plural)
+        symbol: "6B"
+        symbol: "6E"
+        symbol: "6J"
+        symbol: "6P"
+        symbol: "ARE"
+        symbol: "BK"
+        symbol: "BOB"
+        symbol: "BUB"
+        symbol: "BZ"
+        symbol: "CL"
+        symbol: "EH"
+        symbol: "ES"
+        symbol: "FYT"
+        symbol: "FOB"
+        symbol: "GC"
+        symbol: "GE"
+        symbol: "GLI"
+        symbol: "HE"
+        symbol: "HG"
+        symbol: "HO"
+        symbol: "KE"
+        symbol: "LE"
+        symbol: "NB"
+        symbol: "NBY"
+        symbol: "NOB"
+        symbol: "NON"
+        symbol: "NG"
+        symbol: "NQ"
+        symbol: "NUB"
+        symbol: "RB"
+        symbol: "SI"
+        symbol: "SOM"
+        symbol: "SR1"
+        symbol: "SR3"
+        symbol: "TN"
+        symbol: "TUF"
+        symbol: "TUT"
+        symbol: "UB"
+        symbol: "XC"
+        symbol: "XK"
+        symbol: "YM"
+        symbol: "ZB"
+        symbol: "ZC"
+        symbol: "ZCW"
+        symbol: "ZF"
+        symbol: "ZL"
+        symbol: "ZM"
+        symbol: "ZN"
+        symbol: "ZO"
+        symbol: "ZQ"
+        symbol: "ZR"
+        symbol: "ZS"
+        symbol: "ZT"
+        symbol: "ZW"
+    }
+}
+SymbolLists
+{
+   value
    {
       list
       {
          key: "ICE"
-         
          # symbol: "BRN"
          # symbol: "WBS"
          # symbol: "OJ"
@@ -13,31 +73,8 @@ SymbolLists
       list
       {
          key: "GLBX"
-         
-         symbol: "HE"
-         symbol: "LE"
-         symbol: "CL"
-         symbol: "RB"
-         symbol: "HO"
-         symbol: "BZ"
-         symbol: "NG"
-         symbol: "GC"
-         symbol: "SI"
-         symbol: "HG"
-         symbol: "ZW"
-         symbol: "ZC"
-         symbol: "ZS"
-         symbol: "ZM"
-         symbol: "ZL"
-         symbol: "ZO"
-         symbol: "ZR"
-         symbol: "ZB"
-         symbol: "ZN"
-         symbol: "ZF"
-         symbol: "ZT"
-         symbol: "UB"      
-        symbol: "GE"
-      }      
+         symbol: @SymbolList.symbol
+      }
    }
 }
 
@@ -74,6 +111,6 @@ Symbols {
         ice {
             future: @SymbolLists.list[ICE].symbol
             spread: @SymbolLists.list[ICE].symbol
-        }        
+        }
     }
 }

src/lib/config/protos/MarketData.proto

@@ -123,7 +123,8 @@ message CMEOptions
 
       optional bool load_security_db = 33 [default = true];
       reserved 34;
-    //   optional BookType book_type = 34 [default = BOOKTYPE_CONSOLIDATED];
+      // optional BookType book_type = 34 [default = BOOKTYPE_CONSOLIDATED];
+      optional bool validate_mbo = 35 [default = false];
 
    }
    optional string onixs_license = 1 [default = "{default}"];

src/lib/config/protos/toggles.proto

@@ -34,4 +34,10 @@ message SoundNotifications {
 
 message VolumeAtPrice{
    optional bool enabled = 1 [default = true];
+}
+message MBOStats {
+   optional bool enabled = 1 [default = false];
+}
+message PlacementInQueue {
+   optional bool enabled = 1 [default = true];
 }

src/lib/md/Diag/MDPeriodicDumper.cpp

@@ -160,7 +160,7 @@ void run_periodic_dumper(const std::set<UserSymbol>& symbols, uint32_t seq_mod,
    threading::configure_current_thread(thread_name);
 
    SecurityCache cache;
-   MDConsumer consumer(cache, nullptr, &symbols);
+   MDConsumer consumer(cache, nullptr, nullptr, &symbols);
 
    // read all securities into a vector
    std::vector<Entry> entries;

src/lib/md/ExchangeOrderKey.h

@@ -0,0 +1,7 @@
+#pragma once
+#include "lib/utils/ValueWrapper.h"
+namespace bts {
+namespace md { 
+using ExchangeOrderKey = utils::ValueWrapper<struct ExchangeOrderKeyTag,uint64_t>;
+using SparkOrderKey = utils::ValueWrapper<struct SparkOrderKeyTag,uint64_t>;
+} }

src/lib/md/IntPrice.h

@@ -54,9 +54,18 @@ class IntPrice {
          return _mantissa < other._mantissa;
       }
       else {
-         return price() < other.price();
+         // avoiding conversion to double is 4-5% faster
+         // return price() < other.price();
+         auto delta = *this - other;
+         return delta.mantissa() < 0;
       }
    }
+   bool operator <=(const IntPrice& other) {
+      return *this == other || *this < other;
+   }
+   bool operator >=(const IntPrice& other) {
+      return !(*this < other);
+   }
    IntPrice operator - (const IntPrice& o) const {
       return op(o, [](int64_t m1, int64_t m2) { return m1 - m2; });
    }
@@ -169,7 +178,6 @@ inline md::IntPrice round_to_quote_price(const IntPrice& tick, const IntPrice& p
    return price;
 }
 
-
 inline std::ostream& operator<<(std::ostream& os, const IntPrice& p) {
    os << p.price();
    return os;

src/lib/md/MBOProcessor.cpp

@@ -0,0 +1,270 @@
+#include "MBOProcessor.h"
+#include  "lib/utils/Dispatcher.h"
+#include "lib/utils/ConcurrentPool.h"
+#include "lib/config/config.h"
+#include "lib/config/protos/toggles.pb.h"
+
+#include <map>
+#include <functional>
+#include <vector>
+#include <sstream>
+
+#include <tbb/concurrent_hash_map.h>
+
+
+namespace bts {
+namespace md {
+
+struct SecurityKeyHashCompare {
+   static size_t hash(const SecurityKey& k) {
+      return k.value();
+   }
+   static bool equal(const SecurityKey& a, const SecurityKey& b) {
+      return a.value() == b.value();
+   }
+};
+
+void print_pool_stats(const utils::ConcurrentPool<MBOSnapshot>& pool) {
+   size_t count = 0;
+   std::vector<int> bid_sz,ask_sz;
+   pool.unsafe_iterate([&](const MBOSnapshot& s) {
+      if (s.bids.size() > 0 || s.asks.size() > 0) {
+         count++;
+         bid_sz.push_back(s.bids.size());
+         ask_sz.push_back(s.asks.size());
+      }
+   });
+
+   if (count == 0) return;
+
+   std::sort(bid_sz.begin(), bid_sz.end());
+   std::sort(ask_sz.begin(), ask_sz.end());
+   size_t bid_tot =0, ask_tot = 0;
+   for(size_t i : bid_sz) { bid_tot += i; }
+   for(size_t i : ask_sz) { ask_tot += i; }
+
+   std::stringstream bid_percentiles, ask_percentiles;
+   for (double x : { 0.5, 0.75, 0.9, 0.95, 0.99 }) {
+      size_t index = count * x;
+      bid_percentiles << bid_sz[index] << " ";
+      ask_percentiles << ask_sz[index] << " ";
+   }
+   bid_percentiles << bid_sz[count-1] << " ";
+   ask_percentiles << ask_sz[count-1] << " ";
+   LOG(md,Info,
+      "MBO statistics: count: {} - %[50,75,90,95,99,100] bid: [{}] - ask: [{}]",
+      count, bid_percentiles.str(), ask_percentiles.str());
+}
+
+typedef tbb::concurrent_hash_map<SecurityKey, MBOSnapshot*, SecurityKeyHashCompare> MapType;
+struct MBOProcessorImpl {
+   utils::ConcurrentPool<MBOSnapshot> pool;
+   MapType snapshots;
+   utils::Dispatcher& dispatcher;
+   utils::DispatcherTimer timer;
+   MBOProcessorImpl(utils::Dispatcher& d) :
+      pool(2048,512),
+      snapshots(),
+      dispatcher(d),
+      timer(d, 600*1000, [&]() { print_pool_stats(pool); })
+      {
+         if (config::get<config::toggles::MBOStats>().enabled()) {
+            timer.start();
+         }
+      }
+};
+
+//
+// Processor
+
+MBOProcessor::MBOProcessor(utils::Dispatcher* d) :
+   _me(std::make_shared<MBOProcessorImpl>(d == nullptr ? utils::Dispatcher::background() : *d)),
+   _producer(_me),
+   _consumer(_me)
+{}
+
+MBOWriter& MBOProcessor::writer() { return _producer;}
+MBOReader& MBOProcessor::reader() { return _consumer; }
+
+//
+// Writer
+
+MBOWriter::MBOWriter(std::shared_ptr<MBOProcessorImpl>& me): _me(me) { }
+MBOWriter::~MBOWriter() { }
+MBOSnapshot* MBOWriter::allocate_snapshot(size_t, size_t) {
+   return _me->pool.allocate();
+}
+void MBOWriter::release_snapshot(MBOSnapshot** s) {
+   _me->pool.release(s);
+}
+void MBOWriter::publish_snapshot(SecurityKey key, MBOSnapshot** snapshot) {
+   bts_assert(*snapshot != nullptr);
+   MapType::accessor accessor;
+   _me->snapshots.insert(accessor, key);
+   std::swap(accessor->second, *snapshot);
+   if (*snapshot != nullptr) {
+      // Note: relase() must happen inside accessor lifetime to ensure correctness
+      _me->pool.release(snapshot);
+   }
+}
+
+bool MBOWriter::validate_snapshot(uint32_t exchid, SecurityKey key, MBOSnapshot* control) {
+   // bts_assert_msg(false, "TO USE THIS VALIDATION YOU MUST MAKE enqueue_incremental SYNCRONOUS");
+   #define REQUIRE(a,b,y) \
+      if ((a) != (b)) { \
+         LOG(md,Error, "Validation error exch:{} sec:{} {}!={} other:{}", exchid, key.value(), a, b, y); \
+         return false; \
+      }
+
+   // retrieve current snapshot and check they are equal
+   _me->dispatcher.post([this,exchid,key,control]() {
+      MBOReader reader(_me);
+      auto snap = reader.acquire_snapshot(key);
+      if (snap) {
+         REQUIRE (snap->bids.size(), control->bids.size(), '!');
+         REQUIRE (snap->asks.size(), control->asks.size(), '!');
+         for (size_t x = 0; x < snap->bids.size(); ++x) {
+            REQUIRE (snap->bids[x].id, control->bids[x].id, x);
+            REQUIRE (snap->bids[x].size, control->bids[x].size, x);
+         }
+         for (size_t x = 0; x < snap->asks.size(); ++x) {
+            REQUIRE (snap->asks[x].id, control->asks[x].id, x);
+            REQUIRE (snap->asks[x].size, control->asks[x].size, x);
+         }
+      }
+      return true;
+   });
+   #undef REQUIRE
+   return true;
+}
+
+void MBOWriter::enqueue_incremental(MBOWorkingState& state, const inc::Update& update) {
+   LOG(md,Diag, "sec:{} : {}", state.handle.key(), update);
+
+   _me->dispatcher.post([this,&state,update]() {
+      // fast part
+      apply_incremental(state,update);
+      // slow part
+      publish_incremental(state);
+   });
+}
+void MBOWriter::publish_incremental(MBOWorkingState& state) {
+   // determine if we need to publish
+   auto key = SecurityKey(state.handle.key());
+   if (key.value() != invalid_security_key) {
+      auto snap = allocate_snapshot(state.bid.size(), state.ask.size());
+      copy_to_snapshot(state, snap);
+      publish_snapshot(key, &snap);
+   }
+}
+
+void copy_to_snapshot(MBOWorkingState& state, MBOSnapshot* dest) {
+   if (dest == nullptr) return;
+   // copy working state to snapshot
+   dest->updated = state.updated;
+   auto type_map = [](auto pair) {
+         return SnapshotOrder { pair.second.id, std::get<IntPrice>(pair.first), pair.second.size };
+      };
+   std::transform(state.bid.begin(), state.bid.end(), std::back_inserter(dest->bids), type_map);
+   std::transform(state.ask.begin(), state.ask.end(), std::back_inserter(dest->asks), type_map);
+}
+
+template <class C>
+void apply_incremental(uint32_t security_key, C& collection, std::unordered_map<ExchangeOrderKey, MBOWorkingState::key_t>& index, const inc::Update& update) {
+   auto key = std::make_tuple(update.price, update.priority);
+   switch(update.action) {
+      case inc::UpdateAction::New:
+         collection.insert(std::make_pair(key, MBOWorkingState::Data { update.order_id, update.size }));
+         index[update.order_id] = key;
+         break;
+      case inc::UpdateAction::Change: {
+         // look up old key
+         auto r = index.find(update.order_id);
+         if (r != index.end()) {
+            auto oldkey = r->second;
+            collection.erase(oldkey);
+            index.erase(r);
+         }
+         else { 
+            LOG(md,Warn, "MBO Update w/out Add for: sec:{} {}", security_key, update);
+         }
+         collection.insert(std::make_pair(key, MBOWorkingState::Data { update.order_id, update.size }));
+         index[update.order_id] = key;
+      }
+      break;
+      case inc::UpdateAction::Delete:
+         collection.erase(key);
+         index.erase(update.order_id);
+      break;
+   }
+}
+void apply_incremental(MBOWorkingState& state, const inc::Update& update) {
+   // apply the update to the working state
+   if (update.type == inc::UpdateType::Bid) {
+      apply_incremental(state.handle.key(), state.bid, state.orderid_index, update);
+   }
+   else {
+      apply_incremental(state.handle.key(),state.ask, state.orderid_index, update);
+   }
+}
+
+//
+// Reader
+
+MBOReader::MBOReader(std::shared_ptr<MBOProcessorImpl>& me): _me(me) { }
+MBOReader::~MBOReader() { }
+MBOSnapshot* MBOReader::acquire_snapshot(SecurityKey key) const {
+   // get a result eagerly so we don't have to do it while accessor has the entry locked
+   auto result = _me->pool.allocate();
+   MapType::const_accessor accessor;
+   if (_me->snapshots.find(accessor, key) && accessor->second != nullptr) {
+      result->copy_from(*accessor->second);
+   }
+   else {
+      release_snapshot(&result);
+   }
+   return result;
+}
+void MBOReader::release_snapshot(MBOSnapshot**s) const {
+   _me->pool.release(s);
+}
+
+}}
+
+namespace std {
+   using namespace bts;
+   using namespace bts::md;
+
+ostream& operator<<(ostream& os, const inc::Update& update) {
+   os << "Update: { "
+      << "id:" << update.order_id.value() << " "
+      << "pri:" << update.priority << " "
+      << update.price.price() << " "
+      << update.size << " ";
+
+   switch(update.type) {
+      case inc::UpdateType::Bid:
+         os << "B ";
+         break;
+      case inc::UpdateType::Ask:
+         os << "A ";
+         break;
+   }
+
+   switch(update.action) {
+      case inc::UpdateAction::New:
+         os << "+";
+         break;
+      case inc::UpdateAction::Change:
+         os << "C";
+         break;
+      case inc::UpdateAction::Delete:
+         os << "-";
+         break;
+   }
+
+   os << "}";
+   return os;
+}
+
+}

src/lib/md/MBOProcessor.h

@@ -0,0 +1,186 @@
+#pragma once 
+#include "lib/domain_model/SecurityKey.h"
+#include "lib/domain_model/Security.h"
+#include "lib/utils/TimeStamp.h"
+#include "lib/utils/Dispatcher.h"
+#include "IntPrice.h"
+#include "ExchangeOrderKey.h"
+#include "MDProducerHandle.h"
+
+#ifndef _WIN32
+#include "google/btree/btree_map.h"
+#endif
+
+#include <vector>
+#include <memory>
+#include <map>
+#include <tuple>
+
+namespace bts { 
+namespace md { 
+
+struct SnapshotOrder {
+   ExchangeOrderKey id;
+   IntPrice price;
+   int32_t size;
+};
+struct MBOSnapshot {
+   MBOSnapshot() = default;
+   MBOSnapshot(const MBOSnapshot&) = delete;
+   MBOSnapshot(MBOSnapshot&& other) = delete;
+
+   utils::TimeStamp updated;
+   std::vector<SnapshotOrder> bids;
+   std::vector<SnapshotOrder> asks;
+
+   void copy_from(const MBOSnapshot& src) {
+      bids = src.bids;
+      asks = src.asks;
+      updated = src.updated;
+   }
+
+   void reset() {
+      updated = utils::TimeStamp();
+      constexpr int max = 256;
+      if (bids.size() > max) {
+         bids.resize(max);
+      }
+      if (asks.size() > max) {
+         asks.resize(max);
+      }
+      bids.clear();
+      asks.clear();
+   }
+};
+
+namespace inc {
+
+   // according to this link MBO book only has these updates types
+   // https://www.cmegroup.com/confluence/display/EPICSANDBOX/MDP+3.0+-+Market+Data+Snapshot+Full+Refresh#MDP3.0-MarketDataSnapshotFullRefresh-MarketbyOrder(MBO)DataBlock
+   enum class UpdateType {
+      Bid = 0,
+      Ask = 1,
+      // ImpliedBid, // 'E'
+      // ImpliedAsk, // 'F'
+      // BookReset // 'J'
+   };
+
+   // OrderUpdateAction=37708
+   enum class UpdateAction {
+      New = 0,
+      Change = 1,
+      Delete = 2
+   };
+
+   typedef uint64_t Priority;
+   struct Update {
+      ExchangeOrderKey order_id;
+      Priority priority;
+      IntPrice price;
+      int32_t size;
+      UpdateType type;
+      UpdateAction action;
+   };
+}
+
+
+struct MBOWorkingState {
+   typedef std::tuple<IntPrice,inc::Priority> key_t;
+   struct Data {
+      ExchangeOrderKey id;
+      int32_t size;
+   };
+   struct LessBid {
+      bool operator()(const key_t& left, const key_t& right) const {
+         const auto& left_price = std::get<IntPrice>(left);
+         const auto& right_price = std::get<IntPrice>(right);
+         if (left_price == right_price) {
+            return 
+               std::get<inc::Priority>(left) <
+               std::get<inc::Priority>(right);
+         }
+         return !(left_price < right_price);
+      }
+   };
+
+   MBOWorkingState(MDProducerHandle& h) : handle(h), updated(), bid(),ask() {}
+
+   MDProducerHandle handle;
+   utils::TimeStamp updated;
+   // Used for 'change' messages which update price
+   std::unordered_map<ExchangeOrderKey, key_t> orderid_index;
+
+#ifdef _WIN32
+   std::map<key_t,Data,LessBid> bid;
+   std::map<key_t,Data> ask;
+#else
+   btree::btree_map<key_t,Data,LessBid> bid;
+   btree::btree_map<key_t,Data> ask;
+#endif
+};
+
+
+void copy_to_snapshot(MBOWorkingState& state, MBOSnapshot* destination);
+void apply_incremental(MBOWorkingState&, const inc::Update&);
+
+template <class V>
+void apply_incremental(MBOWorkingState& state, const V& vec) {
+   for (const auto& update : vec) {
+      apply_incremental(state, update);
+   }
+}
+
+struct MBOProcessorImpl;
+class MBOProcessor;
+
+class MBOWriter {
+   std::shared_ptr<MBOProcessorImpl> _me;
+   friend class MBOProcessor;
+   MBOWriter(std::shared_ptr<MBOProcessorImpl>&);
+   ~MBOWriter();
+
+   public:
+   //
+   // Snapshot api
+   MBOSnapshot* allocate_snapshot(size_t bid_count=0, size_t ask_count=0);
+   void release_snapshot(MBOSnapshot**);
+   void publish_snapshot(SecurityKey, MBOSnapshot**);
+   bool validate_snapshot(uint32_t, SecurityKey, MBOSnapshot*);
+
+   //
+   // Incremental api
+
+   // Enqueue the update to the mbo work queue.
+   void enqueue_incremental(MBOWorkingState&, const inc::Update&);
+   // Update working state and publish directly.
+   void publish_incremental(MBOWorkingState&);
+};
+
+class MBOReader {
+   std::shared_ptr<MBOProcessorImpl> _me;
+   friend class MBOProcessor;
+   friend class MBOWriter;
+   MBOReader(std::shared_ptr<MBOProcessorImpl>&);
+   ~MBOReader();
+   public:
+   MBOSnapshot* acquire_snapshot(SecurityKey key) const;
+   void release_snapshot(MBOSnapshot**) const;
+};
+
+class MBOProcessor {
+   std::shared_ptr<MBOProcessorImpl> _me;
+   MBOWriter _producer;
+   MBOReader _consumer;
+
+public:
+   MBOProcessor(utils::Dispatcher* _=nullptr);
+   MBOWriter& writer();
+   MBOReader& reader();
+};
+
+
+}} 
+
+namespace std {
+std::ostream& operator<<(std::ostream& os, const bts::md::inc::Update&);
+}

src/lib/md/MBOServiceServer.cpp

@@ -0,0 +1,340 @@
+#ifndef _WIN32
+#include "MBOServiceServer.h" 
+
+#include "lib/api/services/OrderService.client.pb.h"
+#include "lib/utils/Dispatcher.h"
+#include "lib/utils/TimeStamp.h"
+#include "MDClient.h"
+#include "ExchangeOrderKey.h"
+#include "MBOProcessor.h"
+
+#include "lib/utils/LockBox.h"
+
+#include <tuple>
+#include <vector>
+#include <unordered_map>
+
+
+using namespace bts::api::orderman;
+
+namespace bts { 
+namespace md { 
+
+//
+// Simple class to handle MD recovery and maintain an index of SecurityKey=>MDConsumerHandle
+
+class HandleIndex {
+   MDClient& _client;
+   int _sub;
+   std::optional<MDConsumer> _consumer;
+
+public:
+   utils::LockBox<std::unordered_map<SecurityKey,MDConsumerHandle>> index;
+
+   HandleIndex(md::MDClient& client): 
+      _client(client),
+      _sub(0),
+      _consumer(std::nullopt),
+      index() {
+         _sub = _client.on_consumer().subscribe([=](const MDConsumer& c) { on_consumer(c); });
+         on_consumer(_client.current_consumer());
+      }
+
+   ~HandleIndex() { _client.on_consumer().unsubscribe(_sub); }
+
+
+private:
+   void on_consumer(const MDConsumer& c) {
+      MDConsumer temp(c);
+      if (temp.all_heartbeats_ok()) {
+         auto existing = _consumer ? _consumer->id() : 0;
+         LOG(md,Info, "Updateing MBOService MDConsumer. Old {} new {}", existing, temp.id());
+         // clear the index
+         index.lock([](auto& index) { index.clear(); });
+         // Swap to new consumer
+         if (_consumer) {
+            _consumer->swap(temp);
+         }
+         else { _consumer = temp; }
+         // Subscribe to on_security
+         auto id = _consumer->id();
+         _consumer->on_security([this,id](const SecurityPtr&, MDConsumerHandle handle) {
+            if (_consumer->id() == id) {
+               index.lock([handle](auto& index) {
+                  index[SecurityKey(handle.key())] = handle;
+               });
+            }
+         });
+      }
+   }
+};
+
+class ActiveOrder {
+   SecurityKey _security;
+   SparkOrderKey _id;
+   ExchangeOrderKey  _ex_id;
+   int32_t _position;
+
+public:
+   // empty constructor needed for unordered map operator[]
+   // ActiveOrder(): _security(), _id(), _ex_id(), _position(-1) { bts_assert(false); }
+   ActiveOrder(SecurityKey sec, SparkOrderKey s, ExchangeOrderKey ex): _security(sec), _id(s), _ex_id(ex), _position(-1)  {}
+   ActiveOrder(SecurityKey sec, SparkOrderKey s, const std::string& ex): _security(sec), _id(s), _ex_id(), _position(-1) {
+      _ex_id = ExchangeOrderKey(std::stoul(ex));
+   }
+
+   SecurityKey security_id() const { return _security; }
+   SparkOrderKey id() const { return _id; }
+   ExchangeOrderKey exchange_id() const { return _ex_id; }
+   int32_t& position() { return _position; };
+   int32_t position() const { return _position; };
+   bool set_position(int32_t pos) {
+      if (pos != _position) {
+         _position = pos;
+         return true;
+      }
+      return false;
+   }
+   OrderPIQ to_piq() const {
+      OrderPIQ result;
+      result.set_order_id(_id.value());
+      result.set_position(_position);
+      return result;
+   }
+};
+
+//
+// MBOService
+
+struct MBOServiceImpl {
+   MBOServiceImpl(
+      api::ServiceContext& ctx,
+      utils::Dispatcher& d,
+      md::MDClient& c,
+      MBOServiceBase::PiqPublisher&& pub
+      ):
+      dispatcher(d),
+      client(c),
+      orders(ctx.locate<api::orderman::OrderService>()),
+      handle_index(c),
+      publish(pub),
+      timer(d, 200, [&]() {timer_tick();}),
+      orders_sub()
+   {}
+
+   utils::Dispatcher& dispatcher;
+   md::MDClient& client;
+   OrderService orders;
+
+   HandleIndex handle_index;
+   MBOServiceBase::PiqPublisher publish;
+   utils::DispatcherTimer timer;
+
+   rxcpp::composite_subscription orders_sub;
+   std::unordered_map<SparkOrderKey, ActiveOrder> active;
+
+   void timer_tick();
+   void flush();
+};
+
+
+
+///////////////////////
+
+std::optional<uint32_t> search_orders(const ExchangeOrderKey oid, const std::vector<SnapshotOrder>& orders) {
+   IntPrice price = {};
+   size_t position = 0;
+   for (size_t x = 0; x < orders.size(); ++x) {
+      const auto& oo = orders[x];
+      if (oo.id == oid) {
+         if (oo.price == price) return { position };
+         else return 0 ;
+      }
+      else if (price == oo.price) {
+         position += 1;
+      }
+      else {
+         price = oo.price;
+         position = 1;
+      }
+   }
+   return std::nullopt;
+}
+
+bool update_position(const HandleIndex& handles, ActiveOrder& order) {
+   const auto& index = handles.index;
+   auto handle = index.lock([id=order.security_id()](const auto& index) {
+      auto iter = index.find(id);
+      return iter != index.end() ? iter->second : MDConsumerHandle {};
+   });
+
+   if (handle.is_initialized()) {
+      auto mbo = handle.get_mbo();
+      if (mbo) {
+         auto search_result = search_orders(order.exchange_id(), mbo->bids);
+         if (!search_result) {
+            search_result = search_orders(order.exchange_id(), mbo->asks);
+         }
+         if (search_result) {
+            return order.set_position(search_result.value());
+         }
+      }
+   }
+   return false;
+}
+
+void MBOServiceImpl::timer_tick() {
+   //Dispatcher thread
+   std::vector<OrderPIQ> rows;
+   for (auto& pair : active) {
+      if (update_position(handle_index, pair.second)) {
+         rows.push_back(pair.second.to_piq());
+      }
+   }
+   if (!rows.empty()) {
+      publish(nullptr, &rows, nullptr);
+   }
+}
+
+const NativeOrderState* native_order_state(const OrderRecord& record) {
+   switch(record.state().custom_case()) {
+      case OrderState::kMarket:
+         return &record.state().market().native_state();
+      case OrderState::kLimit:
+         return &record.state().limit().native_state();
+      case OrderState::kStop:
+         return &record.state().stop().native_state();
+      case OrderState::kStopLimit:
+         return &record.state().stop_limit().native_state();
+      default:
+      return nullptr;
+   }
+}
+
+bool is_active_exchange_order(const OrderRecord& record) {
+   return record.state().status() == ORDER_STATUS_WORKING &&
+          native_order_state(record) != nullptr;
+}
+
+void try_remove(MBOServiceImpl& me, std::set<uint32_t>& to_remove, const OrderRecord& record) {
+   auto key = SparkOrderKey(record.key());
+   auto iter = me.active.find(key);
+   if (iter != me.active.end()) {
+      me.active.erase(key);
+      to_remove.insert(key.value());
+   }
+}
+void try_add(MBOServiceImpl& me, std::vector<OrderPIQ>& to_add, const OrderRecord& record) {
+   if (is_active_exchange_order(record)) {
+      auto native = native_order_state(record);
+      bts_assert(native);
+      ActiveOrder order(
+         SecurityKey(record.def().security_key()), 
+         SparkOrderKey(record.key()), 
+         native->exchange_order_id());
+      update_position(me.handle_index, order);
+      to_add.push_back(order.to_piq());
+      me.active.insert(std::make_pair(order.id(), order));
+   }
+}
+
+MBOServiceServer::MBOServiceServer(
+      api::ServiceContext& ctx,
+      utils::Dispatcher& d,
+      md::MDClient& c
+      ) : 
+         MBOServiceBase(ctx),
+         _me(new MBOServiceImpl(ctx,d,c,make_piq_publisher()))
+      {
+         // subscribe to mdclient ✔
+         // make publishers ✔
+         // subscribe to orders => stash active orders dict
+         _me->orders_sub =
+         _me->orders.subscribe_orders().subscribe([=](const auto& update) {
+            // api thread
+            const auto& added = update.added();
+            const auto& changed = update.changed();
+            const auto& removed = update.removed();
+            _me->dispatcher.post([this,added,changed,removed]() {
+               // dispatcher thread
+               std::vector<OrderPIQ> to_add, to_change;
+               std::set<uint32_t> to_remove;
+
+               // Remove
+               for(const auto& r : removed) {
+                  // If it was active remove it and add to to_remove
+                  try_remove(*_me, to_remove, *r);
+               }
+               // Added
+               for(const auto& add : added) {
+                  // If active add look up the position and add to the state
+                  try_add(*_me, to_add, *add);
+               }
+               // Changed
+               for(const auto& ch : changed) {
+                  // new order is active and old is not look up piq and add to state
+                  bool is_active = is_active_exchange_order(*ch.current());
+                  bool was_active = is_active_exchange_order(*ch.previous());
+                  if (is_active && !was_active) {
+                     try_add(*_me, to_add, *ch.current());
+                  }
+                  else if (was_active && !is_active) {
+                     try_remove(*_me, to_remove, *ch.current());
+                  }
+                  // is_active && was_active => don't care
+                  // !is_active && !was_active => don't care
+               }
+
+               _me->publish(&to_add, &to_change, &to_remove);
+               if (!_me->timer.running()) _me->timer.start();
+            });
+         });
+      }
+
+MBOServiceServer::~MBOServiceServer() { }
+
+
+void MBOServiceServer::get_orders_for_security(const MBOSecurityQuery& request, const api::Response<PositionData>& reply) {
+   PositionData response;
+   response.set_security_id(request.security_id());
+   auto handle = _me->handle_index.index.lock([id=request.security_id()](const auto& index) {
+      auto iter = index.find(SecurityKey(id));
+      return iter != index.end() ? iter->second : MDConsumerHandle {};
+   });
+
+   std::unordered_map<ExchangeOrderKey, SparkOrderKey> reverse;
+   for (const auto& pair : _me->active) {
+      reverse.insert(std::make_pair(pair.second.exchange_id(), pair.first));
+   }
+
+   if(handle.is_initialized()) {
+      auto mbo = handle.get_mbo();
+      if (mbo) {
+         for (const auto& o : mbo->bids) {
+            auto* entry = response.add_bid();
+            entry->set_exchange_id(o.id.value());
+            entry->set_price(o.price.price());
+            entry->set_size(o.size);
+            auto iter = reverse.find(o.id);
+            if (iter != reverse.end()) {
+               entry->set_order_id(iter->second.value());
+            }
+         }
+         for (const auto& o : mbo->asks) {
+            auto* entry = response.add_ask();
+            entry->set_exchange_id(o.id.value());
+            entry->set_price(o.price.price());
+            entry->set_size(o.size);
+            auto iter = reverse.find(o.id);
+            if (iter != reverse.end()) {
+               entry->set_order_id(iter->second.value());
+            }
+         }
+      }
+   }
+   
+   reply(response);
+}
+
+}} //namespace 
+#endif

src/lib/md/MBOServiceServer.h

@@ -0,0 +1,27 @@
+#pragma once 
+#include "lib/api/public/ServiceContext.h"
+#include "lib/api/services/MBOService.server.pb.h"
+#include <memory>
+
+namespace bts { 
+namespace utils { class Dispatcher; }
+namespace api { namespace orderman { class OrderService; } }
+namespace md { 
+
+class MDClient;
+
+struct MBOServiceImpl;
+class MBOServiceServer : public MBOServiceBase {
+   std::unique_ptr<MBOServiceImpl> _me;
+   public:
+   MBOServiceServer(
+      api::ServiceContext& ctx,
+      utils::Dispatcher&,
+      md::MDClient&);
+   ~MBOServiceServer();
+
+   void get_orders_for_security(const MBOSecurityQuery&, const api::Response<PositionData>&) override;
+
+};
+
+}} //namespace 

src/lib/md/MDClient.cpp

@@ -3,6 +3,7 @@
 #include "SecurityCache.h"
 #include "SecurityOracle.h"
 #include "lib/utils/TimeStamp.h"
+#include "MBOProcessor.h"
 #include <set>
 
 namespace bts {
@@ -15,19 +16,22 @@ struct MDClientImpl{
    std::unique_ptr<utils::DispatcherTimer> _security_timer;
    std::unique_ptr<utils::DispatcherTimer> _recovery_timer;
    std::set<UserSymbol> _symbols;
+   MBOReader* _mbo;
    MDConsumer _consumer;
    utils::Event<MDConsumer> _event;
 
    MDClientImpl(
       utils::Dispatcher* d, 
       api::ISyntheticSecurityService* s,
+      MBOReader* mbo,
       const std::set<UserSymbol>* symbols): 
       _dispatcher(d),
       _ss(s),
       _security_timer(),
       _recovery_timer(),
       _symbols(symbols ? *symbols : get_system_md_symbols()),
-      _consumer(_cache, _ss, &_symbols),
+      _mbo(mbo),
+      _consumer(_cache, _ss, mbo, &_symbols),
       _event()
    {
       if (d) {
@@ -42,7 +46,7 @@ struct MDClientImpl{
 
    void check_heartbeats() {
       if (!_consumer.all_heartbeats_ok()) {
-         MDConsumer temp(_cache,_ss);
+         MDConsumer temp(_cache,_ss,_mbo,&_symbols);
          if (temp.all_heartbeats_ok()) {
             _consumer.swap(temp);
             LOG(md,Info, "Recovering from bad MD. Old consumer {} new consumer {}", _consumer.id(), temp.id());
@@ -71,11 +75,12 @@ struct MDClientImpl{
 MDClient::MDClient(
    utils::Dispatcher* d, 
    api::ISyntheticSecurityService* s,
+   MBOReader* mbo,
    const std::set<UserSymbol>* symbols) : 
-      _self(new MDClientImpl(d,s, symbols)) 
+      _self(new MDClientImpl(d,s,mbo,symbols)) 
    { }
-MDClient::~MDClient() {
-}
+
+MDClient::~MDClient() { }
 
 size_t MDClient::scan_securities() { return _self->scan_securities(); }
 void MDClient::check_heartbeats() { return _self->check_heartbeats(); }

src/lib/md/MDClient.h

@@ -11,7 +11,9 @@ namespace bts {
 namespace md {
 
 class SecurityOracle;
+class MBOReader;
 struct MDClientImpl;
+
 class MDClient {
    std::unique_ptr<MDClientImpl> _self;
 
@@ -20,6 +22,7 @@ public:
    MDClient(
       utils::Dispatcher* d=nullptr, 
       api::ISyntheticSecurityService* ss=nullptr,
+      MBOReader* mbo=nullptr,
       const std::set<UserSymbol>* s=nullptr);
    ~MDClient();
 

src/lib/md/MDConsumer.cpp

@@ -3,6 +3,7 @@
 #include "MemorySegmentUtils.h"
 #include "PlatformLoader.h"
 #include "SecurityCache.h"
+#include "MBOProcessor.h"
 #include "lib/config/config.h"
 #include "lib/utils/SharedList.h"
 #include "lib/utils/WaitFreeQueue.h"
@@ -535,6 +536,7 @@ class MDConsumerImpl {
    Mutex _workmutex;
    vector<shared_ptr<MDConsumerInst>> _instances;
    std::shared_ptr<SyntheticConsumerInst> _sci;
+   MBOReader* _mbo;
    map<security_key, shared_ptr<MDConsumerHandle>> _deferred_handles;
    vector<SecurityCallback> _callbacks;
    mutable SecurityCache _cache;
@@ -545,10 +547,12 @@ public:
       const set<UserSymbol>& symbols, 
       SecurityCache& cache,
       api::ISyntheticSecurityService* ss, 
+      MBOReader* mbo,
       bool shared, co2::Platform platform) :
          _workmutex(),
          _instances(),
          _sci(),
+         _mbo(mbo),
          _deferred_handles(),
          _callbacks(),
          _cache(cache)
@@ -625,7 +629,7 @@ public:
       for (auto& inst : _instances) {
          if (inst->find_handle(key, &cold, &hot)) {
             MDConsumerHandle handle;
-            handle.init(inst->cold_header(), *cold, hot);
+            handle.init(inst->cold_header(), *cold, hot, _mbo);
             return handle;
          }
       }
@@ -657,9 +661,15 @@ public:
       }
 
       // Erase anything that wasn't found from the resulting map
+      // Set the mbo pointer on anything else
       for (auto it = result.begin(); it != result.end(); ) {
-         if (!it->second.is_initialized()) it = result.erase(it);
-         else ++it;
+         if (!it->second.is_initialized()) {
+            it = result.erase(it);
+         }
+         else {
+            it->second.set_mbo(_mbo);
+            ++it;
+         }
       }
 
       return result;
@@ -910,6 +920,7 @@ public:
 MDConsumer::MDConsumer(
    SecurityCache& cache,
    api::ISyntheticSecurityService* ss, 
+   MBOReader* mbo,
    const std::set<UserSymbol>* symbols, 
    bool shared, 
    co2::Platform platform) : _pimpl() {
@@ -920,7 +931,7 @@ MDConsumer::MDConsumer(
          symbols = &all_symbols;
       }
 
-      _pimpl.reset(new MDConsumerImpl(*symbols, cache, ss, shared, platform));
+      _pimpl.reset(new MDConsumerImpl(*symbols, cache, ss, mbo, shared, platform));
 }
 
 MDConsumer::~MDConsumer() {}

src/lib/md/MDConsumer.h

@@ -26,6 +26,7 @@ class MDConsumerImpl;
 class MDConsumerInst;
 class MDConsumerChannel;
 class SecurityCache;
+class MBOReader;
 
 class MDConsumer
 {
@@ -33,6 +34,7 @@ public:
    MDConsumer(
       SecurityCache&,
       api::ISyntheticSecurityService* ss, 
+      MBOReader* mbo=nullptr,
       const std::set<UserSymbol>* sym = nullptr, 
       bool shared=true, 
       co2::Platform platform=co2::PLATFORM_UNSET);

src/lib/md/MDConsumerHandle.cpp

@@ -1,5 +1,6 @@
 #ifndef NO_CPP0X
 #include "MDConsumerHandle.h"
+#include "MBOProcessor.h"
 #include "lib/domain_model/Security.h"
 #include <boost/container/small_vector.hpp>
 
@@ -14,7 +15,7 @@ namespace md {
 MDConsumerHandle::MDConsumerHandle() {
    memset(this, 0, sizeof(MDConsumerHandle));
    _type = HANDLETYPE_VOID;
-   static_assert(sizeof(MDConsumerHandle) <= 32, "");
+   static_assert(sizeof(MDConsumerHandle) <= 40, "");
 }
 MDConsumerHandle::MDConsumerHandle(const std::shared_ptr<const SyntheticStrategy>& def) {
    memset(this, 0, sizeof(MDConsumerHandle));
@@ -33,9 +34,10 @@ MDConsumerHandle::MDConsumerHandle(const MDConsumerHandle& other) {
 
 MDConsumerHandle::~MDConsumerHandle() {}
 
-void MDConsumerHandle::init(const ColdHeader& cheader, const ColdEntry& centry, const void* pvhot) {
+void MDConsumerHandle::init(const ColdHeader& cheader, const ColdEntry& centry, const void* pvhot, const MBOReader* mbo) {
    set_cold(&cheader, &centry);
    set_hot(pvhot);
+   set_mbo(mbo);
 }
 
 TimeStamp copy_bbo(const HotEntry& hot, PriceSize& out_bid, PriceSize& out_ask) {
@@ -371,6 +373,16 @@ CoconutSecurityStatus MDConsumerHandle::status() const {
    never;
 }
 
+std::unique_ptr<MBOSnapshot,std::function<void(MBOSnapshot*)>> MDConsumerHandle::get_mbo() const {
+      if (_type == HANDLETYPE_DEPTH) {
+         MBOSnapshot* snapshot = book.mbo->acquire_snapshot(SecurityKey(key()));
+         if (snapshot != nullptr) {
+            return {snapshot,[mbo=book.mbo](auto* s) { mbo->release_snapshot(&s); }};
+         }
+      }
+      return nullptr;
+   }
+
 }}//namespace
 
 

src/lib/md/MDConsumerHandle.h

@@ -13,6 +13,9 @@ namespace bts {
 namespace md {
 
 class MDConsumerHandle;
+class MBOReader;
+struct MBOSnapshot;
+
 struct SyntheticStrategy {
    std::unique_ptr<spark::ExecutionStrategy> strategy;
    api::Security security;
@@ -141,6 +144,9 @@ TimeStamp optimistic_copy(
    return t;
 }
 
+//
+// MDConsumerHandle
+//
 
 // Can be an invalid handle if you request a security which has not been discovered yet.
 class BTS_EXPORT MDConsumerHandle
@@ -151,7 +157,7 @@ public:
    ~MDConsumerHandle();
    MDConsumerHandle(const MDConsumerHandle& other);
 
-   void init(const ColdHeader&, const ColdEntry&, const void* hot);
+   void init(const ColdHeader&, const ColdEntry&, const void* hot, const MBOReader*);
 
    bool is_initialized() const { return _type != HANDLETYPE_VOID; }
    bool is_synthetic() const { return _type == HANDLETYPE_SYNTHETIC; }
@@ -267,8 +273,11 @@ public:
       return *this;
    }
 
+   std::unique_ptr<MBOSnapshot,std::function<void(MBOSnapshot*)>> get_mbo() const; 
+
 private:
    friend class MDConsumerInst;
+   friend class MDConsumerImpl;
 
    template <class Read>
    HandleReadResult get_book(Read read, TimeStamp& time, uint32_t copydepth, PriceSize* bids, PriceSize* asks) const {
@@ -326,6 +335,19 @@ private:
          _type = HANDLETYPE_BBO;
       }
    }
+   void set_mbo(const MBOReader* mbo) {
+      switch(_type) {
+         case HANDLETYPE_DEPTH:
+            book.mbo = mbo;
+            break;
+         case HANDLETYPE_BBO:
+            bbo.mbo = mbo;
+            break;
+         default:
+         //ok
+         break;
+      }
+   }
 
    enum HandleType : uint8_t {
       HANDLETYPE_VOID,
@@ -337,11 +359,13 @@ private:
       const HotDepthEntry* hot;
       const ColdEntry* cdata;
       const ColdHeader* cheader;
+      const MBOReader* mbo;
    };
    struct BBOHandle {
       const HotEntry* hot;
       const ColdEntry* cdata;
       const ColdHeader* cheader;
+      const MBOReader* mbo;
    };
    struct SyntheticHandle {
       std::shared_ptr<const SyntheticStrategy> def;

src/lib/md/MDConsumerInterop.cpp

@@ -32,7 +32,7 @@ CAPI MDConsumer* create_md_consumer(BlitUserSymbol* usersymbols, uint32_t count,
    for (size_t x = 0; x < count; ++x) {
       symbols.insert(UserSymbol(usersymbols[x].symbol, usersymbols[x].exchange, usersymbols[x].type));
    }
-   return new MDConsumer(cache, nullptr/*synthetics*/, &symbols, shared);
+   return new MDConsumer(cache, nullptr/*synthetics*/, nullptr /*mbo*/, &symbols, shared);
 }
 
 CAPI void destroy_md_consumer(MDConsumer* consumer) {

src/lib/md/ice/ICEUtils.cpp

@@ -26,7 +26,7 @@ std::string ICEChannel::to_string() const
 {
    std::string mt = name;
    for(const auto& x : market_types)
-      mt += " " + x;   
+      mt += " " + std::to_string(x);
    return mt;
 }
 
@@ -123,4 +123,4 @@ const std::vector<ICEChannel> get_configured_ice_channels() {
 
 
 }}} //namespace
-#endif
+#endif

src/lib/md/relay/QueryServer.cpp

@@ -1,4 +1,4 @@
-#ifndef NO_CPP0X
+#ifndef _WIN32
 #include "QueryServer.h"
 #include "lib/config/config.h"
 #include "lib/utils/Dispatcher.h"
@@ -9,6 +9,7 @@
 #include "../MDConsumer.h"
 #include "../MDClient.h"
 #include "../PlatformLoader.h"
+#include "../MBOProcessor.h"
 
 #include "lib/logging/logging.h"
 
@@ -30,6 +31,7 @@
 #include <memory>
 #include <set>
 #include <unordered_map>
+#include <optional>
 
 #include <google/protobuf/arena.h>
 #include <boost/container/flat_map.hpp>
@@ -129,7 +131,6 @@ struct MDConsumerGroup {
 
 };
 
-
 void convert(PriceSize ps, PriceSizeMessage* pps) {
    pps->set_mantissa(ps.mantissa());
    pps->set_exponent(ps.exponent());
@@ -211,6 +212,7 @@ public:
          for (auto& pair : _groups) {
             pair.second->set_consumer(_consumer);
          }
+         // _handle_index.clear();
          subscribe_securities();
          utils::Dispatcher::background().post([temp]() { LOG(md,Info,"QServer destroying consumer {}", temp.id()); }, 60000);
       }
@@ -380,10 +382,7 @@ public:
                google::protobuf::MapPair<uint32_t,std::string> v(h.handle.key(), security.ice().security_desc());
                x->insert(v);
             }    
-            else
-            {
-               LOG(md,Error, "Invalid ExchangeData_case {}", security.ExchangeData_case());
-            }
+            // There is currently no ExchangeData for synthetics
          }
       }
    }
@@ -399,7 +398,6 @@ public:
          }
       }
    }
-
 };
 
 struct MDRelayServer : bts::co2::AuthenticatedMarketDataRelayBase {
@@ -497,6 +495,7 @@ struct MDRelayServer : bts::co2::AuthenticatedMarketDataRelayBase {
     }
 };
 
+
 struct QueryServer {
     QueryServer(utils::Dispatcher& threadpool, 
                 const bts::api::ServiceContext& ctx, 

src/lib/md/test/Co2ManagedTestHelper.cpp

@@ -26,7 +26,7 @@ extern "C"  BTS_EXPORT void co2_get_1234_handle(size_t& size, MDConsumerHandle&
    auto pheader = reinterpret_cast<ColdHeader*>(&buffer[0]);
    pheader->has_depth = false;
    size = sizeof(MDConsumerHandle);
-   handle.init(*pheader, *pcold, phot.get());
+   handle.init(*pheader, *pcold, phot.get(), nullptr);
    // Copy the addresses for comparison
    *hotaddr = (uintptr_t)phot.get();
    *coldaddr = (uintptr_t)pcold.get();

src/lib/md/test/Co2TestInstrument.cpp

@@ -43,7 +43,7 @@ MDProducerHandle Co2TestInstrument::producer() {
 
 MDConsumerHandle Co2TestInstrument::consumer() {
    MDConsumerHandle handle;
-   handle.init(*_cheader, *_cold, _hot.get());
+   handle.init(*_cheader, *_cold, _hot.get(), nullptr/*mbo*/);
    return handle;
 }
 

src/lib/md/test/mbo_tests.cpp

@@ -0,0 +1,295 @@
+#ifndef _WIN32
+#include "catch/catch.hpp"
+#include "../IntPrice.h"
+#include "../MBOProcessor.h"
+#include "Co2TestInstrument.h"
+
+#include <random>
+#include <algorithm>
+
+namespace bts {
+namespace md {
+namespace test {
+int mbo_tests = 0;
+
+TEST_CASE("MBO publish snapshot", "[md][mbo]") {
+   MBOProcessor p;
+   // publish a snapshot
+   auto sec = SecurityKey(99);
+   {
+   auto s = p.writer().allocate_snapshot();
+   s->bids.push_back({ ExchangeOrderKey(1), { 100, 0 }, 42 });
+   p.writer().publish_snapshot(sec, &s);
+   }
+
+   auto s = p.reader().acquire_snapshot(sec);
+   REQUIRE(s);
+   REQUIRE(s->bids.size() == 1);
+   REQUIRE(s->bids[0].id.value() == 1);
+   REQUIRE(s->asks.size() == 0);
+}
+
+std::vector<inc::Update> gen_test_orders(uint64_t id=100, inc::UpdateType type=inc::UpdateType::Bid) {
+   std::vector<inc::Update> result;
+   inc::Priority priority = 10000;
+   for (int p = 100; p < 501; p+=100) {
+      // 6 sizes
+      for (int32_t s = 1; s <= 5; ++s) {
+         IntPrice price { p + 5 , -2};
+         auto order_id = ExchangeOrderKey(++id);
+         auto size = s;
+         priority++;
+         result.push_back( inc::Update { 
+            order_id, 
+            priority,
+            price, 
+            size,
+            type,
+            inc::UpdateAction::New
+         });
+         // std::cout << std::endl << order_id.value() << ": " << price.price() << "-" << size;
+      }
+   }
+   std::random_shuffle(result.begin(), result.end());
+   return result;
+}
+
+TEST_CASE("MBO state to snapshot", "[md][mbo]") {
+   // convert state to snapshot
+   Co2TestInstrument instr { "MBO", api::SECURITYTYPE_FUTURE };
+   auto handle = instr.producer();
+   MBOWorkingState state(handle);
+
+   state.updated = utils::TimeStamp::now();
+
+   auto bids = gen_test_orders(1000, inc::UpdateType::Bid);
+   apply_incremental(state, bids);
+   auto asks = gen_test_orders(2000, inc::UpdateType::Ask);
+   apply_incremental(state, asks);
+
+   MBOSnapshot result;
+   copy_to_snapshot(state, &result);
+
+   REQUIRE(result.updated == state.updated);
+   REQUIRE(result.bids.size() == state.bid.size());
+   REQUIRE(result.asks.size() == state.ask.size());
+   REQUIRE(bids.size() == result.bids.size());
+   REQUIRE(asks.size() == result.asks.size());
+   for (size_t x = 0; x < result.bids.size() - 1; ++x) {
+      REQUIRE(result.bids[x].price >= result.bids[x+1].price);
+   }
+   for (size_t x = 0; x < result.asks.size() - 1; ++x) {
+      REQUIRE(result.asks[x].price <= result.asks[x+1].price);
+   }
+}
+
+template <class T>
+std::vector<T> concat(const std::vector<T>& a, const std::vector<T>& b) {
+   std::vector<T> res;
+   std::copy(a.begin(), a.end(), std::back_inserter(res));
+   std::copy(b.begin(), b.end(), std::back_inserter(res));
+   return res;
+}
+
+TEST_CASE("MBO incremental", "[md][mbo]") {
+   // basic incremental test
+   Co2TestInstrument instr { "MBO", api::SECURITYTYPE_FUTURE };
+   auto handle = instr.producer();
+   MBOWorkingState state(handle);
+   
+   auto orders = 
+      concat( gen_test_orders(128, inc::UpdateType::Bid),
+              gen_test_orders(256, inc::UpdateType::Ask));
+
+   std::random_shuffle(orders.begin(), orders.end());
+   apply_incremental(state, orders);
+
+   REQUIRE(state.bid.size() + state.ask.size() == orders.size());
+
+   // generate some updates/deletes
+   std::vector<inc::Update> updates;
+   // - change all size=2 to size=6
+   // - delete all size=3
+   size_t removed = 0;
+   for (const auto& e : orders) {
+      if (e.size == 2) {
+         updates.push_back( inc::Update {
+            e.order_id,
+            e.priority,
+            e.price,
+            6,
+            e.type,
+            inc::UpdateAction::Change
+         });
+      }
+      else if (e.size == 3) {
+         ++removed;
+         updates.push_back( inc::Update {
+            e.order_id,
+            e.priority,
+            e.price,
+            e.size,
+            e.type,
+            inc::UpdateAction::Delete
+         });
+      }
+   }
+   apply_incremental(state,updates);
+
+   REQUIRE(state.bid.size() + state.ask.size() == (orders.size() - removed));
+
+   MBOSnapshot result;
+   copy_to_snapshot(state, &result);
+
+   auto all = concat (result.bids, result.asks);
+   std::map<int,int> counts;
+   for (const auto& order : all) {
+      REQUIRE(order.size != 3);
+      REQUIRE(order.size != 2);
+      counts[order.size] ++;
+   }
+   REQUIRE(counts[6] > 0);
+   REQUIRE(counts[6] == counts[1]);
+   REQUIRE(counts[5] == counts[4]);
+   REQUIRE(counts[6] == counts[5]);
+}
+
+TEST_CASE("MBO sort order", "[md][mbo]") {
+   Co2TestInstrument instr { "MBO", api::SECURITYTYPE_FUTURE };
+   auto handle = instr.producer();
+   MBOWorkingState state(handle);
+
+   std::vector<inc::Update> updates;
+   // multiple updates at the same price
+   for (unsigned x=0; x < 100; ++x) {
+      // bid
+      auto p = 1000 + x;
+      updates.push_back( inc::Update {
+         ExchangeOrderKey(1000 + x),
+         1000 + x,
+         { 1000, -1 },
+         (int)x+1,
+         inc::UpdateType::Bid,
+         inc::UpdateAction::New
+      });
+      // ask
+      if (x < 75) { // ensure bid.size != ask.size
+         updates.push_back( inc::Update {
+            ExchangeOrderKey(2000 + x),
+            2000 + x,
+            { 2000, -1 },
+            (int)x+1,
+            inc::UpdateType::Ask,
+            inc::UpdateAction::New
+         });
+      }
+   }
+   std::random_shuffle(updates.begin(), updates.end());
+
+   apply_incremental(state, updates);
+
+   MBOSnapshot result;
+   copy_to_snapshot(state, &result);
+
+   // Since all orders are at the same price we can assert they are in priority order
+   // Since priority is not explicit in the shapshot we set order_id to the same value
+
+   for (size_t x = 0; x < result.bids.size() - 1; ++x) {
+      REQUIRE(result.bids[x].id.value() < result.bids[x+1].id.value());
+   }
+   for (size_t x = 0; x < result.asks.size() - 1; ++x) {
+      REQUIRE(result.asks[x].id.value() < result.asks[x+1].id.value());
+   }
+}
+
+TEST_CASE("MBO change price and size", "[md][mbo]") {
+   // When an order's size goes up it gets a new priority
+   Co2TestInstrument instr { "MBO", api::SECURITYTYPE_FUTURE };
+   auto handle = instr.producer();
+   MBOWorkingState state(handle);
+   
+   constexpr auto bid = inc::UpdateType::Bid;
+   constexpr auto ask = inc::UpdateType::Ask;
+   constexpr auto add = inc::UpdateAction::New;
+   constexpr auto remove = inc::UpdateAction::Delete;
+   constexpr auto change = inc::UpdateAction::Change;
+   auto p = [](double price) -> IntPrice { return IntPrice(price * 1000, -3); };
+
+   // snapshot
+   std::vector<inc::Update> snapshot = {
+      { ExchangeOrderKey(701267154552), 1160179309, p(2.975), 1, ask, add},
+      { ExchangeOrderKey(701267991871), 1161068569, p(2.925), 9, bid, add},
+      { ExchangeOrderKey(701267963307), 1161037482, p(2.975), 1, ask, add},
+      { ExchangeOrderKey(701267991967), 1161068670, p(2.900), 1, bid, add},
+      { ExchangeOrderKey(701267928346), 1160999982, p(3.000), 1, ask, add},
+      { ExchangeOrderKey(701267993125), 1161069878, p(2.900), 3, bid, add},
+      { ExchangeOrderKey(701267037972), 1160052467, p(2.875), 1, bid, add},
+      { ExchangeOrderKey(701267099137), 1160119020, p(2.875), 5, bid, add},
+      { ExchangeOrderKey(701267299079), 1160330214, p(2.875), 1, bid, add},
+      { ExchangeOrderKey(701267308619), 1160340068, p(2.875), 5, bid, add},
+      { ExchangeOrderKey(701267310436), 1160341952, p(2.875), 3, bid, add},
+      { ExchangeOrderKey(701267872642), 1160939720, p(2.875), 1, bid, add},
+      { ExchangeOrderKey(701267895533), 1160964252, p(2.875), 1, bid, add},
+      { ExchangeOrderKey(701267953989), 1161027476, p(2.875), 3, bid, add},
+      { ExchangeOrderKey(701267993135), 1161069890, p(2.875), 2, bid, add},
+      { ExchangeOrderKey(701266410447), 1159379621, p(2.850), 3, bid, add},
+      { ExchangeOrderKey(701267324626), 1160356597, p(2.850), 3, bid, add},
+      { ExchangeOrderKey(701267992564), 1161069297, p(2.850), 1, bid, add},
+      { ExchangeOrderKey(701266420294), 1159389827, p(2.775), 5, bid, add},
+      { ExchangeOrderKey(701267334462), 1160366796, p(2.775), 5, bid, add},
+      { ExchangeOrderKey(701266411103), 1159380297, p(2.625), 5, bid, add},
+      { ExchangeOrderKey(701267325284), 1160357275, p(2.625), 5, bid, add},
+   };
+
+   apply_incremental(state, snapshot);
+   REQUIRE(state.bid.size() == 19);
+   REQUIRE(state.ask.size() == 3);
+
+   std::vector<inc::Update> incremental = {
+      { ExchangeOrderKey(701267993738), 1161070510, p(2.925), 2, bid, add},
+      { ExchangeOrderKey(701267993759), 1161070532, p(2.975), 3, ask, add},
+      { ExchangeOrderKey(701267993884), 1161070662, p(2.950), 3, ask, add},
+      { ExchangeOrderKey(701267994228), 1161071029, p(2.925), 2, bid, add},
+      { ExchangeOrderKey(701267994243), 1161071044, p(2.925), 4, bid, add},
+      { ExchangeOrderKey(701267994228), 1161071029, p(2.925), 2, bid, remove},
+      { ExchangeOrderKey(701267993135), 1161071281, p(2.875), 3, bid, change},
+   };
+   apply_incremental(state, incremental);
+   REQUIRE(state.bid.size() == 21);
+   REQUIRE(state.ask.size() == 5);
+   
+   {
+   auto iter = state.bid.begin();
+   auto next = state.bid.begin();
+   next++;
+   for (; next != state.bid.end(); ++iter, ++next) {
+      auto cur_price = std::get<IntPrice>(iter->first);
+      auto cur_prio =  std::get<1>(iter->first);
+      auto next_price = std::get<IntPrice>(next->first);
+      auto next_prio =  std::get<1>(next->first);
+      REQUIRE(cur_price >= next_price);
+      if (cur_price == next_price) {
+         REQUIRE(cur_prio < next_prio);
+      }
+   }
+   }
+   {
+   auto iter = state.ask.begin();
+   auto next = state.ask.begin();
+   next++;
+   for (; next != state.ask.end(); ++iter, ++next) {
+      auto cur_price = std::get<IntPrice>(iter->first);
+      auto cur_prio =  std::get<1>(iter->first);
+      auto next_price = std::get<IntPrice>(next->first);
+      auto next_prio =  std::get<1>(next->first);
+      REQUIRE(cur_price <= next_price);
+      if (cur_price == next_price) {
+         REQUIRE(cur_prio < next_prio);
+      }
+   }
+   }
+   
+}
+
+}}}
+#endif

src/lib/md/test/md2_tests.cpp

@@ -1,3 +1,4 @@
+#ifndef _WIN32
 #include "catch/catch.hpp"
 #include "../SettlementRecorder.h"
 #include "../coconut2.h"
@@ -15,6 +16,7 @@
 
 #include "lib/config/protos/MarketData.pb.h"
 #include "lib/utils/Sqlite_utils.h"
+#include "md_test_utils.h"
 
 #include <vector>
 #include <string>
@@ -55,29 +57,8 @@ Platform TEST_PLATFORM = PLATFORM_SIM;
 //
 // Helper functions for Producer/Consumer tests
 //
-vector<MDProducerHandle> create_test_securities(MDProducer& producer, const char* postfix ="") {
-   FixedFutureSecurity ffs;
-   FixedOptionSecurity fos;
-   ffs.symbol("FOO"+string(postfix));
-   fos.symbol("FOOX"+string(postfix));
-   ffs.exchange = VENUE_BTS;
-   fos.exchange = VENUE_BTS;
-   ffs.key = invalid_security_key;
-   fos.key = invalid_security_key;
-   ffs.expiration_year = fos.expiration_year = 2015;
-   ffs.expiration_day = fos.expiration_day = 27;
-   fos.strike = 1000;
-   fos.call_put = PUTORCALL_CALL;
-   // Add 12 expirations to 2 different shared memory regions
-   vector<MDProducerHandle> result;
-   for (uint32_t x = 1; x <= 12; ++x) {
-      ffs.expiration_month = fos.expiration_month = x;
-      result.push_back(producer.add_security(ffs));
-      result.push_back(producer.add_security(fos));
-      // In a real app these securities would now be passed to the SecurityIDRecorder
-   }
-   return result;
-}
+
+std::vector<MDProducerHandle> create_test_securities(MDProducer& producer, const char* postfix = "");
 
 set<UserSymbol> get_test_symbols(const char* postfix="") {
    set<UserSymbol> symbols;
@@ -146,12 +127,6 @@ Security mkspread(string symbol, security_key key, Security leg1, Security leg2,
    return mkspread(symbol, key, vlegs);
 }
 
-void assign_ids(vector<MDProducerHandle>::iterator begin, vector<MDProducerHandle>::iterator end, security_key start_key = 1000) {
-   for (; begin != end; ++begin) {
-      (*begin).set_key(++start_key);
-   }
-}
-
 
 class Co2Tests 
 {
@@ -370,7 +345,7 @@ public:
       UserSymbol option_symbol( "FOOX", VENUE_BTS, SECURITYTYPE_OPTION );
       symbols.insert(option_symbol);
       SecurityCache cache;
-      MDConsumer consumer(cache, nullptr, &symbols, true, TEST_PLATFORM);
+      MDConsumer consumer(cache, nullptr, nullptr, &symbols, true, TEST_PLATFORM);
 
       vector<Security> consumer_securities;
       // Poll for security list - none visible until they have keys
@@ -441,7 +416,7 @@ public:
       UserSymbol option_symbol( "FOOX", VENUE_BTS, SECURITYTYPE_OPTION );
       symbols.insert(option_symbol);
       SecurityCache cache;
-      MDConsumer consumer(cache, nullptr, &symbols, true, TEST_PLATFORM);
+      MDConsumer consumer(cache, nullptr, nullptr, &symbols, true, TEST_PLATFORM);
 
       // Add the on_security callback
       vector<Security> securities;
@@ -481,7 +456,7 @@ public:
       UserSymbol option_symbol( "FOOX", VENUE_BTS, SECURITYTYPE_OPTION );
       symbols.insert(option_symbol);
       SecurityCache cache;
-      MDConsumer consumer(cache, nullptr, &symbols, true, TEST_PLATFORM);
+      MDConsumer consumer(cache, nullptr, nullptr, &symbols, true, TEST_PLATFORM);
 
       // Get a handle to 24 consumers which haven't been created yet
       vector<MDConsumerHandle> consumers;
@@ -539,7 +514,7 @@ public:
       UserSymbol option_symbol( "FOOX", VENUE_BTS, SECURITYTYPE_OPTION );
       symbols.insert(option_symbol);
       SecurityCache cache;
-      MDConsumer consumer(cache, nullptr, &symbols, true, TEST_PLATFORM);
+      MDConsumer consumer(cache, nullptr, nullptr, &symbols, true, TEST_PLATFORM);
 
       // Get a handle to 24 consumers which haven't been created yet
       vector<MDConsumerHandle> consumers;
@@ -600,7 +575,7 @@ public:
       // Symbol list for the consumer
       auto symbols = get_test_symbols();
       SecurityCache cache;
-      MDConsumer consumer(cache, nullptr, &symbols, true, TEST_PLATFORM);
+      MDConsumer consumer(cache, nullptr, nullptr, &symbols, true, TEST_PLATFORM);
 
       // Add the on_security callback - callback fires for existing securities
       vector<Security> securities;
@@ -628,7 +603,7 @@ public:
       // Symbol list for the consumer
       SecurityCache cache;
       auto symbols = get_test_symbols();
-      MDClient client(nullptr, nullptr, &symbols);
+      MDClient client(nullptr, nullptr, nullptr, &symbols);
       MDConsumer consumer = client.current_consumer();
       map<security_key, MDConsumerHandle> products;
       consumer.on_security([&](const SecurityPtr& security, MDConsumerHandle handle) {
@@ -761,7 +736,7 @@ public:
       UserSymbol option_symbol( "FOOX", VENUE_BTS, SECURITYTYPE_OPTION );
       symbols.insert(option_symbol);
       SecurityCache cache;
-      MDConsumer consumer(cache, nullptr, &symbols, true, TEST_PLATFORM);
+      MDConsumer consumer(cache, nullptr, nullptr, &symbols, true, TEST_PLATFORM);
 
       set<security_key> keys;
       consumer.walk_security_list([&](const SecurityPtr& security, MDConsumerHandle) { keys.insert(security->id()); });
@@ -806,7 +781,7 @@ public:
       csymbols.insert(UserSymbol("S3", VENUE_BTS, SECURITYTYPE_SPREAD));
       csymbols.insert(UserSymbol("S4", VENUE_BTS, SECURITYTYPE_SPREAD));
       SecurityCache cache;
-      MDConsumer consumer(cache, nullptr, &csymbols, true, TEST_PLATFORM);
+      MDConsumer consumer(cache, nullptr, nullptr, &csymbols, true, TEST_PLATFORM);
 
       auto check_sec = [&](const SecurityPtr& security, MDConsumerHandle handle) {
          REQUIRE(security->id() == handle.key());
@@ -853,7 +828,7 @@ public:
       csymbols.insert(UserSymbol("OO", VENUE_BTS, SECURITYTYPE_OPTION));
       csymbols.insert(UserSymbol("SS", VENUE_BTS, SECURITYTYPE_SPREAD));
       SecurityCache cache;
-      MDConsumer consumer(cache, nullptr, &csymbols, true, TEST_PLATFORM);
+      MDConsumer consumer(cache, nullptr, nullptr, &csymbols, true, TEST_PLATFORM);
 
       auto check_sec = [&](const SecurityPtr& security, MDConsumerHandle handle) {
          REQUIRE(security->id() == handle.key());
@@ -905,7 +880,7 @@ public:
       consumer_symbols.insert(UserSymbol("OO", VENUE_BTS, SECURITYTYPE_OPTION));
       consumer_symbols.insert(UserSymbol("SS", VENUE_BTS, SECURITYTYPE_SPREAD));
       SecurityCache cache;
-      MDConsumer consumer(cache, nullptr, &consumer_symbols, true, TEST_PLATFORM);
+      MDConsumer consumer(cache, nullptr, nullptr, &consumer_symbols, true, TEST_PLATFORM);
 
       size_t security_count = 0;
       // Indirection for check function
@@ -974,7 +949,7 @@ public:
       set<UserSymbol> consumer_symbols;
       consumer_symbols.insert(UserSymbol("OO", VENUE_BTS, SECURITYTYPE_OPTION));
       SecurityCache cache;
-      MDConsumer consumer(cache, nullptr, &consumer_symbols, true, TEST_PLATFORM);
+      MDConsumer consumer(cache, nullptr, nullptr, &consumer_symbols, true, TEST_PLATFORM);
       auto h1 = consumer.get_handle(k1);
       REQUIRE(h1.is_initialized());
       auto h2 = consumer.get_handle(k2);
@@ -1260,7 +1235,7 @@ struct Co2RelayTests {
       UserSymbol option_symbol("FOOX", VENUE_BTS, SECURITYTYPE_OPTION);
       symbols.insert(option_symbol);
       SecurityCache cache;
-      MDConsumer consumer(cache, nullptr, &symbols);
+      MDConsumer consumer(cache, nullptr, nullptr, &symbols);
       return consumer;
    }
 
@@ -1327,7 +1302,7 @@ struct Co2RelayTests {
       // todo: setup_query_server should use client.symbols() instead of it's own list
       //
       const auto symbols = setup_symbols();
-      MDClient client(nullptr, nullptr, &symbols);
+      MDClient client(nullptr, nullptr, nullptr, &symbols);
       auto server = setup_query_server(client, producer, &entitlements, &producers);
 
       // assign prices
@@ -1460,7 +1435,7 @@ struct Co2RelayTests {
       vector<MDProducerHandle> producers;
       Entitlements entitlements;
       const auto symbols = setup_symbols();
-      MDClient client(nullptr, nullptr, &symbols);
+      MDClient client(nullptr, nullptr, nullptr, &symbols);
       auto server = setup_query_server(client, producer, &entitlements, &producers);
 
       // assign prices to all levels
@@ -1638,7 +1613,7 @@ namespace ImpliedTests {
       UserSymbol option_symbol( "FOOX", VENUE_BTS, SECURITYTYPE_OPTION );
       symbols.insert(option_symbol);
       SecurityCache cache;
-      MDConsumer consumer(cache, nullptr, &symbols);
+      MDConsumer consumer(cache, nullptr, nullptr, &symbols);
       vector<Security> consumer_securities;
       // Poll for security list - none visible until they have keys
       size_t count = consumer.walk_security_list([&](const SecurityPtr& security, MDConsumerHandle) {consumer_securities.push_back(*security);});
@@ -2417,3 +2392,4 @@ TEST_CASE("IntPrice", "[md]") {
 
 
 }}}
+#endif

src/lib/md/test/md_test_utils.cpp

@@ -0,0 +1,41 @@
+#include "md_test_utils.h"
+#include "../MDProducer.h"
+
+#include <vector>
+#include <string>
+
+namespace bts {
+namespace md {
+namespace test {
+
+std::vector<MDProducerHandle> create_test_securities(MDProducer& producer, const char* postfix ="") {
+   FixedFutureSecurity ffs;
+   FixedOptionSecurity fos;
+   ffs.symbol("FOO"+std::string(postfix));
+   fos.symbol("FOOX"+std::string(postfix));
+   ffs.exchange = api::VENUE_BTS;
+   fos.exchange = api::VENUE_BTS;
+   ffs.key = invalid_security_key;
+   fos.key = invalid_security_key;
+   ffs.expiration_year = fos.expiration_year = 2015;
+   ffs.expiration_day = fos.expiration_day = 27;
+   fos.strike = 1000;
+   fos.call_put = api::PUTORCALL_CALL;
+   // Add 12 expirations to 2 different shared memory regions
+   std::vector<MDProducerHandle> result;
+   for (uint32_t x = 1; x <= 12; ++x) {
+      ffs.expiration_month = fos.expiration_month = x;
+      result.push_back(producer.add_security(ffs));
+      result.push_back(producer.add_security(fos));
+      // In a real app these securities would now be passed to the SecurityIDRecorder
+   }
+   return result;
+}
+
+void assign_ids(std::vector<MDProducerHandle>::iterator begin, std::vector<MDProducerHandle>::iterator end, security_key start_key) {
+   for (; begin != end; ++begin) {
+      (*begin).set_key(++start_key);
+   }
+}
+
+}}}//namespace

src/lib/md/test/md_test_utils.h

@@ -2,6 +2,7 @@
 
 #include "lib/md/IntPrice.h"
 #include "lib/md/PriceSize.h"
+#include "../MDProducerHandle.h"
 #include <vector>
 
 namespace bts {
@@ -28,5 +29,6 @@ Book compute_synthetic_book(const api::Security& security, const std::vector<Boo
 
 std::ostream& operator<< (std::ostream& os, const PS& val);
 
+void assign_ids(std::vector<MDProducerHandle>::iterator begin, std::vector<MDProducerHandle>::iterator end, security_key start_key = 1000);
 
 }}}//namespace

src/lib/orderman/actors/order_manager/OrderServices.h

@@ -166,6 +166,8 @@ class OrderServices {
    void report_state(const api::orderman::OrderState &state,
                      trace::CorID const &trace = trace::CorID()) {
 
+      bts_assert(state.status() != api::orderman::ORDER_STATUS_INACTIVE);
+      
       OrderStateUpdateEvent ev(_owner->key(), _owner->parent_key(),
                                _owner->definition(), state, trace);
       _data->owner.publish(_data->report_order_state_pub, ev);

src/lib/orderman/orders/spreader_orders/SpreaderOrder.cpp

@@ -452,7 +452,7 @@ bts::api::orderman::OrderStatus SpreaderOrder::_get_order_status () const{
             bts::api::orderman::OrderStatus::ORDER_STATUS_COMPLETED;
       case SpreaderState::START:
       case SpreaderState::WAITING_FOR_INIT_DATA:
-         return bts::api::orderman::OrderStatus::ORDER_STATUS_INACTIVE;
+         return bts::api::orderman::OrderStatus::ORDER_STATUS_WORKING;
    default:
       bts_assert(false);
       return bts::api::orderman::OrderStatus::ORDER_STATUS_INACTIVE;

src/lib/os/os.cpp

@@ -1000,6 +1000,31 @@ extern "C" BTS_EXPORT char *bts_data_path() {
 }
 
 extern "C" BTS_EXPORT void free_bts_data_path(char *dp) { delete[] dp; }
+
+extern "C" BTS_EXPORT unsigned int setup_process_job_object()
+{
+#ifdef _WIN32
+   if (IsDebuggerPresent()) 
+      return 0;
+   HANDLE hjob = CreateJobObject(nullptr, nullptr);
+   if (hjob == nullptr) 
+      return 100; //Failed to create JOB object. 
+
+   JOBOBJECT_EXTENDED_LIMIT_INFORMATION jeli = {0};
+   jeli.BasicLimitInformation.LimitFlags = JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE;   
+         
+   BOOL success = SetInformationJobObject(hjob, JobObjectExtendedLimitInformation, &jeli, sizeof(jeli));
+   DWORD error = GetLastError();
+   if(!success) 
+      return error;
+   success = AssignProcessToJobObject(hjob, GetCurrentProcess());
+   error = GetLastError();
+   if(!success) 
+      return error;
+#endif   
+   return 0;
+}
+
 }
 }
 } // end namespaces

src/lib/utils/ConcurrentPool.h

@@ -0,0 +1,69 @@
+#pragma once 
+#include <tbb/concurrent_queue.h>
+#include <tbb/concurrent_vector.h>
+#include <atomic>
+
+namespace bts { 
+namespace utils { 
+
+template <class T>
+class ConcurrentPool {
+   tbb::concurrent_vector<std::unique_ptr<T[]>> _all;
+   tbb::concurrent_queue<T*> _free;
+   size_t _initial_size;
+   size_t _grow_size;
+   size_t _outstanding;
+   public:
+   ConcurrentPool(size_t initial_size=256, size_t grow_size=32): 
+   _all(), _free(), _initial_size(initial_size), _grow_size(grow_size), _outstanding(0) {
+      reserve(initial_size);
+   }
+
+   ~ConcurrentPool() { }
+
+   T* allocate() {
+      T* result = nullptr;
+      ++_outstanding;
+      if (_free.try_pop(result)) {
+         return result;
+      }
+      else {
+         reserve(_grow_size);
+         return allocate();
+      }
+   }
+
+   void reserve(size_t count) {
+      std::unique_ptr<T[]> buffer {new T[count]};
+      _outstanding += count;
+      for (size_t x = 0; x < count; ++x) {
+         T* temp = &buffer[x];
+         release(&temp);
+      }
+      _all.push_back(std::move(buffer));
+   }
+
+   void release(T** item) {
+      if (*item) {
+         (*item)->reset();
+         _free.push(*item);
+         *item = nullptr;
+         --_outstanding;
+      }
+   }
+
+   template <class CB>
+   void unsafe_iterate(CB cb) const {
+      for (auto iter = _all.begin(); iter != _all.end(); ++iter) {
+         size_t size = (iter == _all.begin()) ? _initial_size : _grow_size;
+         for (size_t x = 0; x < size; ++x) {
+            const std::unique_ptr<T[]>& bucket = *iter;
+            const T* start = bucket.get();
+            cb(start[x]);
+         }
+      }
+   }
+
+};
+
+}} //namespace 

src/lib/utils/Dispatcher.cpp

@@ -200,7 +200,7 @@ void Dispatcher::invoke(Action action)
    }
 }
 
-void Dispatcher::post(Action action, int64_t delay_ms)
+void Dispatcher::post(Action&& action, int64_t delay_ms)
 {
    if (delay_ms > 0)
    {

src/lib/utils/Dispatcher.h

@@ -87,7 +87,7 @@ public:
    // Perform an action on the dispatcher thread.  Block until the action is complete.
    void invoke(Action action);
    // Queue an action on the dispatcher thread and return immediately.
-   void post(Action action, int64_t delay_ms=0);
+   void post(Action&& action, int64_t delay_ms=0);
 
    boost::asio::io_service* io_service();
 

src/lib/utils/LockBox.h

@@ -0,0 +1,24 @@
+#pragma once 
+#include <mutex>
+
+namespace bts { 
+namespace utils { 
+
+template <class T>
+class LockBox {
+   mutable std::mutex _mutex;
+   T _data;
+public:
+   template <class CB>
+   auto lock(CB cb) {
+      std::lock_guard<std::mutex> guard(_mutex);
+      return cb(_data);
+   }
+   template <class CB>
+   auto lock(CB cb) const {
+      std::lock_guard<std::mutex> guard(_mutex);
+      return cb(_data);
+   }
+};
+
+}} //namespace 

src/server/sparkd/cmds/Benchmark.cpp

@@ -640,6 +640,21 @@ static void perf_log(benchmark::State &state) {
 }
 BENCHMARK(perf_log);
 
+void int_price_less(benchmark::State& state) {
+   for (auto _ : state) {
+      state.PauseTiming();
+      int64_t count = state.range(0);
+      std::vector<md::IntPrice> prices;
+      prices.reserve((int)count);
+      for (int64_t x = 0; x < count; ++x) {
+         prices.push_back({x % 100, (int8_t)-(x%5)});
+      }
+      state.ResumeTiming();
+      std::sort(prices.begin(), prices.end());
+   }
+}
+BENCHMARK(int_price_less)->Range(8, 8 << 11);
+
 std::string Benchmark::help() {
    return "Run micro-benchmarks. Use --benchmark_filter=regex to run only "
           "specific benchmarks.";

src/server/sparkd/cmds/Run.cpp

@@ -14,6 +14,8 @@
 #include "lib/md/MDClient.h"
 #include "lib/md/SyntheticSecurityService.h"
 #include "lib/md/relay/QueryServer.h"
+#include "lib/md/MBOProcessor.h"
+#include "lib/md/MBOServiceServer.h"
 #include "lib/metrics/metrics.h"
 #include "lib/metrics/system_metrics.h"
 #include "lib/orderman/order_factory/DynamicOrderRegistry.h"
@@ -57,15 +59,19 @@ int Run::execute(std::vector<std::string> const &) {
 
    persistence::PersistenceServiceServer settings_saver(sc, "settings");
    user::AuthServiceServer login_server(sc);
-   md::run_market_datapublishers();
+
+   md::MBOProcessor mbo(&utils::Dispatcher::get_named("MBO"));
+   auto* mbowriter = config::get<config::toggles::PlacementInQueue>().enabled() ? &mbo.writer() : nullptr;
+   md::run_market_datapublishers(mbowriter);
 
    api::synthetic::SyntheticSecurityService synthetic_service(sc);
-   md::MDClient mdclient(&utils::Dispatcher::background(), &synthetic_service);
-
+   md::MDClient mdclient(&utils::Dispatcher::background(), &synthetic_service, &mbo.reader());
 
    md::QueryServerPtr md_query_server(
        md::run_query_server(sc, mdclient, stats));
 
+   md::MBOServiceServer mboserver(sc, utils::Dispatcher::background(), mdclient);
+
    // initialize order plugins
    orderman::DynamicOrderRegistry dyn_order_registry;
    SparkServices plugin_services(dyn_order_registry);

src/server/sparkd/cmds/Web.cpp

@@ -37,7 +37,7 @@ int Web::execute(const std::vector<std::string>& args) {
 
    user::AuthServiceServer login_server(sc);
    if (std::find(args.begin(), args.end(), "-md") != args.end()) {
-      md::run_market_datapublishers();
+      md::run_market_datapublishers(nullptr);
    }
 
    api::synthetic::SyntheticSecurityService synthetic_service(sc);

src/server/sparkd/md/platforms/IMDPlatform.h

@@ -7,6 +7,7 @@ class IMDPlatform
 {
 public:
    virtual void send_heartbeats();
+   virtual ~IMDPlatform() {}
 };
 
-}}
+}}

src/server/sparkd/md/platforms/ice_onixs/ICEListener.cpp

@@ -56,17 +56,14 @@ void ICEListener::onMarketStateChange(const onixs_ice::MarketStateChange& , cons
 void ICEListener::onError(const onixs_ice::Error& error) 
 {
    std::string error_desc(error.description());
-   tracepoint (bts_trace,trace_order,(char*) "onError", 0, (uint16_t) 0, 0);
    LOG(md,Warn, "OnixS ICE Error: {} - {}", error.toString(), error_desc );
 }
 void ICEListener::onFeedError(onixs_ice::FeedId feedid, onixs_ice::ErrorCode code, const std::string& desc)
 {
-   tracepoint (bts_trace,trace_order,(char*) "onFeedError", 0, (uint16_t) 0, 0);
    LOG(md,Warn, "OnixS ICE FeedError: {} - {}", feedid, desc, code);   
 }
 void ICEListener::onWarning(const onixs_ice::Warning& warning) 
 {
-   tracepoint (bts_trace,trace_order,(char*) "onWarning", 0, (uint16_t) 0, 0);
    LOG(md,Warn, "OnixS ICE Warn: {} - {}", warning.toString(), std::string(warning.description()) );
 }
 
@@ -74,31 +71,27 @@ void ICEListener::onWarning(const onixs_ice::Warning& warning)
 // Security discovery
 void ICEListener::onFuturesProductDefinition(const onixs_ice::FuturesProductDefinition& msg, bool& is_interested) 
 {
-   tracepoint (bts_trace,trace_order,(char*) "onFuturesProductDefinition", 0, (uint16_t) 0, 0);
    if(msg.isSpread)
       on_future_strip(msg, is_interested);
    else
       on_future(msg, is_interested);
 }
 
-void ICEListener::onNewExpiry(const onixs_ice::NewExpiry& msg, bool& is_interested, const onixs_ice::MessageInfo& msg_info) 
+void ICEListener::onNewExpiry(const onixs_ice::NewExpiry& msg, bool& is_interested, const onixs_ice::MessageInfo& ) 
 {   
-   tracepoint (bts_trace,trace_order,(char*) "onNewExpiry", (uint32_t)msg_info.sequenceNumber, (uint16_t) msg_info.sequenceNumber, 0);
    on_future(msg, is_interested);
 }
 
 void ICEListener::onFuturesStrategyDefinition(const onixs_ice::FuturesStrategyDefinition& def, bool& is_interested) 
 {
-   tracepoint (bts_trace,trace_order,(char*) "onFuturesStrategyDefinition", 0, (uint16_t) 0, 0);
    //TODO need to find out how to produce data for testing
    //on_uds_future_spread(def, is_interested);
    UNUSED(def);
    UNUSED(is_interested);   
 }
 //This is only for UDS
-void ICEListener::onNewFuturesStrategyDefinition(const onixs_ice::NewFuturesStrategyDefinition& msg, bool& is_interested, const onixs_ice::MessageInfo& msg_info)
+void ICEListener::onNewFuturesStrategyDefinition(const onixs_ice::NewFuturesStrategyDefinition& msg, bool& is_interested, const onixs_ice::MessageInfo& )
 {
-   tracepoint (bts_trace,trace_order,(char*) "onFuturesStrategyDefinition", (uint32_t)msg_info.sequenceNumber, (uint16_t) msg_info.sequenceNumber, 0);
    //TODO need to find out how to produce data for testing
    //on_uds_future_spread(msg, is_interested);
    UNUSED(msg);
@@ -152,7 +145,7 @@ void ICEListener::onMarketSnapshotPriceLevel(const onixs_ice::MarketSnapshotPric
 // Book
 void ICEListener::onOrderBookUpdated(const onixs_ice::OrderBook& onixs_book, const onixs_ice::MessageInfo& msg_info) 
 {
-   tracepoint (bts_trace,trace_order,(char*) "onBookUpdate", (uint32_t)msg_info.sequenceNumber, (uint16_t) msg_info.sequenceNumber, 0);
+   //tracepoint (bts_trace,trace_order,(char*) "onBookUpdate", (uint32_t)msg_info.sequenceNumber, (uint16_t) msg_info.sequenceNumber, 0);
       
    auto data = _index.get_security(onixs_book.marketId());
    if (data == nullptr) 
@@ -214,9 +207,8 @@ void on_trade(ICESecurityData* data, const onixs_ice::Price& price, int size, on
    data->handle.add_trade(ts, ps, date, map_aggressor(agside), size);
 }
 
-void ICEListener::onTrade(const onixs_ice::Trade& msg, const onixs_ice::MessageInfo& msg_info) 
+void ICEListener::onTrade(const onixs_ice::Trade& msg, const onixs_ice::MessageInfo& ) 
 {
-   tracepoint (bts_trace,trace_order,(char*) "onTrade", (uint32_t)msg_info.sequenceNumber, (uint16_t) msg_info.sequenceNumber, 0);
    auto* data = _index.get_security(msg.marketId);
    if (data != NULL) 
    {
@@ -230,9 +222,8 @@ void ICEListener::onTrade(const onixs_ice::Trade& msg, const onixs_ice::MessageI
 
 //
 // RFQ
-void ICEListener::onRfq(const onixs_ice::Rfq& msg, const onixs_ice::MessageInfo& msg_info) 
+void ICEListener::onRfq(const onixs_ice::Rfq& msg, const onixs_ice::MessageInfo& ) 
 {
-   tracepoint (bts_trace,trace_order,(char*) "onRfq", (uint32_t)msg_info.sequenceNumber, (uint16_t) msg_info.sequenceNumber, 0);
    auto* data = _index.get_security(msg.marketId);
    if (data != nullptr) 
    {
@@ -267,9 +258,8 @@ void ICEListener::onRfq(const onixs_ice::Rfq& msg, const onixs_ice::MessageInfo&
 
 //
 // Misc
-void ICEListener::onSettlementPrice(const onixs_ice::SettlementPrice& msg, const onixs_ice::MessageInfo& msg_info) 
+void ICEListener::onSettlementPrice(const onixs_ice::SettlementPrice& msg, const onixs_ice::MessageInfo& ) 
 {   
-   tracepoint (bts_trace,trace_order,(char*) "onSettlementPrice", (uint32_t)msg_info.sequenceNumber, (uint16_t) msg_info.sequenceNumber, 0);
    auto* data = _index.get_security(msg.marketId);
    if (data != nullptr) 
    {
@@ -287,9 +277,8 @@ void ICEListener::onOpenInterest(const onixs_ice::OpenInterest &msg, const onixs
    UNUSED(msg_info);
 }
 
-void ICEListener::onOpenPrice(const onixs_ice::OpenPrice& msg, const onixs_ice::MessageInfo& msg_info) 
+void ICEListener::onOpenPrice(const onixs_ice::OpenPrice& msg, const onixs_ice::MessageInfo& ) 
 {
-   tracepoint (bts_trace,trace_order,(char*) "onOpenPrice", (uint32_t)msg_info.sequenceNumber, (uint16_t) msg_info.sequenceNumber, 0);
    auto* data = _index.get_security(msg.marketId);
    if (data != nullptr) 
    {
@@ -301,9 +290,8 @@ void ICEListener::onOpenPrice(const onixs_ice::OpenPrice& msg, const onixs_ice::
    }   
 }
 
-void ICEListener::onClosePrice(const onixs_ice::ClosePrice& msg, const onixs_ice::MessageInfo& msg_info) 
+void ICEListener::onClosePrice(const onixs_ice::ClosePrice& msg, const onixs_ice::MessageInfo& ) 
 {
-   tracepoint (bts_trace,trace_order,(char*) "onClosePrice", (uint32_t)msg_info.sequenceNumber, (uint16_t) msg_info.sequenceNumber, 0);
    auto* data = _index.get_security(msg.marketId);
    if (data != nullptr) 
    {
@@ -315,9 +303,8 @@ void ICEListener::onClosePrice(const onixs_ice::ClosePrice& msg, const onixs_ice
    }     
 }
 
-void ICEListener::onEndOfDayMarketSummary(const onixs_ice::EndOfDayMarketSummary& msg, const onixs_ice::MessageInfo& msg_info) 
+void ICEListener::onEndOfDayMarketSummary(const onixs_ice::EndOfDayMarketSummary& msg, const onixs_ice::MessageInfo& ) 
 {
-   tracepoint (bts_trace,trace_order,(char*) "onEndOfDayMarketSummary", (uint32_t)msg_info.sequenceNumber, (uint16_t) msg_info.sequenceNumber, 0);
    auto* data = _index.get_security(msg.marketId);
    if (data != nullptr) 
    {
@@ -333,18 +320,15 @@ void ICEListener::onEndOfDayMarketSummary(const onixs_ice::EndOfDayMarketSummary
 
 //
 // Feed
-void ICEListener::onMulticastMessageBlockBegin(onixs_ice::FeedId feedId, const onixs_ice::MessageInfo& msg_info) 
+void ICEListener::onMulticastMessageBlockBegin(onixs_ice::FeedId feedId, const onixs_ice::MessageInfo& ) 
 {
    UNUSED(feedId);
-   tracepoint (bts_trace,trace_order,(char*) "onMulticastMessageBlockBegin", (uint32_t)msg_info.sequenceNumber, (uint16_t) msg_info.sequenceNumber, 0);
 }
 
 void ICEListener::onMulticastMessageBlockEnd(onixs_ice::FeedId feedId, const onixs_ice::MarketIds& marketIds) 
 {
    UNUSED(feedId);
    UNUSED(marketIds);
-   tracepoint (bts_trace,trace_order,(char*) "onMulticastMessageBlockEnd.begin",  0, (uint16_t) 0, 0);
-   tracepoint (bts_trace,trace_order,(char*) "onMulticastMessageBlockEnd.end", 0, (uint16_t) 0, 0);
 }
 
 void ICEListener::onFeedStarted(onixs_ice::FeedId, const onixs_ice::MarketIds&) 

src/server/sparkd/md/platforms/ice_onixs/ICEMain.cpp

@@ -217,7 +217,6 @@ unique_ptr<ICEListener> ICEMain::prepare_listener(onixs_ice::Handler& handler, c
       producer_channels.push_back(_producer.get_channel(symbol.symbol().c_str(), symbol.type(), symbol.exchange()));
    }
 
-   const auto& options = channel_settings_for(channel);
    unique_ptr<ICEListener> listener(new ICEListener(_index, _producer, /*_settlement_recorder,*/ producer_channels));
    for (auto& symbol : channel.symbols) 
       listener->add_filter_symbol(symbol.symbol());   

src/server/sparkd/md/platforms/mdp3_onixs/CMESecurityIndex.h

@@ -1,15 +1,17 @@
 #pragma once
 
 #include "lib/md/SecurityIndex.h"
+#include "lib/md/MBOProcessor.h"
 #include <string>
 
 namespace bts {
 namespace md {
 
 struct SecurityData {
-   SecurityData(uint32_t id, MDProducerHandle& h) : id(id), handle(h), electronic_volume(0) {}
+   SecurityData(uint32_t id, MDProducerHandle& h) : id(id), handle(h), mbo_state(h), electronic_volume(0) { }
    uint32_t id; // exchange id
    MDProducerHandle handle;
+   MBOWorkingState mbo_state;
    uint32_t electronic_volume;
 };
 

src/server/sparkd/md/platforms/mdp3_onixs/MDP3Listener.cpp

@@ -15,7 +15,7 @@ namespace md {
 MDP3Listener::MDP3Listener(
       CMESecurityIndex& index,
       MDProducer& producer,
-      // co2::SettlementRecorder& ,
+      MBOWriter* mbo,
       const std::vector<MDProducerChannel> channels) :
    _index(index),
    _producer(producer),
@@ -27,7 +27,8 @@ MDP3Listener::MDP3Listener(
    _symbol_filter(),
    _channels(channels),
    _concurrency_monitor(0),
-   _book_updates()
+   _book_updates(),
+   _mbo(mbo)
 {
    _book_updates.reserve(32);
 }
@@ -109,7 +110,7 @@ void MDP3Listener::onEndOfPacket(onixs::Handler&, const onixs::PacketArgs&) {
 }
 
 //
-// Security discovery
+// Book updates
 
 void MDP3Listener::onBookUpdate(onixs::Handler&, const onixs::Security& sec, const onixs::DirectBook& book) {
    onBookUpdate<COCONUT_MAX_DEPTH>(sec, book, book_type_direct);
@@ -119,6 +120,24 @@ void MDP3Listener::onBookUpdate(onixs::Handler&, const onixs::Security& sec, con
    onBookUpdate<COCONUT_MAX_IMPLIED_DEPTH>(sec, book, book_type_implied);
 }
 
+void MDP3Listener::onBookUpdate(onixs::Handler&, const onixs::Security& sec, const onixs::MboBook& book) {
+   if (_mbo) {
+      auto data = _index.get_security(sec.id());
+      if (data == nullptr) return;
+      auto key = data->handle.key();
+      if (key == invalid_security_key) return;
+      auto* snapshot = _mbo->allocate_snapshot();
+      // copy from onixs book
+      for (const auto& bid : book.bids()) {
+         snapshot->bids.push_back({ ExchangeOrderKey(bid.id()), convert_price(bid.price(), 0), bid.quantity() });
+      }
+      for (const auto& ask : book.offers()) {
+         snapshot->asks.push_back({ ExchangeOrderKey(ask.id()), convert_price(ask.price(),0), ask.quantity() });
+      }
+      _mbo->validate_snapshot(sec.id(), SecurityKey(key), snapshot);
+   }
+}
+
 //
 // Trades
 

src/server/sparkd/md/platforms/mdp3_onixs/MDP3Listener.h

@@ -2,6 +2,7 @@
 
 #include "lib/utils/bts_no_copy.h"
 #include "lib/md/coconut2.h"
+#include "lib/md/MBOProcessor.h"
 #include "CMESecurityIndex.h"
 #include "lib/tracing/bts_trace.h"
 
@@ -13,11 +14,57 @@
 #include <atomic>
 #include <array>  
 #include <utility>
+#include <optional>
 
 namespace bts {
 namespace md {
 
-   
+inline inc::UpdateType mbo_update_type(onixs::EntryTypeBook::Enum otype) {
+   switch(otype) {
+      case onixs::EntryTypeBook::Bid: 
+      case onixs::EntryTypeBook::ImpliedBid: 
+         return inc::UpdateType::Bid;
+         break;
+      case onixs::EntryTypeBook::Offer: 
+      case onixs::EntryTypeBook::ImpliedOffer: 
+         return inc::UpdateType::Ask;
+         break;
+      default:
+         bts_assert_msg(false, "Unexpected update type");
+   }
+}
+
+inline inc::UpdateAction mbo_update_action(onixs::OrderUpdateAction::Enum a) {
+   switch(a) {
+      case onixs::OrderUpdateAction::New:
+         return inc::UpdateAction::New;
+      case onixs::OrderUpdateAction::Update:
+         return inc::UpdateAction::Change;
+      case onixs::OrderUpdateAction::Delete:
+         return inc::UpdateAction::Delete;
+      default:
+         bts_assert_msg(false, "Unexpected update action");
+   }
+}
+
+inline inc::UpdateAction mbo_update_action(onixs::UpdateAction::Enum a) {
+   switch(a) {
+      case onixs::UpdateAction::New:
+         return inc::UpdateAction::New;
+      case onixs::UpdateAction::Change:
+         return inc::UpdateAction::Change;
+      case onixs::UpdateAction::Delete:
+         return inc::UpdateAction::Delete;
+      case onixs::UpdateAction::DeleteThru:
+      case onixs::UpdateAction::DeleteFrom:
+      case onixs::UpdateAction::Overlay:
+         return inc::UpdateAction::Delete;
+      default:
+         bts_assert_msg(false, "Unexpected update action");
+   }
+}
+
+
 class MDP3Listener :
    public onixs::HandlerListener,
    public onixs::MarketDataListener,
@@ -31,7 +78,7 @@ class MDP3Listener :
 public:
    MDP3Listener(CMESecurityIndex& index, 
                 MDProducer&, 
-               //  co2::SettlementRecorder&, 
+                MBOWriter* mbo,
                 const std::vector<MDProducerChannel>);
    virtual ~MDP3Listener();
 
@@ -54,6 +101,139 @@ public:
    void onPacket(onixs::Handler&, const onixs::PacketArgs&) override;
    void onEndOfPacket(onixs::Handler&, const onixs::PacketArgs&) override;
 
+   //
+   // Handle MBO incremental updates directly
+
+   void onMessage(onixs::Handler&, const onixs::IncrementalRefreshBook32Args& args) override {
+      onIncrementalRefreshBook(args);
+   };
+   void onMessage(onixs::Handler&, const onixs::IncrementalRefreshBook46Args& args) override {
+      onIncrementalRefreshBook(args);
+   }
+   template <class Args>
+   void onIncrementalRefreshBook(const Args& args) {
+      if (!_mbo) return;
+
+      for(const auto& entry : args.message().orderIdEntries()) {
+         int qty;
+         entry.displayQty(qty);
+         uint8_t ref;
+         if (entry.referenceId(ref)) {
+            // entries is indexed by 0 but ref is indexed by 1  :facepalm:
+            const auto& ref_entry = args.message().entries()[(int)ref-1];
+            uint32_t secid = ref_entry.securityId();
+            auto data = _index.get_security(secid);
+            if (!data) continue;
+         
+            auto order_id = ExchangeOrderKey(entry.orderId());
+            inc::Priority priority;
+            bts_assert(entry.orderPriority(priority));
+            
+            onixs::Decimal oprice;
+            ref_entry.entryPx(oprice);
+            auto price = convert_price(oprice, data->handle.security().display_factor);
+
+            int32_t size;
+            bts_assert(entry.displayQty(size));
+            auto type = mbo_update_type(ref_entry.entryType());
+
+            auto action = mbo_update_action(entry.orderUpdateAction());
+
+            _mbo->enqueue_incremental(
+               data->mbo_state,
+               inc::Update {
+                  order_id,
+                  priority,
+                  price,
+                  size,
+                  type,
+                  action
+               });
+         }
+      }
+   }
+
+   void onMessage(onixs::Handler&, const onixs::IncrementalRefreshOrderBook43Args& args) override {
+      onIncrementalRefreshOrderBook(args);
+   }
+   void onMessage(onixs::Handler&, const onixs::IncrementalRefreshOrderBook47Args& args) override {
+      onIncrementalRefreshOrderBook(args);
+   }
+   template <class Args>
+   void onIncrementalRefreshOrderBook(const Args& args) {
+      if (!_mbo) return;
+      for (const auto& entry : args.message().entries()) {
+         auto data = _index.get_security(entry.securityId());
+         if (!data) continue;
+
+         // order id
+         uint64_t temp;
+         bts_assert(
+            entry.orderId(temp));
+         auto order_id = ExchangeOrderKey(temp);
+
+         inc::Priority priority;
+         bts_assert(
+            entry.orderPriority(priority));
+            
+         onixs::Decimal onxprice;
+         bts_assert(
+            entry.entryPx(onxprice));
+         auto price = convert_price(onxprice, data->handle.security().display_factor);
+
+         int32_t size;
+         bts_assert(
+            entry.displayQty(size));
+            
+         auto type = mbo_update_type(entry.entryType());
+         auto action = mbo_update_action(entry.updateAction());
+
+         _mbo->enqueue_incremental(
+            data->mbo_state,
+            inc:: Update {
+               order_id,
+               priority,
+               price,
+               size,
+               type,
+               action
+            });
+      }
+   }
+
+   void onMessage(onixs::Handler&, const onixs::SnapshotFullRefreshOrderBook44Args& args) override {
+      onSnapshotFullRefreshOrderBook(args);
+   }
+   void onMessage(onixs::Handler&, const onixs::SnapshotFullRefreshOrderBook53Args& args) override {
+      onSnapshotFullRefreshOrderBook(args);
+   }
+   template <class Args>
+   void onSnapshotFullRefreshOrderBook(const Args& args) {
+      if (!_mbo) return;
+      auto data = _index.get_security(args.message().securityId());
+      if (!data) return;
+      for (const auto& entry : args.message().entries()) {
+         auto order_id = ExchangeOrderKey(entry.orderId());
+         inc::Priority priority;
+         bts_assert(entry.orderPriority(priority));
+         auto price = convert_price(entry.entryPx(), data->handle.security().display_factor);
+         auto size = entry.displayQty();
+         inc::UpdateType type = mbo_update_type(entry.entryType());
+         _mbo->enqueue_incremental(
+            data->mbo_state,
+            inc::Update {
+               order_id,
+               priority,
+               price,
+               size,
+               type,
+               inc::UpdateAction::New
+            });
+      }
+   }
+
+
+
    //
    // Security discovery
    template <typename InstrumentDefinitionFuture>
@@ -74,9 +254,9 @@ public:
 
    //
    // Book
-   // void onBookUpdate(onixs::Handler&, const onixs::Security&, const onixs::MboBook&) override;
    void onBookUpdate(onixs::Handler&, const onixs::Security&, const onixs::DirectBook&) override;
    void onBookUpdate(onixs::Handler&, const onixs::Security&, const onixs::ImpliedBook&) override;
+   void onBookUpdate(onixs::Handler&, const onixs::Security&, const onixs::MboBook&) override;
    // void onBookUpdate(onixs::Handler&, const onixs::Security&, const onixs::ConsolidatedBook&) override;
 
    template <size_t MAX_DEPTH,class TBook>
@@ -199,6 +379,7 @@ private:
       std::pair<std::array<PriceSize, COCONUT_MAX_DEPTH>, std::array<PriceSize,COCONUT_MAX_IMPLIED_DEPTH>> asks;
    };
    std::vector<BookData> _book_updates;
+   MBOWriter* _mbo;
 };
 
 //////////////////////////////////////////
@@ -228,9 +409,7 @@ void MDP3Listener::onFuture(onixs::Handler&, const onixs::Security& osec, const
    tracepoint (bts_trace,trace_order,(char*) "onDefinition.future", _latest_sequence, (uint16_t) _latest_sequence, 0);
    const auto& sec = def.message();
    auto exchange_id = sec.securityId();
-   // LOG(md, Info, "onDefinition future {} {}",  
-   //    std::string(sec.asset().items(), sec.asset().size()),
-   //    std::string(sec.symbol().items(), sec.symbol().size()));
+   // LOG(md, Info, "onDefinition future exch:{}",  exchange_id);
 
    if (filter_security(exchange_id, sec) != ADD) return;
    bts_assert(exchange_id == osec.id());
@@ -258,7 +437,7 @@ void MDP3Listener::onOption(onixs::Handler&, const onixs::Security&, const Instr
    tracepoint (bts_trace,trace_order,(char*) "onDefinition.option", _latest_sequence, (uint16_t) _latest_sequence, 0);
    const auto& sec = def.message();
    auto exchange_id = sec.securityId();
-   // LOG(Info, MarketData) << "onDefinition option " << exchange_id;
+   // LOG(md, Info, "onDefinition option exch:{}",  exchange_id);
    if (filter_security(exchange_id, sec) != ADD) return;
 
    FixedOptionSecurity fsec;
@@ -302,9 +481,7 @@ void MDP3Listener::onSpread(onixs::Handler&, const onixs::Security&, const Instr
    tracepoint (bts_trace,trace_order,(char*) "onDefinition.spread", _latest_sequence, (uint16_t) _latest_sequence, 0);
    const auto& sec = spread_args.message();
    auto exchange_id = sec.securityId();
-   // LOG(md, Info, "onDefinition spread {} {}",  
-   //    std::string(sec.asset().items(), sec.asset().size()),
-   //    std::string(sec.symbol().items(), sec.symbol().size()));
+   // LOG(md, Info, "onDefinition spread exch:{}",  exchange_id);
    if (filter_security(exchange_id, sec) != ADD) return;
    
    // Filter out any already expired spreads

src/server/sparkd/md/platforms/mdp3_onixs/MDP3Main.cpp

@@ -1,6 +1,7 @@
 #include "MDP3Main.h"
 
 #include "lib/md/cme/MDP3Utils.h"
+#include "lib/md/MBOProcessor.h"
 #include "lib/config/config.h"
 #include "lib/md/coconut2.h"
 #include "lib/os/os.h"
@@ -116,12 +117,23 @@ map<string, unique_ptr<onixs::FeedEngine>> create_feed_engines(const vector<CMEC
    return result;
 }
 
-onixs::SessionSettings session_settings_for_channel(const string& channel_id) {
+onixs::SessionSettings session_settings_for_channel(const string& channel_id, MBOWriter* mbo) {
    const auto& options = channel_settings_for(channel_id);
    onixs::SessionSettings settings;
-   settings.marketRecovery(onixs::MarketRecoveryOptions::MbpOnly);
+
+   if (mbo) {
+      settings.marketRecovery(onixs::MarketRecoveryOptions::BothMboAndMbp);
+   }
+   else {
+      settings.marketRecovery(onixs::MarketRecoveryOptions::MbpOnly);
+   }
+   
    if (!options.join_snapshot_on_gap()) {
+      #ifdef DEBUG
+      settings.gapAndErrorHandling(onixs::GapAndErrorHandlingOptions::RecoverMarketState);
+      #else
       settings.gapAndErrorHandling(onixs::GapAndErrorHandlingOptions::ContinueProcessing);
+      #endif
    }
    if (options.use_natural_refresh()) {
       settings.joinRecovery(onixs::JoinRecoveryOptions::Disabled);
@@ -130,14 +142,14 @@ onixs::SessionSettings session_settings_for_channel(const string& channel_id) {
    return settings;
 }
 
-void MDP3Main::start(std::vector<UserSymbol>& symbols) {
+void MDP3Main::start(std::vector<UserSymbol>& symbols, MBOWriter* mbo) {
    await_cme_config();
    initialize();
    const auto& channels = parse_channels(symbols);
    _engines = create_feed_engines(channels);
    for (const auto& channel : channels) {
-      unique_ptr<onixs::Handler> phandler = prepare_handler(channel.id);
-      _listeners.push_back(prepare_listener(*phandler, channel));
+      unique_ptr<onixs::Handler> phandler = prepare_handler(channel.id, mbo);
+      _listeners.push_back(prepare_listener(*phandler, channel, mbo));
       _handlers.insert(std::make_pair(channel.id, move(phandler)));
       std::stringstream symstr;
       for (const auto& sym : channel.symbols) { symstr << sym.symbol() << "-" << SecurityType_Name(sym.type()) << " "; }
@@ -145,7 +157,7 @@ void MDP3Main::start(std::vector<UserSymbol>& symbols) {
    }
 
    for (auto& pair : _handlers) {
-      onixs::SessionSettings settings = session_settings_for_channel(pair.first);
+      onixs::SessionSettings settings = session_settings_for_channel(pair.first, mbo);
       pair.second->start(settings);
    }
    LOG(md,Info, "** CME MDP3 Publisher started **");
@@ -185,7 +197,7 @@ void MDP3Main::initialize() {
 
 // //
 // Initialize HandlerSettings and allocate the channel Handler
-unique_ptr<onixs::Handler> MDP3Main::prepare_handler(const string& channel_id) {
+unique_ptr<onixs::Handler> MDP3Main::prepare_handler(const string& channel_id, MBOWriter* mbo) {
    const auto& pub_options = config::get<PublisherOptions>();
    const auto& options = channel_settings_for(channel_id);
    LOG(md,Info,"Loaded channel options for channel {}:\n{}",channel_id ,options.DebugString());
@@ -198,6 +210,9 @@ unique_ptr<onixs::Handler> MDP3Main::prepare_handler(const string& channel_id) {
    settings.bookManagement().directBooks().maintain(true);
    settings.bookManagement().impliedBooks().maintain(true);
    settings.bookManagement().consolidatedBooks().maintain(false);
+   if (options.validate_mbo() && mbo != nullptr) {
+      settings.bookManagement().mboBooks().maintain(true);
+   }
 
    // 10k default is *far* too low
    settings.recoveryQueueMaxSize(options.recovery_queue_size());
@@ -320,14 +335,14 @@ unique_ptr<onixs::Handler> MDP3Main::prepare_handler(const string& channel_id) {
 
 //
 // Create a listener for this channel and register callbacks
-unique_ptr<MDP3Listener> MDP3Main::prepare_listener(onixs::Handler& handler, const CMEChannel& channel) {
+unique_ptr<MDP3Listener> MDP3Main::prepare_listener(onixs::Handler& handler, const CMEChannel& channel, MBOWriter* mbo) {
    // We need to get a handler to the producer channels so we can produce heartbeats
    vector<MDProducerChannel> producer_channels;
    for (auto& symbol : channel.symbols) {
       producer_channels.push_back(_producer.get_channel(symbol.symbol().c_str(), symbol.type(), symbol.exchange()));
   }
 
-   unique_ptr<MDP3Listener> listener(new MDP3Listener(_index, _producer, /*_settlement_recorder,*/ producer_channels));
+   unique_ptr<MDP3Listener> listener(new MDP3Listener(_index, _producer, mbo, producer_channels));
    for (auto& symbol : channel.symbols) {
       listener->add_filter_symbol(symbol.symbol());
    }

src/server/sparkd/md/platforms/mdp3_onixs/MDP3Main.h

@@ -16,6 +16,8 @@ namespace md {
 
 // void run_onixs_mdp3_server(UserSymbol* sym, size_t symcount);
 
+class MBOWriter;
+
 struct CMEChannel;
 
 /*
@@ -30,12 +32,12 @@ public:
    MDP3Main();
    ~MDP3Main();
 
-   void start(std::vector<UserSymbol>&);
+   void start(std::vector<UserSymbol>&, MBOWriter*);
    void send_heartbeats() override;
    
    void initialize();
-   std::unique_ptr<onixs::Handler> prepare_handler(const std::string& channel);
-   std::unique_ptr<MDP3Listener> prepare_listener(onixs::Handler& handler, const CMEChannel& channel);
+   std::unique_ptr<onixs::Handler> prepare_handler(const std::string& channel, MBOWriter*);
+   std::unique_ptr<MDP3Listener> prepare_listener(onixs::Handler& handler, const CMEChannel& channel, MBOWriter*);
 
 private:
    BTS_NO_COPY(MDP3Main);

src/server/sparkd/md/run_md.cpp

@@ -23,6 +23,7 @@
 #include <thread>
 #include <memory>
 #include <set>
+#include <optional>
 
 using namespace bts;
 using namespace std;
@@ -68,8 +69,7 @@ void run_replay() {
    replay_recording(file, naggle_ms, fallback_path);
 }
 
-void run_onixs() 
-{
+void run_onixs(MBOWriter* mbo) {
    std::vector<std::shared_ptr<IMDPlatform>> platforms;
    // get all glbx symbols
    std::vector<UserSymbol> cme_symbols;
@@ -78,7 +78,7 @@ void run_onixs()
    {
       MDP3Main* main = new MDP3Main();
       std::shared_ptr<IMDPlatform> ptr(main);
-      main->start(cme_symbols);
+      main->start(cme_symbols, mbo);
       platforms.push_back(ptr);
    }
    
@@ -110,15 +110,14 @@ void run_cleanup() {
 }
 
 
-std::thread* _md_thread;
-void run_market_data_publishers_forever() {
-   os::set_thread_name("md publisher");
+std::optional<std::thread> _md_thread;
+void run_market_data_publishers_forever(MBOWriter* mbo) {
    const auto& conf = config::get<MarketDataConfig>();
    if (conf.use_replay()) {
       run_replay();
    }
    else if (conf.use_onixs()) {
-      run_onixs();
+      run_onixs(mbo);
    }
 }
 
@@ -142,14 +141,15 @@ void boot_sleep() {
    }
 }
 
-void run_market_datapublishers() {
-   bts_assert(_md_thread == nullptr);
+void run_market_datapublishers(MBOWriter* mbo) {
+   bts_assert(_md_thread == std::nullopt);
    run_cleanup();
    if (!config::get<MarketDataConfig>().use_md()) return;
    // start a thread for the md
-   _md_thread = new std::thread([]() {
-      run_market_data_publishers_forever();
-   });
+   _md_thread = std::optional(
+      threading::make_thread("md publisher", [mbo]() {
+      run_market_data_publishers_forever(mbo);
+   }));
 
   if (config::get<MarketDataConfig>().use_sim()) { 
    threading::make_thread("ExchangeSim", []() { sim::start_exchange_sim(); })
@@ -359,7 +359,7 @@ void md_tool(std::vector<std::string> const &args) {
    else if (cmd == "cleanup") run_cleanup();
    else if (cmd == "dump") run_dump(vector<string>(args.begin()+1, args.end()));
    else if (cmd == "replay") run_replay();
-   else if (cmd == "run") run_market_data_publishers_forever();
+   else if (cmd == "run") run_market_data_publishers_forever(nullptr);
    else if (cmd == "relay") relay_everything();
    else if (cmd == "test_relay") relay_everything();
    else if (cmd == "dumpsecurity" || cmd == "dumpsecurities") dump_db(vector<string>(args.begin()+1, args.end()));

src/server/sparkd/md/run_md.h

@@ -5,7 +5,9 @@
 namespace bts {
 namespace md {
 
-void run_market_datapublishers();
+class MBOWriter;
+
+void run_market_datapublishers(MBOWriter*);
 void md_tool(std::vector<std::string> const& args);
 
 }}

src/server/sparkd/position/PositionServiceServer.cpp

@@ -48,6 +48,10 @@ me(new PImpl(context,
                  me->publisher.add_or_update(_on_fill(*f,false));
               for (const auto& f : update.added())
                  me->publisher.add_or_update(_on_fill(*f,true));              
+              for (const auto& f : update.changed()) {
+                 me->publisher.add_or_update(_on_fill(*f.previous(),false));
+                 me->publisher.add_or_update(_on_fill(*f.current(),true));
+              }
               me->publisher.flush();
            });
 }

src/server/sparkd/user/AdminServiceServer.cpp

@@ -66,6 +66,10 @@ AdminServiceServer::AdminServiceServer(bts::api::ServiceContext& context)
                  if(it != _order_map.end())
                     _order_map.erase(it); 
               }
+              for (const auto& ch : update.changed()) {
+                 const auto& o = ch.current();
+                 _order_map[o->key()] = o;
+              }
               for (const auto& o : update.added()){
                  _order_map[o->key()]=o;
               }

testing/manual/forms/fills.txt

@@ -3,9 +3,8 @@ Fills UI Test
 
 Start the form (Trading->Fills)
 
-Make sure the following columns have filtered when the window starts:
+Make sure the following columns have filters when the window starts:
 	Fill Time: Since Open
-	Fill Kind: Standard, Leg
 
 Cross an order, get a fill and make sure it shows up
 Test sorting and filtering

testing/manual/positions_and_fills/PAndL.txt

@@ -17,12 +17,10 @@ and you may also want to clear your orders for a clean slate:
 Throughout the test we'll refer to accounts ACC A and ACC A1. These specific accounts aren't necessary, but rather that we have fills in multiple distinct accounts.
 
 Bring up the pnl form (empty).
-Bring up a SIM:SI ladder form, pick your favorite expiration. It should be completely empty since we're controlling the md, and the ACC A account should be selected in the account drop down list.
+Bring up a SIM:SI ladder form, pick your favorite expiration. It should be completely empty (just prices, no sizes) since we're controlling the md, and the ACC A account should be selected in the account drop down list.
 Bring up an order ticket for your corresponding ladder
 	place a buy order with price 14.000 and qty 10
-		your ladder should populate now
-	place a sell order with price 14.010 at qty 10
-		this will give you a valid Midpoint for your PnL Price in the PnL
+	place a sell order with price 14.010 at qty 10 this will give you a valid Midpoint for your PnL Price in the PnL
 Sell a 1 lot at your bid (14.000)
 	The Start Net Pos should be 0
 	Start Pnl should be 0