move default keycode exp to 12/31/2019 to fix unit tests

add temporary config to ignore nan day pnl to stop the loss limit from keeping traders out of market

.gitignore

@@ -33,3 +33,4 @@ trace_data/
 /src/lib/orderman/perf_analysis/log.csv
 /doc/user/public/*
 .ionide/*
+/.ignore

doc/user/content/bts_spark/Spreader/_index.md

@@ -154,7 +154,7 @@ insert_anchor_links = "right"
 
         -  Pay Less - If enabled by checking the checkbox, the spreader will
             attempt to pay less when the quantity at the level of the calculated
-            hedge price is more than the threshold defined.  The hedge order is
+            hedge price is less than the threshold defined.  The hedge order is
             paid less by the “Payless Ticks” as defined per leg.
         -  Pay Up - If enabled by checking the checkbox, the spreader will pay
             up when the quantity at the level of the calculated hedge price is

doc/user/content/release_notes/_index.md

@@ -9,14 +9,18 @@ insert_anchor_links = "right"
   - Fills now include information about the algo that originated the fill
   - A new, more powerful PnL form is now available
 ### API Changes
-  - Added security state information (pre open, halted, normal, closed, etc)
+  - Added security state information (pre open, halted, normal, closed, etc) 
+### Bug Fixes
+  - Brought back the "select all" check box in the mini order book
+  - Fixed an issue in the market data subsystem that on rare occasions could result in a startup crash
 ### Internal Changes
-  - Fixed an issue that was stopping us from interpreting SOD times as local rather than GMT
-
+  - Fixed an issue that was stopping us from interpreting SOD times as local rather than GMT 
+## 2.9.3
+### Internal Changes
+  - Fixed system test scripts
 ## 2.9.2
 ### Bug Fixes
   - Fixed an issue with the exchange simulator that could lead to crossed markets
-
 ## 2.9.1
 ### Bug Fixes
   - Fixed an issue with trade sound alerts, where in some cases the use of a "In Between" condition could lead to a UI failure.
@@ -26,7 +30,7 @@ insert_anchor_links = "right"
   - Avoided shipping production configuration defaults that lead to trying to download a position file for the development sftp server.
 ### Api
   - C# API sample now uses synchronous calls.
-
+  
 ## 2.9.0
 ### User Visible Changes
   - Improved Workspace management User Interface
@@ -34,11 +38,11 @@ insert_anchor_links = "right"
   - The Spark UI Window group now has a title in the Windows task bar
   - Added sound alerts for trade events, see Launcher -> Settings -> Alerts
   - Allowed composite algos that are attached to existing orders to identify fills that were already present prior to attachment
-  - Added import facilities for GMI position files
+  - Added import facilities for GMI position files 
   - Added the ability for risk managers to enter manual fills
   - Added "raw" P&L columns, that is columns that *do not* include spread legs in their accounting
 ### Bug Fixes
-  - Fixed the settings action menu (Load/Save/Reset to Factory)
+  - Fixed the settings action menu (Load/Save/Reset to Factory) 
   - Fixed the method used to determine if a security is expired to take into account the start of day time
 ### API Changes
   - Fixed an issue in the Client SDK that could lead to occasional failures to connect to the spark server as well as excessive memory usage.
@@ -75,10 +79,10 @@ insert_anchor_links = "right"
   - Made it easier to generate simple UI's for new order types
 ### API Changes
   - Dynamic orders can now use QuantityMeasure parameter types
-
+  
 ## 2.7.2
 ### User Visible Changes
-  - Changed the login dialog to avoid leaking workspace names from other users.
+  - Changed the login dialog to avoid leaking workspace names from other users. 
   - Added new Splitter algo. From the manual: The splitter is useful
     for pro-rata markets where you want to vary your exposure
     independent of the order quantity that you want to fill.  The
@@ -86,13 +90,13 @@ insert_anchor_links = "right"
     the order quantity but it will put exposed_qty total size on the
     market. Splitter favors order size over queue position.
 ### Internal Changes
-  - Made spark more resilient to lack of connectivity to the CME web site for downloading of templates.
-
+  - Made spark more resilient to lack of connectivity to the CME web site for downloading of templates. 
+  
 ## 2.7.1
 ### User Visible Changes
   - Fixed some issues with the Cover algo, where in some circumstances
     it would erroneously cancel launched hedges on completion.
-
+	
 ## 2.7.0
 ### User Visible Changes
   - Allowed algos to process "in-flight" fills while shutting down
@@ -111,25 +115,25 @@ insert_anchor_links = "right"
 ### User Visible Changes
   - Fixed an issue in the ladder where in some cases the mini order book could not be brought up from a badge
   - Fixed a rare race condition that could lead to erroneous limit enforcement
-  - Fixed an issue with the margin limits where an error could occur if an account name with more than 12 chars was used
+  - Fixed an issue with the margin limits where an error could occur if an account name with more than 12 chars was used  
 
 ## 2.6.1
 ### User Visible Changes
   - Fixed an issue in the ladder where an error could occur when trying to change ladder settings for the first time
-
+  
 ## 2.6.0
 ### User Visible Changes
   - In the ladder made last trade more distinctive when overlayed on the volumn column
-  - In the ladder, corrected display of volume tool tip
+  - In the ladder, corrected display of volume tool tip 
   - In the ladder, changed the consolidation of last trade to consider only the price ignoring aggressor changes
   - Added an option for the Top Algo to delete itself if within N ticks from top of book
   - Address a rare issue that could lead to an error report on UI exit
   - Added margin risk limits
   - Added the "Offset" algo. Offset prices a child algo at a parameterizable distance from the top of the book
-  - Avoided switching the current account and order type on open ladders when limits are changed
+  - Avoided switching the current account and order type on open ladders when limits are changed 
 ### Internal Changes
   - Ensure that admin logins are always allowed to login, regardless of keycode status.
-
+  
 ## 2.5.0
 ### User Visible Changes
   - Added loss limit risk check
@@ -159,13 +163,13 @@ insert_anchor_links = "right"
   - Made Cover order more independent from its hedging chidren:
      - Cover continues to operate even if an hedging child is canceled
 	 - When a cover is canceled it will detach from any running hedging children, instead of canceling them
-  - Changed Lean, Join and Cover algos to hide themselves in the ladder once they have completed their work
+  - Changed Lean, Join and Cover algos to hide themselves in the ladder once they have completed their work 
   - Changed manual fills to appear in day PnL rather than in the SOD records
   - Add buy/sell coloring to fill rows in fills window
   - Streamlined the mini order book:
     - Display of the mini order book popup is quicker
     - Only exchange orders are displayed
-	- Update buttons are only displayed for orders that are under control of an algo with the update applying to the entire algo, not just the exchange order
+	- Update buttons are only displayed for orders that are under control of an algo with the update applying to the entire algo, not just the exchange order 
     - We can now select multiple rows to attach, detach, update or cancel multiple orders in a batch
     - A checkbox column has been added to make clearer which rows are selected
 ### API Changes
@@ -175,7 +179,7 @@ insert_anchor_links = "right"
   - Added system requirements to Spark documentation
   - Implemented keycodes
   - Temporarily rejected stop and stop limit orders, pending bug fixes
-
+  
 ## 2.2.0
 ### User Visible Changes
   - Made our orders easier to spot in MBO display
@@ -186,30 +190,30 @@ insert_anchor_links = "right"
   - Improved handling of orders with undiscovered securities
   - Added feedback to the ladder when the user attempts to middle click on bid/ask columns using an order that does not have parameters
   - Fixed display of algo studio algo types in order book
-  - Hid net change setting in synthetic ladders
+  - Hid net change setting in synthetic ladders 
   - Fixed a number of problems with filtering on the Fills form
   - On the Lean algo, OnToward and Away are now calculated with respect to the opposite inside market at time of the order launch
   - Iceberg Spreaders with queue holders are not erroneously cancelled after the first fill
   - Added security symbol column to fills window
   - Re-organized ladder settings dialog to make it clearer and easier to use
-
+  
 ## 2.1.4
 ### User Visible Changes
   - Fixed handling of cancel button in security picker
-
+  
 ## 2.1.3
 ### User Visible Changes
-  - Fixed a simulator bug in the handling of 0 sized orders
+  - Fixed a simulator bug in the handling of 0 sized orders 
 
 ## 2.2.1
 ### User Visible Changes
   - Fixed order book form exception when display a spreader order for whichs the legs were not present yet
-
+  
 ## 2.1.1
 ### User Visible Changes
   - Added PIQ to GLBX Iceberg orders
   - Corrected handling of stop limit orders
-
+  
 ## 2.1.0
 ### User Visible Changes
   - Added support for the GLBX Iceberg order type
@@ -220,7 +224,7 @@ insert_anchor_links = "right"
   - Added PIQ to the mini order book
   - Fixed a rare bug that could lead to server failure when requesting market data
   - Fixed a bug that could cause incorrect price displays for certain types of synthetic securities
-
+ 
 ## 2.0.5
 ### User Visible Changes
   - Fixed a bug in the handling of rejected QHO orders
@@ -230,19 +234,19 @@ insert_anchor_links = "right"
 ## 2.0.4
 ### User Visible Changes
   - Fixed performance regression in order manager
-
+  
 ### Internal changes
   - Improved market data diagnostics for bad configurations
   - Changed market data default buffer size
-
+  
 ## 2.0.3
 ### Internal changes
   - Improved backwards compatibility in UI settings
 
-
+  
 ## 2.0.2
 ### User Visible Changes
-  - Middle click the ladder price column to show the orders for that price level
+  - Middle click the ladder price column to show the orders for that price level 
   - Middle click the bid or ask column in the ladder no longer sends an order
 
 ### Internal changes
@@ -270,8 +274,8 @@ insert_anchor_links = "right"
   - Improved settings manager tool output
   - Added diagnostics in the event MD memory corruption is detected
   - Added systems tests for join,lean, iceberg and spreader orders
-  - Made exchange order recovery more resilient in the presence of database loss
-
+  - Made exchange order recovery more resilient in the presence of database loss 
+  
 ## 1.6.2
 ### User Visible Changes
 ### Exchange simulator
@@ -310,21 +314,21 @@ insert_anchor_links = "right"
     - These columns can be restored by toggling their visibility within the PnL's Settings form
     - A new Traded Today column has been added to the PnL
       - Filtering on this column will allow a user to restrict the report to only show today's position
-### Internal changes
-  - Improved organization of the shell notification architecture
+### Internal changes 
+  - Improved organization of the shell notification architecture 
   - Added logging to help troubleshoot shell exit hang
 
 ## 1.5.6
 ### User Visible Changes
-  - Fixed a bug that could cause limits to be erroneously enforced
+  - Fixed a bug that could cause limits to be erroneously enforced 
   - Improved algo documentation
   - Changed how the spreader uses net change
   - Simplified trigger algo user interface
-### Internal changes
+### Internal changes 
   - Defaulted to not recording sim market data
-  - Improved documentation build
+  - Improved documentation build 
   - Added additional performance tracing
-
+  
 ## 1.5.5
   - Fixed Market View 'stale working orders' bug
 
@@ -332,11 +336,11 @@ insert_anchor_links = "right"
   - Avoided logging when warming up cache
 
 ## 1.5.3
-  - Fixed sounds startup issue
+  - Fixed sounds startup issue 
 
 ## 1.5.2
   - Added better logging to AlgoDefinitionService
-
+  
 ## 1.5.1
   - Fixed missing securities in the Fills and P&L windows.
 
@@ -357,7 +361,7 @@ insert_anchor_links = "right"
 #### Bug Fixes
   - Fixed the "Since Open" time filter to use the exchange open time rather than 8:30 AM
   - Fixed a bug in the order book where sometimes parent orders would not update their last message column.
-
+  
 #### Algos
   - Added OCO Algo
     - An OCO order creates a relationship between a child order and a 'stop' or 'other' order such that when one of these is cancelled, both cancel, or when one is filled, the other order's size is debited in the amount of the fill.  Composing this order with Cover simulates scalping functionality
@@ -365,7 +369,7 @@ insert_anchor_links = "right"
   - Improved SDK documentation and usability
   - Dynamic orders are now properly identified in the order book
 #### Documentation
-  - Information on the Spark API has been added to the Spark manual
+  - Information on the Spark API has been added to the Spark manual 
   - The documentation has been updated and reorganized to make information easier to find
 ### Internal Changes
   - Improved robustness of limit system test

src/client/spark/Modules/Market/MarketView/MarketViewRow.cs

@@ -26,7 +26,6 @@ namespace mbts.spark {
          readonly ObservableFilterCollection<OrderRecordImmutable> _working_orders, _working_bids, _working_asks;
          readonly string _security_text, _security_key_text;
          readonly Property<double> _bid_price, _ask_price, _last_price, _day_change, _settlement, _session_high, _session_low;
-         readonly Property<bool> _last_price_from_spread;
          readonly CellProperty<int> _position;
          readonly CellProperty<uint> _last_size;
          readonly CellProperty<ulong> _volume;
@@ -55,7 +54,6 @@ namespace mbts.spark {
             _day_change = backing(nameof(day_change), double.NaN);
             _session_low = backing(nameof(session_low), double.NaN);
             _session_high = backing(nameof(session_high), double.NaN);
-            _last_price_from_spread = backing(nameof(last_price_from_spread), false);
             _settlement = backing(nameof(settlement), double.NaN);
             _volume = new CellProperty<ulong>(this, nameof(volume), 0);
             _inside_bids = new CellProperty<bool>(this, nameof(inside_bids));
@@ -111,7 +109,6 @@ namespace mbts.spark {
          public int position => _position.value;
          public double raw_day_position => _raw_day_position;
          public double last_price => _last_price.Value;
-         public bool last_price_from_spread => _last_price_from_spread.Value;
          public uint last_size => _last_size.value;
          public double bid_price => _bid_price.Value;
          public uint bid_direct_size => _bid_direct_size.Value;
@@ -170,17 +167,15 @@ namespace mbts.spark {
                _inside_bids.on_next_distinct(_bid_price.Value >= _working_best_bid.value, ref dirty);
                _inside_asks.on_next_distinct(_ask_price.Value <= _working_best_ask.value, ref dirty);
             }
-            if (md.md.get_last_trade(ref _last_ref, out NativePriceSize ps, out DateTime ts, out ulong v, out bool from_spread)) {               
+            if (md.md.get_last_trade(ref _last_ref, out NativePriceSize ps, out DateTime ts, out ulong v)) {
                _last_price.set(ps.price);
-               _last_price_from_spread.set(from_spread);
                _last_size.on_next_distinct(ps.size, ref dirty);
                _volume.on_next_distinct(v, ref dirty);
             }
-            dirty |= _settlement.set_new(md.md.settlement_price);
-            dirty |= _session_low.set_new(md.md.session_low);
-            dirty |= _session_high.set_new(md.md.session_high);
-            dirty |= _day_change.set_new(last_price - settlement);
-            _volume.on_next_distinct(md.md.total_volume, ref dirty);
+            _settlement.set(md.md.settlement_price);
+            _session_low.set(md.md.session_low);
+            _session_high.set(md.md.session_high);
+            _day_change.set(last_price - settlement);
             if (dirty)
                invalidated?.Invoke();
 

src/client/spark/Modules/Market/MarketView/MarketViewVM.cs

@@ -38,8 +38,7 @@ namespace mbts.spark {
          IMultiValueConverter price_converter = new ui.spark.converters.TickDisplayConverter();
          add_col(Columns.Security, "Security", r => r.security.ToString(), nameof(Row.security), DefaultColumnWidths.Security, security_filter, DataGridFilterKnownTemplates.Regex);
          add_col(Columns.Volume, "Volume", r => r.volume.ToString(), nameof(Row.volume), DefaultColumnWidths.Volume, volume_filter, DataGridFilterKnownTemplates.ULongMinMax);
-         add_col(Columns.LastSpd, "Spd", r => r.last_price_from_spread ? "S" : "", nameof(Row.last_price_from_spread), DefaultColumnWidths.Side, last_price_from_spread_filter, DataGridFilterKnownTemplates.Regex);
-         add_col(Columns.Last, "Last", r => r.format_price(r.last_price), nameof(Row.last_price), DefaultColumnWidths.Price, last_price_filter, DataGridFilterKnownTemplates.DoubleMinMax);         
+         add_col(Columns.Last, "Last", r => r.format_price(r.last_price), nameof(Row.last_price), DefaultColumnWidths.Price, last_price_filter, DataGridFilterKnownTemplates.DoubleMinMax);
          add_col(Columns.LastSize, merged_size_col_header, r => r.last_size.ToString(), nameof(Row.last_size), DefaultColumnWidths.Size, last_size_filter, DataGridFilterKnownTemplates.UIntMinMax, h_alignment: System.Windows.HorizontalAlignment.Right);
          add_col(Columns.Position, "Pos", r => r.position.ToString(), nameof(Row.position), DefaultColumnWidths.Size, position_filter, DataGridFilterKnownTemplates.IntMinMax, true, h_alignment: System.Windows.HorizontalAlignment.Right);
          add_col(Columns.RawDayPosition, "Raw Day Pos", r => SparkUtils.fmt_position(r.raw_day_position), nameof(Row.raw_day_position), DefaultColumnWidths.Size, raw_day_position_filter, DataGridFilterKnownTemplates.DoubleMinMax, true, h_alignment: System.Windows.HorizontalAlignment.Right);
@@ -139,7 +138,6 @@ namespace mbts.spark {
                      ask_direct_size_filter.load_state(filters.ask_direct_size);
                      ask_implied_size_filter.load_state(filters.ask_implied_size);
                      last_price_filter.load_state(filters.last_price);
-                     last_price_from_spread_filter.load_state(filters.last_price_from_spread);
                      last_size_filter.load_state(filters.last_size);
                      day_change_filter.load_state(filters.day_change);
                      settlement_filter.load_state(filters.settlement);
@@ -157,7 +155,7 @@ namespace mbts.spark {
             }), DataGridFilterHelper.hook_filters(rows_view, security_filter,
             bid_price_filter, bid_merged_size_filter, bid_direct_size_filter, bid_implied_size_filter,
             ask_price_filter, ask_merged_size_filter, ask_direct_size_filter, ask_implied_size_filter,
-            last_price_filter, last_price_from_spread_filter, last_size_filter, day_change_filter, settlement_filter,
+            last_price_filter, last_size_filter, day_change_filter, settlement_filter,
             session_low_filter, session_high_filter, position_filter, raw_day_position_filter, working_bid_volume_filter, working_ask_volume_filter, volume_filter, security_key_filter),
             settings_manager.subscribe_current().OnDispatcher().Subscribe(grid_settings_manager),
             WorkLoop.Start("MarketView", _refresh)
@@ -220,7 +218,6 @@ namespace mbts.spark {
       public IDataGridColumnPersistentFilter<Row, UIntMinMaxState> ask_direct_size_filter { get; } = new UIntMinMax<Row>(r => r.ask_direct_size, nameof(Row.ask_direct_size));
       public IDataGridColumnPersistentFilter<Row, UIntMinMaxState> ask_implied_size_filter { get; } = new UIntMinMax<Row>(r => r.ask_implied_size, nameof(Row.ask_implied_size));
       public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> last_price_filter { get; } = new DoubleMinMax<Row>(r => r.last_price, nameof(Row.last_price));
-      public IDataGridColumnPersistentFilter<Row, string> last_price_from_spread_filter { get; } = new RegexFilterVM<Row>(r => r.last_price_from_spread.ToString(), nameof(Row.last_price_from_spread));
       public IDataGridColumnPersistentFilter<Row, UIntMinMaxState> last_size_filter { get; } = new UIntMinMax<Row>(r => r.last_size, nameof(Row.last_size));
       public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> day_change_filter { get; } = new DoubleMinMax<Row>(r => r.day_change, nameof(Row.day_change));
       public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> settlement_filter { get; } = new DoubleMinMax<Row>(r => r.settlement, nameof(Row.settlement));
@@ -253,7 +250,6 @@ namespace mbts.spark {
             ask_direct_size = ask_direct_size_filter.get_state(),
             ask_implied_size = ask_implied_size_filter.get_state(),
             last_price = last_price_filter.get_state(),
-            last_price_from_spread = last_price_from_spread_filter.get_state(),
             last_size = last_size_filter.get_state(),
             day_change = day_change_filter.get_state(),
             settlement = settlement_filter.get_state(),

src/client/spark/Modules/Market/MarketView/MarketViewView.xaml.cs

@@ -12,7 +12,6 @@ namespace mbts.ui.spark {
          Security,
          Volume,
          Last,
-         LastSpd,
          LastSize,
          Position,
          RawDayPosition,

src/client/spark/Modules/Market/MarketView/MarketViewWindowFactory.cs

@@ -100,7 +100,6 @@ namespace bts.shell {
             public static GridColumnSettings security => new GridColumnSettings(col_id(Columns.Security), DefaultColumnHeaders.Security, width: DefaultColumnWidths.Security);
             public static GridColumnSettings volume => new GridColumnSettings(col_id(Columns.Volume), DefaultColumnHeaders.Volume, width: DefaultColumnWidths.Volume);
             public static GridColumnSettings last => new GridColumnSettings(col_id(Columns.Last), DefaultColumnHeaders.Last, width: DefaultColumnWidths.Price);
-            public static GridColumnSettings last_from_spread => new GridColumnSettings(col_id(Columns.LastSpd), DefaultColumnHeaders.S, width: DefaultColumnWidths.Price, is_visible: false);
             public static GridColumnSettings last_size => new GridColumnSettings(col_id(Columns.LastSize), DefaultColumnHeaders.Size, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right);
             public static GridColumnSettings position => new GridColumnSettings(col_id(Columns.Position), DefaultColumnHeaders.Pos, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right);
             public static GridColumnSettings raw_day_position => new GridColumnSettings(col_id(Columns.RawDayPosition), DefaultColumnHeaders.RawDayPos, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right, is_visible: false);
@@ -139,7 +138,6 @@ namespace bts.shell {
                   yield return security;
                   yield return volume;
                   yield return last;
-                  yield return last_from_spread;
                   yield return last_size;
                   yield return position;
                   yield return raw_day_position;
@@ -226,8 +224,6 @@ namespace bts.shell {
          [ProtoMember(25)]
          public string strip_name;
          [ProtoMember(26)]
-         public string last_price_from_spread;
-         [ProtoMember(27)]         
          public string mic_code;
       }
 

src/client/spark/Modules/Spreader/ExecutionStrategyEditor.xaml.cs

@@ -227,7 +227,7 @@ namespace mbts.ui.spark {
       }
 
       DataGridColumn _column_factory(LegSpreaderParamsTuple l) {
-         var col = new DataGridColumnInternal(l.leg.Security, ParameterCellTemplateConverter, ParameterCellEditingTemplateConverter) { Width = 75, };
+         var col = new DataGridColumnInternal(l.leg.Security, ParameterCellTemplateConverter, ParameterCellEditingTemplateConverter);
          col.HeaderTemplate = (DataTemplate)FindResource(IsNew ? RemovableColumnHeaderTemplateKey : NonRemovableColumnHeaderTemplateKey);
          DataGridEx.SetRightBorder(col, new DataGridExVerticalLine(false, true, Colors.DarkGray, 1));
          return col;

src/client/spark/Modules/Trading/Fills/FillsVM.cs

@@ -125,7 +125,6 @@ namespace mbts.spark {
          add_col(Columns.Size, DefaultColumnHeaders.Size, r => r.size.ToString(), new DoubleMinMax<FillTuple>(r => r.fill.LastShares), DataGridFilterKnownTemplates.UIntMinMax, DefaultColumnWidths.Size);
          add_col(Columns.Price, DefaultColumnHeaders.Price, r => r.format_price(r.price), new DoubleMinMax<FillTuple>(r => r.fill.LastPrice), DataGridFilterKnownTemplates.DoubleMinMax, DefaultColumnWidths.Price);
          add_col(Columns.FillKind, DefaultColumnHeaders.FillKind, r => r.fill_kind.ToString(), dv_filter(EnumHelper.get_values<FillKind>().ToArray(), r => r.fill.FillKind), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.FillKind);
-         add_col(Columns.FillStatus, DefaultColumnHeaders.FillStatus, r => r.fill_status.ToString(), dv_filter(EnumHelper.get_values<FillStatus>().ToArray(), r => r.fill.FillStatus), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.FillKind);
          add_col(Columns.FillKey, DefaultColumnHeaders.FillKey, r => r.fill_key.ToString(), new RegexFilterVM<FillTuple>(r => r.fill.FillKey), DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.FillKey);
          add_col(Columns.OrderKey, DefaultColumnHeaders.OrderKey, r => r.order_key.ToString(), new RegexFilterVM<FillTuple>(r => r.fill.OrderKey.ToString()), DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.OrderKey);
          add_col(Columns.SecurityKey, DefaultColumnHeaders.SecurityKey, r => r.security_key.ToString(), new RegexFilterVM<FillTuple>(r => r.fill.SecurityKey.ToString()), DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.SecurityKey);
@@ -228,7 +227,6 @@ namespace mbts.spark {
             case Columns.Exchange: return wrap_default(ft => ft.security.exchange_or_default());
             case Columns.FillKey: return wrap(ft => ft.fill.FillKey);
             case Columns.FillKind: return wrap_default(ft => ft.fill.FillKind);
-            case Columns.FillStatus: return wrap_default(ft => ft.fill.FillStatus);
             case Columns.FillTime: return wrap(ft => ft.fill.Timestamp.to_local_time());
             case Columns.OrderKey: return wrap(ft => ft.fill.OrderKey);
             case Columns.OrderType: return wrap_default(ft => ft.order?.order_type() ?? OrderType.None);
@@ -295,7 +293,6 @@ namespace mbts.spark {
          public double price => _fill.LastPrice;
          public double size => _fill.LastShares;
          public FillKind fill_kind => _fill.FillKind;
-         public FillStatus fill_status => _fill.FillStatus;
          public string fill_key => _fill.FillKey;
          public string order_key => _order.Key.ToString();
 

src/client/spark/Modules/Trading/Fills/FillsView.xaml.cs

@@ -21,7 +21,6 @@ namespace mbts.ui.spark {
          Price,
          Size,
          FillKind,
-         FillStatus,
          FillKey,
          OrderKey,
          SecurityKey

src/client/spark/Modules/Trading/Fills/FillsWindowFactory.cs

@@ -37,8 +37,7 @@ namespace bts.shell {
             { nameof(FillsVM.Row.side), Columns.Side },
             { nameof(FillsVM.Row.size), Columns.Size },
             { nameof(FillsVM.Row.time_in_force), Columns.TimeInForce },
-            { nameof(FillsVM.Row.user), Columns.User },
-            { nameof(FillsVM.Row.fill_status ), Columns.FillStatus },
+            { nameof(FillsVM.Row.user), Columns.User }
          };
 
       public FillsWindowFactory(TradingShellModule module, IBus bus, UserSettingsHub settings_hub, OrdersModel orders_model)

src/client/spark/Modules/Trading/Fills/Settings/FillsFilterStates.cs

@@ -74,10 +74,6 @@ namespace bts.shell.fills {
       [ProtoMember(15)]
       [DIYGridColumnID(nameof(Columns.Symbol))]
       public string[] symbol { get; set; }
-
-      [ProtoMember(16)]
-      [DIYGridColumnID(nameof(Columns.FillStatus))]
-      public Fill.Types.FillStatus[] fill_status { get; set; }
    }
 
    static class FillsFilterStatesExtensions {
@@ -100,7 +96,6 @@ namespace bts.shell.fills {
          pb.add_discrete_values(fs.side, ft => ft.fill.Side);
          pb.add_discrete_values(fs.fill_kind, ft => ft.fill.FillKind);
          pb.add_discrete_values(fs.symbol, ft => ft.security?.Symbol ?? string.Empty);
-         pb.add_discrete_values(fs.fill_status, ft => ft.fill.FillStatus);
          // numeric
          pb.add_numeric(fs.size, ft => ft.fill.LastShares);
          pb.add_numeric(fs.price, ft => ft.fill.LastPrice);

src/client/spark/Modules/Trading/Ladder/VM/LadderVM.cs

@@ -451,7 +451,7 @@ namespace mbts.spark {
                price = temp_price;
             }
          }
-         await _model.order_gateway.send_order(order, price, size);
+         await _model.order_gateway.send_order(security, order, price, size);
          if (auto_clear)
             size_calculator.value = 0;
          var fav_algo = order_properties_vm.selected_favorite_algo_definition;

src/client/spark/Modules/Trading/OrderBook/OrderBookColumns.cs

@@ -750,6 +750,7 @@ namespace mbts.spark {
             };
             header_cb.SetBinding(CheckBox.IsCheckedProperty, new Binding(nameof(OrderBookVM.all_rows_selected)));
             this.Header = header_cb;
+            is_visible = true;
          }
 
          public override FrameworkElement view_factory() {

src/client/spark/Modules/Trading/OrderBook/OrderBookVM.cs

@@ -288,7 +288,14 @@ namespace mbts.spark {
                double price = attach_vm.price;
                uint? size = attach_vm.size;
                uint algo_template_key = attach_vm.algo_vm.algo_root.id.is_persisted ? attach_vm.algo_vm.algo_root.id.persisted : 0u;
-               var order_tups = selected_rows.Select(r => r.order_tuple).ToList();
+               // Don't pass any order tuples that don't have a security defined.
+               // The actual security is passed to the OrdersModel - instead of just the security key -
+               // so it can prevent GlbxIceberg orders from being sent to a non-glbx exchange (such as Ice).
+               /// Impl details: <see cref="OrdersModel.send_order(SecurityImmutable, OrderDefinitionImmutable?, double, uint)"/>
+               var order_tups = selected_rows
+                  .Select(r => r.order_tuple)
+                  .Where(ot => ot.order.security.HasValue)
+                  .ToList();
                var mod_algo_vm = attach_vm.algo_vm.algo_root;
                await Task.WhenAll(order_tups.Select(t => {
                   var algo = t.order.order.Def.CustomDef.actual().CloneToMutable();
@@ -302,6 +309,7 @@ namespace mbts.spark {
                      };
                   }
                   return _model.send_order(
+                     t.order.security.Value,
                      t.order.order.Def.with(
                         attach_target_order_key: t.order.order.Key,
                         custom_def: algo

src/client/spark/Modules/Trading/PNL/Data/PNLData.cs

@@ -501,6 +501,12 @@ namespace bts.shell.pnl {
          account_name == other.account_name &&
          sod == other.sod &&
          day == other.day &&
+         bid_price.Equals(other.bid_price) &&
+         ask_price.Equals(other.ask_price) &&
+         last_price.Equals(other.last_price) &&
+         mid_price.Equals(other.mid_price) &&
+         settlement_price.Equals(other.settlement_price) &&
+         calc_type.Equals(other.calc_type) &&
          raw_day == other.raw_day &&
          working.summary == other.working.summary;
 

src/client/spark/Modules/Trading/PNL/VM/PNLRowVM.cs

@@ -97,22 +97,22 @@ namespace bts.shell.pnl {
                if (security.HasValue && security.Value.ContractCase != Security.ContractOneofCase.Future) {
                   day_long = day_short = sod_long = sod_short = 0;
                } else {
-                  day_long = (decimal)DoubleUtil.nan_to_zero(data.day.long_pos);
-                  day_short = (decimal)DoubleUtil.nan_to_zero(data.day.short_pos);
-                  sod_long = (decimal)DoubleUtil.nan_to_zero(data.sod.long_pos);
-                  sod_short = (decimal)DoubleUtil.nan_to_zero(data.sod.short_pos);
+                  day_long = DoubleUtil.round_to_decimal(data.day.long_pos);
+                  day_short = DoubleUtil.round_to_decimal(data.day.short_pos);
+                  sod_long = DoubleUtil.round_to_decimal(data.sod.long_pos);
+                  sod_short = DoubleUtil.round_to_decimal(data.sod.short_pos);
                }
                var nd = new PNLNormalizedData(
-                  (decimal)DoubleUtil.nan_to_zero(day_pnl.value),
+                  DoubleUtil.round_to_decimal(day_pnl.value),
                   day_long,
                   day_short,
-                  (decimal)DoubleUtil.nan_to_zero(data.raw_day.long_pos),
-                  (decimal)DoubleUtil.nan_to_zero(data.raw_day.short_pos),
-                  (decimal)DoubleUtil.nan_to_zero(sod_pnl.value),
+                  DoubleUtil.round_to_decimal(data.raw_day.long_pos),
+                  DoubleUtil.round_to_decimal(data.raw_day.short_pos),
+                  DoubleUtil.round_to_decimal(sod_pnl.value),
                   sod_long,
                   sod_short,
-                  (decimal)DoubleUtil.nan_to_zero(data.working_long_pos),
-                  (decimal)DoubleUtil.nan_to_zero(data.working_short_pos));
+                  DoubleUtil.round_to_decimal(data.working_long_pos),
+                  DoubleUtil.round_to_decimal(data.working_short_pos));
                if (_normalized_data != nd) {
                   _normalized_data = nd;
                   updated?.Invoke(nd);

src/client/spark/Modules/Trading/PNL/VM/PNLVM.cs

@@ -103,7 +103,7 @@ namespace bts.shell.pnl {
          load_from(settings);
          _dispose = Disposable.Create(
             pnl_source.Operate(
-               _calculation_type.observe_current(),
+               _calculation_type.observe_current(true),
                (data, calc) => data.with(calc_type: calc)
             ).FilterWithRaw(filter_src, PNLRowData.key_fn, null)
             .OnDispatcher()

src/client/spark/Modules/Trading/PNL/View/PNLRowCollectionView.cs

@@ -373,7 +373,9 @@ namespace bts.shell.pnl {
                var less_last = data - last_data;
                var plus_new = less_last + new_data;
                data += new_data - last_data;
-               Log.assert(plus_new == data);
+               if (plus_new != data) {
+                  Log.Ui.warn($"Detected PNL arithmetic rounding error in the group sum for '{string.Join(" ", GroupNamePath)}' (this bug is under investigation).");
+               }
                last_data = new_data;
             }
             void on_is_selected_changed(bool is_selected) {

src/client/spark/Orders/NewOrderTicketVM.cs

@@ -53,7 +53,7 @@ namespace mbts.spark {
          send_order = new DelegateCommand<Side>(side => {
             bool price_valid = _validate_price(), size_valid = size_calculator.validate();
             if (price_valid && size_valid && !HasErrors && !properties_vm.HasErrors) {
-               model.send_order(properties_vm.create(side), price, size_calculator.value.Value).ignore();
+               model.send_order(security, properties_vm.create(side), price, size_calculator.value.Value).ignore();
                RequestClose?.Invoke(this, new EventArgs());
             }
          });

src/client/spark/Orders/OrderUtils.cs

@@ -200,6 +200,13 @@ namespace mbts.spark {
       public static string short_description(this OrderRecordImmutable order, ContractSpecificationImmutable? spec) =>
          $"{(order.HasDef ? order.Def.Side.ToString() : "(unknown-side)")} {(order.HasState ? ($@"{spec.format(order.State.Price)} @ {order.State.Quantity}") : "(unknown-price) @ (unknown-quantity)")}";
 
+      public static string short_description(SecurityImmutable security, Side side, double price, uint quantity) =>
+         new StringBuilderEx(
+            security.ToString(), " ",
+            side.ToString(), " ",
+            price_quantity_description(price, quantity, security.spec_or_default())
+         ).ToString();
+
       public static string short_description(this OrderRecordImmutable order, SecurityImmutable? security) {
          var spec = security.spec_or_default();
          return new StringBuilderEx(

src/client/spark/Orders/OrdersModel.cs

@@ -75,7 +75,6 @@ namespace mbts.spark {
             .TableCheckedAutoCorrect(EqualityComparer<(SecurityImmutable? sec, PNLUnitImmutable data)>.Default)
             .Where(t => t.has_changes())
             .Cache();
-         on_account_pnl = _position_service.subscribei_pnl_account_data().Naggle(interval, scheduler).Table().Cache();
          var working_orders = on_orders.Filter(tup => tup.order.is_active());
          this.working_orders =
             on_orders.Scan(
@@ -241,7 +240,6 @@ namespace mbts.spark {
                .Filter(os => os.order.is_active() && os.order.State.DisplayStatus == OrderState.Types.DisplayStatus.Visible, key => key.order.Key)
                .Subscribe(_security_working_orders_with_piq_demuxer),
             on_positions.Subscribe(_security_positions_demuxer),
-            _bus.subscribe(SparkTags.CreateOrder).Subscribe(a => send_order(a.definition, a.price, a.size).ignore()),
             _bus.subscribe(SparkTags.RecreateOrder).Subscribe(a => a.ForEach(x => send_order(x.definition, x.price, x.size).ignore())),
             _bus.subscribe(SparkTags.CancelOrders).Subscribe(x => cancel_orders(x).ignore()),
             _bus.subscribe(SparkTags.DetachOrders).Subscribe(x => detach_orders(x).ignore()),
@@ -266,7 +264,6 @@ namespace mbts.spark {
       public IObservable<IReadOnlyDictionary<uint, OrderSecurity>> working_orders { get; }
       public IObservable<TableUpdate<(SecurityImmutable?, PNLUnitImmutable), int>> on_pnl { get; }
       public IObservable<TableUpdate<PNLRowData, PNLKey>> security_account_pnl { get; }
-      public IObservable<TableUpdate<AccountPNLImmutable, uint>> on_account_pnl { get; }
       public IObservable<TableUpdate<(SecurityImmutable?, PositionUnitImmutable), int>> on_positions { get; }
       public IObservable<TableUpdate<(AlgoData algo, AlgoDefinitionMetadata? meta), AlgoKey>> studio_algos_with_meta { get; }
       public IObservable<TableUpdate<(SecurityImmutable?, ManualFillImmutable), int>> on_manual_fills_securities { get; }
@@ -287,13 +284,33 @@ namespace mbts.spark {
       #endregion
 
       #region Input
-      public async Task<uint?> send_order(OrderDefinitionImmutable? order, double price, uint size) {
+      // the security arg is needed for a temporary hack to prevent GlbxIceberg orders from going out in non-glbx exchanges (like Ice)
+      // eventually, we'll need an algo type model to handle this stuff.
+      // This hack leverages the fact that every place that calls 'send_order' already has the security handy (not just the sec id).
+      public Task<uint?> send_order(SecurityImmutable security, OrderDefinitionImmutable? order, double price, uint size) {
          if (order.HasValue) {
-            var result = (await send_order_get_error(order.Value, price, size));
-            _activity.on_send_order(order.Value, price, size, result.key, result.error);
-            return result.key;
+#if DEBUG
+            if (security.Exchange != Venue.Glbx && security.Exchange != Venue.Sim) {
+#else
+            if (security.Exchange != Venue.Glbx) {
+#endif
+               var algo_structure = OrderDataHelper.enumerate_algo_structure(order.Value.CustomDef);
+               if (algo_structure.Any(t => t.type == CustomOrderDefinition.CustomOneofCase.GlbxIceberg)) {
+                  Log.Ui.info($"OrdersModel prevented a GlbxIceberg order from going out to a non-glbx exchange ({OrderUtils.short_description(security, order.Value.Side, price, size)}).");
+                  return Task.FromResult<uint?>(null);
+               }
+            }
+            return send_order(order.Value, price, size);
          }
-         return 0;
+         return Task.FromResult<uint?>(null);
+      }
+
+      /// Note: this method is private to force outside callers to use this overload <see cref="send_order(SecurityImmutable, OrderDefinitionImmutable?, double, uint)"/>.
+      /// This allows us to control which orders can be sent to which security/exchange (e.g., GlbxIceberg orders are only permitted for securities on the Glbx exchange).
+      async Task<uint?> send_order(OrderDefinitionImmutable order, double price, uint size) {
+         var result = (await send_order_get_error(order, price, size));
+         _activity.on_send_order(order, price, size, result.key, result.error);
+         return result.key;
       }
 
       async Task<(uint? key, string error)> send_order_get_error(OrderDefinitionImmutable order, double price, uint size) {

src/client/spark/Themes/generic.xaml

@@ -339,7 +339,7 @@
     <Setter Property="BorderThickness" Value="1"/>
     <Setter Property="ScrollViewer.HorizontalScrollBarVisibility" Value="Auto"/>
     <Setter Property="ScrollViewer.VerticalScrollBarVisibility" Value="Auto"/>
-    <Setter Property="Padding" Value="3,3,1,3"/>
+    <Setter Property="Padding" Value="6,3,1,3"/>
     <Setter Property="ScrollViewer.CanContentScroll" Value="true"/>
     <Setter Property="ScrollViewer.PanningMode" Value="Both"/>
     <Setter Property="Stylus.IsFlicksEnabled" Value="False"/>

src/client/spark/core/Bus/SparkTags.cs

@@ -17,7 +17,6 @@ namespace mbts.spark {
       #endregion
 
       #region Order
-      public static ITag<(OrderDefinitionImmutable definition, double price, uint size)> CreateOrder { get; } = new SparkTag<(OrderDefinitionImmutable, double, uint)>();
       public static ITag<IEnumerable<(OrderDefinitionImmutable definition, double price, uint size)>> RecreateOrder { get; } = new SparkTag<IEnumerable<(OrderDefinitionImmutable, double, uint)>>();
 
       public static ITag<IEnumerable<uint>> CancelOrders { get; } = new SparkTag<IEnumerable<uint>>();

src/client/spark/view/Controls/DIYGrid/Columns/ColumnBase.cs

@@ -114,7 +114,6 @@ namespace mbts.ui.controls {
             Aggressor = "Aggressor",
             Volume = "Volume",
             Last = "Last",
-            S = "S",
             Pos = "Pos",
             Working = "Working",
             Bid = "Bid",
@@ -125,7 +124,6 @@ namespace mbts.ui.controls {
             SessionLow = "Low",
             SessionHigh = "High",
             FillKind = "Fill Kind",
-            FillStatus = "Fill Status",
             Name = "Name",
             Description = "Description",
             N_Legs = "# Legs",

src/client/spark/view/Controls/DIYGrid/DoubleUtil.cs

@@ -1,10 +1,9 @@
 using System;
 using System.Linq;
 
-namespace mbts.ui.controls
-{
-   internal class DoubleUtil
-   {
+namespace mbts.ui.controls {
+   static class DoubleUtil {
+      public const int API_DECIMAL_PRECISION = 9;
       /* smallest such that 1.0+DBL_EPSILON != 1.0 */
       private const double EPSILON  =   2.2204460492503131e-016;
       private const double BIG_EPSILON = EPSILON * 10;
@@ -36,6 +35,9 @@ namespace mbts.ui.controls
             (double.IsNaN(right) ? double.NaN : -right) :
             (double.IsNaN(right) ? left : left - right);
 
+      public static decimal round_to_decimal(double value, int decimals = API_DECIMAL_PRECISION) =>
+         double.IsNaN(value) ? 0 : (decimal)Math.Round(value, decimals, MidpointRounding.AwayFromZero);
+
       internal static bool AreClose(double v1, double v2) {
          if (v1 == v2) return true;
          double delta = Math.Abs(v1 - v2);

src/client/spark_api/Co2/MDConsumerHandle.cs

@@ -133,12 +133,6 @@ namespace mbts.co2 {
          total_volume = _cold->total_volume;
          return (_cold->last_trade.load_if_new(ref time, out ps));
       }
-      public bool get_last_trade(ref ulong time, out NativePriceSize ps, out DateTime timestamp, out ulong total_volume, out bool from_spread) {
-         timestamp = _cold->last_trade_time;
-         total_volume = _cold->total_volume;
-         from_spread = (_cold->from_spread != 0);
-         return (_cold->last_trade.load_if_new(ref time, out ps));
-      }      
       public ulong misc_timestamp { get { return _cold->misc_timestamp; } }
 
       public ulong total_volume { get { return _cold->total_volume; } }
@@ -265,7 +259,7 @@ namespace mbts.co2 {
          // Last trade
          public AtomicPriceSize last_trade;
          public ulong total_volume;
-         public utils.LLDateTime last_trade_time;         
+         public utils.LLDateTime last_trade_time;
 
          //// Settlement
          public UInt64 misc_timestamp;
@@ -286,7 +280,6 @@ namespace mbts.co2 {
          public double session_low_offer;
          public double fixing_price;
 
-         public int from_spread;
          public int deleted;
          // FixedSecurity will not be directly addressable.  Instead the security is available through the callback
          public Int64 security_size;

src/client/spark_api/api/Update.cs

@@ -8,7 +8,6 @@ using System.Linq;
 using System.Reactive.Concurrency;
 using System.Reactive.Disposables;
 using System.Reactive.Linq;
-using System.Reactive.Subjects;
 using System.Text;
 
 namespace bts.api.core {
@@ -94,38 +93,38 @@ namespace bts.api.core {
          new Update<T, TKey>(new T[0], new T[0], new Change<T>[0], new T[0], key_extractor);
 
       public Update<T,TKey> merge(Update<T, TKey> next) {
-         var comp = new RowEqualityComparer<T, TKey>(_key);
-         var added = this.added as HashSet<T> ?? new HashSet<T>(this.added, comp);
+         var added = this.added as KeyedValueSet<TKey, T> ?? new KeyedValueSet<TKey, T>(this.added, _key);
          var changes = this.changed as ChangeDict<T,TKey> ?? new ChangeDict<T,TKey>(this.changes, _key);
-         var removed = this.removed as HashSet<T> ?? new HashSet<T>(this.removed, comp);
+         var removed = this.removed as KeyedValueSet<TKey, T> ?? new KeyedValueSet<TKey, T>(this.removed, _key);
 
          // Add
-         added.UnionWith(next.added);
+         foreach (var v in next.added) {
+            var k = _key(v);
+            if (removed.Remove(k, out var oldval)) {
+               changes.AddOrMerge(new Change<T>(oldval, v));
+            } else {
+               added[k] = v;
+            }
+         }
 
          // Change
          foreach (var c in next.changes) {
-            if (added.Contains(c.Old)) {
-               added.Remove(c.Old);
-               added.Add(c.New);
-            }
-            else {
+            var k = _key(c.Old);
+            if (!added.Replace(k, c.New))
                changes.AddOrMerge(c);
-            }
          }
 
          // Remove
          foreach (var r in next.removed) {
-            if (!added.Remove(r)) {
-               removed.Add(r);
-            }
-            changes.Remove(_key(r));
+            var k = _key(r);
+            if (!added.Remove(k))
+               removed[k] = changes.TryRemove(k, out var ch) ? ch.Old : r;
          }
 
          return new Update<T, TKey>(
             next.all,
             added, changes, removed, 
-            _key
-            );
+            _key);
       }
 
       public IEnumerable<T> all { get; }
@@ -148,6 +147,39 @@ namespace bts.api.core {
 #endif
    }
 
+   class KeyedValueSet<TKey, TValue> : IEnumerable<TValue> {
+      readonly Dictionary<TKey, TValue> _cache = new Dictionary<TKey, TValue>();
+
+      public KeyedValueSet(IEnumerable<TValue> value, Func<TValue, TKey> key) {
+         foreach (var v in value)
+            _cache[key(v)] = v;
+      }
+
+      public TValue this[TKey key] {
+         set => _cache[key] = value;
+      }
+
+      public bool Remove(TKey key) =>
+         _cache.Remove(key);
+
+      public bool Remove(TKey key, out TValue value) =>
+         _cache.TryGetValue(key, out value) && _cache.Remove(key);
+
+      public bool Replace(TKey key, TValue value) {
+         if (_cache.ContainsKey(key)) {
+            _cache[key] = value;
+            return true;
+         }
+         return false;
+      }
+
+      public IEnumerator<TValue> GetEnumerator() =>
+         _cache.Values.GetEnumerator();
+
+      IEnumerator IEnumerable.GetEnumerator() =>
+         GetEnumerator();
+   }
+
    class ChangeDict<T, TKey> : IEnumerable<Change<T>> {
       Dictionary<TKey, Change<T>> _changes;
       Func<Change<T>, TKey> _key;
@@ -167,9 +199,11 @@ namespace bts.api.core {
          return null;
       }
 
-      public void Remove(TKey key) {
+      public bool Remove(TKey key) =>
          _changes.Remove(key);
-      }
+
+      public bool TryRemove(TKey key, out Change<T> result) =>
+         _changes.TryGetValue(key, out result) && _changes.Remove(key);
 
       #region IEnumerable
       public IEnumerator<Change<T>> GetEnumerator() {
@@ -181,31 +215,6 @@ namespace bts.api.core {
       #endregion
    }
 
-   struct RowEqualityComparer<T,TKey> : IEqualityComparer<T>, IEqualityComparer<Change<T>>
-      where TKey : IEquatable<TKey> {
-      readonly Func<T,TKey> _key;
-
-      public RowEqualityComparer(Func<T,TKey> key) {
-         _key = key;
-      }
-
-      public bool Equals(T x, T y) {
-         return _key(x).Equals(_key(y));
-      }
-
-      public bool Equals(Change<T> x, Change<T> y) {
-         return Equals(x.New, y.New);
-      }
-
-      public int GetHashCode(T obj) {
-         return _key(obj).GetHashCode();
-      }
-
-      public int GetHashCode(Change<T> c) {
-         return GetHashCode(c.New);
-      }
-   }
-
    public class TableUpdate<T, TKey> where TKey : IEquatable<TKey> {
       // warning: this does not have a key_extractor
       public static readonly TableUpdate<T, TKey> empty = new TableUpdate<T, TKey>(ImmutableDictionary<TKey, T>.Empty, Array.Empty<T>(), Array.Empty<Change<T>>(), Array.Empty<T>(), null);
@@ -922,4 +931,4 @@ namespace bts.api.core {
             $"T is '{typeof(T)}', TKey is '{typeof(TKey)}'";
       }
    }
-}
+}

src/client/spark_excel/spark_excel.csproj

@@ -62,7 +62,7 @@
   </ItemGroup>
   
   <ItemGroup>
-    <None Include="spark_excel_example_2018.xlsm"><CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory></None>  
+    <None Include="spark_excel_example_2019.xlsm"><CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory></None>  
   </ItemGroup>
   
   <!-- <Target Name="CopyProjectReferencesToPackage" DependsOnTargets="ResolveReferences">

src/client/spark_sdk/README.md

@@ -62,35 +62,45 @@ include: "http://location:8000/config/"
 * Make sure you can connect to a running server
 * run spark_sdk.exe with no parameters to see help
 ```
-spark_sdk\bin\Debug\net471> spark_sdk.exe
+spark_sdk\bin\Debug\net471> spark_sdk.exe -ConfigServer http://location:8000/config
 
 BTS Spark SDK
 Usages: spark_sdk [command] [arguments]
 
 Commands:
    md <exchange> <security_type> <symbol> [<security_desc>]
-   ui <exchange> <security_type> <symbol> <security_desc> <poll_interval_in_ms>
-   order <exchange> <security_type> <symbol> <security_desc> <signed_quantity> [<price>]
+   order <account> <exchange> <security_type> <symbol> <security_desc> <signed_quantity> [<price>]
 Arguments:
+   --user=<bts user name>
+   --pass=<password>
    --pump=<pump_interval_in_ms=100 default>
    
-Example:
-   To print simulated market data to the screen:
-   spark_sdk.exe md SIM FUTURE ES SIM:ESU9 --user=bts
-```
+```   
+* To print simulated market data to the screen:
+
+   ```spark_sdk.exe md SIM FUTURE ES SIM:ESU9 --user=bts --pass= -ConfigServer http://location:8000/config```
+   
+* To send a 1 lot sell order for ES in account ACC A:
+
+   ```spark_sdk.exe order "ACC A" SIM FUTURE ES SIM:ESU9 -1 4000.5 --user=bts --pass= -ConfigServer http://location:8000/config   ```
+   
+   You should see an order status stream print to the screen after running this command 
+
 
 * Check out sample code
    * [Market Data](src/spark_sdk/samples/MarketDataSample.cs)
    * [Orders](src/spark_sdk/samples/OrderSample.cs)
    * [Driver](src/spark_sdk/samples/SDKMain.cs) 
 
-
+```
  ### Excel
- ```
- spark_sdk\bin\Debug\net471> start spark_excel-addin64.xll 
+ * Most BTS Client API calls can be made through Excel 2016, load the example spreadsheet to see how
+  ```
+ spark_sdk\bin\Debug\net471> start spark_excel-addin64-packed.xll 
  ```
  * This will start excel and load the bts addin
- * Check the [```excel```](excel/) directory for an example spreadsheet
+ * From excel, open  ```spark_sdk\bin\Debug\net471\spark_excel_example_2019.xlsm``` to get started
+ * Check the [```excel```](excel/) directory for other example spreadsheets
  
 
 

src/client/spark_sdk/spark_sdk.csproj

@@ -58,7 +58,7 @@
 
   <ItemGroup Condition= " '$(SparkRoot)' != '' ">
      <None Include="README.md" pack="true" PackagePath="." />
-     <None Include="$(OutputPath)\net471\spark_excel_example_2018.xlsm" pack="true" PackagePath=".\excel\spark_excel_example_2018.xlsm" />
+     <None Include="$(OutputPath)\net471\spark_excel_example_2019.xlsm" pack="true" PackagePath=".\excel\spark_excel_example_2019.xlsm" />
      <None Include="$(OutputPath)\net471\spark_excel-addin64-packed.xll" pack="true" PackagePath=".\excel\spark_excel-addin64-packed.xll" />
      <None Include="$(OutputPath)\net471\spark_excel-addin64.dna" pack="true" PackagePath=".\excel\spark_excel-addin64.dna" />
      <None Include="doc/**" pack="true" PackagePath=".\doc\" />
@@ -96,7 +96,7 @@
 
    <Target Name="copy_lib_deployed471" AfterTargets="PostBuildEvent" Condition= " '$(SparkRoot)' == '' and '$(TargetFramework)' == 'net471'" >
      <Copy SourceFiles="..\..\lib\net471\spark_excel.dll" DestinationFolder="$(OutputPath)\" SkipUnchangedFiles="true" />
-     <Copy SourceFiles="..\..\excel\spark_excel_example_2018.xlsm" DestinationFolder="$(OutputPath)\" SkipUnchangedFiles="true" />
+     <Copy SourceFiles="..\..\excel\spark_excel_example_2019.xlsm" DestinationFolder="$(OutputPath)\" SkipUnchangedFiles="true" />
      <Copy SourceFiles="..\..\excel\spark_excel-addin64-packed.xll" DestinationFolder="$(OutputPath)\" SkipUnchangedFiles="true" />
      <Copy SourceFiles="..\..\excel\spark_excel-addin64.dna" DestinationFolder="$(OutputPath)\" SkipUnchangedFiles="true" />
    </Target>

src/client/spark_test/Co2/Co2Tests.cs

@@ -142,8 +142,7 @@ namespace mbts.co2.tests
          Assert.IsTrue(entry.session_low_offer == 5.5);
          Assert.IsTrue(entry.fixing_price == 6.6);
 
-         Assert.IsTrue(entry.from_spread == 0);
-         Assert.IsTrue(entry.deleted == 1);         
+         Assert.IsTrue(entry.deleted == 1);
       }
 
       [Test]

src/client/spark_test/Utils/TestUtils.cs

@@ -85,8 +85,6 @@ namespace bts.tests.utils {
          if(timeout == null) {
             timeout = TimeSpan.FromMinutes(2);
          }
-         Console.WriteLine("Please Wait... this is going to take some time...");
-         await Task.Delay(180000);
 
          System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch();
          sw.Start();
@@ -133,13 +131,6 @@ namespace bts.tests.utils {
          System.Diagnostics.Stopwatch watch = new System.Diagnostics.Stopwatch();
          watch.Start();
 
-         var prts = mbts.config.Configuration.get<bts.config.Ports>();
-         Console.WriteLine("\n\n\n\n\n\n\n\n\n\n");
-         Console.WriteLine("config_port: " + prts.config_port);
-         Console.WriteLine("qbus_port: " + prts.qbus_port);
-         Console.WriteLine("glbx_sim_fix_port: " + prts.glbx_sim_fix_port);
-         Console.WriteLine("\n\n\n\n\n\n\n\n\n\n");
-
          var ctx = new ServiceContext();
          var mdrelay = ctx.locate<bts.co2.MarketDataRelay>();
 
@@ -152,10 +143,6 @@ namespace bts.tests.utils {
                products = tmpp.value;
                break;
             }
-            else {
-               Console.WriteLine("mdrelay.request_products_async Error: ");
-               Console.WriteLine(tmpp.error_message);
-            }
             await Task.Delay(10000);
          }
 
@@ -250,7 +237,7 @@ namespace bts.tests.utils {
          Result<co2.MDReply>? md_result = await await_with_timeout(
                                                                md_relay.requestmd_async(md_request),
                                                                timeout);
-         Console.Write(md_result);
+
          return md_result != null;
       }
    }

src/client/spark_test/system_test_scripts/test_position_limits.sh

@@ -30,8 +30,6 @@ cat << EOF > $sparkdata_dir/conf/spark.conf
    }
 EOF
 
-echo "Port"
-./sparkd configcheck Ports
 
 nohup ./sparkd run &
 pid=$!

src/data_files/files/onixs_ice/ConnectivityConfiguration.APITest.xml

@@ -11,6 +11,7 @@
 -->
 <connectivityConfiguration>
     <!-- TCP Connection Information -->
+
     <tcp>
         <server name="default">
             <ip>63.247.113.163</ip>
@@ -18,7 +19,7 @@
             <userName>bluetrd-ps</userName>
             <password>Starts123</password>
             <getStripInfoMessages>N</getStripInfoMessages>
-            <strategyPreference>New</strategyPreference>
+            <strategyPreference>Legacy</strategyPreference>
             <localNetworkInterface></localNetworkInterface>
         </server>
         <server name="endex">
@@ -27,7 +28,7 @@
             <userName>Value from ICE Support</userName>
             <password>Value from ICE Support</password>
             <getStripInfoMessages>N</getStripInfoMessages>
-            <strategyPreference>New</strategyPreference>
+            <strategyPreference>Legacy</strategyPreference>
             <localNetworkInterface></localNetworkInterface>
         </server>
         <server name="liffe-eu">
@@ -36,7 +37,7 @@
             <userName>bluetrdl-ps</userName>
             <password>Starts123</password>
             <getStripInfoMessages>N</getStripInfoMessages>
-            <strategyPreference>New</strategyPreference>
+            <strategyPreference>Legacy</strategyPreference>
             <localNetworkInterface></localNetworkInterface>
         </server>
         <settings>

src/data_files/files/onixs_ice/ConnectivityConfiguration.PerfTest.xml

@@ -17,10 +17,10 @@
         <server name="default">
             <ip>63.247.113.214</ip>
             <port>3000</port>
-            <userName>bt_ps</userName>
-            <password>Starts123</password>
+            <userName>Value from ICE Support</userName>
+            <password>Value from ICE Support</password>
             <getStripInfoMessages>N</getStripInfoMessages>
-            <strategyPreference>New</strategyPreference>
+            <strategyPreference>Legacy</strategyPreference>
             <localNetworkInterface></localNetworkInterface>
         </server>
         <settings>

src/lib/CMakeDeps_win.cmake

@@ -1 +1 @@
-set(sources api/apicore.proto;api/apigateway.proto;api/apioptions.proto;api/core/ApiClientInterop.cpp;api/core/ApiServerInterop.cpp;api/public/ApiServer.cpp;api/public/ClientChannel.cpp;api/public/QuickbusClientChannel.cpp;api/public/ServerChannel.cpp;api/public/ServiceContext.cpp;api/service_helpers/AdminServiceUtils.cpp;api/service_helpers/AlgoServiceUtils.cpp;api/service_helpers/PositionUtilities.cpp;api/services/AdminService.proto;api/services/AlgoDefinitionService.proto;api/services/AlgoService.proto;api/services/ApiUnitTestService.proto;api/services/AuthService.proto;api/services/BasePositionService.proto;api/services/ErrorReportService.proto;api/services/HelloService.proto;api/services/MBOService.proto;api/services/MarginService.proto;api/services/MarketDataAuxiliaryService.proto;api/services/MarketDataRelay.proto;api/services/MoneyFileService.proto;api/services/OrderService.proto;api/services/OutOfProcessURLListener.proto;api/services/PersistenceService.proto;api/services/PositionService.proto;api/services/ShellDiagnosticService.proto;api/services/ShellLatencyService.proto;api/services/SyntheticSecurityService.proto;api/test/ApiTests.cpp;api/test/HelloServiceServer.cpp;config/ConfigCheck.cpp;config/ConfigDownloader.cpp;config/ConfigErrorCollector.cpp;config/ConfigLog.cpp;config/ConfigPropertyEvaluator.cpp;config/ConfigValueQueue.cpp;config/ConfigurationException.cpp;config/ConfigurationState.cpp;config/PreprocessInputStream.cpp;config/config.cpp;config/defaults/MarketData.conf;config/defaults/Ports.conf;config/defaults/Symbols.conf;config/defaults/cme_reference_data.conf;config/defaults/edge.conf;config/defaults/jetfix.conf;config/defaults/keycode.conf;config/defaults/logging.conf;config/defaults/orderman_cfg.conf;config/defaults/orders_fills_cleanup.conf;config/defaults/position_cfg.conf;config/defaults/span.conf;config/defaults/spreader.conf;config/defaults/testkeys.conf;config/defaults/threads.conf;config/defaults/toggles.conf;config/extensions/extensions.proto;config/invalid/invalid.proto;config/protos/ExchangeSimCfg.proto;config/protos/MarketData.proto;config/protos/Ports.proto;config/protos/Symbols.proto;config/protos/UI.proto;config/protos/cme_reference_data.proto;config/protos/jetfix.proto;config/protos/keycode.proto;config/protos/logging.proto;config/protos/mercury_cfg.proto;config/protos/orderman_cfg.proto;config/protos/orders_fills_cleanup.proto;config/protos/perflog_cfg.proto;config/protos/position_cfg.proto;config/protos/quickbus_cfg.proto;config/protos/span.proto;config/protos/spreader.proto;config/protos/system.proto;config/protos/testkeys.proto;config/protos/threads.proto;config/protos/toggles.proto;config/tests/HTTPServer.cpp;config/tests/config_ref_tests.cpp;config/tests/config_tests.cpp;config/text_format/text_format.cpp;domain_model/Security.cpp;domain_model/Security.proto;domain_model/SecurityTests.cpp;domain_model/sod.cpp;domain_model/tests/domain_tests.cpp;expression/BooleanGrammar.cpp;expression/ExpressionEvaluator.cpp;expression/ParserExceptions.cpp;expression/Value.cpp;expression/tests/expression_parser_tests.cpp;logging/LoggerHub.cpp;logging/Sink.cpp;logging/logging.cpp;logging/tests/logging_tests.cpp;md/Diag/MDPeriodicDumper.cpp;md/Diag/ShmDumper.cpp;md/MBOProcessor.cpp;md/MBOServiceServer.cpp;md/MDClient.cpp;md/MDConsumer.cpp;md/MDConsumerHandle.cpp;md/MDConsumerInterop.cpp;md/MDProducer.cpp;md/MDProducerHandle.cpp;md/MemorySegmentUtils.cpp;md/PlatformLoader.cpp;md/PriceSize.cpp;md/SecurityCache.cpp;md/SecurityDB.cpp;md/SecurityIDRecorder.cpp;md/SecurityMapper.cpp;md/SecurityOracle.cpp;md/SettlementRecorder.cpp;md/Shm/FixedSecurity.cpp;md/SyntheticSecurityService.cpp;md/cme/MDP3Utils.cpp;md/common.cpp;md/ice/ICEUtils.cpp;md/protos/co2.proto;md/relay/QueryServer.cpp;md/relay/RelayClient.cpp;md/relay/RelayRecorder.cpp;md/test/Co2ManagedTestHelper.cpp;md/test/Co2TestInstrument.cpp;md/test/mbo_tests.cpp;md/test/md2_tests.cpp;md/test/md_test_utils.cpp;md/test/synthetic_book_merge_tests.cpp;md/vendor/OnixsUtils.cpp;os/os.cpp;os/tests/os_tests.cpp;quickbus/core/LocalNode.cpp;quickbus/core/Quickbus.cpp;quickbus/core/SubscriptionID.cpp;quickbus/core/TableID.cpp;quickbus/message_model/Update.cpp;quickbus/message_model/merge_updates.cpp;quickbus/network/QBConnection.cpp;quickbus/network/QBConnectionManager.cpp;quickbus/network/endpoint_description.cpp;quickbus/network/quickbus_msgs.proto;quickbus/pubsub/TablePublisher.cpp;quickbus/pubsub/TableSubscriber.cpp;quickbus/pubsub/tests/qb_pubsub_tests.cpp;quickbus/routing/INode.cpp;quickbus/routing/NodeID.cpp;quickbus/routing/Router.cpp;quickbus/rpc/RPCClient.cpp;quickbus/rpc/RPCServer.cpp;quickbus/rpc/RPCSessionID.cpp;quickbus/rpc/rpc_utils.cpp;quickbus/rpc/tests/qb_rpc_tests.cpp;quickbus/tests/quickbus_tests.cpp;sparkproto/SparkProtos.proto;testing/catch_test_runner.cpp;utils/AuthSessionManager.cpp;utils/AuthToken.cpp;utils/Dispatcher.cpp;utils/GrowableSPSCQueue.cpp;utils/HTTPStream.cpp;utils/PBUtils.cpp;utils/Puid.cpp;utils/SecureLogoLine truncated
+set(sources api/apicore.proto;api/apigateway.proto;api/apioptions.proto;api/core/ApiClientInterop.cpp;api/core/ApiServerInterop.cpp;api/public/ApiServer.cpp;api/public/ClientChannel.cpp;api/public/QuickbusClientChannel.cpp;api/public/ServerChannel.cpp;api/public/ServiceContext.cpp;api/service_helpers/AdminServiceUtils.cpp;api/service_helpers/AlgoServiceUtils.cpp;api/service_helpers/PositionUtilities.cpp;api/services/AdminService.proto;api/services/AlgoDefinitionService.proto;api/services/AlgoService.proto;api/services/ApiUnitTestService.proto;api/services/AuthService.proto;api/services/BasePositionService.proto;api/services/ErrorReportService.proto;api/services/HelloService.proto;api/services/MBOService.proto;api/services/MarginService.proto;api/services/MarketDataAuxiliaryService.proto;api/services/MarketDataRelay.proto;api/services/MoneyFileService.proto;api/services/OrderService.proto;api/services/OutOfProcessURLListener.proto;api/services/PersistenceService.proto;api/services/PositionService.proto;api/services/ShellDiagnosticService.proto;api/services/ShellLatencyService.proto;api/services/SyntheticSecurityService.proto;api/test/ApiTests.cpp;api/test/HelloServiceServer.cpp;config/ConfigCheck.cpp;config/ConfigDownloader.cpp;config/ConfigErrorCollector.cpp;config/ConfigLog.cpp;config/ConfigPropertyEvaluator.cpp;config/ConfigValueQueue.cpp;config/ConfigurationException.cpp;config/ConfigurationState.cpp;config/PreprocessInputStream.cpp;config/config.cpp;config/defaults/MarketData.conf;config/defaults/Ports.conf;config/defaults/Symbols.conf;config/defaults/cme_reference_data.conf;config/defaults/edge.conf;config/defaults/jetfix.conf;config/defaults/keycode.conf;config/defaults/logging.conf;config/defaults/orderman_cfg.conf;config/defaults/orders_fills_cleanup.conf;config/defaults/position_cfg.conf;config/defaults/span.conf;config/defaults/spreader.conf;config/defaults/testkeys.conf;config/defaults/threads.conf;config/defaults/toggles.conf;config/extensions/extensions.proto;config/invalid/invalid.proto;config/protos/ExchangeSimCfg.proto;config/protos/MarketData.proto;config/protos/Ports.proto;config/protos/Symbols.proto;config/protos/UI.proto;config/protos/cme_reference_data.proto;config/protos/jetfix.proto;config/protos/keycode.proto;config/protos/logging.proto;config/protos/mercury_cfg.proto;config/protos/orderman_cfg.proto;config/protos/orders_fills_cleanup.proto;config/protos/perflog_cfg.proto;config/protos/position_cfg.proto;config/protos/quickbus_cfg.proto;config/protos/span.proto;config/protos/spreader.proto;config/protos/system.proto;config/protos/testkeys.proto;config/protos/threads.proto;config/protos/toggles.proto;config/tests/HTTPServer.cpp;config/tests/config_ref_tests.cpp;config/tests/config_tests.cpp;config/text_format/text_format.cpp;domain_model/Security.cpp;domain_model/Security.proto;domain_model/SecurityTests.cpp;domain_model/sod.cpp;domain_model/tests/domain_tests.cpp;expression/BooleanGrammar.cpp;expression/ExpressionEvaluator.cpp;expression/ParserExceptions.cpp;expression/Value.cpp;expression/tests/expression_parser_tests.cpp;logging/LoggerHub.cpp;logging/Sink.cpp;logging/logging.cpp;logging/tests/logging_tests.cpp;md/Diag/MDPeriodicDumper.cpp;md/Diag/ShmDumper.cpp;md/MBOProcessor.cpp;md/MBOServiceServer.cpp;md/MDClient.cpp;md/MDConsumer.cpp;md/MDConsumerHandle.cpp;md/MDConsumerInterop.cpp;md/MDProducer.cpp;md/MDProducerHandle.cpp;md/MemorySegmentUtils.cpp;md/PlatformLoader.cpp;md/PriceSize.cpp;md/SecurityCache.cpp;md/SecurityDB.cpp;md/SecurityIDRecorder.cpp;md/SecurityMapper.cpp;md/SecurityOracle.cpp;md/SettlementRecorder.cpp;md/Shm/FixedSecurity.cpp;md/SyntheticSecurityService.cpp;md/cme/MDP3Utils.cpp;md/common.cpp;md/ice/ICEUtils.cpp;md/protos/co2.proto;md/relay/QueryServer.cpp;md/relay/RelayClient.cpp;md/relay/RelayRecorder.cpp;md/test/Co2ManagedTestHelper.cpp;md/test/Co2TestInstrument.cpp;md/test/mbo_tests.cpp;md/test/md2_tests.cpp;md/test/md_test_utils.cpp;md/test/synthetic_book_merge_tests.cpp;md/vendor/OnixsUtils.cpp;os/os.cpp;os/tests/os_tests.cpp;quickbus/core/LocalNode.cpp;quickbus/core/Quickbus.cpp;quickbus/core/SubscriptionID.cpp;quickbus/core/TableID.cpp;quickbus/message_model/Update.cpp;quickbus/message_model/merge_updates.cpp;quickbus/network/QBConnection.cpp;quickbus/network/QBConnectionManager.cpp;quickbus/network/endpoint_description.cpp;quickbus/network/quickbus_msgs.proto;quickbus/pubsub/TablePublisher.cpp;quickbus/pubsub/TableSubscriber.cpp;quickbus/pubsub/tests/qb_pubsub_tests.cpp;quickbus/routing/INode.cpp;quickbus/routing/NodeID.cpp;quickbus/routing/Router.cpp;quickbus/rpc/RPCClient.cpp;quickbus/rpc/RPCServer.cpp;quickbus/rpc/RPCSessionID.cpp;quickbus/rpc/rpc_utils.cpp;quickbus/rpc/tests/qb_rpc_tests.cpp;quickbus/tests/quickbus_tests.cpp;sparkproto/SparkProtos.proto;testing/catch_test_runner.cpp;utils/AuthSessionManager.cpp;utils/AuthToken.cpp;utils/Dispatcher.cpp;utils/GrowableSPSCQueue.cpp;utils/HTTPStream.cpp;utils/PBUtils.cpp;utils/Puid.cpp;utils/SecureLogoLine truncated

src/lib/api/service_helpers/AdminServiceUtils.cpp

@@ -46,15 +46,12 @@ bool AdminServiceUtils::has_exchange(const ExchangeConnection& conn,
       break;
    case api::Venue::VENUE_UNSET:
    case api::Venue::VENUE_SIM:
-   case api::Venue::VENUE_SIMICE:
       rv = conn.has_sim_connection();
-      break;      
-   case api::Venue::VENUE_ICE:
-      rv = conn.has_ice_connection();
       break;
    case api::Venue::VENUE_BTS:
    case api::Venue::VENUE_CBOE:
    case api::Venue::VENUE_CBOECFE:
+   case api::Venue::VENUE_ICE:
    case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
    case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:
       rv = false;
@@ -92,23 +89,6 @@ const ExchangeConnection* AdminServiceUtils::resolve_cme_connection(
    return nullptr;                                                               
 }
 
-const bts::api::admin::ExchangeConnection*
-AdminServiceUtils::resolve_ice_connection(const bts::api::admin::AdminData& data,
-                                          const bts::api::admin::BTSAccount& account,
-                                          const bts::api::Security&)
-{
-   if(account.has_connections())
-   {
-      for (const std::string& conn_name : account.connections().exchange_connection_names())
-      {
-         const bts::api::admin::ExchangeConnection* conn = find_connection(data, conn_name);
-         if (conn!=nullptr && conn->has_ice_connection())
-            return conn;  
-      }
-   }
-   return nullptr;                                                               
-}
-
 const bts::api::admin::ExchangeConnection*
 AdminServiceUtils::resolve_sim_connection(const bts::api::admin::AdminData& data,
                                           const bts::api::admin::BTSAccount& account,
@@ -137,22 +117,14 @@ AdminServiceUtils::resolve_connection(const bts::api::admin::AdminData& data,
          rv = resolve_sim_connection(data, account, sec);
       break;
    }
-   case api::Venue::VENUE_ICE: 
-   {      
-      rv = resolve_ice_connection(data, account, sec);
-      auto deployment_type = boost::to_lower_copy(config::get<config::DeploymentType>().type());
-      if (rv == nullptr && (deployment_type == "dev" || deployment_type == "demo"))
-         rv = resolve_sim_connection(data, account, sec);
-      break;
-   }   
    case api::Venue::VENUE_UNSET:
    case api::Venue::VENUE_BTS:
    case api::Venue::VENUE_SIM:
-   case api::Venue::VENUE_SIMICE:
       rv = resolve_sim_connection(data, account, sec);
       break;
    case api::Venue::VENUE_CBOE:
    case api::Venue::VENUE_CBOECFE:      
+   case api::Venue::VENUE_ICE:
    case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
       case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:
       rv = nullptr;
@@ -189,15 +161,9 @@ void AdminServiceUtils::fill_connections_from_config(bts::api::admin::AdminData&
       case config::JetFIX::Connection::CBOE:
          break;
       case config::JetFIX::Connection::SIM:
-      case config::JetFIX::Connection::SIMICE:
          connection.mutable_sim_connection();
          add_connection(admin_data,connection);
          break;
-      case config::JetFIX::Connection::ICE:
-         connection.mutable_ice_connection()->set_sender_compid(c.sender_compid());
-         add_connection(admin_data,connection);
-         break;
-         
       }
    }
    for(int i=0;i<admin_data.accounts_size();i++)
@@ -384,16 +350,13 @@ std::string AdminServiceUtils::find_account_exchange_account (
          rv = cme_account_data->clearing_account();
       break;
    }
-   case api::Venue::VENUE_ICE:
-   {
+   case api::Venue::VENUE_ICE:{
       auto ice_account_data = get_account_ice_data(account, data);
       if (ice_account_data != nullptr)
-         rv = ice_account_data->clearing_account();
-      break;      
+         rv = ice_account_data->clearing_account_override();
+      break;
    }
-   case api::Venue::VENUE_SIM: 
-   case api::Venue::VENUE_SIMICE:
-   {
+   case api::Venue::VENUE_SIM: {
       auto sim_account_data = get_account_sim_data(account, data);
       if (sim_account_data != nullptr)
          rv = sim_account_data->clearing_account();
@@ -503,46 +466,6 @@ bts::api::orderman::RoutingRecord AdminServiceUtils::get_routing_record(const bt
       return rv;
    }
    switch (sec.exchange()) {
-   case api::Venue::VENUE_ICE:
-   {
-      const bts::api::admin::ICEUserData* ice_user_data = get_user_ice_data(*user_ptr, data);
-      if(ice_user_data==nullptr)
-      {
-        out_result.set_is_success(false);
-        out_result.set_error_text("Could not find ICE data for user");
-        return rv;
-      }     
-      const bts::api::admin::ICEAccountData* ice_account_data = get_account_ice_data(*account_ptr, data);
-      if(ice_account_data==nullptr)
-      {
-         out_result.set_is_success(false);
-         out_result.set_error_text("Could not find ICE data for account");
-         return rv;
-      }        
-      // check for data corectness
-      if (ice_user_data->operator_id().length() == 0)
-      {
-         out_result.set_is_success(false);
-         out_result.set_error_text("Operator ID is not set");
-         return rv;
-      }
-
-      rv.set_session(conn_ptr->name());
-      rv.set_sender_compid(conn_ptr->ice_connection().sender_compid());
-      rv.set_account(ice_account_data->clearing_account());
-      rv.set_operator_id(ice_user_data->operator_id());   
-      rv.set_cti_code(ice_account_data->cti());
-      rv.set_customer_or_firm(ice_account_data->origin());
-      rv.set_clearing_firm(ice_account_data->clearing_firm());
-      rv.set_customer_account_ref_id(ice_account_data->customer_account_ref_id());
-      rv.set_account_code(ice_account_data->account_code());
-      rv.set_company_id(std::to_string(ice_user_data->company_id()));
-      rv.set_authorized_trader(ice_user_data->authorized_trader());
-      rv.set_authorized_group(ice_user_data->authorized_group());      
-      rv.set_cust_order_handing_inst(ice_user_data->cust_order_handing_inst());
-      
-      return rv;             
-   }
    case bts::api::Venue::VENUE_GLBX: {
      const bts::api::admin::CMEUserData* cme_user_data = get_user_cme_data(*user_ptr, data);
      if(cme_user_data==nullptr){
@@ -593,9 +516,7 @@ bts::api::orderman::RoutingRecord AdminServiceUtils::get_routing_record(const bt
      rv.set_customer_or_firm(cme_account_data->origin());
      return rv;
    }
-   case api::Venue::VENUE_SIM: 
-   case api::Venue::VENUE_SIMICE: 
-   {
+   case api::Venue::VENUE_SIM: {
      const bts::api::admin::SIMUserData* sim_user_data = get_user_sim_data(*user_ptr, data);
      if(sim_user_data==nullptr){
         out_result.set_is_success(false);
@@ -641,7 +562,8 @@ bts::api::orderman::RoutingRecord AdminServiceUtils::get_routing_record(const bt
    case api::Venue::VENUE_UNSET:
    case api::Venue::VENUE_BTS:
    case api::Venue::VENUE_CBOE:
-   case api::Venue::VENUE_CBOECFE:   
+   case api::Venue::VENUE_CBOECFE:
+   case api::Venue::VENUE_ICE:
    case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
    case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:
       out_result.set_is_success(false);
@@ -788,18 +710,6 @@ const bts::api::admin::CMEUserData* AdminServiceUtils::get_group_cme_data(const
    return nullptr;
 }
 
-const bts::api::admin::ICEUserData* AdminServiceUtils::get_group_ice_data(const bts::api::admin::BTSUserGroup& group,
-                                                                          const bts::api::admin::AdminData& data){
-   if (group.has_user_data() && group.user_data().has_exchange_user_data() && group.user_data().exchange_user_data().has_ice_data())
-      return &(group.user_data().exchange_user_data().ice_data());
-   else if (has_id(group.parent_group_id())){
-      const bts::api::admin::BTSUserGroup* parent_group = find_group(data, group.parent_group_id());
-      if (parent_group != nullptr)
-         return get_group_ice_data(*parent_group, data);
-   }
-   return nullptr;
-}
-
 const bts::api::admin::SIMUserData* AdminServiceUtils::get_group_sim_data(const bts::api::admin::BTSUserGroup& group,
                                                                           const bts::api::admin::AdminData& data){
    if (group.has_user_data() && group.user_data().has_exchange_user_data() && group.user_data().exchange_user_data().has_sim_data())
@@ -930,18 +840,6 @@ const bts::api::admin::CMEUserData* AdminServiceUtils::get_user_cme_data(const b
    return nullptr;
 }
 
-const bts::api::admin::ICEUserData* AdminServiceUtils::get_user_ice_data(const bts::api::admin::BTSUser& user, 
-                                                                         const bts::api::admin::AdminData& data){
-   if (user.has_user_data() && user.user_data().has_exchange_user_data() && user.user_data().exchange_user_data().has_ice_data())
-      return &(user.user_data().exchange_user_data().ice_data());
-   else{
-      const bts::api::admin::BTSUserGroup* parent_group = find_group(data, user.parent_group_id());
-         if (parent_group != nullptr)
-            return get_group_ice_data(*parent_group, data);
-   }
-   return nullptr;
-}
-
 const bts::api::admin::SIMUserData* AdminServiceUtils::get_user_sim_data(const bts::api::admin::BTSUser& user, 
                                                                          const bts::api::admin::AdminData& data){
    if (user.has_user_data() && user.user_data().has_exchange_user_data() && user.user_data().exchange_user_data().has_sim_data())
@@ -969,16 +867,7 @@ std::string AdminServiceUtils::find_user_exchange_id (const bts::api::admin::Adm
          rv = cme_user_data->operator_id();
       break;
    }
-   case api::Venue::VENUE_ICE:
-   {
-      auto ice_user_data = get_user_ice_data(user, data);
-      if (ice_user_data != nullptr)
-         rv = ice_user_data->operator_id();
-      break;      
-   }
-   case api::Venue::VENUE_SIM: 
-   case api::Venue::VENUE_SIMICE:
-   {
+   case api::Venue::VENUE_SIM: {
       auto sim_user_data = get_user_sim_data(user, data);
       if (sim_user_data != nullptr)
          rv = sim_user_data->operator_id();
@@ -988,38 +877,8 @@ std::string AdminServiceUtils::find_user_exchange_id (const bts::api::admin::Adm
    case api::Venue::VENUE_UNSET:
    case api::Venue::VENUE_BTS:
    case api::Venue::VENUE_CBOE:
-   case api::Venue::VENUE_CBOECFE:         
-   case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
-   case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:      
-      out_result.set_is_success(false); out_result.set_error_text("Exchange not supported");
-      break;
-   }
-   return rv;
-}
-
-std::string AdminServiceUtils::find_user_company_id (const bts::api::admin::AdminData& data,
-                                                     const bts::api::admin::BTSUser& user,
-                                                     api::Venue exchange,
-                                                     bts::api::admin::AdminServiceOperationResult& out_result)
-{
-   out_result.set_is_success(true);
-   std::string rv = "";
-   switch(exchange)
-   {
+   case api::Venue::VENUE_CBOECFE:      
    case api::Venue::VENUE_ICE:
-   {
-      auto ice_user_data = get_user_ice_data(user, data);
-      if (ice_user_data != nullptr)
-         rv = std::to_string(ice_user_data->company_id());
-      break;      
-   }
-   case api::Venue::VENUE_SIM: 
-   case api::Venue::VENUE_SIMICE:
-   case api::Venue::VENUE_GLBX:
-   case api::Venue::VENUE_UNSET:
-   case api::Venue::VENUE_BTS:
-   case api::Venue::VENUE_CBOE:
-   case api::Venue::VENUE_CBOECFE:         
    case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
    case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:      
       out_result.set_is_success(false); out_result.set_error_text("Exchange not supported");
@@ -1027,7 +886,6 @@ std::string AdminServiceUtils::find_user_company_id (const bts::api::admin::Admi
    }
    return rv;
 }
-
 std::pair<const bts::api::admin::BTSAccount*, const bts::api::admin::BTSUser*>
 AdminServiceUtils::find_user_account_matches (const bts::api::admin::AdminData& admin_data,
                                               const std::string& connection, const std::string& exchange_account,
@@ -1038,22 +896,14 @@ AdminServiceUtils::find_user_account_matches (const bts::api::admin::AdminData&
    auto user_ptrs = users(admin_data, false);
    for(const bts::api::admin::BTSAccount* account_ptr : account_ptrs){
       if (account_has_connection(*account_ptr, connection)){
-         if (boost::algorithm::iequals(exchange_account, find_account_exchange_account(admin_data,*account_ptr,exchange,find_account_result)) &&
-                                       find_account_result.is_success())
-         {
-            for(const bts::api::admin::BTSUser* user_ptr : user_ptrs)
-            {
-               if (boost::algorithm::iequals(exchange_user,find_user_exchange_id(admin_data,*user_ptr,exchange,find_user_result)) &&
+         if (boost::algorithm::iequals(exchange_account,
+                                       find_account_exchange_account(admin_data,*account_ptr,exchange,find_account_result)) &&
+                            find_account_result.is_success()){
+            for(const bts::api::admin::BTSUser* user_ptr : user_ptrs){
+               if (boost::algorithm::iequals(exchange_user,
+                                             find_user_exchange_id(admin_data,*user_ptr,exchange,find_user_result)) &&
                    find_user_result.is_success())
                   return std::pair<const bts::api::admin::BTSAccount*, const bts::api::admin::BTSUser*>(account_ptr,user_ptr);
-                  
-               //This code eventually gets called by get_reverse_routing_record(...)
-               //At the ice, they send back the login level tag50 in exec reports which doesn't match any user level tag50s in the exchange_user
-               //so instead we look for the CompanyID (or client_id, tag 109) field here.  If this field matches we have a good bet that 
-               //the user we find is enough to fulfill the routing record enough to cancel or modify orphaned orders, etc
-               if (boost::algorithm::iequals(exchange_user,find_user_company_id(admin_data,*user_ptr,exchange,find_user_result)) &&
-                   find_user_result.is_success())
-                  return std::pair<const bts::api::admin::BTSAccount*, const bts::api::admin::BTSUser*>(account_ptr,user_ptr);                  
             }
          }
       }
@@ -1145,26 +995,16 @@ void AdminServiceUtils::validate_user_exchange_data (const bts::api::admin::BTSU
 }
 void AdminServiceUtils::get_md_entitlement_report(const AdminData& admin_data,
                           std::vector<MDEntitlementReportUnit>& entitlements){
-   for (const auto& user : admin_data.users())
-   {
-      const CMEUserData* cme_data=AdminServiceUtils::get_user_cme_data(user, admin_data);
-      if(cme_data!=nullptr)
-      {
-         for (const auto& cme_mic_data : cme_data->md_permissions())
-         {
-            MDEntitlementReportUnit rv=get_cme_md_entitlement_unit(user, cme_mic_data);
+   for (const auto& user : admin_data.users()){
+      const CMEUserData* cme_data=AdminServiceUtils::get_user_cme_data(user,
+                                                                  admin_data);
+      if(cme_data!=nullptr){
+         for (const auto& cme_mic_data : cme_data->md_permissions()){
+            MDEntitlementReportUnit rv=get_cme_md_entitlement_unit(user,
+                                                                 cme_mic_data);
             entitlements.push_back(rv);
          }
       }
-      const ICEUserData* ice_data=AdminServiceUtils::get_user_ice_data(user, admin_data);
-      if(ice_data!=nullptr)
-      {
-         for (const auto& ice_mic_data : ice_data->md_permissions())
-         {
-            MDEntitlementReportUnit rv=get_ice_md_entitlement_unit(user, ice_mic_data);
-            entitlements.push_back(rv);
-         }
-      }      
    }   
 }
 MDEntitlementReportUnit AdminServiceUtils::get_cme_md_entitlement_unit(
@@ -1212,22 +1052,6 @@ MDEntitlementReportUnit AdminServiceUtils::get_cme_md_entitlement_unit(
    rv.set_record_id(key.str());
    return rv;
 }
-MDEntitlementReportUnit AdminServiceUtils::get_ice_md_entitlement_unit(
-                                             const BTSUser& user,
-                                             const ICEMicData& ice_mic_data){
-   MDEntitlementReportUnit rv;
-   rv.set_user(user.name());
-   rv.set_user_id(user.id());
-   rv.set_exchange( bts::api::Venue_Name(bts::api::Venue::VENUE_ICE) );
-   rv.set_mic( bts::api::ICEMics_Name( ice_mic_data.mic()) );
-
-   rv.set_enabled(ice_mic_data.enabled());
-   std::ostringstream key;
-   key << rv.user() << rv.exchange() << rv.mic();
-   rv.set_record_id(key.str());
-   return rv;
-}
-
 void AdminServiceUtils::get_user_symbol_permissions(const AdminData& data,
                                                     const BTSUser& user,
                               std::vector<UserSymbolMDPermissionUnit>& rv){
@@ -1302,13 +1126,13 @@ void AdminServiceUtils::set_user_exchange_data(bts::api::admin::BTSUser* user,
    if (!user->user_data().has_exchange_user_data())
       user->mutable_user_data()->mutable_exchange_user_data()->CopyFrom(exchange_user_data);
 }
-// void AdminServiceUtils::set_user_md_entitlement_data(bts::api::admin::BTSUser* user,
-//                                                      const bts::api::admin::ExchangeUserMarketDataEntitlementData& exchange_user_md_entitelemnt_data){
-//    if (!user->has_user_data())
-//       user->mutable_user_data();
-//    if (!user->user_data().has_exchange_md_entitlement_data())
-//       user->mutable_user_data()->mutable_exchange_md_entitlement_data()->CopyFrom(exchange_user_md_entitelemnt_data);
-// }
+void AdminServiceUtils::set_user_md_entitlement_data(bts::api::admin::BTSUser* user,
+                                                     const bts::api::admin::ExchangeUserMarketDataEntitlementData& exchange_user_md_entitelemnt_data){
+   if (!user->has_user_data())
+      user->mutable_user_data();
+   if (!user->user_data().has_exchange_md_entitlement_data())
+      user->mutable_user_data()->mutable_exchange_md_entitlement_data()->CopyFrom(exchange_user_md_entitelemnt_data);
+}
 void AdminServiceUtils::clear_user_accounts (bts::api::admin::BTSUser* user){
    if(user->has_user_data() && user->user_data().has_account_permissions())
       user->mutable_user_data()->clear_account_permissions();
@@ -1553,7 +1377,6 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    auto account_1 = create_account(data, parent_account->id(), std::string("ACC A1"), std::string("description"), true, account_result); account_1->set_is_active(true);      
    auto account_2 = create_account(data, parent_account->id(), std::string("ACC A2"), std::string("description"), true, account_result); account_2->set_is_active(true);
    auto account_3 = create_account(data, parent_account->id(), std::string("ACC A3"), std::string("description"), true, account_result); account_3->set_is_active(true);
-   auto account_4 = create_account(data, parent_account->id(), std::string("BLUETST"), std::string("description"), true, account_result); account_4->set_is_active(true);
    create_account(data, parent_account->id(), std::string("ACC A4"), std::string("description"), true, account_result);
    // add group
    auto def_group = create_group(data, 0, std::string("default"), result);
@@ -1564,7 +1387,8 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    auto user_2 = create_user(data, def_group->id(), std::string("bts_2"), std::string("bts_2"), empty, empty,  false, license_data, license_entitlements, user_result); user_2->set_is_active(true);
    auto user_3 = create_user(data, def_group->id(), std::string("bts_3"), std::string("bts_3"), empty, empty, false, license_data, license_entitlements, user_result); user_3->set_is_active(true);
    //associate users
-   bts::api::admin::UserAccountPermissionUnit permission_unit;   
+   bts::api::admin::UserAccountPermissionUnit permission_unit;
+   permission_unit.mutable_trading_permission()->set_submit_market_orders(true);
    permission_unit.mutable_trading_permission()->set_has_permissions(true);
    permission_unit.mutable_admin_permission()->set_has_permissions(true);
    bts::api::admin::UserAccountPermissions account_permissions;
@@ -1572,7 +1396,6 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair<uint32_t, bts::api::admin::UserAccountPermissionUnit>(account_1->id(), permission_unit));
    account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair<uint32_t, bts::api::admin::UserAccountPermissionUnit>(account_2->id(), permission_unit));
    account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair<uint32_t, bts::api::admin::UserAccountPermissionUnit>(account_3->id(), permission_unit));      
-   account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair<uint32_t, bts::api::admin::UserAccountPermissionUnit>(account_4->id(), permission_unit));      
    set_user_accounts(user_1, account_permissions);
    set_user_accounts(user_2, account_permissions);
    set_user_accounts(user_3, account_permissions);
@@ -1582,12 +1405,6 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    exchange_account_data.mutable_cme_account_data()->set_cti(bts::api::orderman::CTICode::CTI_CODE_CTI_CODE1);
    exchange_account_data.mutable_cme_account_data()->set_origin(bts::api::orderman::CustomerOrFirm::CUSTOMER_OR_FIRM_FIRM);
    exchange_account_data.mutable_cme_account_data()->set_clearing_account(parent_account->name());
-   
-   exchange_account_data.mutable_ice_account_data()->set_cti(bts::api::orderman::CTICode::CTI_CODE_UNDEFINED);
-   exchange_account_data.mutable_ice_account_data()->set_origin(bts::api::orderman::CustomerOrFirm::CUSTOMER_OR_FIRM_FIRM);
-   exchange_account_data.mutable_ice_account_data()->set_clearing_account(parent_account->name());
-   exchange_account_data.mutable_ice_account_data()->set_clearing_firm("649");
-   
    exchange_account_data.mutable_sim_account_data()->set_cti(bts::api::orderman::CTICode::CTI_CODE_CTI_CODE1);
    exchange_account_data.mutable_sim_account_data()->set_origin(bts::api::orderman::CustomerOrFirm::CUSTOMER_OR_FIRM_FIRM);
    exchange_account_data.mutable_sim_account_data()->set_clearing_account(parent_account->name());
@@ -1595,87 +1412,33 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
 
 
    exchange_account_data.mutable_cme_account_data()->set_clearing_account(account_1->name());
-   exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_1->name());
    exchange_account_data.mutable_sim_account_data()->set_clearing_account(account_1->name());
    set_account_exchange_data(account_1, exchange_account_data);
 
    exchange_account_data.mutable_cme_account_data()->set_clearing_account(account_2->name());
-   exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_2->name());
    exchange_account_data.mutable_sim_account_data()->set_clearing_account(account_2->name());
    set_account_exchange_data(account_2, exchange_account_data);
    
    exchange_account_data.mutable_cme_account_data()->set_clearing_account(account_3->name());
-   exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_3->name());
    exchange_account_data.mutable_sim_account_data()->set_clearing_account(account_3->name());
    set_account_exchange_data(account_3, exchange_account_data);
 
-   exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_4->name());
-   set_account_exchange_data(account_4, exchange_account_data);
-   
-
    // add cme and sim user data
    bts::api::admin::ExchangeUserData exchange_user_data;
    exchange_user_data.mutable_cme_data()->set_operator_id("op_id_" + user_1->name());
    exchange_user_data.mutable_cme_data()->set_location("us,nc");
-   
-   exchange_user_data.mutable_ice_data()->set_operator_id("op_id_" + user_1->name());
-   exchange_user_data.mutable_ice_data()->set_company_id(51251);
-   exchange_user_data.mutable_ice_data()->set_authorized_trader("bluetrd-fx");
-   exchange_user_data.mutable_ice_data()->set_authorized_group("Group");
-   exchange_user_data.mutable_ice_data()->set_password("Starts123");
-   exchange_user_data.mutable_ice_data()->set_mifid_direct_electronic_access(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_trading_capacity(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_liquidity_provision(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_commodity_deriv_indicator(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_investment_decision(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_execution_decision(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_client_id_code(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_id(1);
-      
    exchange_user_data.mutable_sim_data()->set_operator_id("op_id_" + user_1->name());
    exchange_user_data.mutable_sim_data()->set_location("us,nc");
    set_user_exchange_data(user_1, exchange_user_data);
 
    exchange_user_data.mutable_cme_data()->set_operator_id("op_id_" + user_2->name());
    exchange_user_data.mutable_cme_data()->set_location("us,nc");
-   
-   exchange_user_data.mutable_ice_data()->set_operator_id("op_id_" + user_2->name());
-   exchange_user_data.mutable_ice_data()->set_company_id(51251);
-   exchange_user_data.mutable_ice_data()->set_authorized_trader("bluetrd-fx");
-   exchange_user_data.mutable_ice_data()->set_authorized_group("Group");
-   exchange_user_data.mutable_ice_data()->set_password("Starts123");
-   exchange_user_data.mutable_ice_data()->set_mifid_direct_electronic_access(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_trading_capacity(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_liquidity_provision(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_commodity_deriv_indicator(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_investment_decision(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_execution_decision(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_client_id_code(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_id(1);
-   
-   
    exchange_user_data.mutable_sim_data()->set_operator_id("op_id_" + user_2->name());
    exchange_user_data.mutable_sim_data()->set_location("us,nc");
    set_user_exchange_data(user_2, exchange_user_data);
 
    exchange_user_data.mutable_cme_data()->set_operator_id("op_id_" + user_3->name());
    exchange_user_data.mutable_cme_data()->set_location("us,nc");
-   
-   exchange_user_data.mutable_ice_data()->set_operator_id("op_id_" + user_3->name());
-   exchange_user_data.mutable_ice_data()->set_company_id(51251);
-   exchange_user_data.mutable_ice_data()->set_authorized_trader("bluetrd-fx");
-   exchange_user_data.mutable_ice_data()->set_authorized_group("Group");
-   exchange_user_data.mutable_ice_data()->set_password("Starts123");
-   exchange_user_data.mutable_ice_data()->set_mifid_direct_electronic_access(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_trading_capacity(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_liquidity_provision(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_commodity_deriv_indicator(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_investment_decision(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_execution_decision(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_client_id_code(-1);
-   exchange_user_data.mutable_ice_data()->set_mifid_id(1);
-   
-   
    exchange_user_data.mutable_sim_data()->set_operator_id("op_id_" + user_3->name());
    exchange_user_data.mutable_sim_data()->set_location("us,nc");
    set_user_exchange_data(user_3, exchange_user_data);
@@ -1717,13 +1480,6 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    add_security_clip_sizes_to_account(account_1, api::Venue::VENUE_GLBX, api::SecurityType::SECURITYTYPE_SPREAD,100);
    add_security_clip_sizes_to_account(account_1, api::Venue::VENUE_SIM, api::SecurityType::SECURITYTYPE_FUTURE,100);
    add_security_clip_sizes_to_account(account_1, api::Venue::VENUE_SIM, api::SecurityType::SECURITYTYPE_SPREAD,100);
-
-   //ICE limits   
-   add_product_position_limits(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_FUTURE,50000);
-   add_product_position_limits(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_SPREAD,50000);
-   add_security_clip_sizes_to_account(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_FUTURE,50);
-   add_security_clip_sizes_to_account(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_SPREAD,50);
-   
 }
 bool AdminServiceUtils::has_id (uint32_t id){return id>0;}
 bool AdminServiceUtils::is_symbol_set (const std::string& symbol){

src/lib/api/service_helpers/AdminServiceUtils.h

@@ -35,10 +35,6 @@ public:
                                                    const AdminData& data,
                                                    const BTSAccount& account,
                                                    const api::Security& sec);
-   static const ExchangeConnection* resolve_ice_connection(
-                                                   const AdminData& data,
-                                                   const BTSAccount& account,
-                                                   const api::Security& sec);                                                   
    static const ExchangeConnection* resolve_sim_connection(
                                                    const AdminData& data,
                                                    const BTSAccount& account,
@@ -168,8 +164,6 @@ public:
    // travels up the group tree until it finds a populated cme data 
    static const CMEUserData* get_group_cme_data(const BTSUserGroup& group,
                                                 const AdminData& data);
-   static const ICEUserData* get_group_ice_data(const BTSUserGroup& group,
-                                                const AdminData& data);                                                
    static const SIMUserData* get_group_sim_data(const BTSUserGroup& group, 
                                                 const AdminData& data);
    static const UserAccountPermissions* const get_group_user_permissions (
@@ -204,8 +198,6 @@ public:
    static uint32_t find_max_user_id (const AdminData& data);
    static const CMEUserData* get_user_cme_data(const BTSUser& user, 
                                                const AdminData& data);
-   static const ICEUserData* get_user_ice_data(const BTSUser& user, 
-                                               const AdminData& data);                                               
    static const SIMUserData* get_user_sim_data(const BTSUser& user, 
                                                const AdminData& data);
    // finds the "exchange user" (ex: "operator_id" at CME or "authorized_trader" 
@@ -215,12 +207,6 @@ public:
                                     const BTSUser& user,
                                     api::Venue exchange,
                                     AdminServiceOperationResult& out_result);
-   static std::string find_user_company_id (
-                                    const AdminData& data,
-                                    const BTSUser& user,
-                                    api::Venue exchange,
-                                    AdminServiceOperationResult& out_result);
-                                    
    static std::pair<const BTSAccount*, const BTSUser*>
    find_user_account_matches (const AdminData& admin_data,
                               const std::string& connection,
@@ -260,9 +246,6 @@ public:
    static MDEntitlementReportUnit get_cme_md_entitlement_unit(
                                              const BTSUser& user,
                                              const CMEMicData& cme_mic_data);
-   static MDEntitlementReportUnit get_ice_md_entitlement_unit(
-                                             const BTSUser& user,
-                                             const ICEMicData& ice_mic_data);                                             
    static void get_user_symbol_permissions(
                                  const AdminData& data,
                                  const BTSUser& user,
@@ -289,6 +272,8 @@ public:
    static void set_user_exchange_data(
                                  BTSUser* user,
                                  const ExchangeUserData& exchange_user_data);
+   static void set_user_md_entitlement_data(BTSUser* user,
+                                 const ExchangeUserMarketDataEntitlementData&);
    static void clear_user_accounts (BTSUser* user);
    static void set_user_accounts (BTSUser* user,
                      const UserAccountPermissions& user_account_permissions);

src/lib/api/services/AdminService.proto

@@ -55,12 +55,11 @@ message BTSUser{
    string name = 30;
    string password = 40;
    string full_name = 42;
-   string company = 44;
+   string company = 44;   
    bool is_active = 50;
    bool can_create = 55;
-   LicenseData license_data = 60;
-   LicenseEntitlements license_entitlements = 70;
-
+   LicenseData license_data = 60; //unused
+   LicenseEntitlements license_entitlements = 70; //unused
    BTSUserDerivableData user_data = 80;
 }
 
@@ -77,7 +76,7 @@ message BTSUserDerivableData{
    // b. else its overridden at the user level
    UserAccountPermissions account_permissions = 70;
    ExchangeUserData exchange_user_data = 80;
-   reserved 90; reserved "exchange_md_entitlement_data"; //ExchangeUserMarketDataEntitlementData exchange_md_entitlement_data = 90;
+   ExchangeUserMarketDataEntitlementData exchange_md_entitlement_data = 90;
 }
 
 message LicenseData{
@@ -92,22 +91,21 @@ message LicenseEntitlements{
    bool spreader = 30;
 }
 
-// message AlgoTradingPermissionsUnit{
-//    bool submit_iceberg_orders = 10; //unused
-//    bool submit_time_box_orders = 20; //unused
-//    bool submit_spreader_orders = 30; //unused
-// }
+message AlgoTradingPermissionsUnit{
+   bool submit_iceberg_orders = 10; //unused
+   bool submit_time_box_orders = 20; //unused
+   bool submit_spreader_orders = 30; //unused
+}
 
 message TradingPermissionUnit{
-   reserved 10; reserved "submit_market_orders"; //bool submit_market_orders = 10; //unused
-   reserved 20; reserved "submit_gtc_orders"; //bool submit_gtc_orders = 20; //unused
-   reserved 30; reserved "algo_trading_permission"; //AlgoTradingPermissionsUnit algo_trading_permission = 30; //unused
-
    bool has_permissions = 5;
+   bool submit_market_orders = 10; //unused
+   bool submit_gtc_orders = 20; //unused  
+   AlgoTradingPermissionsUnit algo_trading_permission = 30; //unused
 }
 
 message AdminPermissionUnit{
-   // TODO: needs to be more ellaborate (account writing,
+   // TODO: needs to be more ellaborate (account writing, 
    // reading etc, routing etc)
    bool has_permissions = 10;
    bool has_md_permissions = 20;
@@ -121,15 +119,15 @@ message UserAccountPermissionUnit{
 
 message UserAccountPermissions{
    //uint32 = account_id
-   map<uint32, UserAccountPermissionUnit> account_permissions = 10;
+   map<uint32, UserAccountPermissionUnit> account_permissions = 10;   
 }
 
-// // unused now
-// message UserSecurityTradingLimit{
-//    SecurityScope security = 10;
-//    uint32 clip_size = 20;
-//    bool can_trade = 30;
-// }
+// unused now
+message UserSecurityTradingLimit{
+   SecurityScope security = 10;
+   uint32 clip_size = 20;
+   bool can_trade = 30;   
+}
 
 /************************************************************************/
 /********************************BTS Account*****************************/
@@ -154,7 +152,7 @@ message BTSAccount{
    AccountLossLimit account_loss_limit = 130;
 }
 
-message RiskLimitEnforcement{
+message RiskLimitEnforcement{   
    bool product_limit_in_net_terms = 10;
    bool ignore_spread_position_limits = 20;
 }
@@ -206,22 +204,23 @@ message AccountLossLimit{
 message CMEMicData{
    api.CMEMics mic=10;
    bool enabled=20;
-   // there are other things here
+   // there are other things here   
 }
 
 message ICEMicData{
-   api.ICEMics mic=10;
+   api.CMEMics mic=10;
    bool enabled=20;
 }
 
+message ExchangeUserMarketDataEntitlementData{
+   CMEUserMarketDataEntitlementData cme_data = 10;
+   ICEUserMarketDataEntitlementData ice_data = 20;
+}
 
 message ExchangeUserData{
    CMEUserData cme_data = 10;
    ICEUserData ice_data = 20;
    SIMUserData sim_data = 30;
-
-   enum AdminDisplay { All = 0; MD = 1; Routing = 2; None = 100; }
-   AdminDisplay admin_display_flag = 120;
 }
 
 message ExchangeAccountData{
@@ -230,6 +229,9 @@ message ExchangeAccountData{
    SIMAccountData sim_account_data = 30;
 }
 
+message MarketDataEntitlement{
+   bool can_view = 10;
+}
 
 message ExchangeConnection{
    uint32 id = 10;
@@ -272,19 +274,19 @@ message CMEMSGWConnection{
    uint32 market_segment_id = 60;
 }
 
-message CMEAccountData {
-   reserved 30; reserved "takeup_member_code"; //string takeup_member_code = 30;
-   reserved 50; reserved "direct_electronic_access"; //string direct_electronic_access = 50;
-   reserved 60; reserved "trading_capacity"; //string trading_capacity = 60;
-   reserved 70; reserved "liquidity_provision"; //string liquidity_provision = 70;
-   reserved 80; reserved "commodity_derivative_indicator"; //string commodity_derivative_indicator = 80;
-   reserved 90; reserved "investment_decision_id"; //string investment_decision_id = 90;
-   reserved 100; reserved "execution_decision_id"; //string execution_decision_id = 100;
-   reserved 110; reserved "client_id"; //string client_id = 110;
-
+message CMEAccountData{
+   // should perhaps be an enum
    api.orderman.CTICode cti = 10;
    api.orderman.CustomerOrFirm origin = 20;
+   string takeup_member_code = 30;
    string clearing_account = 40;
+   string direct_electronic_access = 50;
+   string trading_capacity = 60;
+   string liquidity_provision = 70;
+   string commodity_derivative_indicator = 80;
+   string investment_decision_id = 90;
+   string execution_decision_id = 100;
+   string client_id = 110;
    map<string, string> misc_tags = 120;
 }
 
@@ -307,22 +309,21 @@ enum DisplayDesignation
 }
 
 message CMEUserData{
-   reserved 20; reserved "ats_operator_id"; //string ats_operator_id = 20;
-   reserved 40; reserved "text_a"; //string text_a = 40;
-   reserved 50; reserved "text_b"; //string text_b = 50;
-   reserved 80; reserved "max_order_rate_per_connection"; //string max_order_rate_per_connection = 80;
-   //enum AdminDisplay { All = 0; MD = 1; Routing = 2; None = 100; }
-   reserved 120; reserved "admin_display_flag"; //AdminDisplay admin_display_flag = 120;
-   repeated CMEMicData md_permissions = 1000;
-
    string operator_id = 10;
+   string ats_operator_id = 20;
    string smp_id = 30;
+   string text_a = 40;
+   string text_b = 50;
    map<string, string> misc_tags = 60;
    string smp_instruction = 70;
-
+   string max_order_rate_per_connection = 80;
    string location = 90;
    ProfessionalDesignation pro_nonpro = 100;
    DisplayDesignation display_nondisplay = 110;
+   
+   enum AdminDisplay { All = 0; MD = 1; Routing = 2; None = 100; } 
+   AdminDisplay admin_display_flag = 120; 
+   repeated CMEMicData md_permissions = 1000;
 }
 
 message SIMUserData {
@@ -330,67 +331,58 @@ message SIMUserData {
    string location = 90;
 }
 
+message CMEUserMarketDataEntitlementData{
+   MarketDataEntitlement cbot_entitlement = 10;
+   MarketDataEntitlement cme_entitlement = 20;
+   MarketDataEntitlement nymex_entitlement = 30;
+   MarketDataEntitlement bmandf_entitlement = 40;
+   MarketDataEntitlement comex_entitlement = 50;
+   MarketDataEntitlement cme_europe_entitlement = 60;
+}
+
 /********************************ICE Data*********************************/
 
-message ICEConnection {
-   string sender_compid = 10;
+message ICEConnection{
+   string authorized_member_id = 10;
+   string sender_company_id = 20;
+   string sender_sub_id = 30;
+   string session_id = 40;
+   string session_password = 50;
+   string host_ip = 60;
+   string port = 70;
+   string default_routing_trader = 80;
+   string routing_trader_password = 90;
+   int32 max_order_rate = 100;
 }
 
-message ICEAccountData {
-   reserved 10; reserved "nybit_house"; //string nybit_house = 10;
-   reserved 20; reserved "wce_house"; //int32 wce_house = 20;
-   reserved 30; reserved "clearing_account_override"; //string clearing_account_override = 30;
-   reserved 40; reserved "direct_electronic_access"; //bool direct_electronic_access = 40;
-   reserved 50; reserved "trading_capacity"; //string trading_capacity = 50;
-   reserved 60; reserved "liquidity_provision"; //bool liquidity_provision = 60;
-   reserved 70; reserved "commodity_derivative_indicator"; //bool commodity_derivative_indicator = 70;
-   reserved 80; reserved "investment_decision_id"; //string investment_decision_id = 80;
-   reserved 90; reserved "execution_decision_id"; //string execution_decision_id = 90;
-   reserved 100; reserved "client_id"; //string client_id = 100;
-   reserved 110; reserved "mfid_id"; //string mfid_id = 110;
-
-   api.orderman.CTICode cti = 3;
-   api.orderman.CustomerOrFirm origin = 5;
-   string clearing_firm = 200;
-   string clearing_account = 210;
-   string customer_account_ref_id = 220;
-   string account_code = 230;
-   map<string, string> misc_tags = 500;
+message ICEAccountData{
+   string nybit_house = 10;
+   int32 wce_house = 20;
+   string clearing_account_override = 30;
+   bool direct_electronic_access = 40;
+   string trading_capacity = 50;
+   bool liquidity_provision = 60;
+   bool commodity_derivative_indicator = 70;
+   string investment_decision_id = 80;
+   string execution_decision_id = 90;
+   string client_id = 100;
+   string mfid_id = 110;
 }
 
-message ICEUserData {
-   reserved 20; reserved "memo_field"; //string memo_field = 20;
-   reserved 30; reserved "text_b"; //string text_b = 30;
-   reserved 40; reserved "max_order_rate"; //uint32 max_order_rate = 40;
+message ICEUserData{
+   string authorized_trader = 10;
+   string memo_field = 20;
+   string text_b = 30;
+   uint32 max_order_rate = 40;
+}
 
-   int32 company_id = 3; //tag 109 clientid
-   string operator_id = 5; //tag 50
-   string authorized_trader = 10; //tag 553 user_name
-   string authorized_group = 15; //tag 553 user_name
-   string password = 21; //tag 96 raw data
-   string cust_order_handing_inst = 25;
-
-   //mifid
-   int32 mifid_direct_electronic_access = 500;
-   int32 mifid_trading_capacity = 510;
-   int32 mifid_liquidity_provision = 520;
-   int32 mifid_commodity_deriv_indicator = 530;
-   int64 mifid_investment_decision = 540;
-   int64 mifid_execution_decision = 550;
-   int64 mifid_client_id_code = 560;
-   int64 mifid_id = 570;
-
-   map<string, string> misc_tags = 60;
-   ProfessionalDesignation pro_nonpro = 100;
-   DisplayDesignation display_nondisplay = 110;
-
-   repeated ICEMicData md_permissions = 1000;
+message ICEUserMarketDataEntitlementData{
 }
 
 ///////////////////////////////////////////////////////////
 // Order Auxiliary Service ////////////////////////////////
 
-message OrdersSummaryUnit{
+message OrdersSummaryUnit{   
    string account = 10;
    string user = 20;
    uint32 total_orders = 30;
@@ -401,11 +393,11 @@ message OrdersSummary{
 }
 message OrderSummaryRequest{
 }
-// top level orders that have account
+// top level orders that have account 
 // and user that are present here, will
 // be cancelled. wild cards are not
 // supported.
-message OrderCancelRequest{
+message OrderCancelRequest{   
    repeated string accounts = 10;
    repeated string users = 20;
 }
@@ -441,7 +433,7 @@ message AccountCreateAPIArgument{
    string creator_user = 40;
 }
 message AccountUpdateAPIArgument{
-   BTSAccount complete_data=10;
+   BTSAccount complete_data=10;    
 }
 message UserCreateAPIArgument{
    uint32 parent_group_id = 20;
@@ -485,7 +477,7 @@ service AdminService {
    // users
    rpc create_user(UserCreateAPIArgument) returns (AdminServiceCreateEntityResult);
    rpc update_user(UserUpdateAPIArgument) returns (AdminServiceOperationResult);
-   rpc delete_user(IDAPIArgument) returns (AdminServiceOperationResult);
+   rpc delete_user(IDAPIArgument) returns (AdminServiceOperationResult);   
    // subscription
    rpc admin_data(bts.api.VoidMessage) returns (stream AdminData);
    rpc get_admin_data(bts.api.VoidMessage) returns (AdminData);

src/lib/api/services/AuthService.proto

@@ -57,7 +57,4 @@ service AuthService {
 
    rpc get_users(GetUsersRequest) returns (GetUsersResponse)
    {option (bts.api.method_description)="Returns list of valid users";}
-   
-   rpc clear_all_sessions(bts.api.VoidMessage) returns (bts.api.ResultMessage)
-   {option (bts.api.method_description)="Clears all reserved auth sessions";}   
 }

src/lib/api/services/OrderService.proto

@@ -310,13 +310,6 @@ message Fill {
     FILL_KIND_SOD = 2;
   }
   FillKind fill_kind = 10;
-  
-  enum FillStatus { 
-     FILL_STATUS_ACTIVE = 0;
-     FILL_STATUS_CORRECTED = 1;
-     FILL_STATUS_BUSTED = 2;
-  }
-  FillStatus fill_status = 20;
 }
 
 enum OrderSubType {
@@ -334,16 +327,6 @@ message RoutingRecord {
   string smp_id = 6 ;
   SelfMatchPreventionInstruction smp_instruction = 7;
   string location = 8;
-  
-  string sender_compid = 15;
-  string clearing_firm = 20;
-  string customer_account_ref_id = 25;
-  string account_code = 30;    
-  string company_id = 45;
-  string authorized_trader = 50;  
-  string authorized_group = 60;
-  string cust_order_handing_inst = 70;
-  string order_server_id = 200;
 }
 
 message NativeOrderState {

src/lib/config/ConfigurationState.cpp

@@ -24,7 +24,6 @@
 #include <map>
 #include <tuple>
 
-
 #define URDL_HEADER_ONLY 1
 #define URDL_DISALE_SSL 1
 #include <urdl/url.hpp>
@@ -909,4 +908,3 @@ std::string get_fallback(const proto::Message& message)
 }
 
 }}}//end namespaces
-

src/lib/config/defaults/MarketData.conf

@@ -62,44 +62,15 @@ ICEOptions {
    value {
       channel {
          name: "oil" #these names are arbitrary and are for debugging
-         market_type: 3 
-         uds_market_type: 3
-         market_type: 4 #ICE IPE Gas Oil Futures
-         uds_market_type: 4         
          market_type: 5 #ICE Futures Brent, IPE Brent Futures
-         uds_market_type: 5
-         market_type: 9 #ICE Futures WTI, ICE WTI Crude Futures                  
-         uds_market_type: 9
-         market_type: 13 #ICE Heating Oil Futures         
-         uds_market_type: 13
-         market_type: 15 #ICE NYH (RBOB) Gasoline Futures
-         uds_market_type: 15
-         ##market_type: 21 #ICE Futures US Agricultures, Coffee C
-         ##uds_market_type: 21
-         ##market_type: 23 #ICE Futures US Agricultures, FCOJ A
-         ##uds_market_type: 23               
-         
-         ##market_type: 26 #ICE Futures US Financials DX
-         ##uds_market_type: 26
-         
-         uds_market_type: 11 #ICE WTI-Brent spreads
-         uds_market_type: 14 #ICE Heating Oil/WTI Futures Crack
-         uds_market_type: 16 #ICE NYH (RBOB) Gasoline/WTI Futures Crack         
-         uds_market_type: 58 #ICE Futures UI Heating Oil / Brent
-         uds_market_type: 59 #ICE NYH (RBOB) Gasoline/Brent Futures Crack 
-         uds_market_type: 66 #ICE NYH (RBOB) Gasoline/Heating Oil Spread
-         uds_market_type: 70 #ICE Heating Oil/LS Gasoil Futures Spread
-         uds_market_type: 71 #ICE LS Gasoil/Brent Futures Crack
-         uds_market_type: 72 #ICE NYH (RBOB) Gasoline/LS Gasoil Futures Spread         
-         
-         #onixs_log_mode: Debug
+         market_type: 9 #ICE Futures WTI, ICE WTI Crude Futures
+         #udp_buffer_size: 33554432 #32mb
+      }
+      channel {
+         name: "juice"
+         market_type: 23 #ICE Futures US Agricultures, FCOJ A
       }
-      # channel {
-      #    name: "juice"
-      #    market_type: 26 #ICE Futures US Agricultures, FCOJ A
-      # }
       ice_config_file: "ConnectivityConfiguration.APITest.xml"      
-      #ice_config_file: "ConnectivityConfiguration.PerfTest.xml"      
    }
 }
 

src/lib/config/defaults/Ports.conf

@@ -3,8 +3,7 @@ Ports {
    value {
       config_port: 8000
       qbus_port: 8001
-      glbx_sim_fix_port: 8002
-      ice_sim_fix_port: 8003
+      exchange_sim_fix_port: 8002
       replicator_port: 8080
    }
 }

src/lib/config/defaults/Symbols.conf

@@ -19,6 +19,7 @@ SymbolList {
         symbol: "FOB"
         symbol: "GC"
         symbol: "GE"
+        symbol: "GLI"
         symbol: "HE"
         symbol: "HG"
         symbol: "HO"
@@ -65,16 +66,9 @@ SymbolLists
       list
       {
          key: "ICE"
-         symbol: "BRN"
-         symbol: "WBS"
-         symbol: "OJ"
-         symbol: "G"
-         symbol: "KC"
-         symbol: "UHO"
-         symbol: "UHU"
-         symbol: "DX"
-         
-         symbol: "*"
+         # symbol: "BRN"
+         # symbol: "WBS"
+         # symbol: "OJ"
       }
       list
       {
@@ -118,9 +112,5 @@ Symbols {
             future: @SymbolLists.list[ICE].symbol
             spread: @SymbolLists.list[ICE].symbol
         }
-        simice {
-            future: @SymbolLists.list[ICE].symbol
-            spread: @SymbolLists.list[ICE].symbol            
-        }
     }
 }

src/lib/config/defaults/jetfix.conf

@@ -5,7 +5,7 @@ JetFIXSimConnection {
         name: "sim"
         platform: SIM
         socket_connect_host_ip: "127.0.0.1"
-        socket_connect_port: @Ports.glbx_sim_fix_port
+        socket_connect_port: @Ports.exchange_sim_fix_port
         sender_compid: "SPARK" # this is the FIX session name
         sender_subid: "BTS"
         target_compid: "BTS"
@@ -18,173 +18,9 @@ JetFIXSimConnection {
         seconds_to_pause : 1
 	max_connection_attempts: 9999999
       }
-      sim_ice {
-        name: "sim_ice"
-        platform: SIMICE
-        socket_connect_host_ip: "127.0.0.1"
-        socket_connect_port: @Ports.ice_sim_fix_port
-        sender_compid: "51239" # this is the FIX session name
-        sender_subid: "BTS"
-        target_compid: "BTS"
-        target_subid: "BTS"
-        cme_market_segment_id: 0
-        end_time: "02:00:00"
-        start_time: "02:05:00"
-        raw_data: "1JE9X"
-        use_cme_secure_logon: false 
-        seconds_to_pause : 1
-      }      
    }
 }
 
-JetFIX {
-   rule: "CannedCertSession.name='ice_cert'"
-   value {
-      #connection: @JetFIXSimConnection.sim_ice
-      connection {
-         name: "ice_cert"
-         platform: ICE
-         use_cme_secure_logon: false
-         log_fix_stream: true
-         socket_connect_host_ip: "63.247.113.154"
-         socket_connect_port: 80
-         sender_compid: "51251"
-         target_compid: "ICE"         
-         start_time: "22:55:00"
-         end_time: "22:05:00"
-         sender_subid: "BTS"
-         user_name: "bluetrd-os" #this is the order_server_id
-         raw_data: "Starts123"
-         max_order_msgs_per_window: 300
-         auto_fix_seq_num: true
-         
-         seconds_to_replay: 6000
-      }
-      connection: @JetFIXSimConnection.sim
-      #connection: @JetFIXSimConnection.sim_ice
-      
-      connection {
-         name: "cme_mc5_99"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.199"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "99"
-         cme_market_segment_id: 99
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_58"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.172"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "58"
-         cme_market_segment_id: 58
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_54"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.191"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "54"
-         cme_market_segment_id: 54
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_82"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.179"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "82"
-         cme_market_segment_id: 82
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_56"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.174"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "56"
-         cme_market_segment_id: 56
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_50"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.202"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "50"
-         cme_market_segment_id: 50
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_52"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.181"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "52"
-         cme_market_segment_id: 52
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }      
-   }
-}   
-
 JetFIX {
    rule: "CannedCertSession.name='mc7'"
    value {
@@ -620,193 +456,193 @@ JetFIX {
          raw_data: "S4PMA"
          auto_fix_seq_num: true
       }
-      # connection {
-      #    name: "cme_mc5_60"
-      #    platform: GLBX
-      #    socket_connect_host_ip: "69.50.112.195"
-      #    socket_connect_port: 61922
-      #    sender_compid: "MC5004N"
-      #    target_compid: "CME"
-      #    target_subid: "60"
-      #    cme_market_segment_id: 60
-      #    start_time: "00:30:00"
-      #    end_time: "23:30:00"
-      #    start_day: "sunday"
-      #    end_day: "friday"
-      #    sender_subid: "BTS"
-      #    raw_data: "S4PMA"
-      #    auto_fix_seq_num: true
-      # }
-      # connection {
-      #    name: "cme_mc5_76"
-      #    platform: GLBX
-      #    socket_connect_host_ip: "69.50.112.204"
-      #    socket_connect_port: 61922
-      #    sender_compid: "MC5004N"
-      #    target_compid: "CME"
-      #    target_subid: "76"
-      #    cme_market_segment_id: 76
-      #    start_time: "00:30:00"
-      #    end_time: "23:30:00"
-      #    start_day: "sunday"
-      #    end_day: "friday"
-      #    sender_subid: "BTS"
-      #    raw_data: "S4PMA"
-      #    auto_fix_seq_num: true
-      # }
-      # connection {
-      #    name: "cme_mc5_88"
-      #    platform: GLBX
-      #    socket_connect_host_ip: "69.50.112.169"
-      #    socket_connect_port: 61922
-      #    sender_compid: "MC5004N"
-      #    target_compid: "CME"
-      #    target_subid: "88"
-      #    cme_market_segment_id: 88
-      #    start_time: "00:30:00"
-      #    end_time: "23:30:00"
-      #    start_day: "sunday"
-      #    end_day: "friday"
-      #    sender_subid: "BTS"
-      #    raw_data: "S4PMA"
-      #    auto_fix_seq_num: true
-      # }
-      # connection {
-      #    name: "cme_mc5_74"
-      #    platform: GLBX
-      #    socket_connect_host_ip: "69.50.112.197"
-      #    socket_connect_port: 61922
-      #    sender_compid: "MC5004N"
-      #    target_compid: "CME"
-      #    target_subid: "74"
-      #    cme_market_segment_id: 74
-      #    start_time: "00:30:00"
-      #    end_time: "23:30:00"
-      #    start_day: "sunday"
-      #    end_day: "friday"
-      #    sender_subid: "BTS"
-      #    raw_data: "S4PMA"
-      #    auto_fix_seq_num: true
-      # }
-      # connection {
-      #    name: "cme_mc5_64"
-      #    platform: GLBX
-      #    socket_connect_host_ip: "69.50.112.177"
-      #    socket_connect_port: 61922
-      #    sender_compid: "MC5004N"
-      #    target_compid: "CME"
-      #    target_subid: "64"
-      #    cme_market_segment_id: 64
-      #    start_time: "00:30:00"
-      #    end_time: "23:30:00"
-      #    start_day: "sunday"
-      #    end_day: "friday"
-      #    sender_subid: "BTS"
-      #    raw_data: "S4PMA"
-      #    auto_fix_seq_num: true
-      # }
-      # connection {
-      #    name: "cme_mc5_72"
-      #    platform: GLBX
-      #    socket_connect_host_ip: "69.50.112.189"
-      #    socket_connect_port: 61922
-      #    sender_compid: "MC5004N"
-      #    target_compid: "CME"
-      #    target_subid: "72"
-      #    cme_market_segment_id: 72
-      #    start_time: "00:30:00"
-      #    end_time: "23:30:00"
-      #    start_day: "sunday"
-      #    end_day: "friday"
-      #    sender_subid: "BTS"
-      #    raw_data: "S4PMA"
-      #    auto_fix_seq_num: true
-      # }
-      # connection {
-      #    name: "cme_mc5_70"
-      #    platform: GLBX
-      #    socket_connect_host_ip: "69.50.112.183"
-      #    socket_connect_port: 61922
-      #    sender_compid: "MC5004N"
-      #    target_compid: "CME"
-      #    target_subid: "70"
-      #    cme_market_segment_id: 70
-      #    start_time: "00:30:00"
-      #    end_time: "23:30:00"
-      #    start_day: "sunday"
-      #    end_day: "friday"
-      #    sender_subid: "BTS"
-      #    raw_data: "S4PMA"
-      #    auto_fix_seq_num: true
-      # }
-      # connection {
-      #    name: "cme_mc5_68"
-      #    platform: GLBX
-      #    socket_connect_host_ip: "69.50.112.168"
-      #    socket_connect_port: 61922
-      #    sender_compid: "MC5004N"
-      #    target_compid: "CME"
-      #    target_subid: "68"
-      #    cme_market_segment_id: 68
-      #    start_time: "00:30:00"
-      #    end_time: "23:30:00"
-      #    start_day: "sunday"
-      #    end_day: "friday"
-      #    sender_subid: "BTS"
-      #    raw_data: "S4PMA"
-      #    auto_fix_seq_num: true
-      # }
-      # connection {
-      #    name: "cme_mc5_80"
-      #    platform: GLBX
-      #    socket_connect_host_ip: "69.50.112.187"
-      #    socket_connect_port: 61922
-      #    sender_compid: "MC5004N"
-      #    target_compid: "CME"
-      #    target_subid: "80"
-      #    cme_market_segment_id: 80
-      #    start_time: "00:30:00"
-      #    end_time: "23:30:00"
-      #    start_day: "sunday"
-      #    end_day: "friday"
-      #    sender_subid: "BTS"
-      #    raw_data: "S4PMA"
-      #    auto_fix_seq_num: true
-      # }
-      # connection {
-      #    name: "cme_mc5_84"
-      #    platform: GLBX
-      #    socket_connect_host_ip: "69.50.112.198"
-      #    socket_connect_port: 61922
-      #    sender_compid: "MC5004N"
-      #    target_compid: "CME"
-      #    target_subid: "84"
-      #    cme_market_segment_id: 84
-      #    start_time: "00:30:00"
-      #    end_time: "23:30:00"
-      #    start_day: "sunday"
-      #    end_day: "friday"
-      #    sender_subid: "BTS"
-      #    raw_data: "S4PMA"
-      #    auto_fix_seq_num: true
-      # }
-      # connection {
-      #    name: "cme_mc5_78"
-      #    platform: GLBX
-      #    socket_connect_host_ip: "69.50.112.167"
-      #    socket_connect_port: 61922
-      #    sender_compid: "MC5004N"
-      #    target_compid: "CME"
-      #    target_subid: "78"
-      #    cme_market_segment_id: 78
-      #    start_time: "00:30:00"
-      #    end_time: "23:30:00"
-      #    start_day: "sunday"
-      #    end_day: "friday"
-      #    sender_subid: "BTS"
-      #    raw_data: "S4PMA"
-      #    auto_fix_seq_num: true
-      # }
+      connection {
+         name: "cme_mc5_60"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.195"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "60"
+         cme_market_segment_id: 60
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_76"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.204"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "76"
+         cme_market_segment_id: 76
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_88"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.169"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "88"
+         cme_market_segment_id: 88
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_74"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.197"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "74"
+         cme_market_segment_id: 74
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_64"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.177"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "64"
+         cme_market_segment_id: 64
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_72"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.189"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "72"
+         cme_market_segment_id: 72
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_70"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.183"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "70"
+         cme_market_segment_id: 70
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_68"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.168"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "68"
+         cme_market_segment_id: 68
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_80"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.187"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "80"
+         cme_market_segment_id: 80
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_84"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.198"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "84"
+         cme_market_segment_id: 84
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_78"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.167"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "78"
+         cme_market_segment_id: 78
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
    }
 }
 JetFIX {

src/lib/config/defaults/keycode.conf

@@ -1,9 +1,13 @@
+
+
 #nusers: 1
 #allow concurrent : true
 #expires: 2019-Dec-31 00:00:00   637133472000000000
-#generation_time: 2019-Sep-16 17:50:57.225054   5248728548999638444
+#generation_time: 2019-Sep-30 15:52:10.434845   5248740573731736354
 EncryptedKeyCode {
    value {
-      keycode: "E7sUCxYuveyuvvBfwMdTrUCpFROJWQvsy4pYjwZ+zonjHwEmDkjxbxaqUEKqMzBbAq3YFnxIGOrQpRJte71SEoG5FGy4ZoRrVOjrMdNVLmieUWtZU43FKGJzImj/tJKEXu2xgwYJRZTj/AJ8C30PuVnzm0qUmR+UCgCOxKVeK0OFlo6zSqyDutcy5kLW3va+nTuFp43aknm6Iiu8LFW16wa6QDDCSQd4QgnHmiz5wyMC90yGBE4qObTKRGKDl5mWsjEL16aWK2ZO/0/ebFgGG8WygQfkeFzJS7B4q5Ta0nkSkVyLZjq1LN2/g9AXeCzCJUX5/F96kdfsnfSgAJepeg=="
+      keycode: "fVZlub92ulkXME+uZSyO9+WmyAXYKD735Qzxxfdj4G+SqEDTuwfNxOZVPS3TCFQqWPDmgjKTbbvfqIrGe3icxkVObUcUoS7lBXRXFrSvzOfs9eTz9RqHRvquDidnOzKxmSXsxV2bMb21UqBCJFeTzb4Q6Agp8EtjXJRc57UMVVygvdXg528S3ocBIq1V4W5NYraFn5WpqrTuBhpYbnsZAHi77s/EEwUT+6la3J2OHPtbiQnB6JFb0zsExMuzE4gwp79ojuQ5pnV2DXBZFNwxYvFyBUU/ac6oU89saZ/ay1DbOjz76TNqnNFewZvzdXZl3y6+X1eblX2streH2pn7gg=="
    }
-}
+}
+
+

src/lib/config/defaults/orders_fills_cleanup.conf

@@ -2,14 +2,4 @@ OrdersSoftCleanupCutoff {
    value {
       extra_days: 1
    }
-}
-
-SODTime 
-{
-   rule: "DeploymentType.type == 'dev'"
-   value
-   {
-      sod_hh: 20
-      sod_mm: 30
-   }
 }

src/lib/config/protos/MarketData.proto

@@ -155,7 +155,6 @@ message ICEOptions
    message ICEChannel {
       required string name = 1;
       repeated uint32 market_type = 2;
-      repeated uint32 uds_market_type = 7;
       optional uint32 udp_buffer_size = 3 [default = 4194304]; //4mb      
       optional uint32 heartbeat_interval = 4 [default = 36000];
       optional uint32 sequence_gap_margin = 5 [default = 10000];

src/lib/config/protos/Ports.proto

@@ -5,7 +5,6 @@ message Ports {
   // see ports.conf for actual defaults
   optional int32 config_port = 1 ;
   optional int32 qbus_port = 2 ;
-  optional int32 glbx_sim_fix_port =  3 ;
-  optional int32 ice_sim_fix_port =  5 ;
+  optional int32 exchange_sim_fix_port =  3 ;
   optional int32 replicator_port = 4 ;
 }

src/lib/config/protos/Symbols.proto

@@ -20,5 +20,4 @@ message Symbols {
   optional SecurityExchangeList glbx = 10;
   optional SecurityExchangeList sim = 20;
   optional SecurityExchangeList ice = 30;
-  optional SecurityExchangeList simice = 40;
 }

src/lib/config/protos/jetfix.proto

@@ -7,11 +7,9 @@ message JetFIX {
     required string name = 10;
 
     enum JetFIXPlatform {
-	   GLBX = 1;
-	   CBOE = 6;
-      SIM = 2;
-      ICE = 3;
-      SIMICE = 4;
+	GLBX = 1;
+	CBOE = 6;
+        SIM = 2;
     }
     optional JetFIXPlatform platform = 15                       [default = GLBX];
     
@@ -49,7 +47,6 @@ message JetFIX {
     optional string target_subid = 130				[default = ""];
     optional string sender_locationid = 135			[default = "US,NC"];
     optional string raw_data = 136				[default =""];
-    optional string user_name = 138 				[default =""];
     optional string self_match_prevention_id = 140		[default=""];
     optional string cti_code = 145				[default="1"];
     optional int32  customer_or_firm = 147		        [default=1];
@@ -94,7 +91,6 @@ message JetFIX {
 
 message JetFIXSimConnection {
   required JetFIX.Connection sim = 10;
-  required JetFIX.Connection sim_ice = 20;
 }
 
 message CannedCertSession {

src/lib/config/protos/toggles.proto

@@ -58,3 +58,7 @@ message EnsureOnScreenWithWorkAreaRect {
    // Should verify that the code works with every monitor arrangement before enabling by default.
    optional bool enabled = 1 [default = false];
 }
+
+message IgnoreNansInDayPnL {
+   optional bool enabled = 1 [default = true];
+}

src/lib/domain_model/Security.cpp

@@ -29,10 +29,9 @@ std::string descriptor(api::Security const &sec) {
    else if (sec.has_ice())
       return sec.ice().security_desc();
    else
-      return "No descriptor for non glbx / ice exchange";
+      return "No descriptor for non glbx exchange";
 }
 
-
 boost::gregorian::date smallest_leg(
     ::google::protobuf::RepeatedPtrField<::bts::api::ContractLeg> const &l) {
    bts_assert(l.size());

src/lib/domain_model/Security.h

@@ -191,6 +191,5 @@ inline bool gr_than(double p1, double p2, double tick_size) {
    return gr_than_eq(p1, p2, tick_size) &&
           !test_price_equality(p1, p2, tick_size);
 }
-
-
-}}
+} // namespace securities
+} // namespace bts

src/lib/domain_model/Security.proto

@@ -12,7 +12,6 @@ enum Venue {
      VENUE_CBOECFE = 207;
      VENUE_GLBX = 301;
      VENUE_ICE = 306;
-     VENUE_SIMICE = 3;
 }
 
 enum SecurityType {
@@ -55,12 +54,11 @@ enum CMEMics {
 }
 
 enum ICEMics{
-   IF_UNSET = 0;
-   IFU = 1;
-   IFE = 2;
-   IFL = 3;
-   IFC = 4;
-   IFEN = 5;
+   ICE_UNSET=0;
+   ICE_US = 1;
+   ICE_EU = 2;
+   ICE_LIFFE = 3;
+   ICE_CA = 4;
 }
 
 enum Side {
@@ -105,9 +103,8 @@ message CBOECFEExchangeData {
 message ICEExchangeData {
    int64 exchange_id = 10;
    string security_desc = 15; //contractSymbol
-   int32 price_denominator = 17;
-   int32 deal_denominator = 20;
-   int32 settle_denominator = 30;
+   int32 deal_display_factor = 20;
+   int32 settle_display_factor = 30;
    
    string market_description = 100;
    string cleared_alias = 110;

src/lib/jetfix/codec/FIXEncodingStream.h

@@ -1,11 +1,11 @@
 #pragma once
 
-#include "lib/jetfix/codec/FIXBuffer.h"
-#include "lib/jetfix/codec/FIXFragment.h"
-#include "lib/jetfix/codec/FieldDefinitions.h"
-#include "lib/jetfix/codec/Fields.h"
-#include "lib/jetfix/codec/MessageTypes.h"
 #include <memory>
+#include "lib/jetfix/codec/FIXBuffer.h"
+#include "lib/jetfix/codec/MessageTypes.h"
+#include "lib/jetfix/codec/FIXFragment.h"
+#include "lib/jetfix/codec/Fields.h"
+#include "lib/jetfix/codec/FieldDefinitions.h"
 
 #include "lib/config/protos/jetfix.pb.h"
 
@@ -14,186 +14,185 @@ namespace jetfix {
 
 // Helper to batch the encoding of multiple FIX messages all sharing
 // the same FIX headers.
-class FIXEncodingStream {
- public:
+class FIXEncodingStream
+{
+public:
    FIXEncodingStream(config::JetFIX::Connection const &cfg,
-                     uint32_t initial_capacity = 1024 * 10)
-       : _msg_header_template(), _header_template_size(),
-         _header_template_checksum(), _buffer(initial_capacity), _message_map(),
-         _current_msg(_buffer), _state(State::OPEN), _seq_num_bm(),
-         _last_seq_num_bm(), _sending_time_bm() {
-      _message_map.reserve(10 * 1024);
+                     uint32_t initial_capacity = 1024*10)
+      :
+      _msg_header_template (),
+      _header_template_size (),
+      _header_template_checksum (),
+      _buffer (initial_capacity),
+      _message_map (),
+      _current_msg (_buffer),
+      _state (State::OPEN),
+      _body_length_bm (),
+      _msg_type_bm (),
+      _seq_num_bm (),
+      _last_seq_num_bm (),
+      _sending_time_bm ()
+   {
+      _message_map.reserve (10*1024);
 
-      FIXBuffer header_buffer(&_msg_header_template[0],
-                              _msg_header_template.size());
-      FIXFragment header(header_buffer);
+      FIXBuffer header_buffer (&_msg_header_template[0],_msg_header_template.size ());
+      FIXFragment header (header_buffer);
+      header.add<fix::BeginString>("FIX.4.2");
 
-      // We cannot include body lenght and msg type in the header
-      // because message type is not fixed lenght, and therefore its
-      // not possible to have bookmarks to fields after it, ahead of
-      // knowing the message type value. The rest of the header is
-      // fixed lenght so we form it here.
-      _seq_num_bm = header.add<fix::MsgSeqNum>(0, 9);
-      _last_seq_num_bm = header.add<fix::LastSeqNumProcessed>(0, 9);
+      _body_length_bm = header.add<fix::BodyLength>(0,6);
+      _msg_type_bm = header.add<fix::MsgType>(" ",1);
+      _seq_num_bm = header.add<fix::MsgSeqNum>(0,9);
+      _last_seq_num_bm = header.add<fix::LastSeqNumProcessed>(0,9);
       _sending_time_bm = header.add<SendingTimeStr>(
-          "20150427-19:05:18.923", sizeof("20150427-19:05:18.923") - 1);
+          "20150427-19:05:18.923",sizeof("20150427-19:05:18.923")-1);
 
-      header.add<fix::SenderCompID>(cfg.sender_compid());
-      header.add<fix::TargetCompId>(cfg.target_compid());
-      header.add<fix::TargetSubID>(cfg.target_subid());
-      _header_template_size = header.size();
-      _header_template_checksum = header.checksum();
+      header.add<fix::SenderCompID>(cfg.sender_compid ());
+      header.add<fix::TargetCompId>(cfg.target_compid ());
+      header.add<fix::TargetSubID>(cfg.target_subid ());
+      _header_template_size = header.size ();
+      _header_template_checksum = header.checksum ();
    }
 
-   inline FIXFragment &new_msg(MessageTypes type) {
-      bts_assert(_state == State::OPEN);
-      tidy_last_msg();
-
-      auto msg_offset = _buffer.size();
-
-      _current_msg = FIXFragment(_buffer);
-      _begin_string_bm = _current_msg.add<fix::BeginString>("FIX.4.2");
-      _body_length_bm = _current_msg.add<fix::BodyLength>(0, 6);
-
-      char mt_encoding[4];
-      auto start = encode_message_type(type, mt_encoding);
-      _msg_type_bm = _current_msg.add<fix::MsgType>(
-          mt_encoding + start, sizeof(mt_encoding) - start);
-
-      auto header_offset = _buffer.size();
-      _message_map.emplace_back(msg_offset, header_offset, 0u, 0u, 0u,
-                                FIXFragment::Bookmark<IntField<10u>>());
-      _current_msg.append(&_msg_header_template[0], _header_template_size,
-                          _header_template_checksum);
+   inline FIXFragment &new_msg (MessageTypes type)
+   {
+      bts_assert (_state == State::OPEN);
+      tidy_last_msg ();
+      
+      auto msg_offset = _buffer.size ();
+      _message_map.emplace_back (msg_offset,0u,0u,0u,FIXFragment::Bookmark<IntField<10u>>());
+      _current_msg = FIXFragment (_buffer);
+      _current_msg.append (&_msg_header_template[0],
+                           _header_template_size,
+                           _header_template_checksum);
+      char msg_type_str[1] = {(char) type};
+      _current_msg.overwrite (_msg_type_bm.shift (msg_offset),msg_type_str,1);
       return _current_msg;
    }
-
-   inline void close() {
-      bts_assert(_state == State::OPEN);
-      tidy_last_msg();
+   
+   inline void close ()
+   {
+      bts_assert (_state == State::OPEN);
+      tidy_last_msg ();
       _state = State::CLOSED;
    }
 
-   inline void reopen_empty() {
-      bts_assert(_state == State::CLOSED);
-      bts_assert(_message_map.empty());
+   inline void reopen_empty ()
+   {
+      bts_assert (_state == State::CLOSED);
+      bts_assert (_message_map.empty ());
       _state = State::OPEN;
    }
+   
+   inline uint32_t size () const
+   {return _buffer.size ();}
 
-   inline uint32_t size() const { return _buffer.size(); }
+   inline const char* data () const
+   {return _buffer.data ();}
 
-   inline const char *data() const { return _buffer.data(); }
-
-   struct MessageDesc {
-      MessageDesc(uint32_t offset, uint32_t header_offset, uint32_t size,
-                  uint32_t checksum, uint32_t seq_num,
-                  FIXFragment::Bookmark<fix::Checksum> checksum_bm)
-          : offset(offset), header_offset(header_offset), size(size),
-            checksum(checksum), seq_num(seq_num), checksum_bm(checksum_bm) {}
+   struct MessageDesc
+   {
+      MessageDesc (uint32_t offset,
+                   uint32_t size,
+                   uint32_t checksum,
+                   uint32_t seq_num,
+                   FIXFragment::Bookmark<fix::Checksum> checksum_bm)
+         : offset (offset),
+           size (size),
+           checksum (checksum),
+           seq_num (seq_num),
+           checksum_bm (checksum_bm)
+      {}
+      
 
       uint32_t offset;
-      uint32_t header_offset;
       uint32_t size;
       uint32_t checksum;
       uint32_t seq_num;
       FIXFragment::Bookmark<fix::Checksum> checksum_bm;
    };
-
-   std::string to_str() const {
+   
+   std::string to_str () const 
+   {
       std::string result;
-
-      if (size()) {
+      
+      if (size ())  {
          // return std::string(data(),size() is simpler but valgrind
          // reports a warning in that call
-         result.resize(size());
-         std::copy(data(), data() + size(), result.begin());
+         result.resize (size ());
+         std::copy (data (), data ()+size(),result.begin ());
       }
       return result;
    }
 
-   std::string to_human_str() const {
-      auto s = to_str();
-      std::replace(s.begin(), s.end(), '\001', '|');
+   std::string to_human_str () const 
+   {
+      auto s = to_str ();
+      std::replace( s.begin(), s.end(), '\001', '|' );
       return s;
    }
-
-   inline uint32_t stamp(uint32_t seq_num, uint32_t last_processed,
-                         std::string const &time_str) {
-      bts_assert(_state == State::CLOSED);
+   
+   inline uint32_t stamp (uint32_t seq_num,uint32_t last_processed, std::string const&time_str)
+   {
+      bts_assert (_state == State::CLOSED);
       _state = State::STAMPED;
 
       for (auto &m : _message_map) {
          ++seq_num;
          m.seq_num = seq_num;
 
-         FIXFragment frag(_buffer, m.checksum);
-         frag.overwrite(_seq_num_bm.shift(m.header_offset), m.seq_num, 9);
-         frag.overwrite(_last_seq_num_bm.shift(m.header_offset), last_processed,
-                        9);
-         frag.overwrite(_sending_time_bm.shift(m.header_offset), time_str);
-         frag.overwrite_checksum(m.checksum_bm, frag.checksum() % 256);
+         FIXFragment frag (_buffer,m.checksum);
+         frag.overwrite (_seq_num_bm.shift (m.offset),m.seq_num,9);
+         frag.overwrite (_last_seq_num_bm.shift (m.offset),last_processed,9);
+         frag.overwrite (_sending_time_bm.shift (m.offset),time_str);
+         frag.overwrite_checksum (m.checksum_bm,frag.checksum () % 256);
       }
       return seq_num;
    }
 
-   inline void reset() {
-      _buffer.reset();
-      _message_map.clear();
+   inline void reset ()
+   {
+      _buffer.reset ();
+      _message_map.clear ();
       _state = State::OPEN;
    }
 
-   inline bool is_closed() const { return _state == State::CLOSED; }
+   inline bool is_closed () const 
+   {return _state == State::CLOSED;}
 
-   size_t n_msgs() const { return _message_map.size(); }
-
-   std::vector<MessageDesc> const &msg_map() const { return _message_map; }
-
- private:
-   std::array<char, 256> _msg_header_template;
+   size_t n_msgs () const {return _message_map.size ();}
+   
+   std::vector<MessageDesc> const &msg_map () const {return _message_map;}
+private:
+   std::array<char,256> _msg_header_template;
    uint32_t _header_template_size;
    uint32_t _header_template_checksum;
    FIXBuffer _buffer;
    std::vector<MessageDesc> _message_map;
    FIXFragment _current_msg;
-   enum class State { OPEN, CLOSED, STAMPED };
+   enum class State { OPEN,CLOSED,STAMPED};
    State _state;
-
-   // These bookmarks point to the current msg (not the header)
-   FIXFragment::Bookmark<fix::BeginString> _begin_string_bm;
+   
    FIXFragment::Bookmark<fix::BodyLength> _body_length_bm;
    FIXFragment::Bookmark<fix::MsgType> _msg_type_bm;
-
-   // These bookmarks point to the header and thus have to be shifted
-   // by the offset of the end of the message type.
    FIXFragment::Bookmark<fix::MsgSeqNum> _seq_num_bm;
    FIXFragment::Bookmark<fix::LastSeqNumProcessed> _last_seq_num_bm;
    typedef StringField<fix::SendingTime::tag> SendingTimeStr;
    FIXFragment::Bookmark<SendingTimeStr> _sending_time_bm;
 
-   inline void tidy_last_msg() {
-      if (_message_map.size()) {
-         auto &this_msg_map = _message_map.back();
-         auto start = _msg_type_bm.offset - fix::MsgType::tag_sz - 1;
-         auto end = _buffer.size();
-         _current_msg.overwrite(_body_length_bm, end - start, 6);
-         this_msg_map.checksum = _current_msg.checksum();
-         this_msg_map.checksum_bm = _current_msg.add_checksum(0);
-         this_msg_map.size = _buffer.size() - this_msg_map.offset;
+   inline void tidy_last_msg ()
+   {
+      if (_message_map.size ()) {
+         auto &this_msg_map = _message_map.back ();
+         auto start =
+            this_msg_map.offset + _msg_type_bm.offset - fix::MsgType::tag_sz-1;
+         auto end = _buffer.size ();
+         _current_msg.overwrite (_body_length_bm.shift (this_msg_map.offset),
+                                 end-start,6);
+         this_msg_map.checksum = _current_msg.checksum ();
+         this_msg_map.checksum_bm = _current_msg.add_checksum (0);
+         this_msg_map.size = _buffer.size () - this_msg_map.offset;
       }
    }
-
-   uint32_t encode_message_type(MessageTypes m, char out[4]) {
-      auto mt = (uint32_t)m;
-      for (int i = 0; i < 4; i++) {
-         out[i] = mt >> (8 * (3 - i));
-         mt &= ((1 << (8 * (3 - i))) - 1);
-      }
-      for (int i = 0; i < 4; i++)
-         if (out[i])
-            return i;
-      return 0;
-   }
 };
 
-} // namespace jetfix
-} // namespace bts
+}}

src/lib/jetfix/codec/FieldDefinitions.h

@@ -69,9 +69,6 @@ typedef StringField<143> TargetLocationID;
 typedef StringField<1603> ApplicationSystemName;
 typedef StringField<1604> ApplicationSystemVersion;
 typedef StringField<1605> ApplicationSystemVendor;
-typedef StringField<9450> ICEApplicationSystemName;
-typedef StringField<9451> ICEApplicationSystemVersion;
-typedef StringField<9452> ICEApplicationSystemVendor;
 typedef StringField<167> SecurityType;
 typedef StringField<17> ExecID;
 typedef StringField<19> ExecRefID;
@@ -100,7 +97,6 @@ typedef StringField<7928> SelfMatchPreventionID;
 typedef StringField<8> BeginString;
 typedef StringField<9324> OrderOptionalData;
 typedef StringField<96> RawData;
-typedef StringField<553> UserName;
 typedef StringField<9717> CorrelationClOrdID;
 typedef StringField<9771> MMAccount;
 typedef TimeField<122> OrigSendingTime;
@@ -108,38 +104,6 @@ typedef TimeField<52> SendingTime;
 typedef TimeField<5979> RequestTime;
 typedef TimeField<60> TransactTime;
 
-//ICE
-typedef IntField<9208> Int_CtiCode; 
-typedef BigIntField<11> Int_ClOrdID; 
-typedef BigIntField<37> Int_OrderID; 
-typedef BigIntField<41> Int_OrigClOrdID; 
-typedef IntField<109> CompanyID; 
-typedef IntField<55> Int_Symbol; 
-typedef IntField<439> Int_ClearingFirm;
-typedef StringField<9121> Memo;
-typedef IntField<103> OrdRejReason;
-typedef IntField<285> DeleteReason;
-typedef CharField<9195> AccountCode;
-typedef StringField<9207> CustomerAccountRefId;
-typedef StringField<115> OnBehalfOfCompID;
-typedef StringField<116> OnBehalfOfSubID;
-typedef StringField<144> OnBehalfOfLocationID;
-typedef StringField<9139> OriginatorUserID;
-typedef CharField<18> ExecInst;
-typedef IntField<210> MaxShow;
-typedef IntField<9006> ICEStrategyPreference;
-typedef IntField<9007> ICETimeStampPreference;
-typedef StringField<1031> CustOrderHandlingInst;
-
-typedef IntField<9700> MifidDirectElectronicAccess;
-typedef IntField<9701> MifidTradingCapacity;
-typedef IntField<9702> MifidLiquidityProvision;
-typedef IntField<9703> MifidCommodityDerivIndicator;
-typedef BigIntField<9704> MifidInvestmentDecision;
-typedef BigIntField<9705> MifidExecutionDecision;
-typedef BigIntField<9706> MifidClientIDCode;
-typedef BigIntField<9707> MifidId;
-
 typedef Group<NoQuoteEntries,
               QuoteEntryID,
               Symbol,

src/lib/jetfix/codec/Fields.h

@@ -156,33 +156,6 @@ public:
    }
 };
 
-template <uint16_t tag>
-class BigIntField : public BaseField<tag,int64_t>
-{
-public:
-   typedef BaseField<tag,int64_t> Parent;
-
-   void read (ReaderCursor &cursor)
-   {
-      auto loc = cursor.read_field ();
-      Parent::_value = atoll (&(cursor.data ()[loc.first]));
-   }
-
-   static uint32_t max_value_size (int64_t,int pad = 0)
-   {
-      if (pad == 0)
-         return 20;
-      else
-         return pad;
-   }
-   
-   static std::pair<uint32_t,uint32_t> write (char*buffer,int64_t i,int pad = 0)
-   {
-      auto written = i64_toa (i,buffer,pad);
-      return std::make_pair (written,sum_bytes (buffer,written));
-   }
-};
-
 template <uint16_t tag>
 class StringField : public BaseField<tag,std::string>
 {

src/lib/jetfix/codec/MessageDefinitions.h

@@ -66,58 +66,5 @@ typedef FIXMsg<MessageTypes::CancelReject,
                CorrelationClOrdID>
 CancelRejectMsg;
 
-typedef FIXMsg<MessageTypes::ICETraderLoginResponse, 
-               CompanyID,
-               UserName,
-               Text>
-ICETraderLoginResponseMsg;
-
-typedef FIXMsg<MessageTypes::ICETraderLogoutResponse, 
-               CompanyID,
-               UserName,
-               Text>
-ICETraderLogoutResponseMsg;
-
-typedef FIXMsg<MessageTypes::ExecutionReport,
-               Int_OrderID,
-               Int_ClOrdID,
-               CompanyID,
-               ExecID,               
-               ExecRefID,
-               ExecType,
-               OrdStatus,
-               OrdRejReason,               
-               Int_Symbol,               
-               Side,
-               OrderQty,
-               MaxShow,
-               OrdType,
-               Price,
-               StopPx,
-               TimeInForce,
-               LeavesQty,
-               CumQty,
-               AvgPx,
-               LastPx,
-               LastShares,               
-               TransactTimeStr,
-               MultiLegReportingType,               
-               ClearingAccount,            
-               Memo,                                       
-               Text,
-               DeleteReason,
-               TargetSubID,
-               MsgSeqNum>
-ICEExecutionReportMsg;
-
-typedef FIXMsg<MessageTypes::CancelReject,
-               Int_OrderID,
-               Int_ClOrdID,
-               Int_OrigClOrdID,
-               CXlRejReason,
-               Text,
-               Memo>
-ICECancelRejectMsg;
-
 }}}
 

src/lib/jetfix/codec/MessageTypes.h

@@ -5,7 +5,7 @@
 namespace bts {
 namespace jetfix {
 
-enum class MessageTypes : uint32_t
+enum class MessageTypes
 {
    Advertisement               = '7',
    Allocation                  = 'J',
@@ -48,12 +48,7 @@ enum class MessageTypes : uint32_t
    SettlementInstructions      = 'T',
    TestRequest                 = '1',
    TradingSessionStatus        = 'h',
-   TradingSessionStatusRequest = 'g',
-   ICETraderLogin              = ('U' << 16) + ('C' << 8) + 'G',
-   ICETraderLoginResponse      = ('U' << 16) + ('C' << 8) + 'I',   
-   ICETraderLogout             = ('U' << 16) + ('C' << 8) + 'H',   
-   ICETraderLogoutResponse     = ('U' << 16) + ('C' << 8) + 'J',   
-   
+   TradingSessionStatusRequest = 'g'
 };
 
 }}

src/lib/jetfix/codec/i32_toa.h

@@ -259,199 +259,3 @@ inline int u32_toa(uint32_t u, char* buffer, int pad) {
 }
 
 
-///https://github.com/miloyip/itoa-benchmark
-
-#define BEGIN2(n) \
-    do { \
-        int t = (n); \
-        if(t < 10) *p++ = '0' + t; \
-        else { \
-            t *= 2; \
-            *p++ = gDigitsLut[t]; \
-            *p++ = gDigitsLut[t + 1]; \
-        } \
-    } while(0)
-#define MIDDLE2(n) \
-    do { \
-        int t = (n) * 2; \
-        *p++ = gDigitsLut[t]; \
-        *p++ = gDigitsLut[t + 1]; \
-    } while(0)
-#define BEGIN4(n) \
-    do { \
-        int t4 = (n); \
-        if(t4 < 100) BEGIN2(t4); \
-        else { BEGIN2(t4 / 100); MIDDLE2(t4 % 100); } \
-    } while(0)
-#define MIDDLE4(n) \
-    do { \
-        int t4 = (n); \
-        MIDDLE2(t4 / 100); MIDDLE2(t4 % 100); \
-    } while(0)
-#define BEGIN8(n) \
-    do { \
-        uint32_t t8 = (n); \
-        if(t8 < 10000) BEGIN4(t8); \
-        else { BEGIN4(t8 / 10000); MIDDLE4(t8 % 10000); } \
-    } while(0)
-#define MIDDLE8(n) \
-    do { \
-        uint32_t t8 = (n); \
-        MIDDLE4(t8 / 10000); MIDDLE4(t8 % 10000); \
-    } while(0)
-#define MIDDLE16(n) \
-    do { \
-        uint64_t t16 = (n); \
-        MIDDLE8(t16 / 100000000); MIDDLE8(t16 % 100000000); \
-    } while(0)
-    
-inline int u64_toa(uint64_t value, char* p) 
-{
-   auto start = p;
-   if(value < 100000000) BEGIN8(value);
-   else if(value < 10000000000000000) { BEGIN8(value / 100000000); MIDDLE8(value % 100000000); }
-   else { BEGIN4(value / 10000000000000000); MIDDLE16(value % 10000000000000000); }
-   *p = 0;   
-   return p-start;
-}    
-
-inline int count_digits (int64_t value)
-{
-   static auto P01 = 10ull;
-   static auto P02 = P01*10;
-   static auto P03 = P02*10;
-   static auto P04 = P03*10;
-   static auto P05 = P04*10;
-   static auto P06 = P05*10;
-   static auto P07 = P06*10;
-   static auto P08 = P07*10;
-   static auto P09 = P08*10;
-   static auto P10 = P09*10;
-   static auto P11 = P10*10;
-   static auto P12 = P11*10;
-   static auto P13 = P12*10;
-   static auto P14 = P13*10;
-   static auto P15 = P14*10;
-   static auto P16 = P15*10;
-   static auto P17 = P16*10;
-   static auto P18 = P17*10;
-   static auto P19 = P18*10;
-
-   uint64_t u = static_cast<uint64_t>(value);
-   if (value < 0) 
-      u = ~u + 1;
-
-   if (u < P01) return 1;
-   if (u < P02) return 2;
-   if (u < P03) return 3;
-   if (u < P04) return 4;
-   if (u < P05) return 5;
-   if (u < P06) return 6;
-   if (u < P07) return 7;
-   if (u < P08) return 8;
-   if (u < P09) return 9;
-   if (u < P10) return 10;
-   if (u < P11) return 11;
-   if (u < P12) return 12;
-   if (u < P13) return 13;
-   if (u < P14) return 14;
-   if (u < P15) return 15;
-   if (u < P16) return 16;
-   if (u < P17) return 17;
-   if (u < P18) return 18;
-   if (u < P19) return 19;
-   return 20;
-}
-
-inline int count_digits (uint64_t u)
-{
-   static auto P01 = 10ull;
-   static auto P02 = P01*10;
-   static auto P03 = P02*10;
-   static auto P04 = P03*10;
-   static auto P05 = P04*10;
-   static auto P06 = P05*10;
-   static auto P07 = P06*10;
-   static auto P08 = P07*10;
-   static auto P09 = P08*10;
-   static auto P10 = P09*10;
-   static auto P11 = P10*10;
-   static auto P12 = P11*10;
-   static auto P13 = P12*10;
-   static auto P14 = P13*10;
-   static auto P15 = P14*10;
-   static auto P16 = P15*10;
-   static auto P17 = P16*10;
-   static auto P18 = P17*10;
-   static auto P19 = P18*10;
-
-   if (u < P01) return 1;
-   if (u < P02) return 2;
-   if (u < P03) return 3;
-   if (u < P04) return 4;
-   if (u < P05) return 5;
-   if (u < P06) return 6;
-   if (u < P07) return 7;
-   if (u < P08) return 8;
-   if (u < P09) return 9;
-   if (u < P10) return 10;
-   if (u < P11) return 11;
-   if (u < P12) return 12;
-   if (u < P13) return 13;
-   if (u < P14) return 14;
-   if (u < P15) return 15;
-   if (u < P16) return 16;
-   if (u < P17) return 17;
-   if (u < P18) return 18;
-   if (u < P19) return 19;
-   return 20;
-}
-
-inline int i64_toa(int64_t value, char* buffer, int pad) 
-{
-   auto start = buffer;
-   uint64_t u = static_cast<uint64_t>(value);
-   if (value < 0) {
-      *buffer++ = '-';
-      u = ~u + 1;
-   }
-   if (pad) 
-   {
-      auto digits = count_digits (value);
-      auto offset = pad - digits;
-      if (offset > 0) 
-      {
-         auto start = buffer + offset;
-         while (buffer != start)
-         {
-            *buffer='0';
-            buffer++;
-         }
-      }
-      else if (offset < 0)
-         return 0;
-   }
-   return (buffer-start) + u64_toa(u, buffer);
-}
-
-inline int u64_toa(uint64_t u, char* buffer, int pad) 
-{
-   auto start = buffer;
-   if (pad) 
-   {
-      auto digits = count_digits (u);
-      auto offset = pad - digits;
-      if (offset > 0) 
-      {
-         auto start = buffer + offset;
-         while (buffer != start)
-         {
-            *buffer='0';
-            buffer++;
-         }
-      }
-      else if (offset < 0)
-         return 0;
-   }
-   return (buffer-start) + u64_toa(u, buffer);
-}

src/lib/jetfix/gateway/Session.cpp

@@ -783,63 +783,42 @@ OutFIXPtr make_logon(uint32_t seq_num, uint32_t last_seq_num_processed,
    const char *application = "O2";
 #endif
 
-   if(session_cfg.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_ICE || 
-      session_cfg.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_SIMICE)
-   {
-      buffer.add<ICEApplicationSystemName>(application);
-      buffer.add<ICEApplicationSystemVersion>("1.0");
-      buffer.add<ICEApplicationSystemVendor>("BTS");      
-      
-      buffer.add<UserName>(session_cfg.user_name());      
-      buffer.add<RawDataLength>(session_cfg.raw_data().size());
-      buffer.add<RawData>(session_cfg.raw_data());      
-      buffer.add<EncryptMethod>(0);
-      buffer.add<ICEStrategyPreference>(1);
-      buffer.add<ICETimeStampPreference>(1);
+   buffer.add<ApplicationSystemName>(application);
+   buffer.add<ApplicationSystemVersion>("1.0");
+   buffer.add<ApplicationSystemVendor>("BTS");
+
+   if (session_cfg.has_trading_session_id()) {
+      buffer.add<fix::NoTradingSessions>(1);
+      buffer.add<fix::TradingSessionID>(session_cfg.trading_session_id());
    }
-   else
-   {
-      buffer.add<ApplicationSystemName>(application);
-      buffer.add<ApplicationSystemVersion>("1.0");
-      buffer.add<ApplicationSystemVendor>("BTS");
+   if (session_cfg.has_udf_support_indicator())
+      buffer.add<fix::UDFSupportIndicator>(session_cfg.udf_support_indicator());
 
-      if (session_cfg.has_trading_session_id()) {
-         buffer.add<fix::NoTradingSessions>(1);
-         buffer.add<fix::TradingSessionID>(session_cfg.trading_session_id());
-      }
-      if (session_cfg.has_udf_support_indicator())
-         buffer.add<fix::UDFSupportIndicator>(session_cfg.udf_support_indicator());
+   if (session_cfg.use_cme_secure_logon()) {
+      bts_assert(session_cfg.sender_compid().size() >= 3);
+      auto session_id = session_cfg.sender_compid().substr(0, 3);
+      std::string seq_num_str(9, ' ');
+      i32_toa(seq_num, &seq_num_str[0], 9);
 
-      if (session_cfg.use_cme_secure_logon()) {
-         bts_assert(session_cfg.sender_compid().size() >= 3);
-         auto session_id = session_cfg.sender_compid().substr(0, 3);
-         std::string seq_num_str(9, ' ');
-         i32_toa(seq_num, &seq_num_str[0], 9);
-
-         std::string last_seq_num_str(9, ' ');
-         i32_toa(last_seq_num_processed, &last_seq_num_str[0], 9);
-         auto ld = glbx::calculate_hmac(
-            session_id, seq_num_str, session_cfg.sender_compid(),
-            session_cfg.sender_subid(), time_str, session_cfg.target_subid(),
-            std::to_string(session_cfg.seconds_between_hbs()),
-            session_cfg.sender_locationid(), last_seq_num_str, application, "1.0",
-            "BTS");
-   #ifndef BTS_EDGE
-         LOG(Jetfix, Info, "Canonical string for secure logon: {}", ld.canonical);
-   #endif
-         buffer.add<EncodedTextLen>(ld.access_key.size());
-         buffer.add<EncodedText>(ld.access_key);
-         buffer.add<EncryptedPasswordMethod>("CME-1-SHA-256");
-         buffer.add<EncryptedPasswordLen>(ld.signature.size());
-         buffer.add<EncryptedPassword>(ld.signature);
-      } 
-      else 
-      {
-         buffer.add<RawDataLength>(session_cfg.raw_data().size());
-         buffer.add<RawData>(session_cfg.raw_data());
-         if(session_cfg.user_name() != "")
-            buffer.add<UserName>(session_cfg.user_name());
-      }
+      std::string last_seq_num_str(9, ' ');
+      i32_toa(last_seq_num_processed, &last_seq_num_str[0], 9);
+      auto ld = glbx::calculate_hmac(
+          session_id, seq_num_str, session_cfg.sender_compid(),
+          session_cfg.sender_subid(), time_str, session_cfg.target_subid(),
+          std::to_string(session_cfg.seconds_between_hbs()),
+          session_cfg.sender_locationid(), last_seq_num_str, application, "1.0",
+          "BTS");
+#ifndef BTS_EDGE
+      LOG(Jetfix, Info, "Canonical string for secure logon: {}", ld.canonical);
+#endif
+      buffer.add<EncodedTextLen>(ld.access_key.size());
+      buffer.add<EncodedText>(ld.access_key);
+      buffer.add<EncryptedPasswordMethod>("CME-1-SHA-256");
+      buffer.add<EncryptedPasswordLen>(ld.signature.size());
+      buffer.add<EncryptedPassword>(ld.signature);
+   } else {
+      buffer.add<RawDataLength>(session_cfg.raw_data().size());
+      buffer.add<RawData>(session_cfg.raw_data());
    }
 
    stream.close();

src/lib/md/MBOProcessor.h

@@ -6,9 +6,7 @@
 #include "ExchangeOrderKey.h"
 #include "MDProducerHandle.h"
 
-#ifndef _WIN32
 #include "google/btree/btree_map.h"
-#endif
 
 #include <vector>
 #include <memory>
@@ -109,13 +107,8 @@ struct MBOWorkingState {
    // Used for 'change' messages which update price
    std::unordered_map<ExchangeOrderKey, key_t> orderid_index;
 
-#ifdef _WIN32
-   std::map<key_t,Data,LessBid> bid;
-   std::map<key_t,Data> ask;
-#else
    btree::btree_map<key_t,Data,LessBid> bid;
    btree::btree_map<key_t,Data> ask;
-#endif
 };
 
 

src/lib/md/MDConsumer.cpp

@@ -13,6 +13,7 @@
 #include "lib/utils/sane_algorithm.h"
 
 #include <tbb/queuing_mutex.h>
+#include <google/btree/btree_map.h>
 #include <string>
 #include <map>
 #include <set>
@@ -21,6 +22,7 @@
 
 using namespace std;
 using namespace bts::api;
+using namespace btree;
 namespace bts {
 namespace md {
 
@@ -158,32 +160,36 @@ public:
       return false;
    }
 
-   void search_handles(map<security_key, MDConsumerHandle>& handles) {
+   btree_map<security_key, tuple<const ColdEntry*,const void*>> search_handles(const std::set<security_key>& keys) {
+      btree_map<security_key, tuple<const ColdEntry*, const void*>> result;
       // Walk hot and cold lists and fill in any found handles
-      if (!_cold.is_initialized()) return;
+      if (!_cold.is_initialized()) return result;
       _cold.all_entries([&](const ColdEntry& entry) {
-         auto iter = handles.find(entry.security.key);
-         if (iter != handles.end()) {
-            iter->second.set_cold(&cold_header(), &entry);
+         if (keys.find(entry.security.key) != keys.end()) {
+            result.insert(
+               std::make_pair(
+                  entry.security.key,
+                  make_tuple(&entry,nullptr)));
          }
       });
       // Do the same for hot
       if (_hot_depth) {
          _hot_depth->all_entries([&](const HotDepthEntry& entry) {
-            auto iter = handles.find(entry.key);
-            if (iter != handles.end() && iter->second.is_initialized()) {
-               iter->second.set_hot(&entry);
+            auto iter = result.find(entry.key);
+            if (iter != result.end()) {
+               iter->second = make_tuple(get<0>(iter->second), &entry);
             }
          });
       }
       else if (_hot){
          _hot->all_entries([&](const HotEntry& entry) {
-            auto iter = handles.find(entry.key);
-            if (iter != handles.end() && iter->second.is_initialized()) {
-               iter->second.set_hot(&entry);
+            auto iter = result.find(entry.key);
+            if (iter != result.end()) {
+               iter->second = make_tuple(get<0>(iter->second), &entry);
             }
          });
       }
+      return result;
    }
 
    template <typename CB>
@@ -661,24 +667,24 @@ public:
    map<security_key, MDConsumerHandle> get_handles(const set<security_key>& keys) {
       map<security_key, MDConsumerHandle> result;
       if (keys.empty()) return result;
-      // Initialize the result to match the input
-      for (security_key key : keys) {
-         result.insert(std::make_pair(key, MDConsumerHandle()));
-      }
 
       for (auto& inst : _instances) {
-         inst->search_handles(result);
-      }
-
-      // Erase anything that wasn't found from the resulting map
-      // Set the mbo pointer on anything else
-      for (auto it = result.begin(); it != result.end(); ) {
-         if (!it->second.is_initialized()) {
-            it = result.erase(it);
-         }
-         else {
-            it->second.set_mbo(_mbo);
-            ++it;
+         auto found = inst->search_handles(keys);
+         for (const auto& t : found) {
+            // auto [cold,hot] = t.second;
+            auto cold = get<0>(t.second);
+            auto hot = get<1>(t.second);
+            if (cold && hot) {
+               result.insert(
+                  make_pair(
+                     cold->security.key, 
+                     MDConsumerHandle {
+                        inst->cold_header(), 
+                        *cold,
+                        hot,
+                        _mbo
+                     }));
+            }
          }
       }
 
@@ -719,6 +725,7 @@ public:
       vector<MutSecurityPtr> legs;
       legs.reserve(handles.size());
       for (const auto& pair : handles) {
+         bts_assert_d(pair.second.is_initialized());
          if (pair.second.is_initialized()) {
             legs.push_back(make_shared<Security>(pair.second.get_normalized_security()));
          }

src/lib/md/MDConsumerHandle.cpp

@@ -32,6 +32,9 @@ MDConsumerHandle::MDConsumerHandle(const MDConsumerHandle& other) {
    *this = other;
 }
 
+MDConsumerHandle::MDConsumerHandle(const ColdHeader& cheader, const ColdEntry& centry, const void* pvhot, const MBOReader* mbo)
+{init(cheader,centry,pvhot,mbo);}
+
 MDConsumerHandle::~MDConsumerHandle() {}
 
 void MDConsumerHandle::init(const ColdHeader& cheader, const ColdEntry& centry, const void* pvhot, const MBOReader* mbo) {
@@ -293,7 +296,7 @@ TimeStamp copy_synthetic_depth(const SyntheticStrategy& synth, size_t count, Pri
 // END SYNTHETIC DEPTH
 /////////////////////////////
 
-bool MDConsumerHandle::get_last_trade(TimeStamp& time, PriceSize& trade, datetime::LLDateTime& trade_timestamp, size_t* total_volume, int* from_spread) const {
+bool MDConsumerHandle::get_last_trade(TimeStamp& time, PriceSize& trade, datetime::LLDateTime& trade_timestamp, size_t* total_volume) const {
    switch(_type) {
       case HANDLETYPE_VOID: return false;
       case HANDLETYPE_DEPTH:
@@ -303,7 +306,6 @@ bool MDConsumerHandle::get_last_trade(TimeStamp& time, PriceSize& trade, datetim
          if (result) {
             trade_timestamp = pcold->last_trade_time;
             if (total_volume) { *total_volume = pcold->total_volume; }
-            if (from_spread) { *from_spread = pcold->from_spread; }
          }
          return result;
       }

src/lib/md/MDConsumerHandle.h

@@ -156,6 +156,7 @@ public:
    explicit MDConsumerHandle(const std::shared_ptr<const SyntheticStrategy>&);
    ~MDConsumerHandle();
    MDConsumerHandle(const MDConsumerHandle& other);
+   MDConsumerHandle(const ColdHeader&, const ColdEntry&, const void* hot, const MBOReader*);
 
    void init(const ColdHeader&, const ColdEntry&, const void* hot, const MBOReader*);
 
@@ -260,7 +261,7 @@ public:
       return get_book(read_combined, time, copydepth, bids, asks);
    }
    
-   bool get_last_trade(TimeStamp& time, PriceSize& out_trade, datetime::LLDateTime& out_trade_timestamp, size_t* total_volume=nullptr,int* from_spread=nullptr) const;
+   bool get_last_trade(TimeStamp& time, PriceSize& out_trade, datetime::LLDateTime& out_trade_timestamp, size_t* total_volume=nullptr) const;
 
    IntPrice settlement_price() const {
       switch(_type) {
@@ -297,9 +298,6 @@ public:
    std::unique_ptr<MBOSnapshot,std::function<void(MBOSnapshot*)>> get_mbo() const; 
 
 private:
-   friend class MDConsumerInst;
-   friend class MDConsumerImpl;
-
    template <class Read>
    HandleReadResult get_book(Read read, TimeStamp& time, uint32_t copydepth, PriceSize* bids, PriceSize* asks) const {
       bts_assert_d(copydepth <= COCONUT_MAX_DEPTH);

src/lib/md/MDProducerHandle.cpp

@@ -101,9 +101,8 @@ void MDProducerHandle::set_status(CoconutSecurityStatus status) {
    }
 }
 
-void MDProducerHandle::add_trade(TimeStamp time, const PriceSize price, LLDateTime tradetime, 
-                                 co2::AggressorSide agside, uint32_t volume, bool from_spread) {
-   // Add this trade to the last trade if price/side match
+void MDProducerHandle::add_trade(TimeStamp time, const PriceSize price, LLDateTime tradetime, co2::AggressorSide agside, uint32_t volume) {
+   // Add this trade to the last trade if price match
    auto current = std::get<PriceSize>(cdata().last_trade.load());
    if (current.int_price() == price.int_price()) {
       cdata().last_trade.store(PriceSize { price.int_price(), price.size() + current.size() }, time);
@@ -112,9 +111,7 @@ void MDProducerHandle::add_trade(TimeStamp time, const PriceSize price, LLDateTi
       cdata().last_trade.store(price,time);
    }
    cdata().last_trade_time = tradetime;
-   cdata().from_spread = (from_spread ? 1 : 0);
-   if (volume > 0) 
-      cdata().total_volume = volume;
+   cdata().total_volume = volume;
 
    if (key() != invalid_security_key) {
       co2::TradeMessage trade;
@@ -128,15 +125,9 @@ void MDProducerHandle::add_trade(TimeStamp time, const PriceSize price, LLDateTi
       _channel.add_trade(trade);
    }
 }
-void MDProducerHandle::set_last_trade(TimeStamp time, const PriceSize ps, datetime::LLDateTime tradetime, uint32_t total_volume, bool from_spread) {
+void MDProducerHandle::set_last_trade(TimeStamp time, const PriceSize ps, datetime::LLDateTime tradetime, uint32_t total_volume) {
    cdata().last_trade.store(ps,time);
    cdata().last_trade_time = tradetime;
-   cdata().from_spread = (from_spread ? 1 : 0);
-   if(total_volume > 0)
-      cdata().total_volume = total_volume;
-}
-
-void MDProducerHandle::set_total_volume(uint32_t total_volume) {
    cdata().total_volume = total_volume;
 }
 

src/lib/md/MDProducerHandle.h

@@ -92,11 +92,9 @@ public:
    void set_status(co2::CoconutSecurityStatus status); // hot
 
    // Updates trade log and last trade
-   void add_trade(TimeStamp time, const PriceSize pricesize, datetime::LLDateTime date, 
-                  co2::AggressorSide agside, uint32_t total_volume=0, bool from_spread=false); // cold
+   void add_trade(TimeStamp time, const PriceSize pricesize, datetime::LLDateTime date, co2::AggressorSide agside, uint32_t total_volume=0); // cold
    // Update last_trade without updating trade log
-   void set_last_trade(TimeStamp time, const PriceSize ps, datetime::LLDateTime date, uint32_t total_volume=0,bool from_spread=false);
-   void set_total_volume(uint32_t total_volume);
+   void set_last_trade(TimeStamp time, const PriceSize ps, datetime::LLDateTime date, uint32_t total_volume);
    void add_rfq(TimeStamp time, const co2::QuoteRequestMessage& rfq); // cold
 
    void update_exchange_status(TimeStamp time, TradingStatus status); // cold

src/lib/md/PlatformLoader.cpp

@@ -90,7 +90,7 @@ void add_symbols_to_result(
 
 size_t get_system_md_symbols(api::Venue exchange, SecurityType type,
                              std::set<UserSymbol> &result) {
-   bts_assert(config::system_symbols::Symbols::descriptor()->field_count() == 4);
+   bts_assert(config::system_symbols::Symbols::descriptor()->field_count() == 3);
    {
       auto const &e = config::get<config::system_symbols::Symbols>().glbx();
       if (exchange == api::VENUE_UNSET || exchange == api::VENUE_GLBX) {
@@ -146,47 +146,34 @@ size_t get_system_md_symbols(api::Venue exchange, SecurityType type,
 
    {
       auto const &e = config::get<config::system_symbols::Symbols>().sim();
-      if (exchange == api::VENUE_UNSET || exchange == api::VENUE_SIM) 
-      {
-         if (type == api::SECURITYTYPE_FUTURE || type == api::SECURITYTYPE_UNSET) 
-            add_symbols_to_result(e.future(), api::VENUE_SIM, api::SECURITYTYPE_FUTURE, &result);
-         if (type == api::SECURITYTYPE_OPTION || type == api::SECURITYTYPE_UNSET) 
-            add_symbols_to_result(e.option(), api::VENUE_SIM, api::SECURITYTYPE_OPTION, &result);
-         if (type == api::SECURITYTYPE_SPREAD || type == api::SECURITYTYPE_UNSET) 
-            add_symbols_to_result(e.spread(), api::VENUE_SIM, api::SECURITYTYPE_SPREAD, &result);
+      if (exchange == api::VENUE_UNSET || exchange == api::VENUE_SIM) {
+         if (type == api::SECURITYTYPE_FUTURE ||
+             type == api::SECURITYTYPE_UNSET) {
+            add_symbols_to_result(e.future(), api::VENUE_SIM,
+                                  api::SECURITYTYPE_FUTURE, &result);
+         }
+         if (type == api::SECURITYTYPE_OPTION ||
+             type == api::SECURITYTYPE_UNSET) {
+            add_symbols_to_result(e.option(), api::VENUE_SIM,
+                                  api::SECURITYTYPE_OPTION, &result);
+         }
+         if (type == api::SECURITYTYPE_SPREAD ||
+             type == api::SECURITYTYPE_UNSET) {
+            add_symbols_to_result(e.spread(), api::VENUE_SIM,
+                                  api::SECURITYTYPE_SPREAD, &result);
+         }
       }
-      if (exchange == api::VENUE_BTS || exchange == api::VENUE_UNSET) 
-      {
+      if (exchange == api::VENUE_BTS || exchange == api::VENUE_UNSET) {
          std::set<std::string> all_symbols;
          all_symbols.insert(e.future().begin(), e.future().end());
          all_symbols.insert(e.option().begin(), e.option().end());
          all_symbols.insert(e.spread().begin(), e.spread().end());
-         for (const auto &symbol : all_symbols) 
-            result.insert(UserSymbol(symbol, api::VENUE_BTS, api::SECURITYTYPE_SYNTHETIC));      
+         for (const auto &symbol : all_symbols) {
+            result.insert(UserSymbol(symbol, api::VENUE_BTS,
+                                     api::SECURITYTYPE_SYNTHETIC));
+         }
       }
    }
-   
-   {
-      auto const &e = config::get<config::system_symbols::Symbols>().simice();
-      if (exchange == api::VENUE_UNSET || exchange == api::VENUE_SIMICE) 
-      {
-         if (type == api::SECURITYTYPE_FUTURE || type == api::SECURITYTYPE_UNSET) 
-            add_symbols_to_result(e.future(), api::VENUE_SIMICE, api::SECURITYTYPE_FUTURE, &result);
-         if (type == api::SECURITYTYPE_OPTION || type == api::SECURITYTYPE_UNSET) 
-            add_symbols_to_result(e.option(), api::VENUE_SIMICE, api::SECURITYTYPE_OPTION, &result);
-         if (type == api::SECURITYTYPE_SPREAD || type == api::SECURITYTYPE_UNSET) 
-            add_symbols_to_result(e.spread(), api::VENUE_SIMICE, api::SECURITYTYPE_SPREAD, &result);
-      }
-      if (exchange == api::VENUE_BTS || exchange == api::VENUE_UNSET) 
-      {
-         std::set<std::string> all_symbols;
-         all_symbols.insert(e.future().begin(), e.future().end());
-         all_symbols.insert(e.option().begin(), e.option().end());
-         all_symbols.insert(e.spread().begin(), e.spread().end());
-         for (const auto &symbol : all_symbols) 
-            result.insert(UserSymbol(symbol, api::VENUE_BTS, api::SECURITYTYPE_SYNTHETIC));      
-      }
-   }   
 
    return result.size();
 }

src/lib/md/Shm/FixedSecurity.cpp

@@ -31,9 +31,8 @@ FixedSecurity::FixedSecurity() :
    cme_main_fraction(0),
    cme_sub_fraction(0),
    cme_price_display_format(0),
-   ice_price_denominator(0),
-   ice_deal_denominator(0),
-   ice_settle_denominator(0),
+   ice_deal_display_factor(0),
+   ice_settle_display_factor(0),
    _symbol(),
    _cfi_code(),
    _security_desc(),
@@ -100,10 +99,7 @@ void write_leg_descriptor(std::ostringstream& result, char separator, const api:
 std::string security_descriptor(const api::Security& security) {
    const char separator = ' ';
    std::ostringstream result;
-   if(security.exchange() == Venue::VENUE_ICE || security.exchange() == Venue::VENUE_SIMICE)
-      result << security.ice().security_desc() << separator << Venue_Name(security.exchange());
-   else
-      result << security.symbol() << separator << Venue_Name(security.exchange());
+   result << security.symbol() << separator << Venue_Name(security.exchange());
    switch(security.Contract_case()) {
       case Security::ContractCase::kOption: {
          result << separator;
@@ -185,9 +181,8 @@ void convert_security(const FixedSecurity& source, Security* presult) {
       case Venue::VENUE_ICE:    
          dest.mutable_ice()->set_exchange_id(source.numeric_id);
          dest.mutable_ice()->set_security_desc(source.security_desc());     
-         dest.mutable_ice()->set_price_denominator(source.ice_price_denominator);
-         dest.mutable_ice()->set_deal_denominator(source.ice_deal_denominator);
-         dest.mutable_ice()->set_settle_denominator(source.ice_settle_denominator);
+         dest.mutable_ice()->set_deal_display_factor(source.ice_deal_display_factor);
+         dest.mutable_ice()->set_settle_display_factor(source.ice_settle_display_factor);
          
          dest.mutable_ice()->set_market_description(source.ice_market_description());
          dest.mutable_ice()->set_cleared_alias(source.ice_cleared_alias());
@@ -317,9 +312,8 @@ void convert_security(const Security& src, FixedSecurity& dest, bool set_securit
       case Security::ExchangeDataCase::kIce:
          dest.numeric_id = src.ice().exchange_id();
          dest.security_desc(src.ice().security_desc());      
-         dest.ice_price_denominator = src.ice().price_denominator();
-         dest.ice_deal_denominator = src.ice().deal_denominator();
-         dest.ice_settle_denominator = src.ice().settle_denominator();
+         dest.ice_deal_display_factor = src.ice().deal_display_factor();
+         dest.ice_settle_display_factor = src.ice().settle_display_factor();
          
          dest.ice_market_description(src.ice().market_description());
          dest.ice_cleared_alias(src.ice().cleared_alias());

src/lib/md/Shm/FixedSecurity.h

@@ -59,9 +59,8 @@ public:
    uint8_t cme_main_fraction;
    uint8_t cme_sub_fraction;
    uint8_t cme_price_display_format;
-   int8_t ice_price_denominator;
-   int8_t ice_deal_denominator;
-   int8_t ice_settle_denominator;
+   int8_t ice_deal_display_factor;
+   int8_t ice_settle_display_factor;
    
    STRING_PROPERTY(ice_market_description);   
    STRING_PROPERTY(ice_cleared_alias);

src/lib/md/Shm/MDStructure.h

@@ -26,7 +26,7 @@ namespace md {
 
 using utils::TimeStamp;
 
-const uint32_t shm_version = 107;
+const uint32_t shm_version = 102;
 
 struct ALIGNED_(CACHE_LINE) HotHeader {
    size_t count;
@@ -213,10 +213,9 @@ struct ColdEntry {
       exchange_status(co2::TRADINGSTATUS_UNKNOWN),
       last_trade(),
       total_volume(0),
-      last_trade_time(),      
+      last_trade_time(),
       misc(),
-      from_spread(0),
-      deleted(0)
+      deleted(false)
    {
       this->security.size = security_size;
    }
@@ -228,13 +227,12 @@ struct ColdEntry {
    
    // Last Trade
    AtomicPriceSize last_trade;
-   size_t total_volume;   
+   size_t total_volume;
    datetime::LLDateTime  last_trade_time;
-   
+
    MiscData misc;
 
-   int from_spread;
-   int deleted;
+   bool deleted;
    FixedSecurity security;
 };
 

src/lib/md/ice/ICEUtils.cpp

@@ -27,8 +27,6 @@ std::string ICEChannel::to_string() const
    std::string mt = name;
    for(const auto& x : market_types)
       mt += " " + std::to_string(x);
-   for(const auto& x : uds_market_types)
-      mt += " " + std::to_string(x);      
    return mt;
 }
 
@@ -47,9 +45,6 @@ const vector<ICEChannel> parse_ice_channels(const vector<UserSymbol>& symbols)
       chan.symbols.insert(symbols.begin(), symbols.end());
       for(const auto& xx : x.market_type())
          chan.market_types.push_back(xx); 
-      for(const auto& xx : x.uds_market_type())
-         chan.uds_market_types.push_back(xx); 
-         
       result.push_back(chan);            
    }
    return result;

src/lib/md/ice/ICEUtils.h

@@ -13,7 +13,6 @@ struct ICEChannel
 {
    std::string name;
    std::vector<uint32_t> market_types;
-   std::vector<uint32_t> uds_market_types;
    std::set<UserSymbol> symbols;
    std::string to_string() const;
 };

src/lib/md/protos/co2.proto

@@ -152,7 +152,6 @@ message TradeMessage {
    uint32 total_volume = 7;
    bool gap = 8;
    AggressorSide aggressor_side = 9;
-   bool from_spread = 10;
 }
 message QuoteRequestMessage {
     enum Side {

src/lib/md/relay/QueryServer.cpp

@@ -78,8 +78,7 @@ struct MDConsumerGroup {
       symbol(symbol),
       platform(platform_for_exchange(symbol.exchange())),
       channel(consumer.get_channel(symbol)),
-      handles(),
-      mic()
+      handles()
    {
       set_consumer(consumer);
       
@@ -119,38 +118,9 @@ struct MDConsumerGroup {
       bts_assert_d(handle.is_initialized());
       bts_assert(handle.get_usersymbol() == symbol);
       DatedHandle h = { TimeStamp::now(), handle };
-      handles->push_back(h);   
-      if(this->mic == "")
-         set_mic(h);
+      handles->push_back(h);
    }
-   
-   void set_mic(const DatedHandle& h)
-   {
-      if(channel.exchange() == api::VENUE_ICE || channel.exchange() == api::VENUE_SIMICE)
-      {
-         this->mic = h.handle.get_normalized_security().ice().mic_code();
-      }
-      else if(channel.exchange() == api::VENUE_GLBX || channel.exchange() == api::VENUE_SIM)
-      {
-         const auto& cme_symb_mics = config::get<config::CmeSymbolMics>().cme_mic();
-         for(auto mic=cme_symb_mics.begin();mic!=cme_symb_mics.end();mic++)
-         {
-            const auto& micsym = mic->first;
-            if(micsym == symbol.symbol())
-            {
-               this->mic = api::CMEMics_Name(mic->second);
-               break;
-            }
-         }
-      }
-      else if(channel.exchange() == api::VENUE_BTS || channel.exchange() == api::VENUE_UNSET)
-      {
-         this->mic = "BTS";
-      }
-      else
-         bts_assert_msg(false, "unknown venue in QueryServer");
-   }
-   
+
    UserSymbol symbol;
    Platform platform;
    MDConsumerChannel channel;
@@ -158,7 +128,7 @@ struct MDConsumerGroup {
    // Can add to handles without invalidating readers (no lock needed)
    // When we recover we swap a new vector into this unique_ptr and delayed-delete
    shared_ptr<ConcurrentAppendCollection<DatedHandle>> handles;
-   std::string mic;
+
 };
 
 void convert(PriceSize ps, PriceSizeMessage* pps) {
@@ -249,18 +219,12 @@ public:
    }
 
    // the hot part
-   void fill(const SecurityChannel& chan, MDConsumerGroup& group, ChannelReply* reply, 
-             const Entitlements* entitlements, bool bypass_entitlement) const {      
+   void fill(const SecurityChannel& chan, MDConsumerGroup& group, ChannelReply* reply, bool entitled) const {
       bts_assert(reply);
-
       reply->mutable_channel()->CopyFrom(chan);
       reply->set_platform(group.platform);
       reply->mutable_channel()->set_time(TimeStamp::now().to_uint64());
-      
-      bool entitled = bypass_entitlement || chan.exchange() == api::VENUE_BTS;
-      if(entitlements && !entitled)
-         entitled = entitlements->count(group.mic);
-      reply->mutable_channel()->set_entitled(entitled);      
+      reply->mutable_channel()->set_entitled(entitled);
       auto flags = (MDRequest::Flags)chan.flags();
 
       // Get a copy of handles. If recovery happens during this call
@@ -278,10 +242,8 @@ public:
          ChannelUpdate* pupd = &temp_update;
          pupd->set_key(instrument.handle.key());
          // If the security was just discovered try to ensure all data is sent
-         if (instrument.discovered > TimeStamp(chan.time())) 
-         {
-            auto security = handle.get_normalized_security();
-            pupd->mutable_security()->CopyFrom(security);
+         if (instrument.discovered > TimeStamp(chan.time())) {
+            pupd->mutable_security()->CopyFrom(handle.get_normalized_security());
             snapshot = true;
          }
          //if we are not entitled AND this is the first request since we lost our entitlement, zero out books
@@ -361,13 +323,12 @@ public:
                TimeStamp time(snapshot ? 0 : chan.time());
                PriceSize price;
                datetime::LLDateTime trade_time;
-               size_t volume; int from_spread;
-               if (handle.get_last_trade(time, price, trade_time, &volume, &from_spread)) {
+               size_t volume;
+               if (handle.get_last_trade(time, price, trade_time, &volume)) {
                   auto* trade = pupd->mutable_last_trade();
                   trade->set_key(handle.key());
                   trade->set_time(trade_time);
                   trade->set_total_volume(volume);
-                  trade->set_from_spread(from_spread);
                   set_price(price, trade);
                }
             }
@@ -412,7 +373,10 @@ public:
          auto groupiter = _groups.find(UserSymbol(ch.symbol(), ch.exchange(), ch.type()));
          if (groupiter != _groups.end()) {
             auto& group = groupiter->second;
-            fill(ch, *group, reply->add_channels(), entitlements, bypass_entitlement);
+            bool is_entitled = bypass_entitlement || 
+                               ch.exchange() == api::VENUE_BTS || 
+                               (entitlements && entitlements->count(ch.symbol()));
+            fill(ch, *group, reply->add_channels(), is_entitled);
          }
       }
    }
@@ -574,21 +538,27 @@ QueryServerPtr run_query_server(const bts::api::ServiceContext& ctx, MDClient& c
       container->admin_client.subscribe_md_entitlement_data().subscribe([=](const api::admin::Md_entitlement_dataUpdate& update) {         
          const auto& all_updates = update.all();
          utils::Dispatcher::background().post([all_updates, container]() 
-         {            
+         {
+            const auto& cme_symb_mics = config::get<config::CmeSymbolMics>().cme_mic();
             auto entitlements = std::make_shared<std::unordered_map<uint32_t, Entitlements>>();
-            for (const auto& row : all_updates) 
-            {            
-               if (row && row->enabled()) 
-               {
-                  auto iter = entitlements->find(row->user_id());
-                  if (iter == entitlements->end()) {
-                     Entitlements ents;
-                     ents.insert(row->mic());
-                     entitlements->insert(iter, std::make_pair(row->user_id(), std::move(ents)));
+            for (const auto& row : all_updates) {            
+               if (row && row->enabled()) {                                 
+                  for(auto mic=cme_symb_mics.begin();mic!=cme_symb_mics.end();mic++){
+                     if(api::CMEMics_Name(mic->second) == row->mic()) {
+                        const auto& symbol = mic->first;
+                        auto iter = entitlements->find(row->user_id());
+                        if (iter == entitlements->end()) {
+                           Entitlements ents;
+                           ents.insert(symbol);
+                           ents.insert("SIM:" + symbol);
+                           entitlements->insert(iter, std::make_pair(row->user_id(), std::move(ents)));
+                        }
+                        else {
+                           iter->second.insert(symbol);
+                           iter->second.insert("SIM:" + symbol);
+                        }
+                     }
                   }
-                  else {
-                     iter->second.insert(row->mic());
-                  }                                       
                }
             }
             container->relay.update_entitlements(entitlements);

src/lib/md/relay/RelayClient.cpp

@@ -86,7 +86,7 @@ class RelayProducer : public IRelayWriter {
          handle.update_fixing_price(updated, misc.fixing_price());
       }
       if (update.has_last_trade()) {
-         handle.set_last_trade(updated, trade_pricesize(update.last_trade()), update.last_trade().time(), update.last_trade().total_volume(), update.last_trade().from_spread());
+         handle.set_last_trade(updated, trade_pricesize(update.last_trade()), update.last_trade().time(), update.last_trade().total_volume());
       }
    }
 

src/lib/md/test/Co2ManagedTestHelper.cpp

@@ -87,8 +87,7 @@ extern "C" BTS_EXPORT void co2_get_sample_cold(size_t& size, ColdEntry& entry) {
    entry.misc.session_high_bid = 4.4;
    entry.misc.session_low_offer = 5.5;
    entry.misc.fixing_price = 6.6;
-   entry.deleted = 1;
-   entry.from_spread = 0;
+   entry.deleted = true;
 }
 
 extern "C" BTS_EXPORT void co2_get_sample_cold_header(ColdHeader& header) {

src/lib/orderman/actors/ExecutionGateway.cpp

@@ -1,4 +1,4 @@
-#include "GLBXExecutionGateway.h"
+#include "ExecutionGateway.h"
 #include "lib/jetfix/codec/MessageDefinitions.h"
 #include "lib/jetfix/common/OutFIX.h"
 #include "lib/jetfix/gateway/JetFIXGateway.h"
@@ -24,7 +24,7 @@ using JFGw =
     JetFIXGateway<fix::ExecutionReportMsg, fix::BusinessMessageRejectMsg,
                   fix::RejectMsg, fix::CancelRejectMsg>;
 
-struct GLBXExecutionGateway::PImpl {
+struct ExecutionGateway::PImpl {
    PImpl(config::JetFIX::Connection const &cfg, trace::PerfLog &l,
          md::SecurityOracle o)
        : gw(cfg, l), exe_report_pool(), state(State::WAITING_FOR_SECURITIES),
@@ -46,16 +46,14 @@ struct GLBXExecutionGateway::PImpl {
    md::SecurityOracle sec_oracle;
 };
 
-GLBXExecutionGateway::GLBXExecutionGateway(config::JetFIX::Connection const &cfg,
+ExecutionGateway::ExecutionGateway(config::JetFIX::Connection const &cfg,
                                    trace::PerfLog &log, md::SecurityOracle o)
     : me(new PImpl(cfg, log, o)) {
    me->state = State::WAITING_FOR_SECURITIES;
 }
-GLBXExecutionGateway::~GLBXExecutionGateway() {}
+ExecutionGateway::~ExecutionGateway() {}
 
-void GLBXExecutionGateway::on_admin_data(const mercury::Update<NewAdminDataEvent> &) {}
-
-void GLBXExecutionGateway::on_securities(
+void ExecutionGateway::on_securities(
     const mercury::Update<NewSecuritiesEvent> &update) {
    if (update.events) {
 
@@ -98,19 +96,19 @@ void GLBXExecutionGateway::on_securities(
    }
 }
 
-void GLBXExecutionGateway::start() { me->gw.start(); }
+void ExecutionGateway::start() { me->gw.start(); }
 
-OnExeReport &GLBXExecutionGateway::on_exe_report() { return me->on_exe_report; }
+OnExeReport &ExecutionGateway::on_exe_report() { return me->on_exe_report; }
 
-OnGwStatusChange &GLBXExecutionGateway::on_gw_status_change() {
+OnGwStatusChange &ExecutionGateway::on_gw_status_change() {
    return me->on_gw_status_change;
 }
 
-OnCancelReject &GLBXExecutionGateway::on_cancel_reject() {
+OnCancelReject &ExecutionGateway::on_cancel_reject() {
    return me->on_cancel_reject;
 }
 
-void GLBXExecutionGateway::on_exe_report(
+void ExecutionGateway::on_exe_report(
     const jetfix::fix::ExecutionReportMsg &msg) {
    OrderKey ok;
    SecurityKey sk;
@@ -118,7 +116,7 @@ void GLBXExecutionGateway::on_exe_report(
 
    auto scoped_log =
        trace::log_scope(me->perf_log, msg.trace(),
-                        "GLBXExecutionGateway::on_exe_report", ok.value());
+                        "ExecutionGateway::on_exe_report", ok.value());
 
    LOG(Latency, Info, "Started handling of execution report for order {}",
        ok.value());
@@ -268,7 +266,7 @@ void GLBXExecutionGateway::on_exe_report(
    }
 }
 
-void GLBXExecutionGateway::on_cancel_reject(
+void ExecutionGateway::on_cancel_reject(
     const jetfix::fix::CancelRejectMsg &msg) {
    OrderKey ok;
    SecurityKey sk;
@@ -283,7 +281,7 @@ void GLBXExecutionGateway::on_cancel_reject(
    me->on_cancel_reject(ok, reject_msg);
 }
 
-void GLBXExecutionGateway::on_connection_status_change(bool is_connected) {
+void ExecutionGateway::on_connection_status_change(bool is_connected) {
    LOG(OrderMan, Info, "Connection state of {} changed to {}",
        me->gw_name.value(),
        is_connected ? std::string("connected") : std::string("disconnected"));
@@ -291,7 +289,7 @@ void GLBXExecutionGateway::on_connection_status_change(bool is_connected) {
    me->on_gw_status_change(is_connected);
 }
 
-void GLBXExecutionGateway::on_business_reject(
+void ExecutionGateway::on_business_reject(
     jetfix::fix::BusinessMessageRejectMsg const &m) {
    LOG(OrderMan, Error,
        "Unexpected business reject in {} gateway . Reason: "
@@ -300,7 +298,7 @@ void GLBXExecutionGateway::on_business_reject(
        m.has<fix::Text>() ? m.get<fix::Text>() : std::string("Unknown"));
 }
 
-void GLBXExecutionGateway::on_session_reject(jetfix::fix::RejectMsg const &m) {
+void ExecutionGateway::on_session_reject(jetfix::fix::RejectMsg const &m) {
    LOG(OrderMan, Error,
        "Unexpected reject message in {} gateway . Reason: {}. "
        "This should be investigated",
@@ -308,14 +306,14 @@ void GLBXExecutionGateway::on_session_reject(jetfix::fix::RejectMsg const &m) {
        m.has<fix::Text>() ? m.get<fix::Text>() : std::string("Unknown"));
 }
 
-void GLBXExecutionGateway::dispatch_incoming_fix_messages() {
+void ExecutionGateway::dispatch_incoming_fix_messages() {
    me->gw.dispatch_events();
 }
 
-void GLBXExecutionGateway::send_fix(jetfix::OutFIXPtr const &fix,
+void ExecutionGateway::send_fix(jetfix::OutFIXPtr const &fix,
                                 trace::CorID trace) {
    auto scoped_log =
-       trace::log_scope(me->perf_log, trace, "GLBXExecutionGateway::send_fix");
+       trace::log_scope(me->perf_log, trace, "ExecutionGateway::send_fix");
    me->gw.send(fix, trace);
    me->gw.dispatch_events();
 }

src/lib/orderman/actors/ExecutionGateway.h

@@ -0,0 +1,53 @@
+#pragma once
+
+#include "lib/config/protos/jetfix.pb.h"
+#include "lib/jetfix/codec/MessageDefinitions.h"
+#include "lib/jetfix/common/OutFIXPtr.h"
+#include "lib/mercury/events/Update.h"
+#include "lib/orderman/orderman_utils/orderman_common.h"
+
+#include <memory>
+
+namespace bts {
+namespace md { class SecurityOracle;}
+namespace trace {struct CorID;class PerfLog;}
+namespace orderman {
+class NewSecuritiesEvent;
+class ExecutionReport;
+
+using OnExeReport =
+   std::function<void(ExecutionReport const &, trace::CorID const &)>;
+using OnGwStatusChange = std::function<void(bool)>;
+using OnCancelReject = std::function<void(OrderKey, std::string const &)>;
+
+// fix execution gateway for a single FIX session
+class ExecutionGateway {
+ public:
+   ExecutionGateway(config::JetFIX::Connection const &, trace::PerfLog &,
+                    md::SecurityOracle);
+   ~ExecutionGateway();
+
+   void start();
+   void dispatch_incoming_fix_messages();
+   void on_securities(const mercury::Update<NewSecuritiesEvent> &);
+   void send_fix(jetfix::OutFIXPtr const &fix, trace::CorID trace);
+
+   OnExeReport& on_exe_report();
+   OnGwStatusChange& on_gw_status_change();
+   OnCancelReject &on_cancel_reject();
+   
+ private:
+   enum class State { WAITING_FOR_SECURITIES, READY };
+   struct PImpl;
+   std::unique_ptr<PImpl> me;
+
+   // fix gateway events
+   void on_exe_report(jetfix::fix::ExecutionReportMsg const &);
+   void on_cancel_reject(jetfix::fix::CancelRejectMsg const &);
+   void on_connection_status_change(bool connected);
+   void on_business_reject(jetfix::fix::BusinessMessageRejectMsg const &);
+   void on_session_reject(jetfix::fix::RejectMsg const &);
+};
+
+} // namespace orderman
+} // namespace bts

src/lib/orderman/actors/FillStore.cpp

@@ -58,12 +58,10 @@ void FillStore::init() {
             fill.set_fill_kind(Fill::FILL_KIND_SOD);
          auto sec = me->oracle.get_security(SecurityKey(fill.security_key()));
          if (sec) {
-            if (config::get<config::toggles::FilterExpiredFills>().enabled() &&
-                securities::is_expired(*sec))
-               continue;
-            else {
-               auto fk = kvp.first;
-               me->known_fills.insert(fk);
+            auto fk = kvp.first;
+            me->known_fills.insert(fk);
+            if (!config::get<config::toggles::FilterExpiredFills>().enabled() || 
+                !securities::is_expired(*sec)) {
                stored_batch.emplace_back(fill, *sec);
             }
          } else
@@ -95,13 +93,12 @@ void FillStore::on_execution_report(
    if (u.events) {
       for (auto const &ev : *u.events) {
          auto &er = *ev.data;
-         auto fk = fmt::format("{}:{}", er.order_key.value(),
-                                 to_str(er.exchange_fill_id));
-         auto is_known = me->known_fills.count(fk);         
          if (er.exec_type == api::orderman::EXEC_TYPE_PARTIAL_FILL ||
-             er.exec_type == api::orderman::EXEC_TYPE_FILL ||
-             er.exec_type == api::orderman::EXEC_TYPE_TRADE_CORRECT) {
-                
+             er.exec_type == api::orderman::EXEC_TYPE_FILL) {
+
+            auto fk = fmt::format("{}:{}", er.order_key.value(),
+                                  to_str(er.exchange_fill_id));
+            auto is_known = me->known_fills.count(fk);
             api::orderman::Fill fill;
             if (!is_known) {
                fill.set_order_key(er.order_key.value());
@@ -117,9 +114,6 @@ void FillStore::on_execution_report(
                fill.set_avg_price(er.avg_price.value());
                fill.set_side(er.side);
                fill.set_last_shares(er.last_shares.value());
-               fill.set_fill_status(api::orderman::Fill::FILL_STATUS_ACTIVE);
-               if(er.exec_type == api::orderman::EXEC_TYPE_TRADE_CORRECT)
-                  fill.set_fill_status(api::orderman::Fill::FILL_STATUS_CORRECTED);
                fill.set_fill_key(fk);
 
                auto exchange_ts = utils::pb::to_timestamp(
@@ -143,57 +137,6 @@ void FillStore::on_execution_report(
                    "Discarding known execution report for fill {}", fk);
 
          } // if partial_fill or fill
-      }
-      for (auto const &ev : *u.events) 
-      {
-         auto &er = *ev.data;
-         auto efi = to_str(er.exchange_fill_id);
-         if(std::count(efi.begin(), efi.end(), ':') > 0)
-         {
-            auto cpos = efi.find_last_of(':');
-            efi = efi.substr(0, cpos);
-            ExchangeFillID temp;
-            str_to(efi, &temp);
-            efi = to_str(temp);
-         }
-         if(efi == "")
-            continue;
-         auto fk = fmt::format("{}:{}", er.order_key.value(), efi);
-         auto is_known = me->known_fills.count(fk);         
-      
-         if(er.exec_type == api::orderman::EXEC_TYPE_TRADE_CANCEL) 
-         {
-            auto data = me->store->retrieve_one(fills_ns, fk);
-            Fill existing_fill;
-            if(is_known && data.first && utils::pb::from_string(data.second, &existing_fill))
-            {
-               existing_fill.set_fill_status(api::orderman::Fill::FILL_STATUS_BUSTED);
-               auto fv = utils::pb::to_string(existing_fill);
-               me->store->update(fills_ns, fk, fv);                  
-               publish(me->fill_stream, existing_fill, *er.security);
-            }
-            else
-               LOG(OrderMan, Warn, "Received bust for unknown fill {}", fk);
-         }    
-         else if(er.exec_type == api::orderman::EXEC_TYPE_TRADE_CORRECT)
-         {
-            auto data = me->store->retrieve_one(fills_ns, fk);
-            Fill existing_fill;
-            if(is_known && data.first && utils::pb::from_string(data.second, &existing_fill))
-            {
-               existing_fill.set_fill_status(api::orderman::Fill::FILL_STATUS_CORRECTED);
-               existing_fill.set_last_price(er.last_price.value());
-               existing_fill.set_avg_price(er.avg_price.value());
-               existing_fill.set_side(er.side);
-               existing_fill.set_last_shares(er.last_shares.value());
-
-               auto fv = utils::pb::to_string(existing_fill);
-               me->store->update(fills_ns, fk, fv);                  
-               publish(me->fill_stream, existing_fill, *er.security);
-            }
-            else
-               LOG(OrderMan, Warn, "Received correction for unknown fill {}", fk);            
-         }
       }    // for loop
    }       // if u.events
 }

src/lib/orderman/actors/GLBXExecutionGateway.h

@@ -1,41 +0,0 @@
-#pragma once
-
-#include "IExecutionGateway.h"
-
-namespace bts {
-namespace orderman {
-
-
-// fix execution gateway for a single FIX session
-class GLBXExecutionGateway : public IExecutionGateway {
- public:   
-   GLBXExecutionGateway(config::JetFIX::Connection const &, trace::PerfLog &,
-                        md::SecurityOracle);
-   virtual ~GLBXExecutionGateway();
-
-   void start() override;
-   void dispatch_incoming_fix_messages() override;
-   void on_securities(const mercury::Update<NewSecuritiesEvent> &) override;
-   void on_admin_data(const mercury::Update<NewAdminDataEvent> &) override;
-   void send_fix(jetfix::OutFIXPtr const &fix, trace::CorID trace) override;
-
-   OnExeReport& on_exe_report() override;
-   OnGwStatusChange& on_gw_status_change() override;
-   OnCancelReject &on_cancel_reject() override;
-   
- private:
-   BTS_NO_COPY(GLBXExecutionGateway);
-   enum class State { WAITING_FOR_SECURITIES, READY };
-   struct PImpl;
-   std::unique_ptr<PImpl> me;
-
-   // fix gateway events
-   void on_exe_report(jetfix::fix::ExecutionReportMsg const &);
-   void on_cancel_reject(jetfix::fix::CancelRejectMsg const &);
-   void on_connection_status_change(bool connected);
-   void on_business_reject(jetfix::fix::BusinessMessageRejectMsg const &);
-   void on_session_reject(jetfix::fix::RejectMsg const &);
-};
-
-} // namespace orderman
-} // namespace bts

src/lib/orderman/actors/ICEExecutionGateway.cpp

@@ -1,490 +0,0 @@
-#include "ICEExecutionGateway.h"
-#include "lib/jetfix/codec/MessageDefinitions.h"
-#include "lib/jetfix/codec/i32_toa.h"
-#include "lib/jetfix/common/OutFIX.h"
-#include "lib/jetfix/gateway/JetFIXGateway.h"
-#include "lib/logging/logging.h"
-#include "lib/orderman/events/CancelRejectEvent.h"
-#include "lib/orderman/events/GatewayStatusChangeEvent.h"
-#include "lib/orderman/events/NewExecutionReportEvent.h"
-#include "lib/orderman/events/NewSecuritiesEvent.h"
-#include "lib/orderman/orderman_utils/fix_order_codec.h"
-#include "lib/api/services/AdminService.pb.h"
-#include "lib/orderman/events/NewAdminDataEvent.h"
-#include "lib/tracing/bts_trace.h"
-#include "lib/utils/TimeStamp.h"
-#include "lib/utils/helper_macros.h"
-#include "lib/utils/Dispatcher.h"
-#include "lib/utils/bts_chrono.h"
-
-#include <boost/pool/object_pool.hpp>
-#include <atomic>
-#include <set>
-
-namespace bts {
-namespace orderman {
-
-using namespace jetfix;
-
-using JFGw = JetFIXGateway<fix::ICEExecutionReportMsg, fix::BusinessMessageRejectMsg, fix::RejectMsg, 
-                           fix::ICECancelRejectMsg, fix::ICETraderLoginResponseMsg, fix::ICETraderLogoutResponseMsg>;
-
-
-
-struct ICEExecutionGateway::PImpl 
-{
-   PImpl(const config::JetFIX::Connection &cfg, trace::PerfLog &l,
-         md::SecurityOracle o)
-       : gw(cfg, l), exe_report_pool(), state(State::WAITING_FOR_SECURITIES),
-         securities(), securities_by_exchange_id(), gw_name(cfg.name()),
-         cfg(cfg), perf_log(l), admin_data(), nusers(-1), connection_time(), exec_id_to_memo(), 
-         sec_oracle(o)
-   {}
-
-   JFGw gw;
-   boost::object_pool<ExecutionReport> exe_report_pool;
-   State state;
-   std::unordered_map<SecurityKey::Value, api::Security const *> securities;
-   std::unordered_map<int64_t, api::Security const *> securities_by_exchange_id;
-   GatewayName gw_name;
-   config::JetFIX::Connection cfg;
-   OnExeReport on_exe_report;
-   OnGwStatusChange on_gw_status_change;
-   OnCancelReject on_cancel_reject;
-   trace::PerfLog &perf_log;
-   api::admin::AdminData admin_data;
-   std::atomic<int> nusers;
-   TimeStamp connection_time;
-   std::unordered_map<std::string, std::string> exec_id_to_memo; //for the bullshit adjusts
-   md::SecurityOracle sec_oracle;
-};
-
-ICEExecutionGateway::ICEExecutionGateway(const config::JetFIX::Connection &cfg, trace::PerfLog &log, md::SecurityOracle o)
-    : IExecutionGateway(), me(new PImpl(cfg, log, o)) 
-{
-   me->state = State::WAITING_FOR_ADMIN;
-}
-
-ICEExecutionGateway::~ICEExecutionGateway() {}
-
-void ICEExecutionGateway::on_admin_data(const mercury::Update<NewAdminDataEvent>& update) 
-{
-   if (update.events) 
-   {
-      for (auto const &ev : *(update.events)) 
-         me->admin_data.CopyFrom(*ev.admin_data);
-
-      if (me->state == State::WAITING_FOR_ADMIN) 
-      {
-         LOG(OrderMan, Warn, "Received admin data. Waiting for securities {}", me->gw_name.value());
-         me->state = State::WAITING_FOR_SECURITIES;
-      }
-   }   
-}
-
-void ICEExecutionGateway::on_securities(const mercury::Update<NewSecuritiesEvent> &update) 
-{
-   if (update.events) 
-   {
-      for (auto const &ev : *(update.events)) 
-      {
-         auto sec_ptr = ev.security;
-         me->securities[sec_ptr->id()] = sec_ptr;
-         if(sec_ptr->has_ice())
-            me->securities_by_exchange_id[sec_ptr->ice().exchange_id()] = sec_ptr;
-      }
-
-      if (me->state == State::WAITING_FOR_SECURITIES) 
-      {
-         LOG(OrderMan, Warn, "Received first set of {} securities. Beginning wait 2s for trader login {}",
-             update.events->size(), me->gw_name.value());         
-         // got our security synopsis, ready to start the GW
-         me->state = State::READY_TO_START_GW;
-         
-         // subscribe to gateway events
-         me->gw.on<fix::ICEExecutionReportMsg>() = [this](auto const &msg) 
-         {
-            this->on_exe_report(msg);
-         };
-         me->gw.on<fix::ICECancelRejectMsg>() = [this](auto const &msg) 
-         {
-            this->on_cancel_reject(msg);
-         };
-         me->gw.on_ready() = [this]() 
-         {
-            this->on_connection_status_change(true);
-         };
-         me->gw.on_disconnect() = [this]() 
-         {
-            this->on_connection_status_change(false);
-         };
-         me->gw.on<fix::BusinessMessageRejectMsg>() = [this](auto const &m) 
-         {
-            this->on_business_reject(m);
-         };
-         me->gw.on<fix::RejectMsg>() = [this](auto const &m) 
-         {
-            this->on_session_reject(m);
-         };            
-      }
-   }
-}
-
-void ICEExecutionGateway::start() 
-{ 
-   me->gw.start(); 
-   bts_assert_msg(me->state == State::READY_TO_START_GW, "ICEExecutionGateway statemachine failure");
-}
-
-void ICEExecutionGateway::on_connection_status_change(bool is_connected) 
-{
-   LOG(OrderMan, Info, "Connection state of {} changed to {}",
-       me->gw_name.value(), is_connected ? std::string("connected") : std::string("disconnected"));
-   
-   if(is_connected && me->state == State::READY_TO_START_GW)
-   {
-      me->state = State::TRADER_LOGIN_TIMEOUT;
-      me->connection_time = Timer::now();
-   }   
-   
-   if(me->state == State::READY)
-   {
-      me->on_gw_status_change(is_connected);      
-      if(!is_connected) //if we disconnect, set state so we log in again
-         me->state = State::READY_TO_START_GW;
-   }
-}
-
-
-void ICEExecutionGateway::on_trader_login_timeout_expired()
-{
-   if (me->state == State::TRADER_LOGIN_TIMEOUT) 
-   {
-      LOG(OrderMan, Warn, "Done waiting for login timeout, logging on traders {}", me->gw_name.value());
-
-      // got our security synopsis, ready to start the GW
-      me->state = State::LOGIN_TIMEOUT_COMPLETE;      
-      me->gw.on<fix::ICETraderLoginResponseMsg>() = [&](const fix::ICETraderLoginResponseMsg& lmsg) 
-      {
-         std::string u = lmsg.get<fix::UserName>();
-         std::string m;
-         if(lmsg.has<fix::Text>())
-            m = lmsg.get<fix::Text>();         
-         LOG(OrderMan, Error, "Succesffully logging trader {} into ICE {} {}", u, m, me->gw_name.value());
-         if(--me->nusers == 0)
-            this->on_login();
-      };
-      
-      me->gw.on<fix::ICETraderLogoutResponseMsg>() = [&](const fix::ICETraderLogoutResponseMsg& lmsg) 
-      {
-         std::string u = lmsg.get<fix::UserName>();
-         std::string m;
-         if(lmsg.has<fix::Text>())
-            m = lmsg.get<fix::Text>();
-         
-         LOG(OrderMan, Error, "Error logging trader {} into ICE {} {}", u, m, me->gw_name.value());
-         if(--me->nusers == 0)
-            this->on_login();
-      };    
-       
-      std::set<std::string> auth_traders;
-      for(const auto& u : me->admin_data.users())
-      {         
-         if(u.has_user_data() && u.user_data().has_exchange_user_data() && u.user_data().exchange_user_data().has_ice_data())
-         {
-            const auto& ice = u.user_data().exchange_user_data().ice_data();
-            if(auth_traders.find(ice.authorized_trader()) == auth_traders.end())
-            {
-               if(me->nusers == -1)
-                  me->nusers = 0;
-               me->nusers++;
-               
-               auto result = std::make_shared<OutFIX>(me->cfg);
-               auto &stream = result->fix.back();
-               auto &buffer = stream.new_msg(MessageTypes::ICETraderLogin);               
-               buffer.add<fix::CompanyID>(ice.company_id());      
-               buffer.add<fix::UserName>(ice.authorized_trader());      
-               buffer.add<fix::RawData>(ice.password());     
-               
-               if(ice.mifid_direct_electronic_access() >= 0)
-                  buffer.add<fix::MifidDirectElectronicAccess>(ice.mifid_direct_electronic_access());  
-               if(ice.mifid_trading_capacity() >= 0)   
-                  buffer.add<fix::MifidTradingCapacity>(ice.mifid_trading_capacity());     
-               if(ice.mifid_liquidity_provision() >= 0)
-                  buffer.add<fix::MifidLiquidityProvision>(ice.mifid_liquidity_provision());     
-               if(ice.mifid_commodity_deriv_indicator() >= 0)
-                  buffer.add<fix::MifidCommodityDerivIndicator>(ice.mifid_commodity_deriv_indicator());     
-               if(ice.mifid_investment_decision() > 0)
-                  buffer.add<fix::MifidInvestmentDecision>(ice.mifid_investment_decision());     
-               if(ice.mifid_execution_decision() > 0)
-                  buffer.add<fix::MifidExecutionDecision>(ice.mifid_execution_decision());     
-               if(ice.mifid_client_id_code() > 0)
-                  buffer.add<fix::MifidClientIDCode>(ice.mifid_client_id_code());                   
-               if(ice.mifid_id() > 0)
-                  buffer.add<fix::MifidId>(ice.mifid_id());                   
-               stream.close();
-               
-               trace::CorID trace;            
-               send_fix(result, trace);
-               auth_traders.insert(ice.authorized_trader());
-            }
-         }               
-      }
-      LOG(OrderMan, Warn, "{} traders identified as ICE traders {}", auth_traders.size(), me->gw_name.value());               
-   }
-}
-
-void ICEExecutionGateway::on_login()
-{
-   if (me->state == State::LOGIN_TIMEOUT_COMPLETE) 
-   {
-      LOG(OrderMan, Warn, "Successful login traders {}", me->gw_name.value());
-      me->state = State::READY;     
-      me->on_gw_status_change(true);                
-   }   
-}
-
-OnExeReport &ICEExecutionGateway::on_exe_report() { return me->on_exe_report; }
-
-OnGwStatusChange &ICEExecutionGateway::on_gw_status_change() 
-{
-   return me->on_gw_status_change;
-}
-
-OnCancelReject &ICEExecutionGateway::on_cancel_reject() 
-{
-   return me->on_cancel_reject;
-}
-
-bool ICEExecutionGateway::process_memo(const jetfix::fix::ICEExecutionReportMsg &msg, OrderKey* ok, SecurityKey* sk)
-{
-   if(msg.has<fix::Memo>())
-   {
-      decode_memo(msg.get<fix::Memo>(), ok, sk);   
-      return true;
-   }
-   else
-   {
-      if (msg.get<fix::ExecType>() == api::orderman::EXEC_TYPE_TRADE_CORRECT && msg.has<fix::ExecRefID>()) 
-      {
-         std::string ref_exec_id = msg.get<fix::ExecRefID>();
-         auto found = me->exec_id_to_memo.find(ref_exec_id);
-         if(found == me->exec_id_to_memo.end()) 
-         {
-            auto new_price = Price(0);
-            if(msg.has<fix::LastPx>())
-               new_price = Price(msg.get<fix::LastPx>());
-            
-            LOG(OrderMan, Error,
-               "Received unsupported trade correction msg.  This correction will not be forwarded. The trader should be alerted. "
-               "Partial original fill id: {}. New price: {}", ref_exec_id, new_price.value());
-            return false;
-         }
-         else
-         {
-            decode_memo(found->second, ok, sk);
-            return true;
-         }
-      }      
-   }
-   return false;      
-}
-
-void ICEExecutionGateway::on_exe_report(const jetfix::fix::ICEExecutionReportMsg &msg) 
-{
-   OrderKey ok;
-   SecurityKey sk;
-   if(!process_memo(msg, &ok, &sk))
-   {
-      std::string execid = "";
-      if(msg.has<fix::ExecID>())
-         execid = msg.get<fix::ExecID>();
-      LOG(OrderMan, Error,
-         "Received execution report with no memo execid: {}. Will not be able to process this report. This is likely a bug that could lead "
-         "to lost fills. Please investigate.", execid);
-      return;      
-   }
-   auto scoped_log = trace::log_scope(me->perf_log, msg.trace(), "ICEExecutionGateway::on_exe_report", ok.value());
-
-   LOG(Latency, Info, "Started handling of execution report for order {}", ok.value());
-   auto &sec = me->securities[sk.value()];
-   if (!sec) 
-   {
-      sec = me->sec_oracle.get_security(SecurityKey(sk.value()));
-   }   
-   if (!sec)
-      LOG(OrderMan, Error,
-          "Received execution report for an unknown security {}. Will not be able to process this report. This is likely a bug that could lead "
-          "to lost fills. Please investigate.",
-          sk.value());
-   else 
-   {
-      if(sec->has_ice())
-         me->securities_by_exchange_id[sec->ice().exchange_id()] = sec;
-      
-      api::Security const *leg_security = nullptr;
-      if (msg.has<fix::MultiLegReportingType>() && msg.get<fix::MultiLegReportingType>() == 2) 
-      {
-         auto it = me->securities_by_exchange_id.find(msg.get<fix::Int_Symbol>());
-         if (it == me->securities_by_exchange_id.end()) 
-         {
-            LOG(OrderMan, Error,
-                "Received execution report for an unknown spread leg"
-                "security {}. Will not be able to process this report. "
-                "This is likely a bug that could lead to "
-                "lost leg fills. Please investigate. Spread sec_key: {} leg "
-                "exchange id: {}",
-                sk.value(), msg.get<fix::Int_Symbol>());
-            return;
-         }
-         leg_security = it->second;
-      }      
-      std::string text_msg;
-      if (msg.has<fix::Text>())
-         text_msg = msg.get<fix::Text>();
-      if (msg.has<fix::OrdRejReason>())
-         text_msg += " Rejected OrdRejReason: " + std::to_string(msg.get<fix::OrdRejReason>());
-         
-      if (msg.has<fix::DeleteReason>())
-         text_msg += " Cancelled DeleteReason: " + std::to_string(msg.get<fix::DeleteReason>());
-
-      // allocate from our pool
-      auto data = me->exe_report_pool.construct();
-
-      if(msg.has<fix::Memo>())
-         data->memo = msg.get<fix::Memo>();
-      data->exec_type = api::orderman::ExecType(msg.get<fix::ExecType>());
-      data->msg_seq_num = msg.get<fix::MsgSeqNum>();
-
-      // copy data from fix msg to our structure
-      if (msg.has<fix::ClearingAccount>())
-         str_to(msg.get<fix::ClearingAccount>(), &data->account_name);
-      if (msg.has<fix::TargetSubID>())
-         str_to(msg.get<fix::TargetSubID>(), &data->operator_id);
-      if (msg.has<fix::CompanyID>())
-         data->ice_company_id = msg.get<fix::CompanyID>();         
-
-      data->order_key = ok;
-      data->security = sec;
-      data->leg_security = leg_security;
-      data->side = api::Side(msg.get<fix::Side>());
-      if (msg.has<fix::TimeInForce>())
-         data->time_in_force = api::orderman::TimeInForce(msg.get<fix::TimeInForce>());
-      else
-         data->time_in_force = api::orderman::TIME_IN_FORCE_UNDEFINED;
-
-      if (msg.has<fix::OrdType>())
-         data->order_type = api::orderman::NativeOrderType(msg.get<fix::OrdType>());
-      else
-         data->order_type = api::orderman::NATIVE_ORDER_TYPE_UNDEFINED;
-         
-      data->order_status = api::orderman::NativeOrderStatus(msg.get<fix::OrdStatus>());
-      data->manual_order_indicator = api::orderman::MANUAL_ORDER_INDICATOR_NO;
-      
-      std::string oid_string = get_ice_order_id_string(msg);
-      str_to(oid_string, &data->exchange_order_id);
-      data->ice_cl_ord_id = 0;
-      if(msg.has<fix::Int_ClOrdID>())
-         data->ice_cl_ord_id = msg.get<fix::Int_ClOrdID>();
-      
-      if (msg.has<fix::Price>())
-         data->price = securities::decode_exchange_price(*sec, Price(msg.get<fix::Price>()));
-      else
-         data->price = Price(os::nan());
-
-      if (msg.has<fix::StopPx>())
-         data->stop_price = securities::decode_exchange_price(*sec, Price(msg.get<fix::StopPx>()));
-      else
-         data->stop_price = Price(os::nan());
-         
-      if (msg.has<fix::MaxShow>())
-         data->max_show = Quantity(msg.get<fix::MaxShow>());
-      else
-         data->max_show = Quantity(0);         
-
-      if (msg.has<fix::OrderQty>())
-         data->quantity = Quantity(msg.get<fix::OrderQty>());
-      else
-         data->quantity = Quantity(0);
-
-      data->filled_quantity = Quantity(msg.get<fix::CumQty>());
-      
-      data->last_shares = msg.has<fix::LastShares>()
-                              ? Quantity(msg.get<fix::LastShares>())
-                              : Quantity(0);
-      auto last_px_sec = data->leg_security ? data->leg_security : sec;
-      data->last_price = msg.has<fix::LastPx>()
-                             ? securities::decode_exchange_price(*last_px_sec, Price(msg.get<fix::LastPx>()))
-                             : Price(0);
-      data->avg_price = msg.has<fix::AvgPx>()
-                            ? securities::decode_exchange_price(*last_px_sec, Price(msg.get<fix::AvgPx>()))
-                            : Price(0);
-      if (msg.has<fix::ExecID>())
-      {
-         auto execid = msg.get<fix::ExecID>();
-         str_to(execid, &data->exchange_fill_id);
-         if(msg.has<fix::Memo>())
-            me->exec_id_to_memo[execid] = msg.get<fix::Memo>();
-      }
-               
-      if (msg.has<fix::TransactTimeStr>())
-      {
-         std::string stamp = msg.get<fix::TransactTimeStr>();
-         stamp.resize(sizeof(mercury::TimeStampStr));
-         str_to(stamp, &data->transact_time);
-      }
-      data->cfg = &me->cfg;
-      data->msg = text_msg;
-      data->exchange = api::Venue::VENUE_ICE;
-      me->on_exe_report(*data, msg.trace());
-   }
-}
-
-void ICEExecutionGateway::on_cancel_reject(const jetfix::fix::ICECancelRejectMsg &msg) 
-{
-   OrderKey ok(msg.get<fix::Int_ClOrdID>());
-
-   std::string reject_msg;
-   if (msg.has<fix::Text>())
-      reject_msg = msg.get<fix::Text>();
-   else if (msg.has<fix::CXlRejReason>())
-      reject_msg = fmt::format("Order cancel was rejected with error code {}",
-                               msg.get<fix::CXlRejReason>());
-   me->on_cancel_reject(ok, reject_msg);
-}
-
-
-void ICEExecutionGateway::on_business_reject(const jetfix::fix::BusinessMessageRejectMsg& m) 
-{
-   LOG(OrderMan, Error,
-       "Unexpected business reject in {} gateway . Reason: {}. This should be investigated",
-       me->gw_name.value(),
-       m.has<fix::Text>() ? m.get<fix::Text>() : std::string("Unknown"));
-}
-
-void ICEExecutionGateway::on_session_reject(const jetfix::fix::RejectMsg& m) 
-{
-   LOG(OrderMan, Error,
-       "Unexpected reject message in {} gateway . Reason: {}. This should be investigated",
-       me->gw_name.value(),
-       m.has<fix::Text>() ? m.get<fix::Text>() : std::string("Unknown"));
-}
-
-void ICEExecutionGateway::dispatch_incoming_fix_messages() 
-{
-   me->gw.dispatch_events();   
-   if(me->state == State::TRADER_LOGIN_TIMEOUT && ms_diff(Timer::now(), me->connection_time) >= 2000)
-   {
-      on_trader_login_timeout_expired();
-   }
-}
-
-void ICEExecutionGateway::send_fix(const jetfix::OutFIXPtr& fix, trace::CorID trace) 
-{
-   auto scoped_log = trace::log_scope(me->perf_log,trace,"ICEExecutionGateway::send_fix");
-   me->gw.send(fix,trace);
-   me->gw.dispatch_events();
-}
-
-} // namespace orderman
-} // namespace bts

src/lib/orderman/actors/ICEExecutionGateway.h

@@ -1,45 +0,0 @@
-#pragma once
-
-#include "IExecutionGateway.h"
-
-namespace bts {
-namespace orderman {
-
-// fix execution gateway for a single FIX session
-class ICEExecutionGateway : public IExecutionGateway {
- public:   
-   ICEExecutionGateway(const config::JetFIX::Connection&, trace::PerfLog &,
-                       md::SecurityOracle);
-   virtual ~ICEExecutionGateway();
-
-   void start() override;
-   void dispatch_incoming_fix_messages() override;
-   void on_securities(const mercury::Update<NewSecuritiesEvent> &) override;
-   void on_admin_data(const mercury::Update<NewAdminDataEvent> &) override;
-   void send_fix(const jetfix::OutFIXPtr& fix, trace::CorID trace) override;
-
-   OnExeReport& on_exe_report() override;
-   OnGwStatusChange& on_gw_status_change() override;
-   OnCancelReject &on_cancel_reject() override;
-   
- private:
-   BTS_NO_COPY(ICEExecutionGateway);
-   enum class State { WAITING_FOR_ADMIN, WAITING_FOR_SECURITIES, READY_TO_START_GW, TRADER_LOGIN_TIMEOUT, LOGIN_TIMEOUT_COMPLETE, READY };
-   struct PImpl;
-   std::unique_ptr<PImpl> me;
-   
-   void on_trader_login_timeout_expired();
-   void on_login();
-
-   // fix gateway events
-   void on_exe_report(const jetfix::fix::ICEExecutionReportMsg&);
-   void on_cancel_reject(const jetfix::fix::ICECancelRejectMsg&);
-   void on_connection_status_change(bool connected);
-   void on_business_reject(const jetfix::fix::BusinessMessageRejectMsg&);
-   void on_session_reject(const jetfix::fix::RejectMsg&);
-   
-   bool process_memo(const jetfix::fix::ICEExecutionReportMsg &msg, OrderKey*, SecurityKey*);
-};
-
-} // namespace orderman
-} // namespace bts

src/lib/orderman/actors/IExecutionGateway.cpp

@@ -1,20 +0,0 @@
-
-#include "IExecutionGateway.h"
-#include "ICEExecutionGateway.h"
-#include "GLBXExecutionGateway.h"
-
-namespace bts {
-namespace orderman {
-
-std::unique_ptr<IExecutionGateway>
-ExecutionGatewayFactory::create_gateway(const config::JetFIX::Connection& conn, trace::PerfLog& pl, md::SecurityOracle o)
-{
-   if(conn.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_ICE || 
-      conn.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_SIMICE)
-      return std::make_unique<ICEExecutionGateway>(conn, pl, o);
-   else
-      return std::make_unique<GLBXExecutionGateway>(conn, pl, o);
-}
-
-} // namespace orderman
-} // namespace bts

src/lib/orderman/actors/IExecutionGateway.h

@@ -1,52 +0,0 @@
-#pragma once
-
-#include "lib/config/protos/jetfix.pb.h"
-#include "lib/jetfix/codec/MessageDefinitions.h"
-#include "lib/jetfix/common/OutFIXPtr.h"
-#include "lib/md/SecurityOracle.h"
-#include "lib/mercury/events/Update.h"
-#include "lib/orderman/orderman_utils/orderman_common.h"
-#include "lib/utils/bts_no_copy.h"
-
-#include <memory>
-
-namespace bts {
-namespace trace {struct CorID;class PerfLog;}
-namespace orderman {
-class NewSecuritiesEvent;
-class NewAdminDataEvent;
-class ExecutionReport;
-
-
-using OnGwStatusChange = std::function<void(bool)>;
-using OnExeReport = std::function<void(ExecutionReport const &, trace::CorID const &)>;
-using OnCancelReject = std::function<void(OrderKey, std::string const &)>;
-
-// fix execution gateway for a single FIX session
-class IExecutionGateway {
- public:   
-   IExecutionGateway() = default;
-   virtual ~IExecutionGateway() {}
-
-   virtual void start() = 0;
-   virtual void dispatch_incoming_fix_messages() = 0;
-   virtual void on_securities(const mercury::Update<NewSecuritiesEvent> &) = 0;
-   virtual void on_admin_data(const mercury::Update<NewAdminDataEvent> &) = 0;
-   virtual void send_fix(const jetfix::OutFIXPtr&, trace::CorID) = 0;
-   
-   virtual OnExeReport& on_exe_report() = 0;
-   virtual OnGwStatusChange& on_gw_status_change() = 0;
-   virtual OnCancelReject &on_cancel_reject() = 0;
-private:
-   BTS_NO_COPY(IExecutionGateway);
-};
-
-class ExecutionGatewayFactory
-{
-public:
-   static std::unique_ptr<IExecutionGateway> create_gateway(const config::JetFIX::Connection&, trace::PerfLog&, md::SecurityOracle);
-   
-};
-
-} // namespace orderman
-} // namespace bts

src/lib/orderman/actors/order_manager/OrderManager.cpp

@@ -15,7 +15,6 @@
 #include "lib/orderman/orders/Order.h"
 #include "lib/orderman/securities/ISecurityDB.h"
 #include "lib/utils/inplace_function.h"
-#include "lib/utils/thread_manager.h"
 
 #include <algorithm>
 #include <boost/algorithm/string/trim.hpp>
@@ -45,28 +44,12 @@ struct OrderManager::PImpl {
    mercury::Actor::SubHandle filled_position_sub;
    OrderServicesData services_data;
    mercury::Stream<OrderManagerReadyEvent> pub_ready;
-   std::thread background_thread;
-   std::atomic<bool> shutdown = false;
 };
 
 OrderManager::OrderManager(DynamicOrderRegistry const &b, trace::PerfLog &l,
                            md::SecurityOracle o)
-    : me(new PImpl(*this, b, l, o)) {
-   auto id_gen = &me->services_data.id_gen;
-   auto done = &me->shutdown;
-   me->background_thread =
-       threading::make_thread("oman_background", [id_gen, done]() {
-          while (!(*done)) {
-             id_gen->refill();
-             sleep_for(10ms);
-          }
-       });
-}
-
-OrderManager::~OrderManager() {
-   me->shutdown = true;
-   me->background_thread.join();
-}
+    : me(new PImpl(*this, b, l, o)) {}
+OrderManager::~OrderManager() {}
 
 void OrderManager::init() {
    subscribe<NewAdminDataEvent>(
@@ -80,13 +63,11 @@ void OrderManager::init() {
 
 void OrderManager::on_admin_data(const mercury::Update<NewAdminDataEvent> &m) {
    if (m.events) {
+
       if (me->state == State::WAIT_ADMIN) {
          LOG(OrderMan, Info,
-             "Order manager received admin data. Subscribing to security list "
-             "now ");
-         for (auto &osd : me->services_data.sessions)
-            osd->gw->on_admin_data(m);
-
+             "Order manager received admin data. Subscribing "
+             "to security list now ");
          me->state = State::WAIT_SEC;
          subscribe<NewSecuritiesEvent>([this](auto const &sec_update) {
             this->on_securities(sec_update);
@@ -119,7 +100,7 @@ void OrderManager::on_securities(
 
    // forward to execution gateways
    for (auto &osd : me->services_data.sessions)
-      osd->gw->on_securities(update);
+      osd->gw.on_securities(update);
 
    if (update.events) {
 
@@ -459,20 +440,19 @@ void OrderManager::init_order_services() {
       // reject evets
       auto s_ptr = s.get();
       s->exe_report_pub = make_stream<NewExecutionReportEvent>(gw_name);
-      s->gw->on_exe_report() = [s_ptr, this](auto const &er, auto const &t) {
+      s->gw.on_exe_report() = [s_ptr, this](auto const &er, auto const &t) {
          publish(s_ptr->exe_report_pub, &er, t);
       };
       s->gw_status_change_pub = make_stream<GatewayStatusChangeEvent>(gw_name);
-      s->gw->on_gw_status_change() = [s_ptr, this, gw_name](bool b) {
+      s->gw.on_gw_status_change() = [s_ptr, this, gw_name](bool b) {
          publish(s_ptr->gw_status_change_pub, gw_name, b);
       };
       s->cancel_reject_pub = make_stream<CancelRejectEvent>(gw_name);
-      s->gw->on_cancel_reject() = [s_ptr, this](auto const &k,
-                                                auto const &msg) {
+      s->gw.on_cancel_reject() = [s_ptr, this](auto const &k, auto const &msg) {
          publish(s_ptr->cancel_reject_pub, k, msg);
       };
       // handlers are ready so we can start processing fix events
-      s->gw->start();
+      s->gw.start();
 
       reload_fix_buffer(*s);
    }
@@ -561,7 +541,7 @@ void OrderManager::after_event() {
             auto scoped_do_work = trace::log_scope(me->services_data.log, {},
                                                    "OrderManager::after_event");
             fix_stream.close();
-            osd->gw->send_fix(osd->current_buffer, osd->current_trace);
+            osd->gw.send_fix(osd->current_buffer, osd->current_trace);
             reload_fix_buffer(*osd);
          }
       }
@@ -586,7 +566,7 @@ void OrderManager::on_poll(trace::PerfLog &log) {
        trace::log_scope(log, {}, "OrderManager::dispatch_incoming_fix_messages",
                         utils::TimeStamp::Interval::from_microseconds(2));
    for (auto &osd : me->services_data.sessions)
-      osd->gw->dispatch_incoming_fix_messages();
+      osd->gw.dispatch_incoming_fix_messages();
 }
 
 void OrderManager::publish_order_def_update(

src/lib/orderman/actors/order_manager/OrderServices.h

@@ -47,9 +47,6 @@ class OrderServices {
       return ss.is_gateway_connected();
    }
 
-   uint64_t gen_id() {
-      return _data->id_gen.next();
-   }
    Governor &governor(SessionServices &ss) { return ss.governor(); }
 
    void queue_fix(SessionServices &ss, jetfix::MessageTypes mt,

src/lib/orderman/actors/order_manager/OrderServicesData.cpp

@@ -12,9 +12,7 @@ OrderServicesData::OrderServicesData(OrderManager &owner,
       order_factory(sec_db, order_router, b),
       last_timer_ev(
           Timer::now(),
-          to<mercury::TimeStampStr>(datetime::FIX_timestamp(datetime::now()))),
-      id_gen("orderman_ids",5000)
-{
+          to<mercury::TimeStampStr>(datetime::FIX_timestamp(datetime::now()))) {
 
    auto &all_connections = config::get<config::JetFIX>().connection();
    for (auto const &c : all_connections)

src/lib/orderman/actors/order_manager/OrderServicesData.h

@@ -9,7 +9,7 @@
 
 #include "lib/mercury/events/TimerEvent.h"
 #include "lib/md/SecurityOracle.h"
-#include "lib/orderman/actors/IExecutionGateway.h"
+#include "lib/orderman/actors/ExecutionGateway.h"
 #include "lib/orderman/events/ActivateOrderEvent.h"
 #include "lib/orderman/events/BookUpdateEvent.h"
 #include "lib/orderman/events/TickerUpdateEvent.h"
@@ -34,7 +34,6 @@
 #include "lib/orderman/limiter/Limiter.h"
 #include "lib/orderman/order_router/OrderRouter.h"
 #include "lib/utils/datetime.h"
-#include "lib/utils/Puid.h"
 
 #include <boost/date_time/posix_time/posix_time.hpp>
 
@@ -53,10 +52,7 @@ struct SessionServicesData {
                        trace::PerfLog &log, md::SecurityOracle o)
        : session(cfg.name()), gw_connected(false), cfg(cfg),
          governor(cfg.max_order_msgs_per_window(), cfg.window_duration_msecs()),
-         gw(ExecutionGatewayFactory::create_gateway(cfg, log, o)) 
-   {
-      
-   }
+         gw(cfg, log,o) {}
 
    std::string session;
    std::vector<jetfix::OutFIXPtr> free_buffers;
@@ -73,7 +69,7 @@ struct SessionServicesData {
    Governor governor;
    jetfix::OutFIXPtr current_buffer;
    trace::CorID current_trace;
-   std::unique_ptr<IExecutionGateway> gw;
+   ExecutionGateway gw;
    mercury::Stream<NewExecutionReportEvent> exe_report_pub;
    mercury::Stream<GatewayStatusChangeEvent> gw_status_change_pub;
    mercury::Stream<CancelRejectEvent> cancel_reject_pub;
@@ -111,8 +107,7 @@ class OrderServicesData {
    mercury::Stream<OrderOpResultEvent> report_order_op_result_pub;
    mercury::TimerEvent last_timer_ev;
    std::vector<std::unique_ptr<SessionServicesData>> sessions;
-   utils::IDGenerator id_gen ;
-   
+
    SessionServicesData &get_session(std::string const &session_name) {
       for (auto &s : sessions)
          if (s->session == session_name)

src/lib/orderman/actors/order_manager/SessionServices.h

@@ -57,14 +57,14 @@ class SessionServices {
       _gateway_status_sub = src._gateway_status_sub;
       return *this;
    }
-   
-   config::JetFIX::Connection const &session_cfg() const {
-      return _session_data->cfg;
-   }   
 
  protected:
    std::string const &session_name() const { return _session_data->session; }
 
+   config::JetFIX::Connection const &session_cfg() const {
+      return _session_data->cfg;
+   }
+
    void queue_fix(jetfix::MessageTypes mt, jetfix::FIXFragment const &f,
                   trace::CorID const &corid) {
       auto &stream = _session_data->current_buffer->first_stream();

src/lib/orderman/order_factory/OrderFactory.cpp

@@ -10,8 +10,7 @@
 // order types
 #include "lib/orderman/order_factory/DynamicOrderRegistry.h"
 #include "lib/orderman/orders/DynamicOrder.h"
-#include "lib/orderman/orders/GLBXNativeOrder.h"
-#include "lib/orderman/orders/ICENativeOrder.h"
+#include "lib/orderman/orders/NativeOrder.h"
 #include "lib/orderman/orders/TimeBoxOrder.h"
 #include "lib/orderman/orders/TriggerOrder.h"
 #include "lib/orderman/orders/algo/AlgoWrapperOrder.h"
@@ -89,7 +88,7 @@ Order &OrderFactory::create(OrderKey parent,
    // field correctly. Algos will just propagate the value of this
    // field down so eventually native orders launched by algos will
    // get the right value.
-   if (dynamic_cast<NativeOrderBase *>(&order) == nullptr &&
+   if (dynamic_cast<NativeOrder *>(&order) == nullptr &&
        def.manual_order_indicator() ==
            api::orderman::MANUAL_ORDER_INDICATOR_YES) {
       // An algo order incorrectly marked as a manual order
@@ -114,24 +113,6 @@ Order &OrderFactory::recover(api::orderman::OrderRecord const &rec) {
    return order;
 }
 
-NativeOrderBase *OrderFactory::make_order(const api::Venue exchange) {
-   if (exchange == api::Venue::VENUE_ICE ||
-       exchange == api::Venue::VENUE_SIMICE)
-      return new ICENativeOrder();
-   else
-      return new GLBXNativeOrder();
-}
-
-NativeOrderBase *
-OrderFactory::make_native_order(const api::orderman::OrderDefinition &def) {
-   auto secptr = me->sec_db.resolve(SecurityKey(def.security_key()));
-   if (secptr == nullptr)
-      return new GLBXNativeOrder(); // GLBX Native can handle unresolved
-                                    // securities
-   else
-      return make_order(secptr->exchange());
-}
-
 Order &OrderFactory::recreate_from_exe_report(ExecutionReport const &r) {
    using namespace api::orderman;
 
@@ -140,7 +121,7 @@ Order &OrderFactory::recreate_from_exe_report(ExecutionReport const &r) {
    case NATIVE_ORDER_TYPE_LIMIT:
    case NATIVE_ORDER_TYPE_STOP:
    case NATIVE_ORDER_TYPE_STOP_LIMIT: {
-      std::unique_ptr<NativeOrderBase> native_order(make_order(r.exchange));
+      auto native_order = std::make_unique<NativeOrder>();
       native_order->_key = r.order_key;
       native_order->_parent_key = null_order_key();
       auto def = native_order->on_init_from_exe_report(
@@ -170,7 +151,7 @@ Order &OrderFactory::allocate(OrderKey k, OrderKey pk,
    case CustomOrderDefinition::kStop:
    case CustomOrderDefinition::kStopLimit:
    case CustomOrderDefinition::kGlbxIceberg:
-      order_ptr.reset(make_native_order(def));
+      order_ptr.reset(new NativeOrder());
       break;
    case CustomOrderDefinition::kIceberg:
       order_ptr.reset(

src/lib/orderman/order_factory/OrderFactory.h

@@ -11,7 +11,6 @@ class ISecurityDB;
 }
 namespace orderman {
 class Order;
-class NativeOrderBase;
 class DynamicOrderRegistry;
 } // namespace orderman
 namespace orderman {
@@ -46,8 +45,6 @@ class OrderFactory : public IOrderFactory {
  private:
    Order &allocate(OrderKey k, OrderKey pk,
                    api::orderman::OrderDefinition const &);
-   NativeOrderBase* make_order(const api::Venue exchange);
-   NativeOrderBase* make_native_order(const api::orderman::OrderDefinition& def);   
    struct PImpl;
 
    std::unique_ptr<PImpl> me;

src/lib/orderman/orderman_utils/ExecutionReport.h

@@ -35,10 +35,6 @@ public:
    mercury::TimeStampStr transact_time;
    config::JetFIX::Connection const *cfg;
    std::string msg;
-   api::Venue exchange;
-   int64_t ice_cl_ord_id;
-   int32_t ice_company_id;
-   std::string memo;
 };
 
 }}

src/lib/orderman/orderman_utils/fix_order_codec.h

@@ -7,7 +7,6 @@
 #include "lib/jetfix/codec/i32_toa.h"
 #include "lib/utils/bts_assert.h"
 #include "orderman_common.h"
-#include "lib/utils/base62.h"
 
 namespace bts {
 namespace api {
@@ -20,8 +19,6 @@ constexpr int clordid_sz() { return 20; }
 constexpr int price_sz() { return 17; }
 constexpr int quantity_sz() { return 6; }
 constexpr int max_show_sz() { return 6; }
-constexpr int memo_sz() {return 12;}
-constexpr int ice_orderid_sz() {return 12;}
 constexpr uint32_t corid_sz() {
    return 16;
 } // 5 for channel, 1 separator, 10 sn
@@ -53,49 +50,6 @@ template <class MSG> OrderKey get_order_key(MSG const &msg) {
    return result;
 }
 
-inline void encode_memo(OrderKey ok, SecurityKey sk, std::string *out) 
-{
-   auto ok_string = utils::to_base_62(ok.value());
-   auto sk_string = utils::to_base_62(sk.value());
-   bts_assert_msg(ok_string.size() == 6 && sk_string.size() == 6, "base62 encoding didn't work");
-   *out = ok_string + sk_string;   
-}
-
-inline void decode_memo(const std::string& in, OrderKey* ok, SecurityKey* sk)
-{
-   bts_assert(in.size() == memo_sz());   
-   char buffer[memo_sz()/2+1];
-   buffer[memo_sz()/2] = 0;
-   std::memcpy(buffer,&in[0],memo_sz()/2);
-   *ok = OrderKey( utils::from_base_62(std::string(buffer)) );
-
-   std::memcpy(buffer,&in[memo_sz()/2],memo_sz()/2);
-   *sk = SecurityKey( utils::from_base_62(std::string(buffer)) );
-}
-
-template <class MSG>
-std::string get_ice_order_id_string(const MSG& msg)
-{   
-   bts_assert(msg.template has<jetfix::fix::Int_OrderID>());
-   std::string oid_string(ice_orderid_sz(), ' '); 
-   
-   int64_t id = msg.template get<jetfix::fix::Int_OrderID>();   
-   u64_toa(id, &(oid_string[0]), ice_orderid_sz() );
-   return oid_string;
-}
-
-template <class MSG>
-std::string get_ice_clord_id_string(const MSG& msg)
-{   
-   bts_assert(msg.template has<jetfix::fix::Int_ClOrdID>());
-   std::string oid_string(ice_orderid_sz(), ' '); 
-   
-   int64_t id = msg.template get<jetfix::fix::Int_ClOrdID>();   
-   u64_toa(id, &(oid_string[0]), ice_orderid_sz() );
-   return oid_string;
-}
-
-
 inline void encode_corid(const char *channel, uint32_t sn, std::array<char, 17> &out) {
    memset(&out[0], ' ', out.size());
    auto ch_sz = std::min(strlen(channel), (size_t)5);