Merge branch 'pnl-rounding-error' of bitbucket.org:bluetradesys/spark into ice_exec3

fixed bugs in mview when settle/last/low/high change without a bid or offer changing

doc/user/content/bts_spark/Spreader/_index.md

@@ -154,7 +154,7 @@ insert_anchor_links = "right"
 
         -  Pay Less - If enabled by checking the checkbox, the spreader will
             attempt to pay less when the quantity at the level of the calculated
-            hedge price is more than the threshold defined.  The hedge order is
+            hedge price is less than the threshold defined.  The hedge order is
             paid less by the “Payless Ticks” as defined per leg.
         -  Pay Up - If enabled by checking the checkbox, the spreader will pay
             up when the quantity at the level of the calculated hedge price is

doc/user/content/release_notes/_index.md

@@ -9,12 +9,12 @@ insert_anchor_links = "right"
   - Fills now include information about the algo that originated the fill
   - A new, more powerful PnL form is now available
 ### API Changes
-  - Added security state information (pre open, halted, normal, closed, etc) 
+  - Added security state information (pre open, halted, normal, closed, etc)
 ### Bug Fixes
   - Brought back the "select all" check box in the mini order book
   - Fixed an issue in the market data subsystem that on rare occasions could result in a startup crash
 ### Internal Changes
-  - Fixed an issue that was stopping us from interpreting SOD times as local rather than GMT 
+  - Fixed an issue that was stopping us from interpreting SOD times as local rather than GMT
 ## 2.9.3
 ### Internal Changes
   - Fixed system test scripts
@@ -30,7 +30,7 @@ insert_anchor_links = "right"
   - Avoided shipping production configuration defaults that lead to trying to download a position file for the development sftp server.
 ### Api
   - C# API sample now uses synchronous calls.
-  
+
 ## 2.9.0
 ### User Visible Changes
   - Improved Workspace management User Interface
@@ -38,11 +38,11 @@ insert_anchor_links = "right"
   - The Spark UI Window group now has a title in the Windows task bar
   - Added sound alerts for trade events, see Launcher -> Settings -> Alerts
   - Allowed composite algos that are attached to existing orders to identify fills that were already present prior to attachment
-  - Added import facilities for GMI position files 
+  - Added import facilities for GMI position files
   - Added the ability for risk managers to enter manual fills
   - Added "raw" P&L columns, that is columns that *do not* include spread legs in their accounting
 ### Bug Fixes
-  - Fixed the settings action menu (Load/Save/Reset to Factory) 
+  - Fixed the settings action menu (Load/Save/Reset to Factory)
   - Fixed the method used to determine if a security is expired to take into account the start of day time
 ### API Changes
   - Fixed an issue in the Client SDK that could lead to occasional failures to connect to the spark server as well as excessive memory usage.
@@ -79,10 +79,10 @@ insert_anchor_links = "right"
   - Made it easier to generate simple UI's for new order types
 ### API Changes
   - Dynamic orders can now use QuantityMeasure parameter types
-  
+
 ## 2.7.2
 ### User Visible Changes
-  - Changed the login dialog to avoid leaking workspace names from other users. 
+  - Changed the login dialog to avoid leaking workspace names from other users.
   - Added new Splitter algo. From the manual: The splitter is useful
     for pro-rata markets where you want to vary your exposure
     independent of the order quantity that you want to fill.  The
@@ -90,13 +90,13 @@ insert_anchor_links = "right"
     the order quantity but it will put exposed_qty total size on the
     market. Splitter favors order size over queue position.
 ### Internal Changes
-  - Made spark more resilient to lack of connectivity to the CME web site for downloading of templates. 
-  
+  - Made spark more resilient to lack of connectivity to the CME web site for downloading of templates.
+
 ## 2.7.1
 ### User Visible Changes
   - Fixed some issues with the Cover algo, where in some circumstances
     it would erroneously cancel launched hedges on completion.
-	
+
 ## 2.7.0
 ### User Visible Changes
   - Allowed algos to process "in-flight" fills while shutting down
@@ -115,25 +115,25 @@ insert_anchor_links = "right"
 ### User Visible Changes
   - Fixed an issue in the ladder where in some cases the mini order book could not be brought up from a badge
   - Fixed a rare race condition that could lead to erroneous limit enforcement
-  - Fixed an issue with the margin limits where an error could occur if an account name with more than 12 chars was used  
+  - Fixed an issue with the margin limits where an error could occur if an account name with more than 12 chars was used
 
 ## 2.6.1
 ### User Visible Changes
   - Fixed an issue in the ladder where an error could occur when trying to change ladder settings for the first time
-  
+
 ## 2.6.0
 ### User Visible Changes
   - In the ladder made last trade more distinctive when overlayed on the volumn column
-  - In the ladder, corrected display of volume tool tip 
+  - In the ladder, corrected display of volume tool tip
   - In the ladder, changed the consolidation of last trade to consider only the price ignoring aggressor changes
   - Added an option for the Top Algo to delete itself if within N ticks from top of book
   - Address a rare issue that could lead to an error report on UI exit
   - Added margin risk limits
   - Added the "Offset" algo. Offset prices a child algo at a parameterizable distance from the top of the book
-  - Avoided switching the current account and order type on open ladders when limits are changed 
+  - Avoided switching the current account and order type on open ladders when limits are changed
 ### Internal Changes
   - Ensure that admin logins are always allowed to login, regardless of keycode status.
-  
+
 ## 2.5.0
 ### User Visible Changes
   - Added loss limit risk check
@@ -163,13 +163,13 @@ insert_anchor_links = "right"
   - Made Cover order more independent from its hedging chidren:
      - Cover continues to operate even if an hedging child is canceled
 	 - When a cover is canceled it will detach from any running hedging children, instead of canceling them
-  - Changed Lean, Join and Cover algos to hide themselves in the ladder once they have completed their work 
+  - Changed Lean, Join and Cover algos to hide themselves in the ladder once they have completed their work
   - Changed manual fills to appear in day PnL rather than in the SOD records
   - Add buy/sell coloring to fill rows in fills window
   - Streamlined the mini order book:
     - Display of the mini order book popup is quicker
     - Only exchange orders are displayed
-	- Update buttons are only displayed for orders that are under control of an algo with the update applying to the entire algo, not just the exchange order 
+	- Update buttons are only displayed for orders that are under control of an algo with the update applying to the entire algo, not just the exchange order
     - We can now select multiple rows to attach, detach, update or cancel multiple orders in a batch
     - A checkbox column has been added to make clearer which rows are selected
 ### API Changes
@@ -179,7 +179,7 @@ insert_anchor_links = "right"
   - Added system requirements to Spark documentation
   - Implemented keycodes
   - Temporarily rejected stop and stop limit orders, pending bug fixes
-  
+
 ## 2.2.0
 ### User Visible Changes
   - Made our orders easier to spot in MBO display
@@ -190,30 +190,30 @@ insert_anchor_links = "right"
   - Improved handling of orders with undiscovered securities
   - Added feedback to the ladder when the user attempts to middle click on bid/ask columns using an order that does not have parameters
   - Fixed display of algo studio algo types in order book
-  - Hid net change setting in synthetic ladders 
+  - Hid net change setting in synthetic ladders
   - Fixed a number of problems with filtering on the Fills form
   - On the Lean algo, OnToward and Away are now calculated with respect to the opposite inside market at time of the order launch
   - Iceberg Spreaders with queue holders are not erroneously cancelled after the first fill
   - Added security symbol column to fills window
   - Re-organized ladder settings dialog to make it clearer and easier to use
-  
+
 ## 2.1.4
 ### User Visible Changes
   - Fixed handling of cancel button in security picker
-  
+
 ## 2.1.3
 ### User Visible Changes
-  - Fixed a simulator bug in the handling of 0 sized orders 
+  - Fixed a simulator bug in the handling of 0 sized orders
 
 ## 2.2.1
 ### User Visible Changes
   - Fixed order book form exception when display a spreader order for whichs the legs were not present yet
-  
+
 ## 2.1.1
 ### User Visible Changes
   - Added PIQ to GLBX Iceberg orders
   - Corrected handling of stop limit orders
-  
+
 ## 2.1.0
 ### User Visible Changes
   - Added support for the GLBX Iceberg order type
@@ -224,7 +224,7 @@ insert_anchor_links = "right"
   - Added PIQ to the mini order book
   - Fixed a rare bug that could lead to server failure when requesting market data
   - Fixed a bug that could cause incorrect price displays for certain types of synthetic securities
- 
+
 ## 2.0.5
 ### User Visible Changes
   - Fixed a bug in the handling of rejected QHO orders
@@ -234,19 +234,19 @@ insert_anchor_links = "right"
 ## 2.0.4
 ### User Visible Changes
   - Fixed performance regression in order manager
-  
+
 ### Internal changes
   - Improved market data diagnostics for bad configurations
   - Changed market data default buffer size
-  
+
 ## 2.0.3
 ### Internal changes
   - Improved backwards compatibility in UI settings
 
-  
+
 ## 2.0.2
 ### User Visible Changes
-  - Middle click the ladder price column to show the orders for that price level 
+  - Middle click the ladder price column to show the orders for that price level
   - Middle click the bid or ask column in the ladder no longer sends an order
 
 ### Internal changes
@@ -274,8 +274,8 @@ insert_anchor_links = "right"
   - Improved settings manager tool output
   - Added diagnostics in the event MD memory corruption is detected
   - Added systems tests for join,lean, iceberg and spreader orders
-  - Made exchange order recovery more resilient in the presence of database loss 
-  
+  - Made exchange order recovery more resilient in the presence of database loss
+
 ## 1.6.2
 ### User Visible Changes
 ### Exchange simulator
@@ -314,21 +314,21 @@ insert_anchor_links = "right"
     - These columns can be restored by toggling their visibility within the PnL's Settings form
     - A new Traded Today column has been added to the PnL
       - Filtering on this column will allow a user to restrict the report to only show today's position
-### Internal changes 
-  - Improved organization of the shell notification architecture 
+### Internal changes
+  - Improved organization of the shell notification architecture
   - Added logging to help troubleshoot shell exit hang
 
 ## 1.5.6
 ### User Visible Changes
-  - Fixed a bug that could cause limits to be erroneously enforced 
+  - Fixed a bug that could cause limits to be erroneously enforced
   - Improved algo documentation
   - Changed how the spreader uses net change
   - Simplified trigger algo user interface
-### Internal changes 
+### Internal changes
   - Defaulted to not recording sim market data
-  - Improved documentation build 
+  - Improved documentation build
   - Added additional performance tracing
-  
+
 ## 1.5.5
   - Fixed Market View 'stale working orders' bug
 
@@ -336,11 +336,11 @@ insert_anchor_links = "right"
   - Avoided logging when warming up cache
 
 ## 1.5.3
-  - Fixed sounds startup issue 
+  - Fixed sounds startup issue
 
 ## 1.5.2
   - Added better logging to AlgoDefinitionService
-  
+
 ## 1.5.1
   - Fixed missing securities in the Fills and P&L windows.
 
@@ -361,7 +361,7 @@ insert_anchor_links = "right"
 #### Bug Fixes
   - Fixed the "Since Open" time filter to use the exchange open time rather than 8:30 AM
   - Fixed a bug in the order book where sometimes parent orders would not update their last message column.
-  
+
 #### Algos
   - Added OCO Algo
     - An OCO order creates a relationship between a child order and a 'stop' or 'other' order such that when one of these is cancelled, both cancel, or when one is filled, the other order's size is debited in the amount of the fill.  Composing this order with Cover simulates scalping functionality
@@ -369,7 +369,7 @@ insert_anchor_links = "right"
   - Improved SDK documentation and usability
   - Dynamic orders are now properly identified in the order book
 #### Documentation
-  - Information on the Spark API has been added to the Spark manual 
+  - Information on the Spark API has been added to the Spark manual
   - The documentation has been updated and reorganized to make information easier to find
 ### Internal Changes
   - Improved robustness of limit system test

src/client/spark/Modules/Market/MarketView/MarketViewRow.cs

@@ -26,6 +26,7 @@ namespace mbts.spark {
          readonly ObservableFilterCollection<OrderRecordImmutable> _working_orders, _working_bids, _working_asks;
          readonly string _security_text, _security_key_text;
          readonly Property<double> _bid_price, _ask_price, _last_price, _day_change, _settlement, _session_high, _session_low;
+         readonly Property<bool> _last_price_from_spread;
          readonly CellProperty<int> _position;
          readonly CellProperty<uint> _last_size;
          readonly CellProperty<ulong> _volume;
@@ -54,6 +55,7 @@ namespace mbts.spark {
             _day_change = backing(nameof(day_change), double.NaN);
             _session_low = backing(nameof(session_low), double.NaN);
             _session_high = backing(nameof(session_high), double.NaN);
+            _last_price_from_spread = backing(nameof(last_price_from_spread), false);
             _settlement = backing(nameof(settlement), double.NaN);
             _volume = new CellProperty<ulong>(this, nameof(volume), 0);
             _inside_bids = new CellProperty<bool>(this, nameof(inside_bids));
@@ -109,6 +111,7 @@ namespace mbts.spark {
          public int position => _position.value;
          public double raw_day_position => _raw_day_position;
          public double last_price => _last_price.Value;
+         public bool last_price_from_spread => _last_price_from_spread.Value;
          public uint last_size => _last_size.value;
          public double bid_price => _bid_price.Value;
          public uint bid_direct_size => _bid_direct_size.Value;
@@ -167,15 +170,17 @@ namespace mbts.spark {
                _inside_bids.on_next_distinct(_bid_price.Value >= _working_best_bid.value, ref dirty);
                _inside_asks.on_next_distinct(_ask_price.Value <= _working_best_ask.value, ref dirty);
             }
-            if (md.md.get_last_trade(ref _last_ref, out NativePriceSize ps, out DateTime ts, out ulong v)) {
+            if (md.md.get_last_trade(ref _last_ref, out NativePriceSize ps, out DateTime ts, out ulong v, out bool from_spread)) {               
                _last_price.set(ps.price);
+               _last_price_from_spread.set(from_spread);
                _last_size.on_next_distinct(ps.size, ref dirty);
                _volume.on_next_distinct(v, ref dirty);
             }
-            _settlement.set(md.md.settlement_price);
-            _session_low.set(md.md.session_low);
-            _session_high.set(md.md.session_high);
-            _day_change.set(last_price - settlement);
+            dirty |= _settlement.set_new(md.md.settlement_price);
+            dirty |= _session_low.set_new(md.md.session_low);
+            dirty |= _session_high.set_new(md.md.session_high);
+            dirty |= _day_change.set_new(last_price - settlement);
+            _volume.on_next_distinct(md.md.total_volume, ref dirty);
             if (dirty)
                invalidated?.Invoke();
 

src/client/spark/Modules/Market/MarketView/MarketViewVM.cs

@@ -38,7 +38,8 @@ namespace mbts.spark {
          IMultiValueConverter price_converter = new ui.spark.converters.TickDisplayConverter();
          add_col(Columns.Security, "Security", r => r.security.ToString(), nameof(Row.security), DefaultColumnWidths.Security, security_filter, DataGridFilterKnownTemplates.Regex);
          add_col(Columns.Volume, "Volume", r => r.volume.ToString(), nameof(Row.volume), DefaultColumnWidths.Volume, volume_filter, DataGridFilterKnownTemplates.ULongMinMax);
-         add_col(Columns.Last, "Last", r => r.format_price(r.last_price), nameof(Row.last_price), DefaultColumnWidths.Price, last_price_filter, DataGridFilterKnownTemplates.DoubleMinMax);
+         add_col(Columns.LastSpd, "Spd", r => r.last_price_from_spread ? "S" : "", nameof(Row.last_price_from_spread), DefaultColumnWidths.Side, last_price_from_spread_filter, DataGridFilterKnownTemplates.Regex);
+         add_col(Columns.Last, "Last", r => r.format_price(r.last_price), nameof(Row.last_price), DefaultColumnWidths.Price, last_price_filter, DataGridFilterKnownTemplates.DoubleMinMax);         
          add_col(Columns.LastSize, merged_size_col_header, r => r.last_size.ToString(), nameof(Row.last_size), DefaultColumnWidths.Size, last_size_filter, DataGridFilterKnownTemplates.UIntMinMax, h_alignment: System.Windows.HorizontalAlignment.Right);
          add_col(Columns.Position, "Pos", r => r.position.ToString(), nameof(Row.position), DefaultColumnWidths.Size, position_filter, DataGridFilterKnownTemplates.IntMinMax, true, h_alignment: System.Windows.HorizontalAlignment.Right);
          add_col(Columns.RawDayPosition, "Raw Day Pos", r => SparkUtils.fmt_position(r.raw_day_position), nameof(Row.raw_day_position), DefaultColumnWidths.Size, raw_day_position_filter, DataGridFilterKnownTemplates.DoubleMinMax, true, h_alignment: System.Windows.HorizontalAlignment.Right);
@@ -138,6 +139,7 @@ namespace mbts.spark {
                      ask_direct_size_filter.load_state(filters.ask_direct_size);
                      ask_implied_size_filter.load_state(filters.ask_implied_size);
                      last_price_filter.load_state(filters.last_price);
+                     last_price_from_spread_filter.load_state(filters.last_price_from_spread);
                      last_size_filter.load_state(filters.last_size);
                      day_change_filter.load_state(filters.day_change);
                      settlement_filter.load_state(filters.settlement);
@@ -155,7 +157,7 @@ namespace mbts.spark {
             }), DataGridFilterHelper.hook_filters(rows_view, security_filter,
             bid_price_filter, bid_merged_size_filter, bid_direct_size_filter, bid_implied_size_filter,
             ask_price_filter, ask_merged_size_filter, ask_direct_size_filter, ask_implied_size_filter,
-            last_price_filter, last_size_filter, day_change_filter, settlement_filter,
+            last_price_filter, last_price_from_spread_filter, last_size_filter, day_change_filter, settlement_filter,
             session_low_filter, session_high_filter, position_filter, raw_day_position_filter, working_bid_volume_filter, working_ask_volume_filter, volume_filter, security_key_filter),
             settings_manager.subscribe_current().OnDispatcher().Subscribe(grid_settings_manager),
             WorkLoop.Start("MarketView", _refresh)
@@ -218,6 +220,7 @@ namespace mbts.spark {
       public IDataGridColumnPersistentFilter<Row, UIntMinMaxState> ask_direct_size_filter { get; } = new UIntMinMax<Row>(r => r.ask_direct_size, nameof(Row.ask_direct_size));
       public IDataGridColumnPersistentFilter<Row, UIntMinMaxState> ask_implied_size_filter { get; } = new UIntMinMax<Row>(r => r.ask_implied_size, nameof(Row.ask_implied_size));
       public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> last_price_filter { get; } = new DoubleMinMax<Row>(r => r.last_price, nameof(Row.last_price));
+      public IDataGridColumnPersistentFilter<Row, string> last_price_from_spread_filter { get; } = new RegexFilterVM<Row>(r => r.last_price_from_spread.ToString(), nameof(Row.last_price_from_spread));
       public IDataGridColumnPersistentFilter<Row, UIntMinMaxState> last_size_filter { get; } = new UIntMinMax<Row>(r => r.last_size, nameof(Row.last_size));
       public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> day_change_filter { get; } = new DoubleMinMax<Row>(r => r.day_change, nameof(Row.day_change));
       public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> settlement_filter { get; } = new DoubleMinMax<Row>(r => r.settlement, nameof(Row.settlement));
@@ -250,6 +253,7 @@ namespace mbts.spark {
             ask_direct_size = ask_direct_size_filter.get_state(),
             ask_implied_size = ask_implied_size_filter.get_state(),
             last_price = last_price_filter.get_state(),
+            last_price_from_spread = last_price_from_spread_filter.get_state(),
             last_size = last_size_filter.get_state(),
             day_change = day_change_filter.get_state(),
             settlement = settlement_filter.get_state(),

src/client/spark/Modules/Market/MarketView/MarketViewView.xaml.cs

@@ -12,6 +12,7 @@ namespace mbts.ui.spark {
          Security,
          Volume,
          Last,
+         LastSpd,
          LastSize,
          Position,
          RawDayPosition,

src/client/spark/Modules/Market/MarketView/MarketViewWindowFactory.cs

@@ -100,6 +100,7 @@ namespace bts.shell {
             public static GridColumnSettings security => new GridColumnSettings(col_id(Columns.Security), DefaultColumnHeaders.Security, width: DefaultColumnWidths.Security);
             public static GridColumnSettings volume => new GridColumnSettings(col_id(Columns.Volume), DefaultColumnHeaders.Volume, width: DefaultColumnWidths.Volume);
             public static GridColumnSettings last => new GridColumnSettings(col_id(Columns.Last), DefaultColumnHeaders.Last, width: DefaultColumnWidths.Price);
+            public static GridColumnSettings last_from_spread => new GridColumnSettings(col_id(Columns.LastSpd), DefaultColumnHeaders.S, width: DefaultColumnWidths.Price, is_visible: false);
             public static GridColumnSettings last_size => new GridColumnSettings(col_id(Columns.LastSize), DefaultColumnHeaders.Size, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right);
             public static GridColumnSettings position => new GridColumnSettings(col_id(Columns.Position), DefaultColumnHeaders.Pos, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right);
             public static GridColumnSettings raw_day_position => new GridColumnSettings(col_id(Columns.RawDayPosition), DefaultColumnHeaders.RawDayPos, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right, is_visible: false);
@@ -138,6 +139,7 @@ namespace bts.shell {
                   yield return security;
                   yield return volume;
                   yield return last;
+                  yield return last_from_spread;
                   yield return last_size;
                   yield return position;
                   yield return raw_day_position;
@@ -224,6 +226,8 @@ namespace bts.shell {
          [ProtoMember(25)]
          public string strip_name;
          [ProtoMember(26)]
+         public string last_price_from_spread;
+         [ProtoMember(28)]         
          public string mic_code;
       }
 

src/client/spark/Modules/Spreader/ExecutionStrategyEditor.xaml.cs

@@ -227,7 +227,7 @@ namespace mbts.ui.spark {
       }
 
       DataGridColumn _column_factory(LegSpreaderParamsTuple l) {
-         var col = new DataGridColumnInternal(l.leg.Security, ParameterCellTemplateConverter, ParameterCellEditingTemplateConverter) { Width = 75, };
+         var col = new DataGridColumnInternal(l.leg.Security, ParameterCellTemplateConverter, ParameterCellEditingTemplateConverter);
          col.HeaderTemplate = (DataTemplate)FindResource(IsNew ? RemovableColumnHeaderTemplateKey : NonRemovableColumnHeaderTemplateKey);
          DataGridEx.SetRightBorder(col, new DataGridExVerticalLine(false, true, Colors.DarkGray, 1));
          return col;

src/client/spark/Modules/Trading/Fills/FillsVM.cs

@@ -125,6 +125,7 @@ namespace mbts.spark {
          add_col(Columns.Size, DefaultColumnHeaders.Size, r => r.size.ToString(), new DoubleMinMax<FillTuple>(r => r.fill.LastShares), DataGridFilterKnownTemplates.UIntMinMax, DefaultColumnWidths.Size);
          add_col(Columns.Price, DefaultColumnHeaders.Price, r => r.format_price(r.price), new DoubleMinMax<FillTuple>(r => r.fill.LastPrice), DataGridFilterKnownTemplates.DoubleMinMax, DefaultColumnWidths.Price);
          add_col(Columns.FillKind, DefaultColumnHeaders.FillKind, r => r.fill_kind.ToString(), dv_filter(EnumHelper.get_values<FillKind>().ToArray(), r => r.fill.FillKind), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.FillKind);
+         add_col(Columns.FillStatus, DefaultColumnHeaders.FillStatus, r => r.fill_status.ToString(), dv_filter(EnumHelper.get_values<FillStatus>().ToArray(), r => r.fill.FillStatus), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.FillKind);
          add_col(Columns.FillKey, DefaultColumnHeaders.FillKey, r => r.fill_key.ToString(), new RegexFilterVM<FillTuple>(r => r.fill.FillKey), DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.FillKey);
          add_col(Columns.OrderKey, DefaultColumnHeaders.OrderKey, r => r.order_key.ToString(), new RegexFilterVM<FillTuple>(r => r.fill.OrderKey.ToString()), DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.OrderKey);
          add_col(Columns.SecurityKey, DefaultColumnHeaders.SecurityKey, r => r.security_key.ToString(), new RegexFilterVM<FillTuple>(r => r.fill.SecurityKey.ToString()), DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.SecurityKey);
@@ -227,6 +228,7 @@ namespace mbts.spark {
             case Columns.Exchange: return wrap_default(ft => ft.security.exchange_or_default());
             case Columns.FillKey: return wrap(ft => ft.fill.FillKey);
             case Columns.FillKind: return wrap_default(ft => ft.fill.FillKind);
+            case Columns.FillStatus: return wrap_default(ft => ft.fill.FillStatus);
             case Columns.FillTime: return wrap(ft => ft.fill.Timestamp.to_local_time());
             case Columns.OrderKey: return wrap(ft => ft.fill.OrderKey);
             case Columns.OrderType: return wrap_default(ft => ft.order?.order_type() ?? OrderType.None);
@@ -293,6 +295,7 @@ namespace mbts.spark {
          public double price => _fill.LastPrice;
          public double size => _fill.LastShares;
          public FillKind fill_kind => _fill.FillKind;
+         public FillStatus fill_status => _fill.FillStatus;
          public string fill_key => _fill.FillKey;
          public string order_key => _order.Key.ToString();
 

src/client/spark/Modules/Trading/Fills/FillsView.xaml.cs

@@ -21,6 +21,7 @@ namespace mbts.ui.spark {
          Price,
          Size,
          FillKind,
+         FillStatus,
          FillKey,
          OrderKey,
          SecurityKey

src/client/spark/Modules/Trading/Fills/FillsWindowFactory.cs

@@ -37,7 +37,8 @@ namespace bts.shell {
             { nameof(FillsVM.Row.side), Columns.Side },
             { nameof(FillsVM.Row.size), Columns.Size },
             { nameof(FillsVM.Row.time_in_force), Columns.TimeInForce },
-            { nameof(FillsVM.Row.user), Columns.User }
+            { nameof(FillsVM.Row.user), Columns.User },
+            { nameof(FillsVM.Row.fill_status ), Columns.FillStatus },
          };
 
       public FillsWindowFactory(TradingShellModule module, IBus bus, UserSettingsHub settings_hub, OrdersModel orders_model)

src/client/spark/Modules/Trading/Fills/Settings/FillsFilterStates.cs

@@ -74,6 +74,10 @@ namespace bts.shell.fills {
       [ProtoMember(15)]
       [DIYGridColumnID(nameof(Columns.Symbol))]
       public string[] symbol { get; set; }
+
+      [ProtoMember(17)]
+      [DIYGridColumnID(nameof(Columns.FillStatus))]
+      public Fill.Types.FillStatus[] fill_status { get; set; }
    }
 
    static class FillsFilterStatesExtensions {
@@ -96,6 +100,7 @@ namespace bts.shell.fills {
          pb.add_discrete_values(fs.side, ft => ft.fill.Side);
          pb.add_discrete_values(fs.fill_kind, ft => ft.fill.FillKind);
          pb.add_discrete_values(fs.symbol, ft => ft.security?.Symbol ?? string.Empty);
+         pb.add_discrete_values(fs.fill_status, ft => ft.fill.FillStatus);
          // numeric
          pb.add_numeric(fs.size, ft => ft.fill.LastShares);
          pb.add_numeric(fs.price, ft => ft.fill.LastPrice);

src/client/spark/Modules/Trading/PNL/Data/PNLData.cs

@@ -501,6 +501,12 @@ namespace bts.shell.pnl {
          account_name == other.account_name &&
          sod == other.sod &&
          day == other.day &&
+         bid_price.Equals(other.bid_price) &&
+         ask_price.Equals(other.ask_price) &&
+         last_price.Equals(other.last_price) &&
+         mid_price.Equals(other.mid_price) &&
+         settlement_price.Equals(other.settlement_price) &&
+         calc_type.Equals(other.calc_type) &&
          raw_day == other.raw_day &&
          working.summary == other.working.summary;
 

src/client/spark/Modules/Trading/PNL/VM/PNLRowVM.cs

@@ -97,22 +97,22 @@ namespace bts.shell.pnl {
                if (security.HasValue && security.Value.ContractCase != Security.ContractOneofCase.Future) {
                   day_long = day_short = sod_long = sod_short = 0;
                } else {
-                  day_long = (decimal)DoubleUtil.nan_to_zero(data.day.long_pos);
-                  day_short = (decimal)DoubleUtil.nan_to_zero(data.day.short_pos);
-                  sod_long = (decimal)DoubleUtil.nan_to_zero(data.sod.long_pos);
-                  sod_short = (decimal)DoubleUtil.nan_to_zero(data.sod.short_pos);
+                  day_long = DoubleUtil.round_to_decimal(data.day.long_pos);
+                  day_short = DoubleUtil.round_to_decimal(data.day.short_pos);
+                  sod_long = DoubleUtil.round_to_decimal(data.sod.long_pos);
+                  sod_short = DoubleUtil.round_to_decimal(data.sod.short_pos);
                }
                var nd = new PNLNormalizedData(
-                  (decimal)DoubleUtil.nan_to_zero(day_pnl.value),
+                  DoubleUtil.round_to_decimal(day_pnl.value),
                   day_long,
                   day_short,
-                  (decimal)DoubleUtil.nan_to_zero(data.raw_day.long_pos),
-                  (decimal)DoubleUtil.nan_to_zero(data.raw_day.short_pos),
-                  (decimal)DoubleUtil.nan_to_zero(sod_pnl.value),
+                  DoubleUtil.round_to_decimal(data.raw_day.long_pos),
+                  DoubleUtil.round_to_decimal(data.raw_day.short_pos),
+                  DoubleUtil.round_to_decimal(sod_pnl.value),
                   sod_long,
                   sod_short,
-                  (decimal)DoubleUtil.nan_to_zero(data.working_long_pos),
-                  (decimal)DoubleUtil.nan_to_zero(data.working_short_pos));
+                  DoubleUtil.round_to_decimal(data.working_long_pos),
+                  DoubleUtil.round_to_decimal(data.working_short_pos));
                if (_normalized_data != nd) {
                   _normalized_data = nd;
                   updated?.Invoke(nd);

src/client/spark/Modules/Trading/PNL/VM/PNLVM.cs

@@ -103,7 +103,7 @@ namespace bts.shell.pnl {
          load_from(settings);
          _dispose = Disposable.Create(
             pnl_source.Operate(
-               _calculation_type.observe_current(),
+               _calculation_type.observe_current(true),
                (data, calc) => data.with(calc_type: calc)
             ).FilterWithRaw(filter_src, PNLRowData.key_fn, null)
             .OnDispatcher()

src/client/spark/Orders/OrdersModel.cs

@@ -75,7 +75,6 @@ namespace mbts.spark {
             .TableCheckedAutoCorrect(EqualityComparer<(SecurityImmutable? sec, PNLUnitImmutable data)>.Default)
             .Where(t => t.has_changes())
             .Cache();
-         on_account_pnl = _position_service.subscribei_pnl_account_data().Naggle(interval, scheduler).Table().Cache();
          var working_orders = on_orders.Filter(tup => tup.order.is_active());
          this.working_orders =
             on_orders.Scan(
@@ -265,7 +264,6 @@ namespace mbts.spark {
       public IObservable<IReadOnlyDictionary<uint, OrderSecurity>> working_orders { get; }
       public IObservable<TableUpdate<(SecurityImmutable?, PNLUnitImmutable), int>> on_pnl { get; }
       public IObservable<TableUpdate<PNLRowData, PNLKey>> security_account_pnl { get; }
-      public IObservable<TableUpdate<AccountPNLImmutable, uint>> on_account_pnl { get; }
       public IObservable<TableUpdate<(SecurityImmutable?, PositionUnitImmutable), int>> on_positions { get; }
       public IObservable<TableUpdate<(AlgoData algo, AlgoDefinitionMetadata? meta), AlgoKey>> studio_algos_with_meta { get; }
       public IObservable<TableUpdate<(SecurityImmutable?, ManualFillImmutable), int>> on_manual_fills_securities { get; }

src/client/spark/view/Controls/DIYGrid/Columns/ColumnBase.cs

@@ -114,6 +114,7 @@ namespace mbts.ui.controls {
             Aggressor = "Aggressor",
             Volume = "Volume",
             Last = "Last",
+            S = "S",
             Pos = "Pos",
             Working = "Working",
             Bid = "Bid",
@@ -124,6 +125,7 @@ namespace mbts.ui.controls {
             SessionLow = "Low",
             SessionHigh = "High",
             FillKind = "Fill Kind",
+            FillStatus = "Fill Status",
             Name = "Name",
             Description = "Description",
             N_Legs = "# Legs",

src/client/spark/view/Controls/DIYGrid/DoubleUtil.cs

@@ -1,10 +1,9 @@
 using System;
 using System.Linq;
 
-namespace mbts.ui.controls
-{
-   internal class DoubleUtil
-   {
+namespace mbts.ui.controls {
+   static class DoubleUtil {
+      public const int API_DECIMAL_PRECISION = 9;
       /* smallest such that 1.0+DBL_EPSILON != 1.0 */
       private const double EPSILON  =   2.2204460492503131e-016;
       private const double BIG_EPSILON = EPSILON * 10;
@@ -36,6 +35,9 @@ namespace mbts.ui.controls
             (double.IsNaN(right) ? double.NaN : -right) :
             (double.IsNaN(right) ? left : left - right);
 
+      public static decimal round_to_decimal(double value, int decimals = API_DECIMAL_PRECISION) =>
+         double.IsNaN(value) ? 0 : (decimal)Math.Round(value, decimals, MidpointRounding.AwayFromZero);
+
       internal static bool AreClose(double v1, double v2) {
          if (v1 == v2) return true;
          double delta = Math.Abs(v1 - v2);

src/client/spark_api/Co2/MDConsumerHandle.cs

@@ -133,6 +133,12 @@ namespace mbts.co2 {
          total_volume = _cold->total_volume;
          return (_cold->last_trade.load_if_new(ref time, out ps));
       }
+      public bool get_last_trade(ref ulong time, out NativePriceSize ps, out DateTime timestamp, out ulong total_volume, out bool from_spread) {
+         timestamp = _cold->last_trade_time;
+         total_volume = _cold->total_volume;
+         from_spread = (_cold->from_spread != 0);
+         return (_cold->last_trade.load_if_new(ref time, out ps));
+      }      
       public ulong misc_timestamp { get { return _cold->misc_timestamp; } }
 
       public ulong total_volume { get { return _cold->total_volume; } }
@@ -259,7 +265,7 @@ namespace mbts.co2 {
          // Last trade
          public AtomicPriceSize last_trade;
          public ulong total_volume;
-         public utils.LLDateTime last_trade_time;
+         public utils.LLDateTime last_trade_time;         
 
          //// Settlement
          public UInt64 misc_timestamp;
@@ -280,6 +286,7 @@ namespace mbts.co2 {
          public double session_low_offer;
          public double fixing_price;
 
+         public int from_spread;
          public int deleted;
          // FixedSecurity will not be directly addressable.  Instead the security is available through the callback
          public Int64 security_size;

src/client/spark_api/api/Update.cs

@@ -8,7 +8,6 @@ using System.Linq;
 using System.Reactive.Concurrency;
 using System.Reactive.Disposables;
 using System.Reactive.Linq;
-using System.Reactive.Subjects;
 using System.Text;
 
 namespace bts.api.core {
@@ -94,38 +93,38 @@ namespace bts.api.core {
          new Update<T, TKey>(new T[0], new T[0], new Change<T>[0], new T[0], key_extractor);
 
       public Update<T,TKey> merge(Update<T, TKey> next) {
-         var comp = new RowEqualityComparer<T, TKey>(_key);
-         var added = this.added as HashSet<T> ?? new HashSet<T>(this.added, comp);
+         var added = this.added as KeyedValueSet<TKey, T> ?? new KeyedValueSet<TKey, T>(this.added, _key);
          var changes = this.changed as ChangeDict<T,TKey> ?? new ChangeDict<T,TKey>(this.changes, _key);
-         var removed = this.removed as HashSet<T> ?? new HashSet<T>(this.removed, comp);
+         var removed = this.removed as KeyedValueSet<TKey, T> ?? new KeyedValueSet<TKey, T>(this.removed, _key);
 
          // Add
-         added.UnionWith(next.added);
+         foreach (var v in next.added) {
+            var k = _key(v);
+            if (removed.TryPop(k, out var oldval)) {
+               changes.AddOrMerge(new Change<T>(oldval, v));
+            } else {
+               added[k] = v;
+            }
+         }
 
          // Change
          foreach (var c in next.changes) {
-            if (added.Contains(c.Old)) {
-               added.Remove(c.Old);
-               added.Add(c.New);
-            }
-            else {
+            var k = _key(c.Old);
+            if (!added.TryReplace(k, c.New))
                changes.AddOrMerge(c);
-            }
          }
 
          // Remove
          foreach (var r in next.removed) {
-            if (!added.Remove(r)) {
-               removed.Add(r);
-            }
-            changes.Remove(_key(r));
+            var k = _key(r);
+            if (!added.Remove(k))
+               removed[k] = changes.TryRemove(k, out var ch) ? ch.Old : r;
          }
 
          return new Update<T, TKey>(
             next.all,
             added, changes, removed, 
-            _key
-            );
+            _key);
       }
 
       public IEnumerable<T> all { get; }
@@ -148,6 +147,39 @@ namespace bts.api.core {
 #endif
    }
 
+   class KeyedValueSet<TKey, TValue> : IEnumerable<TValue> {
+      readonly Dictionary<TKey, TValue> _cache = new Dictionary<TKey, TValue>();
+
+      public KeyedValueSet(IEnumerable<TValue> value, Func<TValue, TKey> key) {
+         foreach (var v in value)
+            _cache[key(v)] = v;
+      }
+
+      public TValue this[TKey key] {
+         set => _cache[key] = value;
+      }
+
+      public bool Remove(TKey key) =>
+         _cache.Remove(key);
+
+      public bool TryPop(TKey key, out TValue value) =>
+         _cache.TryGetValue(key, out value) && _cache.Remove(key);
+
+      public bool TryReplace(TKey key, TValue value) {
+         if (_cache.ContainsKey(key)) {
+            _cache[key] = value;
+            return true;
+         }
+         return false;
+      }
+
+      public IEnumerator<TValue> GetEnumerator() =>
+         _cache.Values.GetEnumerator();
+
+      IEnumerator IEnumerable.GetEnumerator() =>
+         GetEnumerator();
+   }
+
    class ChangeDict<T, TKey> : IEnumerable<Change<T>> {
       Dictionary<TKey, Change<T>> _changes;
       Func<Change<T>, TKey> _key;
@@ -167,9 +199,11 @@ namespace bts.api.core {
          return null;
       }
 
-      public void Remove(TKey key) {
+      public bool Remove(TKey key) =>
          _changes.Remove(key);
-      }
+
+      public bool TryRemove(TKey key, out Change<T> result) =>
+         _changes.TryGetValue(key, out result) && _changes.Remove(key);
 
       #region IEnumerable
       public IEnumerator<Change<T>> GetEnumerator() {
@@ -181,31 +215,6 @@ namespace bts.api.core {
       #endregion
    }
 
-   struct RowEqualityComparer<T,TKey> : IEqualityComparer<T>, IEqualityComparer<Change<T>>
-      where TKey : IEquatable<TKey> {
-      readonly Func<T,TKey> _key;
-
-      public RowEqualityComparer(Func<T,TKey> key) {
-         _key = key;
-      }
-
-      public bool Equals(T x, T y) {
-         return _key(x).Equals(_key(y));
-      }
-
-      public bool Equals(Change<T> x, Change<T> y) {
-         return Equals(x.New, y.New);
-      }
-
-      public int GetHashCode(T obj) {
-         return _key(obj).GetHashCode();
-      }
-
-      public int GetHashCode(Change<T> c) {
-         return GetHashCode(c.New);
-      }
-   }
-
    public class TableUpdate<T, TKey> where TKey : IEquatable<TKey> {
       // warning: this does not have a key_extractor
       public static readonly TableUpdate<T, TKey> empty = new TableUpdate<T, TKey>(ImmutableDictionary<TKey, T>.Empty, Array.Empty<T>(), Array.Empty<Change<T>>(), Array.Empty<T>(), null);
@@ -922,4 +931,4 @@ namespace bts.api.core {
             $"T is '{typeof(T)}', TKey is '{typeof(TKey)}'";
       }
    }
-}
+}

src/client/spark_test/Co2/Co2Tests.cs

@@ -142,7 +142,8 @@ namespace mbts.co2.tests
          Assert.IsTrue(entry.session_low_offer == 5.5);
          Assert.IsTrue(entry.fixing_price == 6.6);
 
-         Assert.IsTrue(entry.deleted == 1);
+         Assert.IsTrue(entry.from_spread == 0);
+         Assert.IsTrue(entry.deleted == 1);         
       }
 
       [Test]

src/client/spark_test/Utils/TestUtils.cs

@@ -85,6 +85,8 @@ namespace bts.tests.utils {
          if(timeout == null) {
             timeout = TimeSpan.FromMinutes(2);
          }
+         Console.WriteLine("Please Wait... this is going to take some time...");
+         await Task.Delay(180000);
 
          System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch();
          sw.Start();
@@ -131,6 +133,13 @@ namespace bts.tests.utils {
          System.Diagnostics.Stopwatch watch = new System.Diagnostics.Stopwatch();
          watch.Start();
 
+         var prts = mbts.config.Configuration.get<bts.config.Ports>();
+         Console.WriteLine("\n\n\n\n\n\n\n\n\n\n");
+         Console.WriteLine("config_port: " + prts.config_port);
+         Console.WriteLine("qbus_port: " + prts.qbus_port);
+         Console.WriteLine("glbx_sim_fix_port: " + prts.glbx_sim_fix_port);
+         Console.WriteLine("\n\n\n\n\n\n\n\n\n\n");
+
          var ctx = new ServiceContext();
          var mdrelay = ctx.locate<bts.co2.MarketDataRelay>();
 
@@ -143,6 +152,10 @@ namespace bts.tests.utils {
                products = tmpp.value;
                break;
             }
+            else {
+               Console.WriteLine("mdrelay.request_products_async Error: ");
+               Console.WriteLine(tmpp.error_message);
+            }
             await Task.Delay(10000);
          }
 
@@ -237,7 +250,7 @@ namespace bts.tests.utils {
          Result<co2.MDReply>? md_result = await await_with_timeout(
                                                                md_relay.requestmd_async(md_request),
                                                                timeout);
-
+         Console.Write(md_result);
          return md_result != null;
       }
    }

src/client/spark_test/system_test_scripts/test_position_limits.sh

@@ -30,6 +30,8 @@ cat << EOF > $sparkdata_dir/conf/spark.conf
    }
 EOF
 
+echo "Port"
+./sparkd configcheck Ports
 
 nohup ./sparkd run &
 pid=$!

src/data_files/files/onixs_ice/ConnectivityConfiguration.APITest.xml

@@ -11,7 +11,6 @@
 -->
 <connectivityConfiguration>
     <!-- TCP Connection Information -->
-
     <tcp>
         <server name="default">
             <ip>63.247.113.163</ip>
@@ -19,7 +18,7 @@
             <userName>bluetrd-ps</userName>
             <password>Starts123</password>
             <getStripInfoMessages>N</getStripInfoMessages>
-            <strategyPreference>Legacy</strategyPreference>
+            <strategyPreference>New</strategyPreference>
             <localNetworkInterface></localNetworkInterface>
         </server>
         <server name="endex">
@@ -28,7 +27,7 @@
             <userName>Value from ICE Support</userName>
             <password>Value from ICE Support</password>
             <getStripInfoMessages>N</getStripInfoMessages>
-            <strategyPreference>Legacy</strategyPreference>
+            <strategyPreference>New</strategyPreference>
             <localNetworkInterface></localNetworkInterface>
         </server>
         <server name="liffe-eu">
@@ -37,7 +36,7 @@
             <userName>bluetrdl-ps</userName>
             <password>Starts123</password>
             <getStripInfoMessages>N</getStripInfoMessages>
-            <strategyPreference>Legacy</strategyPreference>
+            <strategyPreference>New</strategyPreference>
             <localNetworkInterface></localNetworkInterface>
         </server>
         <settings>

src/data_files/files/onixs_ice/ConnectivityConfiguration.PerfTest.xml

@@ -17,10 +17,10 @@
         <server name="default">
             <ip>63.247.113.214</ip>
             <port>3000</port>
-            <userName>Value from ICE Support</userName>
-            <password>Value from ICE Support</password>
+            <userName>bt_ps</userName>
+            <password>Starts123</password>
             <getStripInfoMessages>N</getStripInfoMessages>
-            <strategyPreference>Legacy</strategyPreference>
+            <strategyPreference>New</strategyPreference>
             <localNetworkInterface></localNetworkInterface>
         </server>
         <settings>

src/lib/CMakeDeps_win.cmake

@@ -1 +1 @@
-set(sources api/apicore.proto;api/apigateway.proto;api/apioptions.proto;api/core/ApiClientInterop.cpp;api/core/ApiServerInterop.cpp;api/public/ApiServer.cpp;api/public/ClientChannel.cpp;api/public/QuickbusClientChannel.cpp;api/public/ServerChannel.cpp;api/public/ServiceContext.cpp;api/service_helpers/AdminServiceUtils.cpp;api/service_helpers/AlgoServiceUtils.cpp;api/service_helpers/PositionUtilities.cpp;api/services/AdminService.proto;api/services/AlgoDefinitionService.proto;api/services/AlgoService.proto;api/services/ApiUnitTestService.proto;api/services/AuthService.proto;api/services/BasePositionService.proto;api/services/ErrorReportService.proto;api/services/HelloService.proto;api/services/MBOService.proto;api/services/MarginService.proto;api/services/MarketDataAuxiliaryService.proto;api/services/MarketDataRelay.proto;api/services/MoneyFileService.proto;api/services/OrderService.proto;api/services/OutOfProcessURLListener.proto;api/services/PersistenceService.proto;api/services/PositionService.proto;api/services/ShellDiagnosticService.proto;api/services/ShellLatencyService.proto;api/services/SyntheticSecurityService.proto;api/test/ApiTests.cpp;api/test/HelloServiceServer.cpp;config/ConfigCheck.cpp;config/ConfigDownloader.cpp;config/ConfigErrorCollector.cpp;config/ConfigLog.cpp;config/ConfigPropertyEvaluator.cpp;config/ConfigValueQueue.cpp;config/ConfigurationException.cpp;config/ConfigurationState.cpp;config/PreprocessInputStream.cpp;config/config.cpp;config/defaults/MarketData.conf;config/defaults/Ports.conf;config/defaults/Symbols.conf;config/defaults/cme_reference_data.conf;config/defaults/edge.conf;config/defaults/jetfix.conf;config/defaults/keycode.conf;config/defaults/logging.conf;config/defaults/orderman_cfg.conf;config/defaults/orders_fills_cleanup.conf;config/defaults/position_cfg.conf;config/defaults/span.conf;config/defaults/spreader.conf;config/defaults/testkeys.conf;config/defaults/threads.conf;config/defaults/toggles.conf;config/extensions/extensions.proto;config/invalid/invalid.proto;config/protos/ExchangeSimCfg.proto;config/protos/MarketData.proto;config/protos/Ports.proto;config/protos/Symbols.proto;config/protos/UI.proto;config/protos/cme_reference_data.proto;config/protos/jetfix.proto;config/protos/keycode.proto;config/protos/logging.proto;config/protos/mercury_cfg.proto;config/protos/orderman_cfg.proto;config/protos/orders_fills_cleanup.proto;config/protos/perflog_cfg.proto;config/protos/position_cfg.proto;config/protos/quickbus_cfg.proto;config/protos/span.proto;config/protos/spreader.proto;config/protos/system.proto;config/protos/testkeys.proto;config/protos/threads.proto;config/protos/toggles.proto;config/tests/HTTPServer.cpp;config/tests/config_ref_tests.cpp;config/tests/config_tests.cpp;config/text_format/text_format.cpp;domain_model/Security.cpp;domain_model/Security.proto;domain_model/SecurityTests.cpp;domain_model/sod.cpp;domain_model/tests/domain_tests.cpp;expression/BooleanGrammar.cpp;expression/ExpressionEvaluator.cpp;expression/ParserExceptions.cpp;expression/Value.cpp;expression/tests/expression_parser_tests.cpp;logging/LoggerHub.cpp;logging/Sink.cpp;logging/logging.cpp;logging/tests/logging_tests.cpp;md/Diag/MDPeriodicDumper.cpp;md/Diag/ShmDumper.cpp;md/MBOProcessor.cpp;md/MBOServiceServer.cpp;md/MDClient.cpp;md/MDConsumer.cpp;md/MDConsumerHandle.cpp;md/MDConsumerInterop.cpp;md/MDProducer.cpp;md/MDProducerHandle.cpp;md/MemorySegmentUtils.cpp;md/PlatformLoader.cpp;md/PriceSize.cpp;md/SecurityCache.cpp;md/SecurityDB.cpp;md/SecurityIDRecorder.cpp;md/SecurityMapper.cpp;md/SecurityOracle.cpp;md/SettlementRecorder.cpp;md/Shm/FixedSecurity.cpp;md/SyntheticSecurityService.cpp;md/cme/MDP3Utils.cpp;md/common.cpp;md/ice/ICEUtils.cpp;md/protos/co2.proto;md/relay/QueryServer.cpp;md/relay/RelayClient.cpp;md/relay/RelayRecorder.cpp;md/test/Co2ManagedTestHelper.cpp;md/test/Co2TestInstrument.cpp;md/test/mbo_tests.cpp;md/test/md2_tests.cpp;md/test/md_test_utils.cpp;md/test/synthetic_book_merge_tests.cpp;md/vendor/OnixsUtils.cpp;os/os.cpp;os/tests/os_tests.cpp;quickbus/core/LocalNode.cpp;quickbus/core/Quickbus.cpp;quickbus/core/SubscriptionID.cpp;quickbus/core/TableID.cpp;quickbus/message_model/Update.cpp;quickbus/message_model/merge_updates.cpp;quickbus/network/QBConnection.cpp;quickbus/network/QBConnectionManager.cpp;quickbus/network/endpoint_description.cpp;quickbus/network/quickbus_msgs.proto;quickbus/pubsub/TablePublisher.cpp;quickbus/pubsub/TableSubscriber.cpp;quickbus/pubsub/tests/qb_pubsub_tests.cpp;quickbus/routing/INode.cpp;quickbus/routing/NodeID.cpp;quickbus/routing/Router.cpp;quickbus/rpc/RPCClient.cpp;quickbus/rpc/RPCServer.cpp;quickbus/rpc/RPCSessionID.cpp;quickbus/rpc/rpc_utils.cpp;quickbus/rpc/tests/qb_rpc_tests.cpp;quickbus/tests/quickbus_tests.cpp;sparkproto/SparkProtos.proto;testing/catch_test_runner.cpp;utils/AuthSessionManager.cpp;utils/AuthToken.cpp;utils/Dispatcher.cpp;utils/GrowableSPSCQueue.cpp;utils/HTTPStream.cpp;utils/PBUtils.cpp;utils/Puid.cpp;utils/SecureLogoLine truncated
+set(sources api/apicore.proto;api/apigateway.proto;api/apioptions.proto;api/core/ApiClientInterop.cpp;api/core/ApiServerInterop.cpp;api/public/ApiServer.cpp;api/public/ClientChannel.cpp;api/public/QuickbusClientChannel.cpp;api/public/ServerChannel.cpp;api/public/ServiceContext.cpp;api/service_helpers/AdminServiceUtils.cpp;api/service_helpers/AlgoServiceUtils.cpp;api/service_helpers/PositionUtilities.cpp;api/services/AdminService.proto;api/services/AlgoDefinitionService.proto;api/services/AlgoService.proto;api/services/ApiUnitTestService.proto;api/services/AuthService.proto;api/services/BasePositionService.proto;api/services/ErrorReportService.proto;api/services/HelloService.proto;api/services/MBOService.proto;api/services/MarginService.proto;api/services/MarketDataAuxiliaryService.proto;api/services/MarketDataRelay.proto;api/services/MoneyFileService.proto;api/services/OrderService.proto;api/services/OutOfProcessURLListener.proto;api/services/PersistenceService.proto;api/services/PositionService.proto;api/services/ShellDiagnosticService.proto;api/services/ShellLatencyService.proto;api/services/SyntheticSecurityService.proto;api/test/ApiTests.cpp;api/test/HelloServiceServer.cpp;config/ConfigCheck.cpp;config/ConfigDownloader.cpp;config/ConfigErrorCollector.cpp;config/ConfigLog.cpp;config/ConfigPropertyEvaluator.cpp;config/ConfigValueQueue.cpp;config/ConfigurationException.cpp;config/ConfigurationState.cpp;config/PreprocessInputStream.cpp;config/config.cpp;config/defaults/MarketData.conf;config/defaults/Ports.conf;config/defaults/Symbols.conf;config/defaults/cme_reference_data.conf;config/defaults/edge.conf;config/defaults/jetfix.conf;config/defaults/keycode.conf;config/defaults/logging.conf;config/defaults/orderman_cfg.conf;config/defaults/orders_fills_cleanup.conf;config/defaults/position_cfg.conf;config/defaults/span.conf;config/defaults/spreader.conf;config/defaults/testkeys.conf;config/defaults/threads.conf;config/defaults/toggles.conf;config/extensions/extensions.proto;config/invalid/invalid.proto;config/protos/ExchangeSimCfg.proto;config/protos/MarketData.proto;config/protos/Ports.proto;config/protos/Symbols.proto;config/protos/UI.proto;config/protos/cme_reference_data.proto;config/protos/jetfix.proto;config/protos/keycode.proto;config/protos/logging.proto;config/protos/mercury_cfg.proto;config/protos/orderman_cfg.proto;config/protos/orders_fills_cleanup.proto;config/protos/perflog_cfg.proto;config/protos/position_cfg.proto;config/protos/quickbus_cfg.proto;config/protos/span.proto;config/protos/spreader.proto;config/protos/system.proto;config/protos/testkeys.proto;config/protos/threads.proto;config/protos/toggles.proto;config/tests/HTTPServer.cpp;config/tests/config_ref_tests.cpp;config/tests/config_tests.cpp;config/text_format/text_format.cpp;domain_model/Security.cpp;domain_model/Security.proto;domain_model/SecurityTests.cpp;domain_model/sod.cpp;domain_model/tests/domain_tests.cpp;expression/BooleanGrammar.cpp;expression/ExpressionEvaluator.cpp;expression/ParserExceptions.cpp;expression/Value.cpp;expression/tests/expression_parser_tests.cpp;logging/LoggerHub.cpp;logging/Sink.cpp;logging/logging.cpp;logging/tests/logging_tests.cpp;md/Diag/MDPeriodicDumper.cpp;md/Diag/ShmDumper.cpp;md/MBOProcessor.cpp;md/MBOServiceServer.cpp;md/MDClient.cpp;md/MDConsumer.cpp;md/MDConsumerHandle.cpp;md/MDConsumerInterop.cpp;md/MDProducer.cpp;md/MDProducerHandle.cpp;md/MemorySegmentUtils.cpp;md/PlatformLoader.cpp;md/PriceSize.cpp;md/SecurityCache.cpp;md/SecurityDB.cpp;md/SecurityIDRecorder.cpp;md/SecurityMapper.cpp;md/SecurityOracle.cpp;md/SettlementRecorder.cpp;md/Shm/FixedSecurity.cpp;md/SyntheticSecurityService.cpp;md/cme/MDP3Utils.cpp;md/common.cpp;md/ice/ICEUtils.cpp;md/protos/co2.proto;md/relay/QueryServer.cpp;md/relay/RelayClient.cpp;md/relay/RelayRecorder.cpp;md/test/Co2ManagedTestHelper.cpp;md/test/Co2TestInstrument.cpp;md/test/mbo_tests.cpp;md/test/md2_tests.cpp;md/test/md_test_utils.cpp;md/test/synthetic_book_merge_tests.cpp;md/vendor/OnixsUtils.cpp;os/os.cpp;os/tests/os_tests.cpp;quickbus/core/LocalNode.cpp;quickbus/core/Quickbus.cpp;quickbus/core/SubscriptionID.cpp;quickbus/core/TableID.cpp;quickbus/message_model/Update.cpp;quickbus/message_model/merge_updates.cpp;quickbus/network/QBConnection.cpp;quickbus/network/QBConnectionManager.cpp;quickbus/network/endpoint_description.cpp;quickbus/network/quickbus_msgs.proto;quickbus/pubsub/TablePublisher.cpp;quickbus/pubsub/TableSubscriber.cpp;quickbus/pubsub/tests/qb_pubsub_tests.cpp;quickbus/routing/INode.cpp;quickbus/routing/NodeID.cpp;quickbus/routing/Router.cpp;quickbus/rpc/RPCClient.cpp;quickbus/rpc/RPCServer.cpp;quickbus/rpc/RPCSessionID.cpp;quickbus/rpc/rpc_utils.cpp;quickbus/rpc/tests/qb_rpc_tests.cpp;quickbus/tests/quickbus_tests.cpp;sparkproto/SparkProtos.proto;testing/catch_test_runner.cpp;utils/AuthSessionManager.cpp;utils/AuthToken.cpp;utils/Dispatcher.cpp;utils/GrowableSPSCQueue.cpp;utils/HTTPStream.cpp;utils/PBUtils.cpp;utils/Puid.cpp;utils/SecureLogoLine truncated

src/lib/api/service_helpers/AdminServiceUtils.cpp

@@ -46,12 +46,15 @@ bool AdminServiceUtils::has_exchange(const ExchangeConnection& conn,
       break;
    case api::Venue::VENUE_UNSET:
    case api::Venue::VENUE_SIM:
+   case api::Venue::VENUE_SIMICE:
       rv = conn.has_sim_connection();
+      break;      
+   case api::Venue::VENUE_ICE:
+      rv = conn.has_ice_connection();
       break;
    case api::Venue::VENUE_BTS:
    case api::Venue::VENUE_CBOE:
    case api::Venue::VENUE_CBOECFE:
-   case api::Venue::VENUE_ICE:
    case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
    case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:
       rv = false;
@@ -89,6 +92,23 @@ const ExchangeConnection* AdminServiceUtils::resolve_cme_connection(
    return nullptr;                                                               
 }
 
+const bts::api::admin::ExchangeConnection*
+AdminServiceUtils::resolve_ice_connection(const bts::api::admin::AdminData& data,
+                                          const bts::api::admin::BTSAccount& account,
+                                          const bts::api::Security&)
+{
+   if(account.has_connections())
+   {
+      for (const std::string& conn_name : account.connections().exchange_connection_names())
+      {
+         const bts::api::admin::ExchangeConnection* conn = find_connection(data, conn_name);
+         if (conn!=nullptr && conn->has_ice_connection())
+            return conn;  
+      }
+   }
+   return nullptr;                                                               
+}
+
 const bts::api::admin::ExchangeConnection*
 AdminServiceUtils::resolve_sim_connection(const bts::api::admin::AdminData& data,
                                           const bts::api::admin::BTSAccount& account,
@@ -117,14 +137,22 @@ AdminServiceUtils::resolve_connection(const bts::api::admin::AdminData& data,
          rv = resolve_sim_connection(data, account, sec);
       break;
    }
+   case api::Venue::VENUE_ICE: 
+   {      
+      rv = resolve_ice_connection(data, account, sec);
+      auto deployment_type = boost::to_lower_copy(config::get<config::DeploymentType>().type());
+      if (rv == nullptr && (deployment_type == "dev" || deployment_type == "demo"))
+         rv = resolve_sim_connection(data, account, sec);
+      break;
+   }   
    case api::Venue::VENUE_UNSET:
    case api::Venue::VENUE_BTS:
    case api::Venue::VENUE_SIM:
+   case api::Venue::VENUE_SIMICE:
       rv = resolve_sim_connection(data, account, sec);
       break;
    case api::Venue::VENUE_CBOE:
    case api::Venue::VENUE_CBOECFE:      
-   case api::Venue::VENUE_ICE:
    case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
       case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:
       rv = nullptr;
@@ -161,9 +189,15 @@ void AdminServiceUtils::fill_connections_from_config(bts::api::admin::AdminData&
       case config::JetFIX::Connection::CBOE:
          break;
       case config::JetFIX::Connection::SIM:
+      case config::JetFIX::Connection::SIMICE:
          connection.mutable_sim_connection();
          add_connection(admin_data,connection);
          break;
+      case config::JetFIX::Connection::ICE:
+         connection.mutable_ice_connection()->set_sender_compid(c.sender_compid());
+         add_connection(admin_data,connection);
+         break;
+         
       }
    }
    for(int i=0;i<admin_data.accounts_size();i++)
@@ -350,13 +384,16 @@ std::string AdminServiceUtils::find_account_exchange_account (
          rv = cme_account_data->clearing_account();
       break;
    }
-   case api::Venue::VENUE_ICE:{
+   case api::Venue::VENUE_ICE:
+   {
       auto ice_account_data = get_account_ice_data(account, data);
       if (ice_account_data != nullptr)
-         rv = ice_account_data->clearing_account_override();
-      break;
+         rv = ice_account_data->clearing_account();
+      break;      
    }
-   case api::Venue::VENUE_SIM: {
+   case api::Venue::VENUE_SIM: 
+   case api::Venue::VENUE_SIMICE:
+   {
       auto sim_account_data = get_account_sim_data(account, data);
       if (sim_account_data != nullptr)
          rv = sim_account_data->clearing_account();
@@ -466,6 +503,46 @@ bts::api::orderman::RoutingRecord AdminServiceUtils::get_routing_record(const bt
       return rv;
    }
    switch (sec.exchange()) {
+   case api::Venue::VENUE_ICE:
+   {
+      const bts::api::admin::ICEUserData* ice_user_data = get_user_ice_data(*user_ptr, data);
+      if(ice_user_data==nullptr)
+      {
+        out_result.set_is_success(false);
+        out_result.set_error_text("Could not find ICE data for user");
+        return rv;
+      }     
+      const bts::api::admin::ICEAccountData* ice_account_data = get_account_ice_data(*account_ptr, data);
+      if(ice_account_data==nullptr)
+      {
+         out_result.set_is_success(false);
+         out_result.set_error_text("Could not find ICE data for account");
+         return rv;
+      }        
+      // check for data corectness
+      if (ice_user_data->operator_id().length() == 0)
+      {
+         out_result.set_is_success(false);
+         out_result.set_error_text("Operator ID is not set");
+         return rv;
+      }
+
+      rv.set_session(conn_ptr->name());
+      rv.set_sender_compid(conn_ptr->ice_connection().sender_compid());
+      rv.set_account(ice_account_data->clearing_account());
+      rv.set_operator_id(ice_user_data->operator_id());   
+      rv.set_cti_code(ice_account_data->cti());
+      rv.set_customer_or_firm(ice_account_data->origin());
+      rv.set_clearing_firm(ice_account_data->clearing_firm());
+      rv.set_customer_account_ref_id(ice_account_data->customer_account_ref_id());
+      rv.set_account_code(ice_account_data->account_code());
+      rv.set_company_id(std::to_string(ice_user_data->company_id()));
+      rv.set_authorized_trader(ice_user_data->authorized_trader());
+      rv.set_authorized_group(ice_user_data->authorized_group());      
+      rv.set_cust_order_handing_inst(ice_user_data->cust_order_handing_inst());
+      
+      return rv;             
+   }
    case bts::api::Venue::VENUE_GLBX: {
      const bts::api::admin::CMEUserData* cme_user_data = get_user_cme_data(*user_ptr, data);
      if(cme_user_data==nullptr){
@@ -516,7 +593,9 @@ bts::api::orderman::RoutingRecord AdminServiceUtils::get_routing_record(const bt
      rv.set_customer_or_firm(cme_account_data->origin());
      return rv;
    }
-   case api::Venue::VENUE_SIM: {
+   case api::Venue::VENUE_SIM: 
+   case api::Venue::VENUE_SIMICE: 
+   {
      const bts::api::admin::SIMUserData* sim_user_data = get_user_sim_data(*user_ptr, data);
      if(sim_user_data==nullptr){
         out_result.set_is_success(false);
@@ -562,8 +641,7 @@ bts::api::orderman::RoutingRecord AdminServiceUtils::get_routing_record(const bt
    case api::Venue::VENUE_UNSET:
    case api::Venue::VENUE_BTS:
    case api::Venue::VENUE_CBOE:
-   case api::Venue::VENUE_CBOECFE:
-   case api::Venue::VENUE_ICE:
+   case api::Venue::VENUE_CBOECFE:   
    case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
    case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:
       out_result.set_is_success(false);
@@ -710,6 +788,18 @@ const bts::api::admin::CMEUserData* AdminServiceUtils::get_group_cme_data(const
    return nullptr;
 }
 
+const bts::api::admin::ICEUserData* AdminServiceUtils::get_group_ice_data(const bts::api::admin::BTSUserGroup& group,
+                                                                          const bts::api::admin::AdminData& data){
+   if (group.has_user_data() && group.user_data().has_exchange_user_data() && group.user_data().exchange_user_data().has_ice_data())
+      return &(group.user_data().exchange_user_data().ice_data());
+   else if (has_id(group.parent_group_id())){
+      const bts::api::admin::BTSUserGroup* parent_group = find_group(data, group.parent_group_id());
+      if (parent_group != nullptr)
+         return get_group_ice_data(*parent_group, data);
+   }
+   return nullptr;
+}
+
 const bts::api::admin::SIMUserData* AdminServiceUtils::get_group_sim_data(const bts::api::admin::BTSUserGroup& group,
                                                                           const bts::api::admin::AdminData& data){
    if (group.has_user_data() && group.user_data().has_exchange_user_data() && group.user_data().exchange_user_data().has_sim_data())
@@ -840,6 +930,18 @@ const bts::api::admin::CMEUserData* AdminServiceUtils::get_user_cme_data(const b
    return nullptr;
 }
 
+const bts::api::admin::ICEUserData* AdminServiceUtils::get_user_ice_data(const bts::api::admin::BTSUser& user, 
+                                                                         const bts::api::admin::AdminData& data){
+   if (user.has_user_data() && user.user_data().has_exchange_user_data() && user.user_data().exchange_user_data().has_ice_data())
+      return &(user.user_data().exchange_user_data().ice_data());
+   else{
+      const bts::api::admin::BTSUserGroup* parent_group = find_group(data, user.parent_group_id());
+         if (parent_group != nullptr)
+            return get_group_ice_data(*parent_group, data);
+   }
+   return nullptr;
+}
+
 const bts::api::admin::SIMUserData* AdminServiceUtils::get_user_sim_data(const bts::api::admin::BTSUser& user, 
                                                                          const bts::api::admin::AdminData& data){
    if (user.has_user_data() && user.user_data().has_exchange_user_data() && user.user_data().exchange_user_data().has_sim_data())
@@ -867,7 +969,16 @@ std::string AdminServiceUtils::find_user_exchange_id (const bts::api::admin::Adm
          rv = cme_user_data->operator_id();
       break;
    }
-   case api::Venue::VENUE_SIM: {
+   case api::Venue::VENUE_ICE:
+   {
+      auto ice_user_data = get_user_ice_data(user, data);
+      if (ice_user_data != nullptr)
+         rv = ice_user_data->operator_id();
+      break;      
+   }
+   case api::Venue::VENUE_SIM: 
+   case api::Venue::VENUE_SIMICE:
+   {
       auto sim_user_data = get_user_sim_data(user, data);
       if (sim_user_data != nullptr)
          rv = sim_user_data->operator_id();
@@ -877,8 +988,7 @@ std::string AdminServiceUtils::find_user_exchange_id (const bts::api::admin::Adm
    case api::Venue::VENUE_UNSET:
    case api::Venue::VENUE_BTS:
    case api::Venue::VENUE_CBOE:
-   case api::Venue::VENUE_CBOECFE:      
-   case api::Venue::VENUE_ICE:
+   case api::Venue::VENUE_CBOECFE:         
    case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
    case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:      
       out_result.set_is_success(false); out_result.set_error_text("Exchange not supported");
@@ -886,6 +996,38 @@ std::string AdminServiceUtils::find_user_exchange_id (const bts::api::admin::Adm
    }
    return rv;
 }
+
+std::string AdminServiceUtils::find_user_company_id (const bts::api::admin::AdminData& data,
+                                                     const bts::api::admin::BTSUser& user,
+                                                     api::Venue exchange,
+                                                     bts::api::admin::AdminServiceOperationResult& out_result)
+{
+   out_result.set_is_success(true);
+   std::string rv = "";
+   switch(exchange)
+   {
+   case api::Venue::VENUE_ICE:
+   {
+      auto ice_user_data = get_user_ice_data(user, data);
+      if (ice_user_data != nullptr)
+         rv = std::to_string(ice_user_data->company_id());
+      break;      
+   }
+   case api::Venue::VENUE_SIM: 
+   case api::Venue::VENUE_SIMICE:
+   case api::Venue::VENUE_GLBX:
+   case api::Venue::VENUE_UNSET:
+   case api::Venue::VENUE_BTS:
+   case api::Venue::VENUE_CBOE:
+   case api::Venue::VENUE_CBOECFE:         
+   case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
+   case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:      
+      out_result.set_is_success(false); out_result.set_error_text("Exchange not supported");
+      break;
+   }
+   return rv;
+}
+
 std::pair<const bts::api::admin::BTSAccount*, const bts::api::admin::BTSUser*>
 AdminServiceUtils::find_user_account_matches (const bts::api::admin::AdminData& admin_data,
                                               const std::string& connection, const std::string& exchange_account,
@@ -896,14 +1038,22 @@ AdminServiceUtils::find_user_account_matches (const bts::api::admin::AdminData&
    auto user_ptrs = users(admin_data, false);
    for(const bts::api::admin::BTSAccount* account_ptr : account_ptrs){
       if (account_has_connection(*account_ptr, connection)){
-         if (boost::algorithm::iequals(exchange_account,
-                                       find_account_exchange_account(admin_data,*account_ptr,exchange,find_account_result)) &&
-                            find_account_result.is_success()){
-            for(const bts::api::admin::BTSUser* user_ptr : user_ptrs){
-               if (boost::algorithm::iequals(exchange_user,
-                                             find_user_exchange_id(admin_data,*user_ptr,exchange,find_user_result)) &&
+         if (boost::algorithm::iequals(exchange_account, find_account_exchange_account(admin_data,*account_ptr,exchange,find_account_result)) &&
+                                       find_account_result.is_success())
+         {
+            for(const bts::api::admin::BTSUser* user_ptr : user_ptrs)
+            {
+               if (boost::algorithm::iequals(exchange_user,find_user_exchange_id(admin_data,*user_ptr,exchange,find_user_result)) &&
                    find_user_result.is_success())
                   return std::pair<const bts::api::admin::BTSAccount*, const bts::api::admin::BTSUser*>(account_ptr,user_ptr);
+                  
+               //This code eventually gets called by get_reverse_routing_record(...)
+               //At the ice, they send back the login level tag50 in exec reports which doesn't match any user level tag50s in the exchange_user
+               //so instead we look for the CompanyID (or client_id, tag 109) field here.  If this field matches we have a good bet that 
+               //the user we find is enough to fulfill the routing record enough to cancel or modify orphaned orders, etc
+               if (boost::algorithm::iequals(exchange_user,find_user_company_id(admin_data,*user_ptr,exchange,find_user_result)) &&
+                   find_user_result.is_success())
+                  return std::pair<const bts::api::admin::BTSAccount*, const bts::api::admin::BTSUser*>(account_ptr,user_ptr);                  
             }
          }
       }
@@ -995,16 +1145,26 @@ void AdminServiceUtils::validate_user_exchange_data (const bts::api::admin::BTSU
 }
 void AdminServiceUtils::get_md_entitlement_report(const AdminData& admin_data,
                           std::vector<MDEntitlementReportUnit>& entitlements){
-   for (const auto& user : admin_data.users()){
-      const CMEUserData* cme_data=AdminServiceUtils::get_user_cme_data(user,
-                                                                  admin_data);
-      if(cme_data!=nullptr){
-         for (const auto& cme_mic_data : cme_data->md_permissions()){
-            MDEntitlementReportUnit rv=get_cme_md_entitlement_unit(user,
-                                                                 cme_mic_data);
+   for (const auto& user : admin_data.users())
+   {
+      const CMEUserData* cme_data=AdminServiceUtils::get_user_cme_data(user, admin_data);
+      if(cme_data!=nullptr)
+      {
+         for (const auto& cme_mic_data : cme_data->md_permissions())
+         {
+            MDEntitlementReportUnit rv=get_cme_md_entitlement_unit(user, cme_mic_data);
             entitlements.push_back(rv);
          }
       }
+      const ICEUserData* ice_data=AdminServiceUtils::get_user_ice_data(user, admin_data);
+      if(ice_data!=nullptr)
+      {
+         for (const auto& ice_mic_data : ice_data->md_permissions())
+         {
+            MDEntitlementReportUnit rv=get_ice_md_entitlement_unit(user, ice_mic_data);
+            entitlements.push_back(rv);
+         }
+      }      
    }   
 }
 MDEntitlementReportUnit AdminServiceUtils::get_cme_md_entitlement_unit(
@@ -1052,6 +1212,22 @@ MDEntitlementReportUnit AdminServiceUtils::get_cme_md_entitlement_unit(
    rv.set_record_id(key.str());
    return rv;
 }
+MDEntitlementReportUnit AdminServiceUtils::get_ice_md_entitlement_unit(
+                                             const BTSUser& user,
+                                             const ICEMicData& ice_mic_data){
+   MDEntitlementReportUnit rv;
+   rv.set_user(user.name());
+   rv.set_user_id(user.id());
+   rv.set_exchange( bts::api::Venue_Name(bts::api::Venue::VENUE_ICE) );
+   rv.set_mic( bts::api::ICEMics_Name( ice_mic_data.mic()) );
+
+   rv.set_enabled(ice_mic_data.enabled());
+   std::ostringstream key;
+   key << rv.user() << rv.exchange() << rv.mic();
+   rv.set_record_id(key.str());
+   return rv;
+}
+
 void AdminServiceUtils::get_user_symbol_permissions(const AdminData& data,
                                                     const BTSUser& user,
                               std::vector<UserSymbolMDPermissionUnit>& rv){
@@ -1126,13 +1302,13 @@ void AdminServiceUtils::set_user_exchange_data(bts::api::admin::BTSUser* user,
    if (!user->user_data().has_exchange_user_data())
       user->mutable_user_data()->mutable_exchange_user_data()->CopyFrom(exchange_user_data);
 }
-void AdminServiceUtils::set_user_md_entitlement_data(bts::api::admin::BTSUser* user,
-                                                     const bts::api::admin::ExchangeUserMarketDataEntitlementData& exchange_user_md_entitelemnt_data){
-   if (!user->has_user_data())
-      user->mutable_user_data();
-   if (!user->user_data().has_exchange_md_entitlement_data())
-      user->mutable_user_data()->mutable_exchange_md_entitlement_data()->CopyFrom(exchange_user_md_entitelemnt_data);
-}
+// void AdminServiceUtils::set_user_md_entitlement_data(bts::api::admin::BTSUser* user,
+//                                                      const bts::api::admin::ExchangeUserMarketDataEntitlementData& exchange_user_md_entitelemnt_data){
+//    if (!user->has_user_data())
+//       user->mutable_user_data();
+//    if (!user->user_data().has_exchange_md_entitlement_data())
+//       user->mutable_user_data()->mutable_exchange_md_entitlement_data()->CopyFrom(exchange_user_md_entitelemnt_data);
+// }
 void AdminServiceUtils::clear_user_accounts (bts::api::admin::BTSUser* user){
    if(user->has_user_data() && user->user_data().has_account_permissions())
       user->mutable_user_data()->clear_account_permissions();
@@ -1377,6 +1553,7 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    auto account_1 = create_account(data, parent_account->id(), std::string("ACC A1"), std::string("description"), true, account_result); account_1->set_is_active(true);      
    auto account_2 = create_account(data, parent_account->id(), std::string("ACC A2"), std::string("description"), true, account_result); account_2->set_is_active(true);
    auto account_3 = create_account(data, parent_account->id(), std::string("ACC A3"), std::string("description"), true, account_result); account_3->set_is_active(true);
+   auto account_4 = create_account(data, parent_account->id(), std::string("BLUETST"), std::string("description"), true, account_result); account_4->set_is_active(true);
    create_account(data, parent_account->id(), std::string("ACC A4"), std::string("description"), true, account_result);
    // add group
    auto def_group = create_group(data, 0, std::string("default"), result);
@@ -1387,8 +1564,7 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    auto user_2 = create_user(data, def_group->id(), std::string("bts_2"), std::string("bts_2"), empty, empty,  false, license_data, license_entitlements, user_result); user_2->set_is_active(true);
    auto user_3 = create_user(data, def_group->id(), std::string("bts_3"), std::string("bts_3"), empty, empty, false, license_data, license_entitlements, user_result); user_3->set_is_active(true);
    //associate users
-   bts::api::admin::UserAccountPermissionUnit permission_unit;
-   permission_unit.mutable_trading_permission()->set_submit_market_orders(true);
+   bts::api::admin::UserAccountPermissionUnit permission_unit;   
    permission_unit.mutable_trading_permission()->set_has_permissions(true);
    permission_unit.mutable_admin_permission()->set_has_permissions(true);
    bts::api::admin::UserAccountPermissions account_permissions;
@@ -1396,6 +1572,7 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair<uint32_t, bts::api::admin::UserAccountPermissionUnit>(account_1->id(), permission_unit));
    account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair<uint32_t, bts::api::admin::UserAccountPermissionUnit>(account_2->id(), permission_unit));
    account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair<uint32_t, bts::api::admin::UserAccountPermissionUnit>(account_3->id(), permission_unit));      
+   account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair<uint32_t, bts::api::admin::UserAccountPermissionUnit>(account_4->id(), permission_unit));      
    set_user_accounts(user_1, account_permissions);
    set_user_accounts(user_2, account_permissions);
    set_user_accounts(user_3, account_permissions);
@@ -1405,6 +1582,12 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    exchange_account_data.mutable_cme_account_data()->set_cti(bts::api::orderman::CTICode::CTI_CODE_CTI_CODE1);
    exchange_account_data.mutable_cme_account_data()->set_origin(bts::api::orderman::CustomerOrFirm::CUSTOMER_OR_FIRM_FIRM);
    exchange_account_data.mutable_cme_account_data()->set_clearing_account(parent_account->name());
+   
+   exchange_account_data.mutable_ice_account_data()->set_cti(bts::api::orderman::CTICode::CTI_CODE_UNDEFINED);
+   exchange_account_data.mutable_ice_account_data()->set_origin(bts::api::orderman::CustomerOrFirm::CUSTOMER_OR_FIRM_FIRM);
+   exchange_account_data.mutable_ice_account_data()->set_clearing_account(parent_account->name());
+   exchange_account_data.mutable_ice_account_data()->set_clearing_firm("649");
+   
    exchange_account_data.mutable_sim_account_data()->set_cti(bts::api::orderman::CTICode::CTI_CODE_CTI_CODE1);
    exchange_account_data.mutable_sim_account_data()->set_origin(bts::api::orderman::CustomerOrFirm::CUSTOMER_OR_FIRM_FIRM);
    exchange_account_data.mutable_sim_account_data()->set_clearing_account(parent_account->name());
@@ -1412,33 +1595,87 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
 
 
    exchange_account_data.mutable_cme_account_data()->set_clearing_account(account_1->name());
+   exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_1->name());
    exchange_account_data.mutable_sim_account_data()->set_clearing_account(account_1->name());
    set_account_exchange_data(account_1, exchange_account_data);
 
    exchange_account_data.mutable_cme_account_data()->set_clearing_account(account_2->name());
+   exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_2->name());
    exchange_account_data.mutable_sim_account_data()->set_clearing_account(account_2->name());
    set_account_exchange_data(account_2, exchange_account_data);
    
    exchange_account_data.mutable_cme_account_data()->set_clearing_account(account_3->name());
+   exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_3->name());
    exchange_account_data.mutable_sim_account_data()->set_clearing_account(account_3->name());
    set_account_exchange_data(account_3, exchange_account_data);
 
+   exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_4->name());
+   set_account_exchange_data(account_4, exchange_account_data);
+   
+
    // add cme and sim user data
    bts::api::admin::ExchangeUserData exchange_user_data;
    exchange_user_data.mutable_cme_data()->set_operator_id("op_id_" + user_1->name());
    exchange_user_data.mutable_cme_data()->set_location("us,nc");
+   
+   exchange_user_data.mutable_ice_data()->set_operator_id("op_id_" + user_1->name());
+   exchange_user_data.mutable_ice_data()->set_company_id(51251);
+   exchange_user_data.mutable_ice_data()->set_authorized_trader("bluetrd-fx");
+   exchange_user_data.mutable_ice_data()->set_authorized_group("Group");
+   exchange_user_data.mutable_ice_data()->set_password("Starts123");
+   exchange_user_data.mutable_ice_data()->set_mifid_direct_electronic_access(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_trading_capacity(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_liquidity_provision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_commodity_deriv_indicator(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_investment_decision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_execution_decision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_client_id_code(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_id(1);
+      
    exchange_user_data.mutable_sim_data()->set_operator_id("op_id_" + user_1->name());
    exchange_user_data.mutable_sim_data()->set_location("us,nc");
    set_user_exchange_data(user_1, exchange_user_data);
 
    exchange_user_data.mutable_cme_data()->set_operator_id("op_id_" + user_2->name());
    exchange_user_data.mutable_cme_data()->set_location("us,nc");
+   
+   exchange_user_data.mutable_ice_data()->set_operator_id("op_id_" + user_2->name());
+   exchange_user_data.mutable_ice_data()->set_company_id(51251);
+   exchange_user_data.mutable_ice_data()->set_authorized_trader("bluetrd-fx");
+   exchange_user_data.mutable_ice_data()->set_authorized_group("Group");
+   exchange_user_data.mutable_ice_data()->set_password("Starts123");
+   exchange_user_data.mutable_ice_data()->set_mifid_direct_electronic_access(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_trading_capacity(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_liquidity_provision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_commodity_deriv_indicator(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_investment_decision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_execution_decision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_client_id_code(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_id(1);
+   
+   
    exchange_user_data.mutable_sim_data()->set_operator_id("op_id_" + user_2->name());
    exchange_user_data.mutable_sim_data()->set_location("us,nc");
    set_user_exchange_data(user_2, exchange_user_data);
 
    exchange_user_data.mutable_cme_data()->set_operator_id("op_id_" + user_3->name());
    exchange_user_data.mutable_cme_data()->set_location("us,nc");
+   
+   exchange_user_data.mutable_ice_data()->set_operator_id("op_id_" + user_3->name());
+   exchange_user_data.mutable_ice_data()->set_company_id(51251);
+   exchange_user_data.mutable_ice_data()->set_authorized_trader("bluetrd-fx");
+   exchange_user_data.mutable_ice_data()->set_authorized_group("Group");
+   exchange_user_data.mutable_ice_data()->set_password("Starts123");
+   exchange_user_data.mutable_ice_data()->set_mifid_direct_electronic_access(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_trading_capacity(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_liquidity_provision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_commodity_deriv_indicator(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_investment_decision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_execution_decision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_client_id_code(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_id(1);
+   
+   
    exchange_user_data.mutable_sim_data()->set_operator_id("op_id_" + user_3->name());
    exchange_user_data.mutable_sim_data()->set_location("us,nc");
    set_user_exchange_data(user_3, exchange_user_data);
@@ -1480,6 +1717,13 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    add_security_clip_sizes_to_account(account_1, api::Venue::VENUE_GLBX, api::SecurityType::SECURITYTYPE_SPREAD,100);
    add_security_clip_sizes_to_account(account_1, api::Venue::VENUE_SIM, api::SecurityType::SECURITYTYPE_FUTURE,100);
    add_security_clip_sizes_to_account(account_1, api::Venue::VENUE_SIM, api::SecurityType::SECURITYTYPE_SPREAD,100);
+
+   //ICE limits   
+   add_product_position_limits(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_FUTURE,50000);
+   add_product_position_limits(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_SPREAD,50000);
+   add_security_clip_sizes_to_account(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_FUTURE,50);
+   add_security_clip_sizes_to_account(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_SPREAD,50);
+   
 }
 bool AdminServiceUtils::has_id (uint32_t id){return id>0;}
 bool AdminServiceUtils::is_symbol_set (const std::string& symbol){

src/lib/api/service_helpers/AdminServiceUtils.h

@@ -35,6 +35,10 @@ public:
                                                    const AdminData& data,
                                                    const BTSAccount& account,
                                                    const api::Security& sec);
+   static const ExchangeConnection* resolve_ice_connection(
+                                                   const AdminData& data,
+                                                   const BTSAccount& account,
+                                                   const api::Security& sec);                                                   
    static const ExchangeConnection* resolve_sim_connection(
                                                    const AdminData& data,
                                                    const BTSAccount& account,
@@ -164,6 +168,8 @@ public:
    // travels up the group tree until it finds a populated cme data 
    static const CMEUserData* get_group_cme_data(const BTSUserGroup& group,
                                                 const AdminData& data);
+   static const ICEUserData* get_group_ice_data(const BTSUserGroup& group,
+                                                const AdminData& data);                                                
    static const SIMUserData* get_group_sim_data(const BTSUserGroup& group, 
                                                 const AdminData& data);
    static const UserAccountPermissions* const get_group_user_permissions (
@@ -198,6 +204,8 @@ public:
    static uint32_t find_max_user_id (const AdminData& data);
    static const CMEUserData* get_user_cme_data(const BTSUser& user, 
                                                const AdminData& data);
+   static const ICEUserData* get_user_ice_data(const BTSUser& user, 
+                                               const AdminData& data);                                               
    static const SIMUserData* get_user_sim_data(const BTSUser& user, 
                                                const AdminData& data);
    // finds the "exchange user" (ex: "operator_id" at CME or "authorized_trader" 
@@ -207,6 +215,12 @@ public:
                                     const BTSUser& user,
                                     api::Venue exchange,
                                     AdminServiceOperationResult& out_result);
+   static std::string find_user_company_id (
+                                    const AdminData& data,
+                                    const BTSUser& user,
+                                    api::Venue exchange,
+                                    AdminServiceOperationResult& out_result);
+                                    
    static std::pair<const BTSAccount*, const BTSUser*>
    find_user_account_matches (const AdminData& admin_data,
                               const std::string& connection,
@@ -246,6 +260,9 @@ public:
    static MDEntitlementReportUnit get_cme_md_entitlement_unit(
                                              const BTSUser& user,
                                              const CMEMicData& cme_mic_data);
+   static MDEntitlementReportUnit get_ice_md_entitlement_unit(
+                                             const BTSUser& user,
+                                             const ICEMicData& ice_mic_data);                                             
    static void get_user_symbol_permissions(
                                  const AdminData& data,
                                  const BTSUser& user,
@@ -272,8 +289,6 @@ public:
    static void set_user_exchange_data(
                                  BTSUser* user,
                                  const ExchangeUserData& exchange_user_data);
-   static void set_user_md_entitlement_data(BTSUser* user,
-                                 const ExchangeUserMarketDataEntitlementData&);
    static void clear_user_accounts (BTSUser* user);
    static void set_user_accounts (BTSUser* user,
                      const UserAccountPermissions& user_account_permissions);

src/lib/api/services/AdminService.proto

@@ -55,11 +55,12 @@ message BTSUser{
    string name = 30;
    string password = 40;
    string full_name = 42;
-   string company = 44;   
+   string company = 44;
    bool is_active = 50;
    bool can_create = 55;
-   LicenseData license_data = 60; //unused
-   LicenseEntitlements license_entitlements = 70; //unused
+   LicenseData license_data = 60;
+   LicenseEntitlements license_entitlements = 70;
+
    BTSUserDerivableData user_data = 80;
 }
 
@@ -76,7 +77,7 @@ message BTSUserDerivableData{
    // b. else its overridden at the user level
    UserAccountPermissions account_permissions = 70;
    ExchangeUserData exchange_user_data = 80;
-   ExchangeUserMarketDataEntitlementData exchange_md_entitlement_data = 90;
+   reserved 90; reserved "exchange_md_entitlement_data"; //ExchangeUserMarketDataEntitlementData exchange_md_entitlement_data = 90;
 }
 
 message LicenseData{
@@ -91,21 +92,22 @@ message LicenseEntitlements{
    bool spreader = 30;
 }
 
-message AlgoTradingPermissionsUnit{
-   bool submit_iceberg_orders = 10; //unused
-   bool submit_time_box_orders = 20; //unused
-   bool submit_spreader_orders = 30; //unused
-}
+// message AlgoTradingPermissionsUnit{
+//    bool submit_iceberg_orders = 10; //unused
+//    bool submit_time_box_orders = 20; //unused
+//    bool submit_spreader_orders = 30; //unused
+// }
 
 message TradingPermissionUnit{
+   reserved 10; reserved "submit_market_orders"; //bool submit_market_orders = 10; //unused
+   reserved 20; reserved "submit_gtc_orders"; //bool submit_gtc_orders = 20; //unused
+   reserved 30; reserved "algo_trading_permission"; //AlgoTradingPermissionsUnit algo_trading_permission = 30; //unused
+
    bool has_permissions = 5;
-   bool submit_market_orders = 10; //unused
-   bool submit_gtc_orders = 20; //unused  
-   AlgoTradingPermissionsUnit algo_trading_permission = 30; //unused
 }
 
 message AdminPermissionUnit{
-   // TODO: needs to be more ellaborate (account writing, 
+   // TODO: needs to be more ellaborate (account writing,
    // reading etc, routing etc)
    bool has_permissions = 10;
    bool has_md_permissions = 20;
@@ -119,15 +121,15 @@ message UserAccountPermissionUnit{
 
 message UserAccountPermissions{
    //uint32 = account_id
-   map<uint32, UserAccountPermissionUnit> account_permissions = 10;   
+   map<uint32, UserAccountPermissionUnit> account_permissions = 10;
 }
 
-// unused now
-message UserSecurityTradingLimit{
-   SecurityScope security = 10;
-   uint32 clip_size = 20;
-   bool can_trade = 30;   
-}
+// // unused now
+// message UserSecurityTradingLimit{
+//    SecurityScope security = 10;
+//    uint32 clip_size = 20;
+//    bool can_trade = 30;
+// }
 
 /************************************************************************/
 /********************************BTS Account*****************************/
@@ -152,7 +154,7 @@ message BTSAccount{
    AccountLossLimit account_loss_limit = 130;
 }
 
-message RiskLimitEnforcement{   
+message RiskLimitEnforcement{
    bool product_limit_in_net_terms = 10;
    bool ignore_spread_position_limits = 20;
 }
@@ -204,23 +206,22 @@ message AccountLossLimit{
 message CMEMicData{
    api.CMEMics mic=10;
    bool enabled=20;
-   // there are other things here   
+   // there are other things here
 }
 
 message ICEMicData{
-   api.CMEMics mic=10;
+   api.ICEMics mic=10;
    bool enabled=20;
 }
 
-message ExchangeUserMarketDataEntitlementData{
-   CMEUserMarketDataEntitlementData cme_data = 10;
-   ICEUserMarketDataEntitlementData ice_data = 20;
-}
 
 message ExchangeUserData{
    CMEUserData cme_data = 10;
    ICEUserData ice_data = 20;
    SIMUserData sim_data = 30;
+
+   enum AdminDisplay { All = 0; MD = 1; Routing = 2; None = 100; }
+   AdminDisplay admin_display_flag = 120;
 }
 
 message ExchangeAccountData{
@@ -229,9 +230,6 @@ message ExchangeAccountData{
    SIMAccountData sim_account_data = 30;
 }
 
-message MarketDataEntitlement{
-   bool can_view = 10;
-}
 
 message ExchangeConnection{
    uint32 id = 10;
@@ -274,19 +272,19 @@ message CMEMSGWConnection{
    uint32 market_segment_id = 60;
 }
 
-message CMEAccountData{
-   // should perhaps be an enum
+message CMEAccountData {
+   reserved 30; reserved "takeup_member_code"; //string takeup_member_code = 30;
+   reserved 50; reserved "direct_electronic_access"; //string direct_electronic_access = 50;
+   reserved 60; reserved "trading_capacity"; //string trading_capacity = 60;
+   reserved 70; reserved "liquidity_provision"; //string liquidity_provision = 70;
+   reserved 80; reserved "commodity_derivative_indicator"; //string commodity_derivative_indicator = 80;
+   reserved 90; reserved "investment_decision_id"; //string investment_decision_id = 90;
+   reserved 100; reserved "execution_decision_id"; //string execution_decision_id = 100;
+   reserved 110; reserved "client_id"; //string client_id = 110;
+
    api.orderman.CTICode cti = 10;
    api.orderman.CustomerOrFirm origin = 20;
-   string takeup_member_code = 30;
    string clearing_account = 40;
-   string direct_electronic_access = 50;
-   string trading_capacity = 60;
-   string liquidity_provision = 70;
-   string commodity_derivative_indicator = 80;
-   string investment_decision_id = 90;
-   string execution_decision_id = 100;
-   string client_id = 110;
    map<string, string> misc_tags = 120;
 }
 
@@ -309,21 +307,22 @@ enum DisplayDesignation
 }
 
 message CMEUserData{
+   reserved 20; reserved "ats_operator_id"; //string ats_operator_id = 20;
+   reserved 40; reserved "text_a"; //string text_a = 40;
+   reserved 50; reserved "text_b"; //string text_b = 50;
+   reserved 80; reserved "max_order_rate_per_connection"; //string max_order_rate_per_connection = 80;
+   //enum AdminDisplay { All = 0; MD = 1; Routing = 2; None = 100; }
+   reserved 120; reserved "admin_display_flag"; //AdminDisplay admin_display_flag = 120;
+   repeated CMEMicData md_permissions = 1000;
+
    string operator_id = 10;
-   string ats_operator_id = 20;
    string smp_id = 30;
-   string text_a = 40;
-   string text_b = 50;
    map<string, string> misc_tags = 60;
    string smp_instruction = 70;
-   string max_order_rate_per_connection = 80;
+
    string location = 90;
    ProfessionalDesignation pro_nonpro = 100;
    DisplayDesignation display_nondisplay = 110;
-   
-   enum AdminDisplay { All = 0; MD = 1; Routing = 2; None = 100; } 
-   AdminDisplay admin_display_flag = 120; 
-   repeated CMEMicData md_permissions = 1000;
 }
 
 message SIMUserData {
@@ -331,58 +330,67 @@ message SIMUserData {
    string location = 90;
 }
 
-message CMEUserMarketDataEntitlementData{
-   MarketDataEntitlement cbot_entitlement = 10;
-   MarketDataEntitlement cme_entitlement = 20;
-   MarketDataEntitlement nymex_entitlement = 30;
-   MarketDataEntitlement bmandf_entitlement = 40;
-   MarketDataEntitlement comex_entitlement = 50;
-   MarketDataEntitlement cme_europe_entitlement = 60;
-}
-
 /********************************ICE Data*********************************/
 
-message ICEConnection{
-   string authorized_member_id = 10;
-   string sender_company_id = 20;
-   string sender_sub_id = 30;
-   string session_id = 40;
-   string session_password = 50;
-   string host_ip = 60;
-   string port = 70;
-   string default_routing_trader = 80;
-   string routing_trader_password = 90;
-   int32 max_order_rate = 100;
+message ICEConnection {
+   string sender_compid = 10;
 }
 
-message ICEAccountData{
-   string nybit_house = 10;
-   int32 wce_house = 20;
-   string clearing_account_override = 30;
-   bool direct_electronic_access = 40;
-   string trading_capacity = 50;
-   bool liquidity_provision = 60;
-   bool commodity_derivative_indicator = 70;
-   string investment_decision_id = 80;
-   string execution_decision_id = 90;
-   string client_id = 100;
-   string mfid_id = 110;
+message ICEAccountData {
+   reserved 10; reserved "nybit_house"; //string nybit_house = 10;
+   reserved 20; reserved "wce_house"; //int32 wce_house = 20;
+   reserved 30; reserved "clearing_account_override"; //string clearing_account_override = 30;
+   reserved 40; reserved "direct_electronic_access"; //bool direct_electronic_access = 40;
+   reserved 50; reserved "trading_capacity"; //string trading_capacity = 50;
+   reserved 60; reserved "liquidity_provision"; //bool liquidity_provision = 60;
+   reserved 70; reserved "commodity_derivative_indicator"; //bool commodity_derivative_indicator = 70;
+   reserved 80; reserved "investment_decision_id"; //string investment_decision_id = 80;
+   reserved 90; reserved "execution_decision_id"; //string execution_decision_id = 90;
+   reserved 100; reserved "client_id"; //string client_id = 100;
+   reserved 110; reserved "mfid_id"; //string mfid_id = 110;
+
+   api.orderman.CTICode cti = 3;
+   api.orderman.CustomerOrFirm origin = 5;
+   string clearing_firm = 200;
+   string clearing_account = 210;
+   string customer_account_ref_id = 220;
+   string account_code = 230;
+   map<string, string> misc_tags = 500;
 }
 
-message ICEUserData{
-   string authorized_trader = 10;
-   string memo_field = 20;
-   string text_b = 30;
-   uint32 max_order_rate = 40;
-}
+message ICEUserData {
+   reserved 20; reserved "memo_field"; //string memo_field = 20;
+   reserved 30; reserved "text_b"; //string text_b = 30;
+   reserved 40; reserved "max_order_rate"; //uint32 max_order_rate = 40;
 
-message ICEUserMarketDataEntitlementData{
+   int32 company_id = 3; //tag 109 clientid
+   string operator_id = 5; //tag 50
+   string authorized_trader = 10; //tag 553 user_name
+   string authorized_group = 15; //tag 553 user_name
+   string password = 21; //tag 96 raw data
+   string cust_order_handing_inst = 25;
+
+   //mifid
+   int32 mifid_direct_electronic_access = 500;
+   int32 mifid_trading_capacity = 510;
+   int32 mifid_liquidity_provision = 520;
+   int32 mifid_commodity_deriv_indicator = 530;
+   int64 mifid_investment_decision = 540;
+   int64 mifid_execution_decision = 550;
+   int64 mifid_client_id_code = 560;
+   int64 mifid_id = 570;
+
+   map<string, string> misc_tags = 60;
+   ProfessionalDesignation pro_nonpro = 100;
+   DisplayDesignation display_nondisplay = 110;
+
+   repeated ICEMicData md_permissions = 1000;
 }
 
 ///////////////////////////////////////////////////////////
 // Order Auxiliary Service ////////////////////////////////
 
-message OrdersSummaryUnit{   
+message OrdersSummaryUnit{
    string account = 10;
    string user = 20;
    uint32 total_orders = 30;
@@ -393,11 +401,11 @@ message OrdersSummary{
 }
 message OrderSummaryRequest{
 }
-// top level orders that have account 
+// top level orders that have account
 // and user that are present here, will
 // be cancelled. wild cards are not
 // supported.
-message OrderCancelRequest{   
+message OrderCancelRequest{
    repeated string accounts = 10;
    repeated string users = 20;
 }
@@ -433,7 +441,7 @@ message AccountCreateAPIArgument{
    string creator_user = 40;
 }
 message AccountUpdateAPIArgument{
-   BTSAccount complete_data=10;    
+   BTSAccount complete_data=10;
 }
 message UserCreateAPIArgument{
    uint32 parent_group_id = 20;
@@ -477,7 +485,7 @@ service AdminService {
    // users
    rpc create_user(UserCreateAPIArgument) returns (AdminServiceCreateEntityResult);
    rpc update_user(UserUpdateAPIArgument) returns (AdminServiceOperationResult);
-   rpc delete_user(IDAPIArgument) returns (AdminServiceOperationResult);   
+   rpc delete_user(IDAPIArgument) returns (AdminServiceOperationResult);
    // subscription
    rpc admin_data(bts.api.VoidMessage) returns (stream AdminData);
    rpc get_admin_data(bts.api.VoidMessage) returns (AdminData);

src/lib/api/services/AuthService.proto

@@ -57,4 +57,7 @@ service AuthService {
 
    rpc get_users(GetUsersRequest) returns (GetUsersResponse)
    {option (bts.api.method_description)="Returns list of valid users";}
+   
+   rpc clear_all_sessions(bts.api.VoidMessage) returns (bts.api.ResultMessage)
+   {option (bts.api.method_description)="Clears all reserved auth sessions";}   
 }

src/lib/api/services/OrderService.proto

@@ -310,6 +310,13 @@ message Fill {
     FILL_KIND_SOD = 2;
   }
   FillKind fill_kind = 10;
+  
+  enum FillStatus { 
+     FILL_STATUS_ACTIVE = 0;
+     FILL_STATUS_CORRECTED = 1;
+     FILL_STATUS_BUSTED = 2;
+  }
+  FillStatus fill_status = 20;
 }
 
 enum OrderSubType {
@@ -327,6 +334,16 @@ message RoutingRecord {
   string smp_id = 6 ;
   SelfMatchPreventionInstruction smp_instruction = 7;
   string location = 8;
+  
+  string sender_compid = 15;
+  string clearing_firm = 20;
+  string customer_account_ref_id = 25;
+  string account_code = 30;    
+  string company_id = 45;
+  string authorized_trader = 50;  
+  string authorized_group = 60;
+  string cust_order_handing_inst = 70;
+  string order_server_id = 200;
 }
 
 message NativeOrderState {

src/lib/config/ConfigurationState.cpp

@@ -24,6 +24,7 @@
 #include <map>
 #include <tuple>
 
+
 #define URDL_HEADER_ONLY 1
 #define URDL_DISALE_SSL 1
 #include <urdl/url.hpp>
@@ -908,3 +909,4 @@ std::string get_fallback(const proto::Message& message)
 }
 
 }}}//end namespaces
+

src/lib/config/defaults/MarketData.conf

@@ -62,15 +62,44 @@ ICEOptions {
    value {
       channel {
          name: "oil" #these names are arbitrary and are for debugging
+         market_type: 3 
+         uds_market_type: 3
+         market_type: 4 #ICE IPE Gas Oil Futures
+         uds_market_type: 4         
          market_type: 5 #ICE Futures Brent, IPE Brent Futures
-         market_type: 9 #ICE Futures WTI, ICE WTI Crude Futures
-         #udp_buffer_size: 33554432 #32mb
-      }
-      channel {
-         name: "juice"
-         market_type: 23 #ICE Futures US Agricultures, FCOJ A
+         uds_market_type: 5
+         market_type: 9 #ICE Futures WTI, ICE WTI Crude Futures                  
+         uds_market_type: 9
+         market_type: 13 #ICE Heating Oil Futures         
+         uds_market_type: 13
+         market_type: 15 #ICE NYH (RBOB) Gasoline Futures
+         uds_market_type: 15
+         ##market_type: 21 #ICE Futures US Agricultures, Coffee C
+         ##uds_market_type: 21
+         ##market_type: 23 #ICE Futures US Agricultures, FCOJ A
+         ##uds_market_type: 23               
+         
+         ##market_type: 26 #ICE Futures US Financials DX
+         ##uds_market_type: 26
+         
+         uds_market_type: 11 #ICE WTI-Brent spreads
+         uds_market_type: 14 #ICE Heating Oil/WTI Futures Crack
+         uds_market_type: 16 #ICE NYH (RBOB) Gasoline/WTI Futures Crack         
+         uds_market_type: 58 #ICE Futures UI Heating Oil / Brent
+         uds_market_type: 59 #ICE NYH (RBOB) Gasoline/Brent Futures Crack 
+         uds_market_type: 66 #ICE NYH (RBOB) Gasoline/Heating Oil Spread
+         uds_market_type: 70 #ICE Heating Oil/LS Gasoil Futures Spread
+         uds_market_type: 71 #ICE LS Gasoil/Brent Futures Crack
+         uds_market_type: 72 #ICE NYH (RBOB) Gasoline/LS Gasoil Futures Spread         
+         
+         #onixs_log_mode: Debug
       }
+      # channel {
+      #    name: "juice"
+      #    market_type: 26 #ICE Futures US Agricultures, FCOJ A
+      # }
       ice_config_file: "ConnectivityConfiguration.APITest.xml"      
+      #ice_config_file: "ConnectivityConfiguration.PerfTest.xml"      
    }
 }
 

src/lib/config/defaults/Ports.conf

@@ -3,7 +3,8 @@ Ports {
    value {
       config_port: 8000
       qbus_port: 8001
-      exchange_sim_fix_port: 8002
+      glbx_sim_fix_port: 8002
+      ice_sim_fix_port: 8003
       replicator_port: 8080
    }
 }

src/lib/config/defaults/Symbols.conf

@@ -19,7 +19,6 @@ SymbolList {
         symbol: "FOB"
         symbol: "GC"
         symbol: "GE"
-        symbol: "GLI"
         symbol: "HE"
         symbol: "HG"
         symbol: "HO"
@@ -66,9 +65,16 @@ SymbolLists
       list
       {
          key: "ICE"
-         # symbol: "BRN"
-         # symbol: "WBS"
-         # symbol: "OJ"
+         symbol: "BRN"
+         symbol: "WBS"
+         symbol: "OJ"
+         symbol: "G"
+         symbol: "KC"
+         symbol: "UHO"
+         symbol: "UHU"
+         symbol: "DX"
+         
+         symbol: "*"
       }
       list
       {
@@ -112,5 +118,9 @@ Symbols {
             future: @SymbolLists.list[ICE].symbol
             spread: @SymbolLists.list[ICE].symbol
         }
+        simice {
+            future: @SymbolLists.list[ICE].symbol
+            spread: @SymbolLists.list[ICE].symbol            
+        }
     }
 }

src/lib/config/defaults/cme_reference_data.conf

@@ -3,9 +3,9 @@
 # source: tools/cpp_helper/generate-cpps.ps1
 #
 
-CppConfig
-{
-	value
+CppConfig
+{
+	value
 	{
 		cpp_entry { key: "098" value: 2000 }
 		cpp_entry { key: "0A" value: 100 }
@@ -354,6 +354,7 @@ CppConfig
 		cpp_entry { key: "BOB" value: 1000 }
 		cpp_entry { key: "BOO" value: 6350 }
 		cpp_entry { key: "BOS" value: 250 }
+		cpp_entry { key: "BPU" value: 1000 }
 		cpp_entry { key: "BR7" value: 1000 }
 		cpp_entry { key: "BSF" value: 25 }
 		cpp_entry { key: "BTC" value: 5 }
@@ -442,12 +443,12 @@ CppConfig
 		cpp_entry { key: "D3L" value: 80 }
 		cpp_entry { key: "D4L" value: 5 }
 		cpp_entry { key: "D7L" value: 80 }
+		cpp_entry { key: "DAB" value: 1000 }
 		cpp_entry { key: "DAX" value: 5 }
 		cpp_entry { key: "DAZ" value: 80 }
 		cpp_entry { key: "DBB" value: 1000 }
 		cpp_entry { key: "DBI" value: 1000 }
 		cpp_entry { key: "DBL" value: 100 }
-		cpp_entry { key: "DBR" value: 1000 }
 		cpp_entry { key: "DBT" value: 100 }
 		cpp_entry { key: "DC" value: 2000 }
 		cpp_entry { key: "DCB" value: 1000 }
@@ -458,14 +459,13 @@ CppConfig
 		cpp_entry { key: "DDE" value: 1000 }
 		cpp_entry { key: "DDI" value: 1000 }
 		cpp_entry { key: "DDO" value: 1000 }
-		cpp_entry { key: "DEB" value: 744 }
+		cpp_entry { key: "DEB" value: 720 }
 		cpp_entry { key: "DEN" value: 250 }
-		cpp_entry { key: "DEP" value: 264 }
+		cpp_entry { key: "DEP" value: 252 }
 		cpp_entry { key: "DFL" value: 100 }
 		cpp_entry { key: "DFN" value: 100 }
 		cpp_entry { key: "DG2" value: 100 }
 		cpp_entry { key: "DG3" value: 100 }
-		cpp_entry { key: "DGH" value: 1000 }
 		cpp_entry { key: "DGI" value: 100 }
 		cpp_entry { key: "DGS" value: 100 }
 		cpp_entry { key: "DGT" value: 100 }
@@ -473,19 +473,13 @@ CppConfig
 		cpp_entry { key: "DHI" value: 1000 }
 		cpp_entry { key: "DJR" value: 100 }
 		cpp_entry { key: "DKR" value: 1000 }
-		cpp_entry { key: "DMB" value: 100 }
-		cpp_entry { key: "DOB" value: 1000 }
-		cpp_entry { key: "DOP" value: 1000 }
-		cpp_entry { key: "DRB" value: 1000 }
 		cpp_entry { key: "DRI" value: 100 }
 		cpp_entry { key: "DRS" value: 100 }
 		cpp_entry { key: "DRT" value: 100 }
 		cpp_entry { key: "DSD" value: 1000 }
 		cpp_entry { key: "DSE" value: 1000 }
-		cpp_entry { key: "DSG" value: 1000 }
 		cpp_entry { key: "DTR" value: 2 }
 		cpp_entry { key: "DTT" value: 2 }
-		cpp_entry { key: "DUA" value: 6350 }
 		cpp_entry { key: "DUP" value: 1000 }
 		cpp_entry { key: "DVE" value: 200 }
 		cpp_entry { key: "DVT" value: 200 }
@@ -530,8 +524,11 @@ CppConfig
 		cpp_entry { key: "EP1" value: 1000 }
 		cpp_entry { key: "EPN" value: 1000 }
 		cpp_entry { key: "EPZ" value: 500000 }
+		cpp_entry { key: "EQ1" value: 50 }
 		cpp_entry { key: "ERL" value: 5 }
 		cpp_entry { key: "ES" value: 50 }
+		cpp_entry { key: "ES1" value: 50 }
+		cpp_entry { key: "ES2" value: 50 }
 		cpp_entry { key: "ESB" value: 1000 }
 		cpp_entry { key: "ESK" value: 125000 }
 		cpp_entry { key: "ESQ" value: 50 }
@@ -733,8 +730,8 @@ CppConfig
 		cpp_entry { key: "IPO" value: 10 }
 		cpp_entry { key: "IPT" value: 10 }
 		cpp_entry { key: "IS" value: 50 }
-		cpp_entry { key: "ITB" value: 744 }
-		cpp_entry { key: "ITP" value: 264 }
+		cpp_entry { key: "ITB" value: 720 }
+		cpp_entry { key: "ITP" value: 252 }
 		cpp_entry { key: "IW" value: 50 }
 		cpp_entry { key: "J4L" value: 80 }
 		cpp_entry { key: "J7" value: 6250000 }
@@ -748,7 +745,7 @@ CppConfig
 		cpp_entry { key: "JKB" value: 10000 }
 		cpp_entry { key: "JKF" value: 10000 }
 		cpp_entry { key: "JKM" value: 10000 }
-		cpp_entry { key: "JML" value: 480 }
+		cpp_entry { key: "JML" value: 240 }
 		cpp_entry { key: "JNC" value: 1000 }
 		cpp_entry { key: "JNG" value: 10000 }
 		cpp_entry { key: "JNL" value: 80 }
@@ -776,6 +773,7 @@ CppConfig
 		cpp_entry { key: "K5" value: 20 }
 		cpp_entry { key: "K5K" value: 20 }
 		cpp_entry { key: "K5N" value: 20 }
+		cpp_entry { key: "KC" value: 375 }
 		cpp_entry { key: "KE" value: 50 }
 		cpp_entry { key: "KET" value: 50 }
 		cpp_entry { key: "KQ" value: 20 }
@@ -1005,6 +1003,7 @@ CppConfig
 		cpp_entry { key: "OAD" value: 1 }
 		cpp_entry { key: "ODO" value: 1000 }
 		cpp_entry { key: "OFF" value: 5 }
+		cpp_entry { key: "OJ" value: 150 }
 		cpp_entry { key: "OML" value: 5 }
 		cpp_entry { key: "OMM" value: 80 }
 		cpp_entry { key: "OOD" value: 5 }
@@ -1147,7 +1146,7 @@ CppConfig
 		cpp_entry { key: "SBT" value: 50 }
 		cpp_entry { key: "SD" value: 1000 }
 		cpp_entry { key: "SDA" value: 250 }
-		cpp_entry { key: "SDD" value: 1760 }
+		cpp_entry { key: "SDD" value: 1600 }
 		cpp_entry { key: "SDG" value: 250 }
 		cpp_entry { key: "SDI" value: 1000 }
 		cpp_entry { key: "SDM" value: 100 }
@@ -1186,7 +1185,7 @@ CppConfig
 		cpp_entry { key: "SR3" value: 2500 }
 		cpp_entry { key: "SR5" value: 1000 }
 		cpp_entry { key: "SRB" value: 1000 }
-		cpp_entry { key: "SRT" value: 1760 }
+		cpp_entry { key: "SRT" value: 1600 }
 		cpp_entry { key: "SSI" value: 1000 }
 		cpp_entry { key: "SSP" value: 5000 }
 		cpp_entry { key: "SSW" value: 1000 }
@@ -1197,27 +1196,28 @@ CppConfig
 		cpp_entry { key: "STY" value: 1000 }
 		cpp_entry { key: "SU" value: 250 }
 		cpp_entry { key: "SUT" value: 250 }
+		cpp_entry { key: "T01" value: 800 }
+		cpp_entry { key: "T02" value: 800 }
 		cpp_entry { key: "T03" value: 800 }
 		cpp_entry { key: "T04" value: 800 }
-		cpp_entry { key: "T06" value: 800 }
 		cpp_entry { key: "T07" value: 800 }
 		cpp_entry { key: "T08" value: 800 }
 		cpp_entry { key: "T09" value: 800 }
+		cpp_entry { key: "T10" value: 800 }
+		cpp_entry { key: "T11" value: 800 }
 		cpp_entry { key: "T12" value: 800 }
 		cpp_entry { key: "T13" value: 800 }
-		cpp_entry { key: "T14" value: 800 }
-		cpp_entry { key: "T15" value: 800 }
 		cpp_entry { key: "T16" value: 800 }
+		cpp_entry { key: "T17" value: 800 }
+		cpp_entry { key: "T18" value: 800 }
 		cpp_entry { key: "T19" value: 800 }
 		cpp_entry { key: "T1U" value: 1000 }
 		cpp_entry { key: "T20" value: 800 }
-		cpp_entry { key: "T21" value: 800 }
-		cpp_entry { key: "T22" value: 800 }
 		cpp_entry { key: "T23" value: 800 }
+		cpp_entry { key: "T24" value: 800 }
+		cpp_entry { key: "T25" value: 800 }
 		cpp_entry { key: "T26" value: 800 }
 		cpp_entry { key: "T27" value: 800 }
-		cpp_entry { key: "T28" value: 800 }
-		cpp_entry { key: "T29" value: 800 }
 		cpp_entry { key: "T2B" value: 1000 }
 		cpp_entry { key: "T2D" value: 1000 }
 		cpp_entry { key: "T2M" value: 100 }
@@ -1276,7 +1276,7 @@ CppConfig
 		cpp_entry { key: "TT" value: 50000 }
 		cpp_entry { key: "TTD" value: 24 }
 		cpp_entry { key: "TTE" value: 10000 }
-		cpp_entry { key: "TTF" value: 744 }
+		cpp_entry { key: "TTF" value: 720 }
 		cpp_entry { key: "TUB" value: 1000 }
 		cpp_entry { key: "TUF" value: 1000 }
 		cpp_entry { key: "TUL" value: 1000 }
@@ -1295,12 +1295,11 @@ CppConfig
 		cpp_entry { key: "UDL" value: 80 }
 		cpp_entry { key: "UFB" value: 100 }
 		cpp_entry { key: "UFE" value: 100 }
-		cpp_entry { key: "UFG" value: 100 }
 		cpp_entry { key: "UFN" value: 100 }
 		cpp_entry { key: "UFU" value: 100 }
 		cpp_entry { key: "UFV" value: 100 }
 		cpp_entry { key: "UGG" value: 42000 }
-		cpp_entry { key: "UKG" value: 310 }
+		cpp_entry { key: "UKG" value: 300 }
 		cpp_entry { key: "ULB" value: 42000 }
 		cpp_entry { key: "UMB" value: 1000 }
 		cpp_entry { key: "UMD" value: 1000 }
@@ -1308,6 +1307,9 @@ CppConfig
 		cpp_entry { key: "UN" value: 1000 }
 		cpp_entry { key: "UNB" value: 1000 }
 		cpp_entry { key: "UNS" value: 1000 }
+		cpp_entry { key: "UP5" value: 1000 }
+		cpp_entry { key: "UPB" value: 1000 }
+		cpp_entry { key: "UPM" value: 1000 }
 		cpp_entry { key: "UR" value: 1000 }
 		cpp_entry { key: "USC" value: 1000 }
 		cpp_entry { key: "USDBRL" value: 1 }
@@ -1455,12 +1457,12 @@ CppConfig
 		cpp_entry { key: "ZT" value: 2000 }
 		cpp_entry { key: "ZW" value: 50 }
 		cpp_entry { key: "ZWT" value: 50 }
-	}
-}
+	}
+}
 
-CmeSymbolMics
-{
-	value
+CmeSymbolMics
+{
+	value
 	{
 		cme_mic { key: "098" value: CBT }
 		cme_mic { key: "0A" value: NYMEX }
@@ -1809,6 +1811,7 @@ CmeSymbolMics
 		cme_mic { key: "BOB" value: CBT }
 		cme_mic { key: "BOO" value: NYMEX }
 		cme_mic { key: "BOS" value: CME }
+		cme_mic { key: "BPU" value: NYMEX }
 		cme_mic { key: "BR7" value: NYMEX }
 		cme_mic { key: "BSF" value: CBT }
 		cme_mic { key: "BTC" value: CME }
@@ -1897,12 +1900,12 @@ CmeSymbolMics
 		cme_mic { key: "D3L" value: NYMEX }
 		cme_mic { key: "D4L" value: NYMEX }
 		cme_mic { key: "D7L" value: NYMEX }
+		cme_mic { key: "DAB" value: NYMEX }
 		cme_mic { key: "DAX" value: NYMEX }
 		cme_mic { key: "DAZ" value: NYMEX }
 		cme_mic { key: "DBB" value: NYMEX }
 		cme_mic { key: "DBI" value: DME }
 		cme_mic { key: "DBL" value: NYMEX }
-		cme_mic { key: "DBR" value: DME }
 		cme_mic { key: "DBT" value: NYMEX }
 		cme_mic { key: "DC" value: CME }
 		cme_mic { key: "DCB" value: NYMEX }
@@ -1920,7 +1923,6 @@ CmeSymbolMics
 		cme_mic { key: "DFN" value: CBT }
 		cme_mic { key: "DG2" value: CBT }
 		cme_mic { key: "DG3" value: CBT }
-		cme_mic { key: "DGH" value: DME }
 		cme_mic { key: "DGI" value: CBT }
 		cme_mic { key: "DGS" value: CBT }
 		cme_mic { key: "DGT" value: CBT }
@@ -1928,19 +1930,13 @@ CmeSymbolMics
 		cme_mic { key: "DHI" value: DME }
 		cme_mic { key: "DJR" value: CBT }
 		cme_mic { key: "DKR" value: DME }
-		cme_mic { key: "DMB" value: DME }
-		cme_mic { key: "DOB" value: DME }
-		cme_mic { key: "DOP" value: DME }
-		cme_mic { key: "DRB" value: DME }
 		cme_mic { key: "DRI" value: CBT }
 		cme_mic { key: "DRS" value: CBT }
 		cme_mic { key: "DRT" value: CBT }
 		cme_mic { key: "DSD" value: DME }
 		cme_mic { key: "DSE" value: DME }
-		cme_mic { key: "DSG" value: DME }
 		cme_mic { key: "DTR" value: CBT }
 		cme_mic { key: "DTT" value: CBT }
-		cme_mic { key: "DUA" value: DME }
 		cme_mic { key: "DUP" value: DME }
 		cme_mic { key: "DVE" value: CME }
 		cme_mic { key: "DVT" value: CME }
@@ -1985,8 +1981,11 @@ CmeSymbolMics
 		cme_mic { key: "EP1" value: NYMEX }
 		cme_mic { key: "EPN" value: NYMEX }
 		cme_mic { key: "EPZ" value: CME }
+		cme_mic { key: "EQ1" value: CME }
 		cme_mic { key: "ERL" value: NYMEX }
 		cme_mic { key: "ES" value: CME }
+		cme_mic { key: "ES1" value: CME }
+		cme_mic { key: "ES2" value: CME }
 		cme_mic { key: "ESB" value: NYMEX }
 		cme_mic { key: "ESK" value: CME }
 		cme_mic { key: "ESQ" value: CME }
@@ -2652,27 +2651,28 @@ CmeSymbolMics
 		cme_mic { key: "STY" value: NYMEX }
 		cme_mic { key: "SU" value: CME }
 		cme_mic { key: "SUT" value: CME }
+		cme_mic { key: "T01" value: NYMEX }
+		cme_mic { key: "T02" value: NYMEX }
 		cme_mic { key: "T03" value: NYMEX }
 		cme_mic { key: "T04" value: NYMEX }
-		cme_mic { key: "T06" value: NYMEX }
 		cme_mic { key: "T07" value: NYMEX }
 		cme_mic { key: "T08" value: NYMEX }
 		cme_mic { key: "T09" value: NYMEX }
+		cme_mic { key: "T10" value: NYMEX }
+		cme_mic { key: "T11" value: NYMEX }
 		cme_mic { key: "T12" value: NYMEX }
 		cme_mic { key: "T13" value: NYMEX }
-		cme_mic { key: "T14" value: NYMEX }
-		cme_mic { key: "T15" value: NYMEX }
 		cme_mic { key: "T16" value: NYMEX }
+		cme_mic { key: "T17" value: NYMEX }
+		cme_mic { key: "T18" value: NYMEX }
 		cme_mic { key: "T19" value: NYMEX }
 		cme_mic { key: "T1U" value: CBT }
 		cme_mic { key: "T20" value: NYMEX }
-		cme_mic { key: "T21" value: NYMEX }
-		cme_mic { key: "T22" value: NYMEX }
 		cme_mic { key: "T23" value: NYMEX }
+		cme_mic { key: "T24" value: NYMEX }
+		cme_mic { key: "T25" value: NYMEX }
 		cme_mic { key: "T26" value: NYMEX }
 		cme_mic { key: "T27" value: NYMEX }
-		cme_mic { key: "T28" value: NYMEX }
-		cme_mic { key: "T29" value: NYMEX }
 		cme_mic { key: "T2B" value: NYMEX }
 		cme_mic { key: "T2D" value: NYMEX }
 		cme_mic { key: "T2M" value: NYMEX }
@@ -2750,7 +2750,6 @@ CmeSymbolMics
 		cme_mic { key: "UDL" value: NYMEX }
 		cme_mic { key: "UFB" value: CBT }
 		cme_mic { key: "UFE" value: CBT }
-		cme_mic { key: "UFG" value: CBT }
 		cme_mic { key: "UFN" value: CBT }
 		cme_mic { key: "UFU" value: CBT }
 		cme_mic { key: "UFV" value: CBT }
@@ -2763,6 +2762,9 @@ CmeSymbolMics
 		cme_mic { key: "UN" value: NYMEX }
 		cme_mic { key: "UNB" value: NYMEX }
 		cme_mic { key: "UNS" value: NYMEX }
+		cme_mic { key: "UP5" value: NYMEX }
+		cme_mic { key: "UPB" value: NYMEX }
+		cme_mic { key: "UPM" value: NYMEX }
 		cme_mic { key: "UR" value: NYMEX }
 		cme_mic { key: "USC" value: NYMEX }
 		cme_mic { key: "USDBRL" value: CME }
@@ -2910,5 +2912,5 @@ CmeSymbolMics
 		cme_mic { key: "ZT" value: CBT }
 		cme_mic { key: "ZW" value: CBT }
 		cme_mic { key: "ZWT" value: CBT }
-	}
+	}
 }

src/lib/config/defaults/jetfix.conf

@@ -5,7 +5,7 @@ JetFIXSimConnection {
         name: "sim"
         platform: SIM
         socket_connect_host_ip: "127.0.0.1"
-        socket_connect_port: @Ports.exchange_sim_fix_port
+        socket_connect_port: @Ports.glbx_sim_fix_port
         sender_compid: "SPARK" # this is the FIX session name
         sender_subid: "BTS"
         target_compid: "BTS"
@@ -18,9 +18,173 @@ JetFIXSimConnection {
         seconds_to_pause : 1
 	max_connection_attempts: 9999999
       }
+      sim_ice {
+        name: "sim_ice"
+        platform: SIMICE
+        socket_connect_host_ip: "127.0.0.1"
+        socket_connect_port: @Ports.ice_sim_fix_port
+        sender_compid: "51239" # this is the FIX session name
+        sender_subid: "BTS"
+        target_compid: "BTS"
+        target_subid: "BTS"
+        cme_market_segment_id: 0
+        end_time: "02:00:00"
+        start_time: "02:05:00"
+        raw_data: "1JE9X"
+        use_cme_secure_logon: false 
+        seconds_to_pause : 1
+      }      
    }
 }
 
+JetFIX {
+   rule: "CannedCertSession.name='ice_cert'"
+   value {
+      #connection: @JetFIXSimConnection.sim_ice
+      connection {
+         name: "ice_cert"
+         platform: ICE
+         use_cme_secure_logon: false
+         log_fix_stream: true
+         socket_connect_host_ip: "63.247.113.154"
+         socket_connect_port: 80
+         sender_compid: "51251"
+         target_compid: "ICE"         
+         start_time: "22:55:00"
+         end_time: "22:05:00"
+         sender_subid: "BTS"
+         user_name: "bluetrd-os" #this is the order_server_id
+         raw_data: "Starts123"
+         max_order_msgs_per_window: 300
+         auto_fix_seq_num: true
+         
+         seconds_to_replay: 6000
+      }
+      connection: @JetFIXSimConnection.sim
+      #connection: @JetFIXSimConnection.sim_ice
+      
+      connection {
+         name: "cme_mc5_99"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.199"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "99"
+         cme_market_segment_id: 99
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_58"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.172"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "58"
+         cme_market_segment_id: 58
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_54"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.191"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "54"
+         cme_market_segment_id: 54
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_82"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.179"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "82"
+         cme_market_segment_id: 82
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_56"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.174"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "56"
+         cme_market_segment_id: 56
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_50"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.202"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "50"
+         cme_market_segment_id: 50
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_52"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.181"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "52"
+         cme_market_segment_id: 52
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }      
+   }
+}   
+
 JetFIX {
    rule: "CannedCertSession.name='mc7'"
    value {
@@ -456,193 +620,193 @@ JetFIX {
          raw_data: "S4PMA"
          auto_fix_seq_num: true
       }
-      connection {
-         name: "cme_mc5_60"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.195"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "60"
-         cme_market_segment_id: 60
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_76"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.204"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "76"
-         cme_market_segment_id: 76
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_88"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.169"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "88"
-         cme_market_segment_id: 88
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_74"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.197"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "74"
-         cme_market_segment_id: 74
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_64"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.177"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "64"
-         cme_market_segment_id: 64
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_72"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.189"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "72"
-         cme_market_segment_id: 72
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_70"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.183"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "70"
-         cme_market_segment_id: 70
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_68"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.168"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "68"
-         cme_market_segment_id: 68
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_80"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.187"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "80"
-         cme_market_segment_id: 80
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_84"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.198"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "84"
-         cme_market_segment_id: 84
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_78"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.167"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "78"
-         cme_market_segment_id: 78
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
+      # connection {
+      #    name: "cme_mc5_60"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.195"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "60"
+      #    cme_market_segment_id: 60
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_76"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.204"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "76"
+      #    cme_market_segment_id: 76
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_88"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.169"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "88"
+      #    cme_market_segment_id: 88
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_74"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.197"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "74"
+      #    cme_market_segment_id: 74
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_64"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.177"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "64"
+      #    cme_market_segment_id: 64
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_72"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.189"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "72"
+      #    cme_market_segment_id: 72
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_70"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.183"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "70"
+      #    cme_market_segment_id: 70
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_68"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.168"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "68"
+      #    cme_market_segment_id: 68
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_80"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.187"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "80"
+      #    cme_market_segment_id: 80
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_84"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.198"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "84"
+      #    cme_market_segment_id: 84
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_78"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.167"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "78"
+      #    cme_market_segment_id: 78
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
    }
 }
 JetFIX {

src/lib/config/defaults/keycode.conf

@@ -1,9 +1,9 @@
 #nusers: 1
-#allow concurrent : false
-#expires: 2019-Sep-30 00:00:00   637053984000000000
-#generation_time: 2019-Jun-18 18:25:44.333280   5248650809870720704
+#allow concurrent : true
+#expires: 2019-Dec-31 00:00:00   637133472000000000
+#generation_time: 2019-Sep-16 17:50:57.225054   5248728548999638444
 EncryptedKeyCode {
    value {
-      keycode: "pgtv/OmievnpmNlgxTYuN801Hbetgu7ZGkHnlGWtK+DBrKSEgKxr5lGvCW1ld/jsztkWhKHDh0iediMDjrPm4+C5qmXlt4JJlAFvKolMN8HVC59m82fECvakcqKq28WSEjf6/DuPJNP1ZwxjcmKouxx1LZ15VlNkECtNrC9pkCnifazbfQZGV3kD08zMEyw/RPmOaVLFFeav8lwI5UpcjPYBIK1rJowXdGSvm1CJViwG5Z2JbdHR39QbKjVx112d5VbVtBq9nO3i0FEKvobjN/Gzcz/5x4TgQtEShKsbmgEN+MBVL3AItQlcFzZvE/05JLmfACUFmk5Eu0a3DlLs3A=="
+      keycode: "E7sUCxYuveyuvvBfwMdTrUCpFROJWQvsy4pYjwZ+zonjHwEmDkjxbxaqUEKqMzBbAq3YFnxIGOrQpRJte71SEoG5FGy4ZoRrVOjrMdNVLmieUWtZU43FKGJzImj/tJKEXu2xgwYJRZTj/AJ8C30PuVnzm0qUmR+UCgCOxKVeK0OFlo6zSqyDutcy5kLW3va+nTuFp43aknm6Iiu8LFW16wa6QDDCSQd4QgnHmiz5wyMC90yGBE4qObTKRGKDl5mWsjEL16aWK2ZO/0/ebFgGG8WygQfkeFzJS7B4q5Ta0nkSkVyLZjq1LN2/g9AXeCzCJUX5/F96kdfsnfSgAJepeg=="
    }
 }

src/lib/config/defaults/orders_fills_cleanup.conf

@@ -2,4 +2,14 @@ OrdersSoftCleanupCutoff {
    value {
       extra_days: 1
    }
+}
+
+SODTime 
+{
+   rule: "DeploymentType.type == 'dev'"
+   value
+   {
+      sod_hh: 20
+      sod_mm: 30
+   }
 }

src/lib/config/protos/MarketData.proto

@@ -155,6 +155,7 @@ message ICEOptions
    message ICEChannel {
       required string name = 1;
       repeated uint32 market_type = 2;
+      repeated uint32 uds_market_type = 7;
       optional uint32 udp_buffer_size = 3 [default = 4194304]; //4mb      
       optional uint32 heartbeat_interval = 4 [default = 36000];
       optional uint32 sequence_gap_margin = 5 [default = 10000];

src/lib/config/protos/Ports.proto

@@ -5,6 +5,7 @@ message Ports {
   // see ports.conf for actual defaults
   optional int32 config_port = 1 ;
   optional int32 qbus_port = 2 ;
-  optional int32 exchange_sim_fix_port =  3 ;
+  optional int32 glbx_sim_fix_port =  3 ;
+  optional int32 ice_sim_fix_port =  5 ;
   optional int32 replicator_port = 4 ;
 }

src/lib/config/protos/Symbols.proto

@@ -20,4 +20,5 @@ message Symbols {
   optional SecurityExchangeList glbx = 10;
   optional SecurityExchangeList sim = 20;
   optional SecurityExchangeList ice = 30;
+  optional SecurityExchangeList simice = 40;
 }

src/lib/config/protos/jetfix.proto

@@ -7,9 +7,11 @@ message JetFIX {
     required string name = 10;
 
     enum JetFIXPlatform {
-	GLBX = 1;
-	CBOE = 6;
-        SIM = 2;
+	   GLBX = 1;
+	   CBOE = 6;
+      SIM = 2;
+      ICE = 3;
+      SIMICE = 4;
     }
     optional JetFIXPlatform platform = 15                       [default = GLBX];
     
@@ -47,6 +49,7 @@ message JetFIX {
     optional string target_subid = 130				[default = ""];
     optional string sender_locationid = 135			[default = "US,NC"];
     optional string raw_data = 136				[default =""];
+    optional string user_name = 138 				[default =""];
     optional string self_match_prevention_id = 140		[default=""];
     optional string cti_code = 145				[default="1"];
     optional int32  customer_or_firm = 147		        [default=1];
@@ -91,6 +94,7 @@ message JetFIX {
 
 message JetFIXSimConnection {
   required JetFIX.Connection sim = 10;
+  required JetFIX.Connection sim_ice = 20;
 }
 
 message CannedCertSession {

src/lib/domain_model/Security.cpp

@@ -29,9 +29,10 @@ std::string descriptor(api::Security const &sec) {
    else if (sec.has_ice())
       return sec.ice().security_desc();
    else
-      return "No descriptor for non glbx exchange";
+      return "No descriptor for non glbx / ice exchange";
 }
 
+
 boost::gregorian::date smallest_leg(
     ::google::protobuf::RepeatedPtrField<::bts::api::ContractLeg> const &l) {
    bts_assert(l.size());

src/lib/domain_model/Security.h

@@ -191,5 +191,6 @@ inline bool gr_than(double p1, double p2, double tick_size) {
    return gr_than_eq(p1, p2, tick_size) &&
           !test_price_equality(p1, p2, tick_size);
 }
-} // namespace securities
-} // namespace bts
+
+
+}}

src/lib/domain_model/Security.proto

@@ -12,6 +12,7 @@ enum Venue {
      VENUE_CBOECFE = 207;
      VENUE_GLBX = 301;
      VENUE_ICE = 306;
+     VENUE_SIMICE = 3;
 }
 
 enum SecurityType {
@@ -54,11 +55,12 @@ enum CMEMics {
 }
 
 enum ICEMics{
-   ICE_UNSET=0;
-   ICE_US = 1;
-   ICE_EU = 2;
-   ICE_LIFFE = 3;
-   ICE_CA = 4;
+   IF_UNSET = 0;
+   IFU = 1;
+   IFE = 2;
+   IFL = 3;
+   IFC = 4;
+   IFEN = 5;
 }
 
 enum Side {
@@ -103,8 +105,9 @@ message CBOECFEExchangeData {
 message ICEExchangeData {
    int64 exchange_id = 10;
    string security_desc = 15; //contractSymbol
-   int32 deal_display_factor = 20;
-   int32 settle_display_factor = 30;
+   int32 price_denominator = 17;
+   int32 deal_denominator = 20;
+   int32 settle_denominator = 30;
    
    string market_description = 100;
    string cleared_alias = 110;

src/lib/jetfix/codec/FIXEncodingStream.h

@@ -1,11 +1,11 @@
 #pragma once
 
-#include <memory>
 #include "lib/jetfix/codec/FIXBuffer.h"
-#include "lib/jetfix/codec/MessageTypes.h"
 #include "lib/jetfix/codec/FIXFragment.h"
-#include "lib/jetfix/codec/Fields.h"
 #include "lib/jetfix/codec/FieldDefinitions.h"
+#include "lib/jetfix/codec/Fields.h"
+#include "lib/jetfix/codec/MessageTypes.h"
+#include <memory>
 
 #include "lib/config/protos/jetfix.pb.h"
 
@@ -14,185 +14,186 @@ namespace jetfix {
 
 // Helper to batch the encoding of multiple FIX messages all sharing
 // the same FIX headers.
-class FIXEncodingStream
-{
-public:
+class FIXEncodingStream {
+ public:
    FIXEncodingStream(config::JetFIX::Connection const &cfg,
-                     uint32_t initial_capacity = 1024*10)
-      :
-      _msg_header_template (),
-      _header_template_size (),
-      _header_template_checksum (),
-      _buffer (initial_capacity),
-      _message_map (),
-      _current_msg (_buffer),
-      _state (State::OPEN),
-      _body_length_bm (),
-      _msg_type_bm (),
-      _seq_num_bm (),
-      _last_seq_num_bm (),
-      _sending_time_bm ()
-   {
-      _message_map.reserve (10*1024);
+                     uint32_t initial_capacity = 1024 * 10)
+       : _msg_header_template(), _header_template_size(),
+         _header_template_checksum(), _buffer(initial_capacity), _message_map(),
+         _current_msg(_buffer), _state(State::OPEN), _seq_num_bm(),
+         _last_seq_num_bm(), _sending_time_bm() {
+      _message_map.reserve(10 * 1024);
 
-      FIXBuffer header_buffer (&_msg_header_template[0],_msg_header_template.size ());
-      FIXFragment header (header_buffer);
-      header.add<fix::BeginString>("FIX.4.2");
+      FIXBuffer header_buffer(&_msg_header_template[0],
+                              _msg_header_template.size());
+      FIXFragment header(header_buffer);
 
-      _body_length_bm = header.add<fix::BodyLength>(0,6);
-      _msg_type_bm = header.add<fix::MsgType>(" ",1);
-      _seq_num_bm = header.add<fix::MsgSeqNum>(0,9);
-      _last_seq_num_bm = header.add<fix::LastSeqNumProcessed>(0,9);
+      // We cannot include body lenght and msg type in the header
+      // because message type is not fixed lenght, and therefore its
+      // not possible to have bookmarks to fields after it, ahead of
+      // knowing the message type value. The rest of the header is
+      // fixed lenght so we form it here.
+      _seq_num_bm = header.add<fix::MsgSeqNum>(0, 9);
+      _last_seq_num_bm = header.add<fix::LastSeqNumProcessed>(0, 9);
       _sending_time_bm = header.add<SendingTimeStr>(
-          "20150427-19:05:18.923",sizeof("20150427-19:05:18.923")-1);
+          "20150427-19:05:18.923", sizeof("20150427-19:05:18.923") - 1);
 
-      header.add<fix::SenderCompID>(cfg.sender_compid ());
-      header.add<fix::TargetCompId>(cfg.target_compid ());
-      header.add<fix::TargetSubID>(cfg.target_subid ());
-      _header_template_size = header.size ();
-      _header_template_checksum = header.checksum ();
+      header.add<fix::SenderCompID>(cfg.sender_compid());
+      header.add<fix::TargetCompId>(cfg.target_compid());
+      header.add<fix::TargetSubID>(cfg.target_subid());
+      _header_template_size = header.size();
+      _header_template_checksum = header.checksum();
    }
 
-   inline FIXFragment &new_msg (MessageTypes type)
-   {
-      bts_assert (_state == State::OPEN);
-      tidy_last_msg ();
-      
-      auto msg_offset = _buffer.size ();
-      _message_map.emplace_back (msg_offset,0u,0u,0u,FIXFragment::Bookmark<IntField<10u>>());
-      _current_msg = FIXFragment (_buffer);
-      _current_msg.append (&_msg_header_template[0],
-                           _header_template_size,
-                           _header_template_checksum);
-      char msg_type_str[1] = {(char) type};
-      _current_msg.overwrite (_msg_type_bm.shift (msg_offset),msg_type_str,1);
+   inline FIXFragment &new_msg(MessageTypes type) {
+      bts_assert(_state == State::OPEN);
+      tidy_last_msg();
+
+      auto msg_offset = _buffer.size();
+
+      _current_msg = FIXFragment(_buffer);
+      _begin_string_bm = _current_msg.add<fix::BeginString>("FIX.4.2");
+      _body_length_bm = _current_msg.add<fix::BodyLength>(0, 6);
+
+      char mt_encoding[4];
+      auto start = encode_message_type(type, mt_encoding);
+      _msg_type_bm = _current_msg.add<fix::MsgType>(
+          mt_encoding + start, sizeof(mt_encoding) - start);
+
+      auto header_offset = _buffer.size();
+      _message_map.emplace_back(msg_offset, header_offset, 0u, 0u, 0u,
+                                FIXFragment::Bookmark<IntField<10u>>());
+      _current_msg.append(&_msg_header_template[0], _header_template_size,
+                          _header_template_checksum);
       return _current_msg;
    }
-   
-   inline void close ()
-   {
-      bts_assert (_state == State::OPEN);
-      tidy_last_msg ();
+
+   inline void close() {
+      bts_assert(_state == State::OPEN);
+      tidy_last_msg();
       _state = State::CLOSED;
    }
 
-   inline void reopen_empty ()
-   {
-      bts_assert (_state == State::CLOSED);
-      bts_assert (_message_map.empty ());
+   inline void reopen_empty() {
+      bts_assert(_state == State::CLOSED);
+      bts_assert(_message_map.empty());
       _state = State::OPEN;
    }
-   
-   inline uint32_t size () const
-   {return _buffer.size ();}
 
-   inline const char* data () const
-   {return _buffer.data ();}
+   inline uint32_t size() const { return _buffer.size(); }
 
-   struct MessageDesc
-   {
-      MessageDesc (uint32_t offset,
-                   uint32_t size,
-                   uint32_t checksum,
-                   uint32_t seq_num,
-                   FIXFragment::Bookmark<fix::Checksum> checksum_bm)
-         : offset (offset),
-           size (size),
-           checksum (checksum),
-           seq_num (seq_num),
-           checksum_bm (checksum_bm)
-      {}
-      
+   inline const char *data() const { return _buffer.data(); }
+
+   struct MessageDesc {
+      MessageDesc(uint32_t offset, uint32_t header_offset, uint32_t size,
+                  uint32_t checksum, uint32_t seq_num,
+                  FIXFragment::Bookmark<fix::Checksum> checksum_bm)
+          : offset(offset), header_offset(header_offset), size(size),
+            checksum(checksum), seq_num(seq_num), checksum_bm(checksum_bm) {}
 
       uint32_t offset;
+      uint32_t header_offset;
       uint32_t size;
       uint32_t checksum;
       uint32_t seq_num;
       FIXFragment::Bookmark<fix::Checksum> checksum_bm;
    };
-   
-   std::string to_str () const 
-   {
+
+   std::string to_str() const {
       std::string result;
-      
-      if (size ())  {
+
+      if (size()) {
          // return std::string(data(),size() is simpler but valgrind
          // reports a warning in that call
-         result.resize (size ());
-         std::copy (data (), data ()+size(),result.begin ());
+         result.resize(size());
+         std::copy(data(), data() + size(), result.begin());
       }
       return result;
    }
 
-   std::string to_human_str () const 
-   {
-      auto s = to_str ();
-      std::replace( s.begin(), s.end(), '\001', '|' );
+   std::string to_human_str() const {
+      auto s = to_str();
+      std::replace(s.begin(), s.end(), '\001', '|');
       return s;
    }
-   
-   inline uint32_t stamp (uint32_t seq_num,uint32_t last_processed, std::string const&time_str)
-   {
-      bts_assert (_state == State::CLOSED);
+
+   inline uint32_t stamp(uint32_t seq_num, uint32_t last_processed,
+                         std::string const &time_str) {
+      bts_assert(_state == State::CLOSED);
       _state = State::STAMPED;
 
       for (auto &m : _message_map) {
          ++seq_num;
          m.seq_num = seq_num;
 
-         FIXFragment frag (_buffer,m.checksum);
-         frag.overwrite (_seq_num_bm.shift (m.offset),m.seq_num,9);
-         frag.overwrite (_last_seq_num_bm.shift (m.offset),last_processed,9);
-         frag.overwrite (_sending_time_bm.shift (m.offset),time_str);
-         frag.overwrite_checksum (m.checksum_bm,frag.checksum () % 256);
+         FIXFragment frag(_buffer, m.checksum);
+         frag.overwrite(_seq_num_bm.shift(m.header_offset), m.seq_num, 9);
+         frag.overwrite(_last_seq_num_bm.shift(m.header_offset), last_processed,
+                        9);
+         frag.overwrite(_sending_time_bm.shift(m.header_offset), time_str);
+         frag.overwrite_checksum(m.checksum_bm, frag.checksum() % 256);
       }
       return seq_num;
    }
 
-   inline void reset ()
-   {
-      _buffer.reset ();
-      _message_map.clear ();
+   inline void reset() {
+      _buffer.reset();
+      _message_map.clear();
       _state = State::OPEN;
    }
 
-   inline bool is_closed () const 
-   {return _state == State::CLOSED;}
+   inline bool is_closed() const { return _state == State::CLOSED; }
 
-   size_t n_msgs () const {return _message_map.size ();}
-   
-   std::vector<MessageDesc> const &msg_map () const {return _message_map;}
-private:
-   std::array<char,256> _msg_header_template;
+   size_t n_msgs() const { return _message_map.size(); }
+
+   std::vector<MessageDesc> const &msg_map() const { return _message_map; }
+
+ private:
+   std::array<char, 256> _msg_header_template;
    uint32_t _header_template_size;
    uint32_t _header_template_checksum;
    FIXBuffer _buffer;
    std::vector<MessageDesc> _message_map;
    FIXFragment _current_msg;
-   enum class State { OPEN,CLOSED,STAMPED};
+   enum class State { OPEN, CLOSED, STAMPED };
    State _state;
-   
+
+   // These bookmarks point to the current msg (not the header)
+   FIXFragment::Bookmark<fix::BeginString> _begin_string_bm;
    FIXFragment::Bookmark<fix::BodyLength> _body_length_bm;
    FIXFragment::Bookmark<fix::MsgType> _msg_type_bm;
+
+   // These bookmarks point to the header and thus have to be shifted
+   // by the offset of the end of the message type.
    FIXFragment::Bookmark<fix::MsgSeqNum> _seq_num_bm;
    FIXFragment::Bookmark<fix::LastSeqNumProcessed> _last_seq_num_bm;
    typedef StringField<fix::SendingTime::tag> SendingTimeStr;
    FIXFragment::Bookmark<SendingTimeStr> _sending_time_bm;
 
-   inline void tidy_last_msg ()
-   {
-      if (_message_map.size ()) {
-         auto &this_msg_map = _message_map.back ();
-         auto start =
-            this_msg_map.offset + _msg_type_bm.offset - fix::MsgType::tag_sz-1;
-         auto end = _buffer.size ();
-         _current_msg.overwrite (_body_length_bm.shift (this_msg_map.offset),
-                                 end-start,6);
-         this_msg_map.checksum = _current_msg.checksum ();
-         this_msg_map.checksum_bm = _current_msg.add_checksum (0);
-         this_msg_map.size = _buffer.size () - this_msg_map.offset;
+   inline void tidy_last_msg() {
+      if (_message_map.size()) {
+         auto &this_msg_map = _message_map.back();
+         auto start = _msg_type_bm.offset - fix::MsgType::tag_sz - 1;
+         auto end = _buffer.size();
+         _current_msg.overwrite(_body_length_bm, end - start, 6);
+         this_msg_map.checksum = _current_msg.checksum();
+         this_msg_map.checksum_bm = _current_msg.add_checksum(0);
+         this_msg_map.size = _buffer.size() - this_msg_map.offset;
       }
    }
+
+   uint32_t encode_message_type(MessageTypes m, char out[4]) {
+      auto mt = (uint32_t)m;
+      for (int i = 0; i < 4; i++) {
+         out[i] = mt >> (8 * (3 - i));
+         mt &= ((1 << (8 * (3 - i))) - 1);
+      }
+      for (int i = 0; i < 4; i++)
+         if (out[i])
+            return i;
+      return 0;
+   }
 };
 
-}}
+} // namespace jetfix
+} // namespace bts

src/lib/jetfix/codec/FieldDefinitions.h

@@ -69,6 +69,9 @@ typedef StringField<143> TargetLocationID;
 typedef StringField<1603> ApplicationSystemName;
 typedef StringField<1604> ApplicationSystemVersion;
 typedef StringField<1605> ApplicationSystemVendor;
+typedef StringField<9450> ICEApplicationSystemName;
+typedef StringField<9451> ICEApplicationSystemVersion;
+typedef StringField<9452> ICEApplicationSystemVendor;
 typedef StringField<167> SecurityType;
 typedef StringField<17> ExecID;
 typedef StringField<19> ExecRefID;
@@ -97,6 +100,7 @@ typedef StringField<7928> SelfMatchPreventionID;
 typedef StringField<8> BeginString;
 typedef StringField<9324> OrderOptionalData;
 typedef StringField<96> RawData;
+typedef StringField<553> UserName;
 typedef StringField<9717> CorrelationClOrdID;
 typedef StringField<9771> MMAccount;
 typedef TimeField<122> OrigSendingTime;
@@ -104,6 +108,38 @@ typedef TimeField<52> SendingTime;
 typedef TimeField<5979> RequestTime;
 typedef TimeField<60> TransactTime;
 
+//ICE
+typedef IntField<9208> Int_CtiCode; 
+typedef BigIntField<11> Int_ClOrdID; 
+typedef BigIntField<37> Int_OrderID; 
+typedef BigIntField<41> Int_OrigClOrdID; 
+typedef IntField<109> CompanyID; 
+typedef IntField<55> Int_Symbol; 
+typedef IntField<439> Int_ClearingFirm;
+typedef StringField<9121> Memo;
+typedef IntField<103> OrdRejReason;
+typedef IntField<285> DeleteReason;
+typedef CharField<9195> AccountCode;
+typedef StringField<9207> CustomerAccountRefId;
+typedef StringField<115> OnBehalfOfCompID;
+typedef StringField<116> OnBehalfOfSubID;
+typedef StringField<144> OnBehalfOfLocationID;
+typedef StringField<9139> OriginatorUserID;
+typedef CharField<18> ExecInst;
+typedef IntField<210> MaxShow;
+typedef IntField<9006> ICEStrategyPreference;
+typedef IntField<9007> ICETimeStampPreference;
+typedef StringField<1031> CustOrderHandlingInst;
+
+typedef IntField<9700> MifidDirectElectronicAccess;
+typedef IntField<9701> MifidTradingCapacity;
+typedef IntField<9702> MifidLiquidityProvision;
+typedef IntField<9703> MifidCommodityDerivIndicator;
+typedef BigIntField<9704> MifidInvestmentDecision;
+typedef BigIntField<9705> MifidExecutionDecision;
+typedef BigIntField<9706> MifidClientIDCode;
+typedef BigIntField<9707> MifidId;
+
 typedef Group<NoQuoteEntries,
               QuoteEntryID,
               Symbol,

src/lib/jetfix/codec/Fields.h

@@ -156,6 +156,33 @@ public:
    }
 };
 
+template <uint16_t tag>
+class BigIntField : public BaseField<tag,int64_t>
+{
+public:
+   typedef BaseField<tag,int64_t> Parent;
+
+   void read (ReaderCursor &cursor)
+   {
+      auto loc = cursor.read_field ();
+      Parent::_value = atoll (&(cursor.data ()[loc.first]));
+   }
+
+   static uint32_t max_value_size (int64_t,int pad = 0)
+   {
+      if (pad == 0)
+         return 20;
+      else
+         return pad;
+   }
+   
+   static std::pair<uint32_t,uint32_t> write (char*buffer,int64_t i,int pad = 0)
+   {
+      auto written = i64_toa (i,buffer,pad);
+      return std::make_pair (written,sum_bytes (buffer,written));
+   }
+};
+
 template <uint16_t tag>
 class StringField : public BaseField<tag,std::string>
 {

src/lib/jetfix/codec/MessageDefinitions.h

@@ -66,5 +66,58 @@ typedef FIXMsg<MessageTypes::CancelReject,
                CorrelationClOrdID>
 CancelRejectMsg;
 
+typedef FIXMsg<MessageTypes::ICETraderLoginResponse, 
+               CompanyID,
+               UserName,
+               Text>
+ICETraderLoginResponseMsg;
+
+typedef FIXMsg<MessageTypes::ICETraderLogoutResponse, 
+               CompanyID,
+               UserName,
+               Text>
+ICETraderLogoutResponseMsg;
+
+typedef FIXMsg<MessageTypes::ExecutionReport,
+               Int_OrderID,
+               Int_ClOrdID,
+               CompanyID,
+               ExecID,               
+               ExecRefID,
+               ExecType,
+               OrdStatus,
+               OrdRejReason,               
+               Int_Symbol,               
+               Side,
+               OrderQty,
+               MaxShow,
+               OrdType,
+               Price,
+               StopPx,
+               TimeInForce,
+               LeavesQty,
+               CumQty,
+               AvgPx,
+               LastPx,
+               LastShares,               
+               TransactTimeStr,
+               MultiLegReportingType,               
+               ClearingAccount,            
+               Memo,                                       
+               Text,
+               DeleteReason,
+               TargetSubID,
+               MsgSeqNum>
+ICEExecutionReportMsg;
+
+typedef FIXMsg<MessageTypes::CancelReject,
+               Int_OrderID,
+               Int_ClOrdID,
+               Int_OrigClOrdID,
+               CXlRejReason,
+               Text,
+               Memo>
+ICECancelRejectMsg;
+
 }}}
 

src/lib/jetfix/codec/MessageTypes.h

@@ -5,7 +5,7 @@
 namespace bts {
 namespace jetfix {
 
-enum class MessageTypes
+enum class MessageTypes : uint32_t
 {
    Advertisement               = '7',
    Allocation                  = 'J',
@@ -48,7 +48,12 @@ enum class MessageTypes
    SettlementInstructions      = 'T',
    TestRequest                 = '1',
    TradingSessionStatus        = 'h',
-   TradingSessionStatusRequest = 'g'
+   TradingSessionStatusRequest = 'g',
+   ICETraderLogin              = ('U' << 16) + ('C' << 8) + 'G',
+   ICETraderLoginResponse      = ('U' << 16) + ('C' << 8) + 'I',   
+   ICETraderLogout             = ('U' << 16) + ('C' << 8) + 'H',   
+   ICETraderLogoutResponse     = ('U' << 16) + ('C' << 8) + 'J',   
+   
 };
 
 }}

src/lib/jetfix/codec/i32_toa.h

@@ -259,3 +259,199 @@ inline int u32_toa(uint32_t u, char* buffer, int pad) {
 }
 
 
+///https://github.com/miloyip/itoa-benchmark
+
+#define BEGIN2(n) \
+    do { \
+        int t = (n); \
+        if(t < 10) *p++ = '0' + t; \
+        else { \
+            t *= 2; \
+            *p++ = gDigitsLut[t]; \
+            *p++ = gDigitsLut[t + 1]; \
+        } \
+    } while(0)
+#define MIDDLE2(n) \
+    do { \
+        int t = (n) * 2; \
+        *p++ = gDigitsLut[t]; \
+        *p++ = gDigitsLut[t + 1]; \
+    } while(0)
+#define BEGIN4(n) \
+    do { \
+        int t4 = (n); \
+        if(t4 < 100) BEGIN2(t4); \
+        else { BEGIN2(t4 / 100); MIDDLE2(t4 % 100); } \
+    } while(0)
+#define MIDDLE4(n) \
+    do { \
+        int t4 = (n); \
+        MIDDLE2(t4 / 100); MIDDLE2(t4 % 100); \
+    } while(0)
+#define BEGIN8(n) \
+    do { \
+        uint32_t t8 = (n); \
+        if(t8 < 10000) BEGIN4(t8); \
+        else { BEGIN4(t8 / 10000); MIDDLE4(t8 % 10000); } \
+    } while(0)
+#define MIDDLE8(n) \
+    do { \
+        uint32_t t8 = (n); \
+        MIDDLE4(t8 / 10000); MIDDLE4(t8 % 10000); \
+    } while(0)
+#define MIDDLE16(n) \
+    do { \
+        uint64_t t16 = (n); \
+        MIDDLE8(t16 / 100000000); MIDDLE8(t16 % 100000000); \
+    } while(0)
+    
+inline int u64_toa(uint64_t value, char* p) 
+{
+   auto start = p;
+   if(value < 100000000) BEGIN8(value);
+   else if(value < 10000000000000000) { BEGIN8(value / 100000000); MIDDLE8(value % 100000000); }
+   else { BEGIN4(value / 10000000000000000); MIDDLE16(value % 10000000000000000); }
+   *p = 0;   
+   return p-start;
+}    
+
+inline int count_digits (int64_t value)
+{
+   static auto P01 = 10ull;
+   static auto P02 = P01*10;
+   static auto P03 = P02*10;
+   static auto P04 = P03*10;
+   static auto P05 = P04*10;
+   static auto P06 = P05*10;
+   static auto P07 = P06*10;
+   static auto P08 = P07*10;
+   static auto P09 = P08*10;
+   static auto P10 = P09*10;
+   static auto P11 = P10*10;
+   static auto P12 = P11*10;
+   static auto P13 = P12*10;
+   static auto P14 = P13*10;
+   static auto P15 = P14*10;
+   static auto P16 = P15*10;
+   static auto P17 = P16*10;
+   static auto P18 = P17*10;
+   static auto P19 = P18*10;
+
+   uint64_t u = static_cast<uint64_t>(value);
+   if (value < 0) 
+      u = ~u + 1;
+
+   if (u < P01) return 1;
+   if (u < P02) return 2;
+   if (u < P03) return 3;
+   if (u < P04) return 4;
+   if (u < P05) return 5;
+   if (u < P06) return 6;
+   if (u < P07) return 7;
+   if (u < P08) return 8;
+   if (u < P09) return 9;
+   if (u < P10) return 10;
+   if (u < P11) return 11;
+   if (u < P12) return 12;
+   if (u < P13) return 13;
+   if (u < P14) return 14;
+   if (u < P15) return 15;
+   if (u < P16) return 16;
+   if (u < P17) return 17;
+   if (u < P18) return 18;
+   if (u < P19) return 19;
+   return 20;
+}
+
+inline int count_digits (uint64_t u)
+{
+   static auto P01 = 10ull;
+   static auto P02 = P01*10;
+   static auto P03 = P02*10;
+   static auto P04 = P03*10;
+   static auto P05 = P04*10;
+   static auto P06 = P05*10;
+   static auto P07 = P06*10;
+   static auto P08 = P07*10;
+   static auto P09 = P08*10;
+   static auto P10 = P09*10;
+   static auto P11 = P10*10;
+   static auto P12 = P11*10;
+   static auto P13 = P12*10;
+   static auto P14 = P13*10;
+   static auto P15 = P14*10;
+   static auto P16 = P15*10;
+   static auto P17 = P16*10;
+   static auto P18 = P17*10;
+   static auto P19 = P18*10;
+
+   if (u < P01) return 1;
+   if (u < P02) return 2;
+   if (u < P03) return 3;
+   if (u < P04) return 4;
+   if (u < P05) return 5;
+   if (u < P06) return 6;
+   if (u < P07) return 7;
+   if (u < P08) return 8;
+   if (u < P09) return 9;
+   if (u < P10) return 10;
+   if (u < P11) return 11;
+   if (u < P12) return 12;
+   if (u < P13) return 13;
+   if (u < P14) return 14;
+   if (u < P15) return 15;
+   if (u < P16) return 16;
+   if (u < P17) return 17;
+   if (u < P18) return 18;
+   if (u < P19) return 19;
+   return 20;
+}
+
+inline int i64_toa(int64_t value, char* buffer, int pad) 
+{
+   auto start = buffer;
+   uint64_t u = static_cast<uint64_t>(value);
+   if (value < 0) {
+      *buffer++ = '-';
+      u = ~u + 1;
+   }
+   if (pad) 
+   {
+      auto digits = count_digits (value);
+      auto offset = pad - digits;
+      if (offset > 0) 
+      {
+         auto start = buffer + offset;
+         while (buffer != start)
+         {
+            *buffer='0';
+            buffer++;
+         }
+      }
+      else if (offset < 0)
+         return 0;
+   }
+   return (buffer-start) + u64_toa(u, buffer);
+}
+
+inline int u64_toa(uint64_t u, char* buffer, int pad) 
+{
+   auto start = buffer;
+   if (pad) 
+   {
+      auto digits = count_digits (u);
+      auto offset = pad - digits;
+      if (offset > 0) 
+      {
+         auto start = buffer + offset;
+         while (buffer != start)
+         {
+            *buffer='0';
+            buffer++;
+         }
+      }
+      else if (offset < 0)
+         return 0;
+   }
+   return (buffer-start) + u64_toa(u, buffer);
+}

src/lib/jetfix/gateway/Session.cpp

@@ -783,42 +783,63 @@ OutFIXPtr make_logon(uint32_t seq_num, uint32_t last_seq_num_processed,
    const char *application = "O2";
 #endif
 
-   buffer.add<ApplicationSystemName>(application);
-   buffer.add<ApplicationSystemVersion>("1.0");
-   buffer.add<ApplicationSystemVendor>("BTS");
-
-   if (session_cfg.has_trading_session_id()) {
-      buffer.add<fix::NoTradingSessions>(1);
-      buffer.add<fix::TradingSessionID>(session_cfg.trading_session_id());
-   }
-   if (session_cfg.has_udf_support_indicator())
-      buffer.add<fix::UDFSupportIndicator>(session_cfg.udf_support_indicator());
-
-   if (session_cfg.use_cme_secure_logon()) {
-      bts_assert(session_cfg.sender_compid().size() >= 3);
-      auto session_id = session_cfg.sender_compid().substr(0, 3);
-      std::string seq_num_str(9, ' ');
-      i32_toa(seq_num, &seq_num_str[0], 9);
-
-      std::string last_seq_num_str(9, ' ');
-      i32_toa(last_seq_num_processed, &last_seq_num_str[0], 9);
-      auto ld = glbx::calculate_hmac(
-          session_id, seq_num_str, session_cfg.sender_compid(),
-          session_cfg.sender_subid(), time_str, session_cfg.target_subid(),
-          std::to_string(session_cfg.seconds_between_hbs()),
-          session_cfg.sender_locationid(), last_seq_num_str, application, "1.0",
-          "BTS");
-#ifndef BTS_EDGE
-      LOG(Jetfix, Info, "Canonical string for secure logon: {}", ld.canonical);
-#endif
-      buffer.add<EncodedTextLen>(ld.access_key.size());
-      buffer.add<EncodedText>(ld.access_key);
-      buffer.add<EncryptedPasswordMethod>("CME-1-SHA-256");
-      buffer.add<EncryptedPasswordLen>(ld.signature.size());
-      buffer.add<EncryptedPassword>(ld.signature);
-   } else {
+   if(session_cfg.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_ICE || 
+      session_cfg.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_SIMICE)
+   {
+      buffer.add<ICEApplicationSystemName>(application);
+      buffer.add<ICEApplicationSystemVersion>("1.0");
+      buffer.add<ICEApplicationSystemVendor>("BTS");      
+      
+      buffer.add<UserName>(session_cfg.user_name());      
       buffer.add<RawDataLength>(session_cfg.raw_data().size());
-      buffer.add<RawData>(session_cfg.raw_data());
+      buffer.add<RawData>(session_cfg.raw_data());      
+      buffer.add<EncryptMethod>(0);
+      buffer.add<ICEStrategyPreference>(1);
+      buffer.add<ICETimeStampPreference>(1);
+   }
+   else
+   {
+      buffer.add<ApplicationSystemName>(application);
+      buffer.add<ApplicationSystemVersion>("1.0");
+      buffer.add<ApplicationSystemVendor>("BTS");
+
+      if (session_cfg.has_trading_session_id()) {
+         buffer.add<fix::NoTradingSessions>(1);
+         buffer.add<fix::TradingSessionID>(session_cfg.trading_session_id());
+      }
+      if (session_cfg.has_udf_support_indicator())
+         buffer.add<fix::UDFSupportIndicator>(session_cfg.udf_support_indicator());
+
+      if (session_cfg.use_cme_secure_logon()) {
+         bts_assert(session_cfg.sender_compid().size() >= 3);
+         auto session_id = session_cfg.sender_compid().substr(0, 3);
+         std::string seq_num_str(9, ' ');
+         i32_toa(seq_num, &seq_num_str[0], 9);
+
+         std::string last_seq_num_str(9, ' ');
+         i32_toa(last_seq_num_processed, &last_seq_num_str[0], 9);
+         auto ld = glbx::calculate_hmac(
+            session_id, seq_num_str, session_cfg.sender_compid(),
+            session_cfg.sender_subid(), time_str, session_cfg.target_subid(),
+            std::to_string(session_cfg.seconds_between_hbs()),
+            session_cfg.sender_locationid(), last_seq_num_str, application, "1.0",
+            "BTS");
+   #ifndef BTS_EDGE
+         LOG(Jetfix, Info, "Canonical string for secure logon: {}", ld.canonical);
+   #endif
+         buffer.add<EncodedTextLen>(ld.access_key.size());
+         buffer.add<EncodedText>(ld.access_key);
+         buffer.add<EncryptedPasswordMethod>("CME-1-SHA-256");
+         buffer.add<EncryptedPasswordLen>(ld.signature.size());
+         buffer.add<EncryptedPassword>(ld.signature);
+      } 
+      else 
+      {
+         buffer.add<RawDataLength>(session_cfg.raw_data().size());
+         buffer.add<RawData>(session_cfg.raw_data());
+         if(session_cfg.user_name() != "")
+            buffer.add<UserName>(session_cfg.user_name());
+      }
    }
 
    stream.close();

src/lib/md/MDConsumerHandle.cpp

@@ -296,7 +296,7 @@ TimeStamp copy_synthetic_depth(const SyntheticStrategy& synth, size_t count, Pri
 // END SYNTHETIC DEPTH
 /////////////////////////////
 
-bool MDConsumerHandle::get_last_trade(TimeStamp& time, PriceSize& trade, datetime::LLDateTime& trade_timestamp, size_t* total_volume) const {
+bool MDConsumerHandle::get_last_trade(TimeStamp& time, PriceSize& trade, datetime::LLDateTime& trade_timestamp, size_t* total_volume, int* from_spread) const {
    switch(_type) {
       case HANDLETYPE_VOID: return false;
       case HANDLETYPE_DEPTH:
@@ -306,6 +306,7 @@ bool MDConsumerHandle::get_last_trade(TimeStamp& time, PriceSize& trade, datetim
          if (result) {
             trade_timestamp = pcold->last_trade_time;
             if (total_volume) { *total_volume = pcold->total_volume; }
+            if (from_spread) { *from_spread = pcold->from_spread; }
          }
          return result;
       }

src/lib/md/MDConsumerHandle.h

@@ -261,7 +261,7 @@ public:
       return get_book(read_combined, time, copydepth, bids, asks);
    }
    
-   bool get_last_trade(TimeStamp& time, PriceSize& out_trade, datetime::LLDateTime& out_trade_timestamp, size_t* total_volume=nullptr) const;
+   bool get_last_trade(TimeStamp& time, PriceSize& out_trade, datetime::LLDateTime& out_trade_timestamp, size_t* total_volume=nullptr,int* from_spread=nullptr) const;
 
    IntPrice settlement_price() const {
       switch(_type) {

src/lib/md/MDProducerHandle.cpp

@@ -101,8 +101,9 @@ void MDProducerHandle::set_status(CoconutSecurityStatus status) {
    }
 }
 
-void MDProducerHandle::add_trade(TimeStamp time, const PriceSize price, LLDateTime tradetime, co2::AggressorSide agside, uint32_t volume) {
-   // Add this trade to the last trade if price match
+void MDProducerHandle::add_trade(TimeStamp time, const PriceSize price, LLDateTime tradetime, 
+                                 co2::AggressorSide agside, uint32_t volume, bool from_spread) {
+   // Add this trade to the last trade if price/side match
    auto current = std::get<PriceSize>(cdata().last_trade.load());
    if (current.int_price() == price.int_price()) {
       cdata().last_trade.store(PriceSize { price.int_price(), price.size() + current.size() }, time);
@@ -111,7 +112,9 @@ void MDProducerHandle::add_trade(TimeStamp time, const PriceSize price, LLDateTi
       cdata().last_trade.store(price,time);
    }
    cdata().last_trade_time = tradetime;
-   cdata().total_volume = volume;
+   cdata().from_spread = (from_spread ? 1 : 0);
+   if (volume > 0) 
+      cdata().total_volume = volume;
 
    if (key() != invalid_security_key) {
       co2::TradeMessage trade;
@@ -125,9 +128,15 @@ void MDProducerHandle::add_trade(TimeStamp time, const PriceSize price, LLDateTi
       _channel.add_trade(trade);
    }
 }
-void MDProducerHandle::set_last_trade(TimeStamp time, const PriceSize ps, datetime::LLDateTime tradetime, uint32_t total_volume) {
+void MDProducerHandle::set_last_trade(TimeStamp time, const PriceSize ps, datetime::LLDateTime tradetime, uint32_t total_volume, bool from_spread) {
    cdata().last_trade.store(ps,time);
    cdata().last_trade_time = tradetime;
+   cdata().from_spread = (from_spread ? 1 : 0);
+   if(total_volume > 0)
+      cdata().total_volume = total_volume;
+}
+
+void MDProducerHandle::set_total_volume(uint32_t total_volume) {
    cdata().total_volume = total_volume;
 }
 

src/lib/md/MDProducerHandle.h

@@ -92,9 +92,11 @@ public:
    void set_status(co2::CoconutSecurityStatus status); // hot
 
    // Updates trade log and last trade
-   void add_trade(TimeStamp time, const PriceSize pricesize, datetime::LLDateTime date, co2::AggressorSide agside, uint32_t total_volume=0); // cold
+   void add_trade(TimeStamp time, const PriceSize pricesize, datetime::LLDateTime date, 
+                  co2::AggressorSide agside, uint32_t total_volume=0, bool from_spread=false); // cold
    // Update last_trade without updating trade log
-   void set_last_trade(TimeStamp time, const PriceSize ps, datetime::LLDateTime date, uint32_t total_volume);
+   void set_last_trade(TimeStamp time, const PriceSize ps, datetime::LLDateTime date, uint32_t total_volume=0,bool from_spread=false);
+   void set_total_volume(uint32_t total_volume);
    void add_rfq(TimeStamp time, const co2::QuoteRequestMessage& rfq); // cold
 
    void update_exchange_status(TimeStamp time, TradingStatus status); // cold

src/lib/md/PlatformLoader.cpp

@@ -90,7 +90,7 @@ void add_symbols_to_result(
 
 size_t get_system_md_symbols(api::Venue exchange, SecurityType type,
                              std::set<UserSymbol> &result) {
-   bts_assert(config::system_symbols::Symbols::descriptor()->field_count() == 3);
+   bts_assert(config::system_symbols::Symbols::descriptor()->field_count() == 4);
    {
       auto const &e = config::get<config::system_symbols::Symbols>().glbx();
       if (exchange == api::VENUE_UNSET || exchange == api::VENUE_GLBX) {
@@ -146,34 +146,47 @@ size_t get_system_md_symbols(api::Venue exchange, SecurityType type,
 
    {
       auto const &e = config::get<config::system_symbols::Symbols>().sim();
-      if (exchange == api::VENUE_UNSET || exchange == api::VENUE_SIM) {
-         if (type == api::SECURITYTYPE_FUTURE ||
-             type == api::SECURITYTYPE_UNSET) {
-            add_symbols_to_result(e.future(), api::VENUE_SIM,
-                                  api::SECURITYTYPE_FUTURE, &result);
-         }
-         if (type == api::SECURITYTYPE_OPTION ||
-             type == api::SECURITYTYPE_UNSET) {
-            add_symbols_to_result(e.option(), api::VENUE_SIM,
-                                  api::SECURITYTYPE_OPTION, &result);
-         }
-         if (type == api::SECURITYTYPE_SPREAD ||
-             type == api::SECURITYTYPE_UNSET) {
-            add_symbols_to_result(e.spread(), api::VENUE_SIM,
-                                  api::SECURITYTYPE_SPREAD, &result);
-         }
+      if (exchange == api::VENUE_UNSET || exchange == api::VENUE_SIM) 
+      {
+         if (type == api::SECURITYTYPE_FUTURE || type == api::SECURITYTYPE_UNSET) 
+            add_symbols_to_result(e.future(), api::VENUE_SIM, api::SECURITYTYPE_FUTURE, &result);
+         if (type == api::SECURITYTYPE_OPTION || type == api::SECURITYTYPE_UNSET) 
+            add_symbols_to_result(e.option(), api::VENUE_SIM, api::SECURITYTYPE_OPTION, &result);
+         if (type == api::SECURITYTYPE_SPREAD || type == api::SECURITYTYPE_UNSET) 
+            add_symbols_to_result(e.spread(), api::VENUE_SIM, api::SECURITYTYPE_SPREAD, &result);
       }
-      if (exchange == api::VENUE_BTS || exchange == api::VENUE_UNSET) {
+      if (exchange == api::VENUE_BTS || exchange == api::VENUE_UNSET) 
+      {
          std::set<std::string> all_symbols;
          all_symbols.insert(e.future().begin(), e.future().end());
          all_symbols.insert(e.option().begin(), e.option().end());
          all_symbols.insert(e.spread().begin(), e.spread().end());
-         for (const auto &symbol : all_symbols) {
-            result.insert(UserSymbol(symbol, api::VENUE_BTS,
-                                     api::SECURITYTYPE_SYNTHETIC));
-         }
+         for (const auto &symbol : all_symbols) 
+            result.insert(UserSymbol(symbol, api::VENUE_BTS, api::SECURITYTYPE_SYNTHETIC));      
       }
    }
+   
+   {
+      auto const &e = config::get<config::system_symbols::Symbols>().simice();
+      if (exchange == api::VENUE_UNSET || exchange == api::VENUE_SIMICE) 
+      {
+         if (type == api::SECURITYTYPE_FUTURE || type == api::SECURITYTYPE_UNSET) 
+            add_symbols_to_result(e.future(), api::VENUE_SIMICE, api::SECURITYTYPE_FUTURE, &result);
+         if (type == api::SECURITYTYPE_OPTION || type == api::SECURITYTYPE_UNSET) 
+            add_symbols_to_result(e.option(), api::VENUE_SIMICE, api::SECURITYTYPE_OPTION, &result);
+         if (type == api::SECURITYTYPE_SPREAD || type == api::SECURITYTYPE_UNSET) 
+            add_symbols_to_result(e.spread(), api::VENUE_SIMICE, api::SECURITYTYPE_SPREAD, &result);
+      }
+      if (exchange == api::VENUE_BTS || exchange == api::VENUE_UNSET) 
+      {
+         std::set<std::string> all_symbols;
+         all_symbols.insert(e.future().begin(), e.future().end());
+         all_symbols.insert(e.option().begin(), e.option().end());
+         all_symbols.insert(e.spread().begin(), e.spread().end());
+         for (const auto &symbol : all_symbols) 
+            result.insert(UserSymbol(symbol, api::VENUE_BTS, api::SECURITYTYPE_SYNTHETIC));      
+      }
+   }   
 
    return result.size();
 }

src/lib/md/Shm/FixedSecurity.cpp

@@ -31,8 +31,9 @@ FixedSecurity::FixedSecurity() :
    cme_main_fraction(0),
    cme_sub_fraction(0),
    cme_price_display_format(0),
-   ice_deal_display_factor(0),
-   ice_settle_display_factor(0),
+   ice_price_denominator(0),
+   ice_deal_denominator(0),
+   ice_settle_denominator(0),
    _symbol(),
    _cfi_code(),
    _security_desc(),
@@ -99,7 +100,10 @@ void write_leg_descriptor(std::ostringstream& result, char separator, const api:
 std::string security_descriptor(const api::Security& security) {
    const char separator = ' ';
    std::ostringstream result;
-   result << security.symbol() << separator << Venue_Name(security.exchange());
+   if(security.exchange() == Venue::VENUE_ICE || security.exchange() == Venue::VENUE_SIMICE)
+      result << security.ice().security_desc() << separator << Venue_Name(security.exchange());
+   else
+      result << security.symbol() << separator << Venue_Name(security.exchange());
    switch(security.Contract_case()) {
       case Security::ContractCase::kOption: {
          result << separator;
@@ -181,8 +185,9 @@ void convert_security(const FixedSecurity& source, Security* presult) {
       case Venue::VENUE_ICE:    
          dest.mutable_ice()->set_exchange_id(source.numeric_id);
          dest.mutable_ice()->set_security_desc(source.security_desc());     
-         dest.mutable_ice()->set_deal_display_factor(source.ice_deal_display_factor);
-         dest.mutable_ice()->set_settle_display_factor(source.ice_settle_display_factor);
+         dest.mutable_ice()->set_price_denominator(source.ice_price_denominator);
+         dest.mutable_ice()->set_deal_denominator(source.ice_deal_denominator);
+         dest.mutable_ice()->set_settle_denominator(source.ice_settle_denominator);
          
          dest.mutable_ice()->set_market_description(source.ice_market_description());
          dest.mutable_ice()->set_cleared_alias(source.ice_cleared_alias());
@@ -312,8 +317,9 @@ void convert_security(const Security& src, FixedSecurity& dest, bool set_securit
       case Security::ExchangeDataCase::kIce:
          dest.numeric_id = src.ice().exchange_id();
          dest.security_desc(src.ice().security_desc());      
-         dest.ice_deal_display_factor = src.ice().deal_display_factor();
-         dest.ice_settle_display_factor = src.ice().settle_display_factor();
+         dest.ice_price_denominator = src.ice().price_denominator();
+         dest.ice_deal_denominator = src.ice().deal_denominator();
+         dest.ice_settle_denominator = src.ice().settle_denominator();
          
          dest.ice_market_description(src.ice().market_description());
          dest.ice_cleared_alias(src.ice().cleared_alias());

src/lib/md/Shm/FixedSecurity.h

@@ -59,8 +59,9 @@ public:
    uint8_t cme_main_fraction;
    uint8_t cme_sub_fraction;
    uint8_t cme_price_display_format;
-   int8_t ice_deal_display_factor;
-   int8_t ice_settle_display_factor;
+   int8_t ice_price_denominator;
+   int8_t ice_deal_denominator;
+   int8_t ice_settle_denominator;
    
    STRING_PROPERTY(ice_market_description);   
    STRING_PROPERTY(ice_cleared_alias);

src/lib/md/Shm/MDStructure.h

@@ -26,7 +26,7 @@ namespace md {
 
 using utils::TimeStamp;
 
-const uint32_t shm_version = 102;
+const uint32_t shm_version = 107;
 
 struct ALIGNED_(CACHE_LINE) HotHeader {
    size_t count;
@@ -213,9 +213,10 @@ struct ColdEntry {
       exchange_status(co2::TRADINGSTATUS_UNKNOWN),
       last_trade(),
       total_volume(0),
-      last_trade_time(),
+      last_trade_time(),      
       misc(),
-      deleted(false)
+      from_spread(0),
+      deleted(0)
    {
       this->security.size = security_size;
    }
@@ -227,12 +228,13 @@ struct ColdEntry {
    
    // Last Trade
    AtomicPriceSize last_trade;
-   size_t total_volume;
+   size_t total_volume;   
    datetime::LLDateTime  last_trade_time;
-
+   
    MiscData misc;
 
-   bool deleted;
+   int from_spread;
+   int deleted;
    FixedSecurity security;
 };
 

src/lib/md/ice/ICEUtils.cpp

@@ -27,6 +27,8 @@ std::string ICEChannel::to_string() const
    std::string mt = name;
    for(const auto& x : market_types)
       mt += " " + std::to_string(x);
+   for(const auto& x : uds_market_types)
+      mt += " " + std::to_string(x);      
    return mt;
 }
 
@@ -45,6 +47,9 @@ const vector<ICEChannel> parse_ice_channels(const vector<UserSymbol>& symbols)
       chan.symbols.insert(symbols.begin(), symbols.end());
       for(const auto& xx : x.market_type())
          chan.market_types.push_back(xx); 
+      for(const auto& xx : x.uds_market_type())
+         chan.uds_market_types.push_back(xx); 
+         
       result.push_back(chan);            
    }
    return result;

src/lib/md/ice/ICEUtils.h

@@ -13,6 +13,7 @@ struct ICEChannel
 {
    std::string name;
    std::vector<uint32_t> market_types;
+   std::vector<uint32_t> uds_market_types;
    std::set<UserSymbol> symbols;
    std::string to_string() const;
 };

src/lib/md/protos/co2.proto

@@ -152,6 +152,7 @@ message TradeMessage {
    uint32 total_volume = 7;
    bool gap = 8;
    AggressorSide aggressor_side = 9;
+   bool from_spread = 10;
 }
 message QuoteRequestMessage {
     enum Side {

src/lib/md/relay/QueryServer.cpp

@@ -78,7 +78,8 @@ struct MDConsumerGroup {
       symbol(symbol),
       platform(platform_for_exchange(symbol.exchange())),
       channel(consumer.get_channel(symbol)),
-      handles()
+      handles(),
+      mic()
    {
       set_consumer(consumer);
       
@@ -118,9 +119,38 @@ struct MDConsumerGroup {
       bts_assert_d(handle.is_initialized());
       bts_assert(handle.get_usersymbol() == symbol);
       DatedHandle h = { TimeStamp::now(), handle };
-      handles->push_back(h);
+      handles->push_back(h);   
+      if(this->mic == "")
+         set_mic(h);
    }
-
+   
+   void set_mic(const DatedHandle& h)
+   {
+      if(channel.exchange() == api::VENUE_ICE || channel.exchange() == api::VENUE_SIMICE)
+      {
+         this->mic = h.handle.get_normalized_security().ice().mic_code();
+      }
+      else if(channel.exchange() == api::VENUE_GLBX || channel.exchange() == api::VENUE_SIM)
+      {
+         const auto& cme_symb_mics = config::get<config::CmeSymbolMics>().cme_mic();
+         for(auto mic=cme_symb_mics.begin();mic!=cme_symb_mics.end();mic++)
+         {
+            const auto& micsym = mic->first;
+            if(micsym == symbol.symbol())
+            {
+               this->mic = api::CMEMics_Name(mic->second);
+               break;
+            }
+         }
+      }
+      else if(channel.exchange() == api::VENUE_BTS || channel.exchange() == api::VENUE_UNSET)
+      {
+         this->mic = "BTS";
+      }
+      else
+         bts_assert_msg(false, "unknown venue in QueryServer");
+   }
+   
    UserSymbol symbol;
    Platform platform;
    MDConsumerChannel channel;
@@ -128,7 +158,7 @@ struct MDConsumerGroup {
    // Can add to handles without invalidating readers (no lock needed)
    // When we recover we swap a new vector into this unique_ptr and delayed-delete
    shared_ptr<ConcurrentAppendCollection<DatedHandle>> handles;
-
+   std::string mic;
 };
 
 void convert(PriceSize ps, PriceSizeMessage* pps) {
@@ -219,12 +249,18 @@ public:
    }
 
    // the hot part
-   void fill(const SecurityChannel& chan, MDConsumerGroup& group, ChannelReply* reply, bool entitled) const {
+   void fill(const SecurityChannel& chan, MDConsumerGroup& group, ChannelReply* reply, 
+             const Entitlements* entitlements, bool bypass_entitlement) const {      
       bts_assert(reply);
+
       reply->mutable_channel()->CopyFrom(chan);
       reply->set_platform(group.platform);
       reply->mutable_channel()->set_time(TimeStamp::now().to_uint64());
-      reply->mutable_channel()->set_entitled(entitled);
+      
+      bool entitled = bypass_entitlement || chan.exchange() == api::VENUE_BTS;
+      if(entitlements && !entitled)
+         entitled = entitlements->count(group.mic);
+      reply->mutable_channel()->set_entitled(entitled);      
       auto flags = (MDRequest::Flags)chan.flags();
 
       // Get a copy of handles. If recovery happens during this call
@@ -242,8 +278,10 @@ public:
          ChannelUpdate* pupd = &temp_update;
          pupd->set_key(instrument.handle.key());
          // If the security was just discovered try to ensure all data is sent
-         if (instrument.discovered > TimeStamp(chan.time())) {
-            pupd->mutable_security()->CopyFrom(handle.get_normalized_security());
+         if (instrument.discovered > TimeStamp(chan.time())) 
+         {
+            auto security = handle.get_normalized_security();
+            pupd->mutable_security()->CopyFrom(security);
             snapshot = true;
          }
          //if we are not entitled AND this is the first request since we lost our entitlement, zero out books
@@ -323,12 +361,13 @@ public:
                TimeStamp time(snapshot ? 0 : chan.time());
                PriceSize price;
                datetime::LLDateTime trade_time;
-               size_t volume;
-               if (handle.get_last_trade(time, price, trade_time, &volume)) {
+               size_t volume; int from_spread;
+               if (handle.get_last_trade(time, price, trade_time, &volume, &from_spread)) {
                   auto* trade = pupd->mutable_last_trade();
                   trade->set_key(handle.key());
                   trade->set_time(trade_time);
                   trade->set_total_volume(volume);
+                  trade->set_from_spread(from_spread);
                   set_price(price, trade);
                }
             }
@@ -373,10 +412,7 @@ public:
          auto groupiter = _groups.find(UserSymbol(ch.symbol(), ch.exchange(), ch.type()));
          if (groupiter != _groups.end()) {
             auto& group = groupiter->second;
-            bool is_entitled = bypass_entitlement || 
-                               ch.exchange() == api::VENUE_BTS || 
-                               (entitlements && entitlements->count(ch.symbol()));
-            fill(ch, *group, reply->add_channels(), is_entitled);
+            fill(ch, *group, reply->add_channels(), entitlements, bypass_entitlement);
          }
       }
    }
@@ -538,27 +574,21 @@ QueryServerPtr run_query_server(const bts::api::ServiceContext& ctx, MDClient& c
       container->admin_client.subscribe_md_entitlement_data().subscribe([=](const api::admin::Md_entitlement_dataUpdate& update) {         
          const auto& all_updates = update.all();
          utils::Dispatcher::background().post([all_updates, container]() 
-         {
-            const auto& cme_symb_mics = config::get<config::CmeSymbolMics>().cme_mic();
+         {            
             auto entitlements = std::make_shared<std::unordered_map<uint32_t, Entitlements>>();
-            for (const auto& row : all_updates) {            
-               if (row && row->enabled()) {                                 
-                  for(auto mic=cme_symb_mics.begin();mic!=cme_symb_mics.end();mic++){
-                     if(api::CMEMics_Name(mic->second) == row->mic()) {
-                        const auto& symbol = mic->first;
-                        auto iter = entitlements->find(row->user_id());
-                        if (iter == entitlements->end()) {
-                           Entitlements ents;
-                           ents.insert(symbol);
-                           ents.insert("SIM:" + symbol);
-                           entitlements->insert(iter, std::make_pair(row->user_id(), std::move(ents)));
-                        }
-                        else {
-                           iter->second.insert(symbol);
-                           iter->second.insert("SIM:" + symbol);
-                        }
-                     }
+            for (const auto& row : all_updates) 
+            {            
+               if (row && row->enabled()) 
+               {
+                  auto iter = entitlements->find(row->user_id());
+                  if (iter == entitlements->end()) {
+                     Entitlements ents;
+                     ents.insert(row->mic());
+                     entitlements->insert(iter, std::make_pair(row->user_id(), std::move(ents)));
                   }
+                  else {
+                     iter->second.insert(row->mic());
+                  }                                       
                }
             }
             container->relay.update_entitlements(entitlements);

src/lib/md/relay/RelayClient.cpp

@@ -86,7 +86,7 @@ class RelayProducer : public IRelayWriter {
          handle.update_fixing_price(updated, misc.fixing_price());
       }
       if (update.has_last_trade()) {
-         handle.set_last_trade(updated, trade_pricesize(update.last_trade()), update.last_trade().time(), update.last_trade().total_volume());
+         handle.set_last_trade(updated, trade_pricesize(update.last_trade()), update.last_trade().time(), update.last_trade().total_volume(), update.last_trade().from_spread());
       }
    }
 

src/lib/md/test/Co2ManagedTestHelper.cpp

@@ -87,7 +87,8 @@ extern "C" BTS_EXPORT void co2_get_sample_cold(size_t& size, ColdEntry& entry) {
    entry.misc.session_high_bid = 4.4;
    entry.misc.session_low_offer = 5.5;
    entry.misc.fixing_price = 6.6;
-   entry.deleted = true;
+   entry.deleted = 1;
+   entry.from_spread = 0;
 }
 
 extern "C" BTS_EXPORT void co2_get_sample_cold_header(ColdHeader& header) {

src/lib/orderman/actors/ExecutionGateway.h

@@ -1,53 +0,0 @@
-#pragma once
-
-#include "lib/config/protos/jetfix.pb.h"
-#include "lib/jetfix/codec/MessageDefinitions.h"
-#include "lib/jetfix/common/OutFIXPtr.h"
-#include "lib/mercury/events/Update.h"
-#include "lib/orderman/orderman_utils/orderman_common.h"
-
-#include <memory>
-
-namespace bts {
-namespace md { class SecurityOracle;}
-namespace trace {struct CorID;class PerfLog;}
-namespace orderman {
-class NewSecuritiesEvent;
-class ExecutionReport;
-
-using OnExeReport =
-   std::function<void(ExecutionReport const &, trace::CorID const &)>;
-using OnGwStatusChange = std::function<void(bool)>;
-using OnCancelReject = std::function<void(OrderKey, std::string const &)>;
-
-// fix execution gateway for a single FIX session
-class ExecutionGateway {
- public:
-   ExecutionGateway(config::JetFIX::Connection const &, trace::PerfLog &,
-                    md::SecurityOracle);
-   ~ExecutionGateway();
-
-   void start();
-   void dispatch_incoming_fix_messages();
-   void on_securities(const mercury::Update<NewSecuritiesEvent> &);
-   void send_fix(jetfix::OutFIXPtr const &fix, trace::CorID trace);
-
-   OnExeReport& on_exe_report();
-   OnGwStatusChange& on_gw_status_change();
-   OnCancelReject &on_cancel_reject();
-   
- private:
-   enum class State { WAITING_FOR_SECURITIES, READY };
-   struct PImpl;
-   std::unique_ptr<PImpl> me;
-
-   // fix gateway events
-   void on_exe_report(jetfix::fix::ExecutionReportMsg const &);
-   void on_cancel_reject(jetfix::fix::CancelRejectMsg const &);
-   void on_connection_status_change(bool connected);
-   void on_business_reject(jetfix::fix::BusinessMessageRejectMsg const &);
-   void on_session_reject(jetfix::fix::RejectMsg const &);
-};
-
-} // namespace orderman
-} // namespace bts

src/lib/orderman/actors/FillStore.cpp

@@ -93,12 +93,12 @@ void FillStore::on_execution_report(
    if (u.events) {
       for (auto const &ev : *u.events) {
          auto &er = *ev.data;
+         auto fk = fmt::format("{}:{}", er.order_key.value(),
+                                 to_str(er.exchange_fill_id));
+         auto is_known = me->known_fills.count(fk);         
          if (er.exec_type == api::orderman::EXEC_TYPE_PARTIAL_FILL ||
              er.exec_type == api::orderman::EXEC_TYPE_FILL) {
-
-            auto fk = fmt::format("{}:{}", er.order_key.value(),
-                                  to_str(er.exchange_fill_id));
-            auto is_known = me->known_fills.count(fk);
+                
             api::orderman::Fill fill;
             if (!is_known) {
                fill.set_order_key(er.order_key.value());
@@ -114,6 +114,9 @@ void FillStore::on_execution_report(
                fill.set_avg_price(er.avg_price.value());
                fill.set_side(er.side);
                fill.set_last_shares(er.last_shares.value());
+               fill.set_fill_status(api::orderman::Fill::FILL_STATUS_ACTIVE);
+               if(er.exec_type == api::orderman::EXEC_TYPE_TRADE_CORRECT)
+                  fill.set_fill_status(api::orderman::Fill::FILL_STATUS_CORRECTED);
                fill.set_fill_key(fk);
 
                auto exchange_ts = utils::pb::to_timestamp(
@@ -137,10 +140,12 @@ void FillStore::on_execution_report(
                    "Discarding known execution report for fill {}", fk);
 
          } // if partial_fill or fill
-      }    // for loop
+      }
    }       // if u.events
 }
 
+
+
 void FillStore::on_order_definition_update(
     mercury::Update<OrderDefinitionUpdateEvent> const &u) {
    if (u.events)

src/lib/orderman/actors/GLBXExecutionGateway.cpp

@@ -1,4 +1,4 @@
-#include "ExecutionGateway.h"
+#include "GLBXExecutionGateway.h"
 #include "lib/jetfix/codec/MessageDefinitions.h"
 #include "lib/jetfix/common/OutFIX.h"
 #include "lib/jetfix/gateway/JetFIXGateway.h"
@@ -24,7 +24,7 @@ using JFGw =
     JetFIXGateway<fix::ExecutionReportMsg, fix::BusinessMessageRejectMsg,
                   fix::RejectMsg, fix::CancelRejectMsg>;
 
-struct ExecutionGateway::PImpl {
+struct GLBXExecutionGateway::PImpl {
    PImpl(config::JetFIX::Connection const &cfg, trace::PerfLog &l,
          md::SecurityOracle o)
        : gw(cfg, l), exe_report_pool(), state(State::WAITING_FOR_SECURITIES),
@@ -46,14 +46,16 @@ struct ExecutionGateway::PImpl {
    md::SecurityOracle sec_oracle;
 };
 
-ExecutionGateway::ExecutionGateway(config::JetFIX::Connection const &cfg,
+GLBXExecutionGateway::GLBXExecutionGateway(config::JetFIX::Connection const &cfg,
                                    trace::PerfLog &log, md::SecurityOracle o)
     : me(new PImpl(cfg, log, o)) {
    me->state = State::WAITING_FOR_SECURITIES;
 }
-ExecutionGateway::~ExecutionGateway() {}
+GLBXExecutionGateway::~GLBXExecutionGateway() {}
 
-void ExecutionGateway::on_securities(
+void GLBXExecutionGateway::on_admin_data(const mercury::Update<NewAdminDataEvent> &) {}
+
+void GLBXExecutionGateway::on_securities(
     const mercury::Update<NewSecuritiesEvent> &update) {
    if (update.events) {
 
@@ -96,19 +98,19 @@ void ExecutionGateway::on_securities(
    }
 }
 
-void ExecutionGateway::start() { me->gw.start(); }
+void GLBXExecutionGateway::start() { me->gw.start(); }
 
-OnExeReport &ExecutionGateway::on_exe_report() { return me->on_exe_report; }
+OnExeReport &GLBXExecutionGateway::on_exe_report() { return me->on_exe_report; }
 
-OnGwStatusChange &ExecutionGateway::on_gw_status_change() {
+OnGwStatusChange &GLBXExecutionGateway::on_gw_status_change() {
    return me->on_gw_status_change;
 }
 
-OnCancelReject &ExecutionGateway::on_cancel_reject() {
+OnCancelReject &GLBXExecutionGateway::on_cancel_reject() {
    return me->on_cancel_reject;
 }
 
-void ExecutionGateway::on_exe_report(
+void GLBXExecutionGateway::on_exe_report(
     const jetfix::fix::ExecutionReportMsg &msg) {
    OrderKey ok;
    SecurityKey sk;
@@ -116,7 +118,7 @@ void ExecutionGateway::on_exe_report(
 
    auto scoped_log =
        trace::log_scope(me->perf_log, msg.trace(),
-                        "ExecutionGateway::on_exe_report", ok.value());
+                        "GLBXExecutionGateway::on_exe_report", ok.value());
 
    LOG(Latency, Info, "Started handling of execution report for order {}",
        ok.value());
@@ -262,11 +264,12 @@ void ExecutionGateway::on_exe_report(
       str_to(msg.get<fix::TransactTimeStr>(), &data->transact_time);
       data->cfg = &me->cfg;
       data->msg = text_msg;
+      data->exchange = api::Venue::VENUE_GLBX;
       me->on_exe_report(*data, msg.trace());
    }
 }
 
-void ExecutionGateway::on_cancel_reject(
+void GLBXExecutionGateway::on_cancel_reject(
     const jetfix::fix::CancelRejectMsg &msg) {
    OrderKey ok;
    SecurityKey sk;
@@ -281,7 +284,7 @@ void ExecutionGateway::on_cancel_reject(
    me->on_cancel_reject(ok, reject_msg);
 }
 
-void ExecutionGateway::on_connection_status_change(bool is_connected) {
+void GLBXExecutionGateway::on_connection_status_change(bool is_connected) {
    LOG(OrderMan, Info, "Connection state of {} changed to {}",
        me->gw_name.value(),
        is_connected ? std::string("connected") : std::string("disconnected"));
@@ -289,7 +292,7 @@ void ExecutionGateway::on_connection_status_change(bool is_connected) {
    me->on_gw_status_change(is_connected);
 }
 
-void ExecutionGateway::on_business_reject(
+void GLBXExecutionGateway::on_business_reject(
     jetfix::fix::BusinessMessageRejectMsg const &m) {
    LOG(OrderMan, Error,
        "Unexpected business reject in {} gateway . Reason: "
@@ -298,7 +301,7 @@ void ExecutionGateway::on_business_reject(
        m.has<fix::Text>() ? m.get<fix::Text>() : std::string("Unknown"));
 }
 
-void ExecutionGateway::on_session_reject(jetfix::fix::RejectMsg const &m) {
+void GLBXExecutionGateway::on_session_reject(jetfix::fix::RejectMsg const &m) {
    LOG(OrderMan, Error,
        "Unexpected reject message in {} gateway . Reason: {}. "
        "This should be investigated",
@@ -306,14 +309,14 @@ void ExecutionGateway::on_session_reject(jetfix::fix::RejectMsg const &m) {
        m.has<fix::Text>() ? m.get<fix::Text>() : std::string("Unknown"));
 }
 
-void ExecutionGateway::dispatch_incoming_fix_messages() {
+void GLBXExecutionGateway::dispatch_incoming_fix_messages() {
    me->gw.dispatch_events();
 }
 
-void ExecutionGateway::send_fix(jetfix::OutFIXPtr const &fix,
+void GLBXExecutionGateway::send_fix(jetfix::OutFIXPtr const &fix,
                                 trace::CorID trace) {
    auto scoped_log =
-       trace::log_scope(me->perf_log, trace, "ExecutionGateway::send_fix");
+       trace::log_scope(me->perf_log, trace, "GLBXExecutionGateway::send_fix");
    me->gw.send(fix, trace);
    me->gw.dispatch_events();
 }

src/lib/orderman/actors/GLBXExecutionGateway.h

@@ -0,0 +1,41 @@
+#pragma once
+
+#include "IExecutionGateway.h"
+
+namespace bts {
+namespace orderman {
+
+
+// fix execution gateway for a single FIX session
+class GLBXExecutionGateway : public IExecutionGateway {
+ public:   
+   GLBXExecutionGateway(config::JetFIX::Connection const &, trace::PerfLog &,
+                        md::SecurityOracle);
+   virtual ~GLBXExecutionGateway();
+
+   void start() override;
+   void dispatch_incoming_fix_messages() override;
+   void on_securities(const mercury::Update<NewSecuritiesEvent> &) override;
+   void on_admin_data(const mercury::Update<NewAdminDataEvent> &) override;
+   void send_fix(jetfix::OutFIXPtr const &fix, trace::CorID trace) override;
+
+   OnExeReport& on_exe_report() override;
+   OnGwStatusChange& on_gw_status_change() override;
+   OnCancelReject &on_cancel_reject() override;
+   
+ private:
+   BTS_NO_COPY(GLBXExecutionGateway);
+   enum class State { WAITING_FOR_SECURITIES, READY };
+   struct PImpl;
+   std::unique_ptr<PImpl> me;
+
+   // fix gateway events
+   void on_exe_report(jetfix::fix::ExecutionReportMsg const &);
+   void on_cancel_reject(jetfix::fix::CancelRejectMsg const &);
+   void on_connection_status_change(bool connected);
+   void on_business_reject(jetfix::fix::BusinessMessageRejectMsg const &);
+   void on_session_reject(jetfix::fix::RejectMsg const &);
+};
+
+} // namespace orderman
+} // namespace bts

src/lib/orderman/actors/ICEExecutionGateway.cpp

@@ -0,0 +1,490 @@
+#include "ICEExecutionGateway.h"
+#include "lib/jetfix/codec/MessageDefinitions.h"
+#include "lib/jetfix/codec/i32_toa.h"
+#include "lib/jetfix/common/OutFIX.h"
+#include "lib/jetfix/gateway/JetFIXGateway.h"
+#include "lib/logging/logging.h"
+#include "lib/orderman/events/CancelRejectEvent.h"
+#include "lib/orderman/events/GatewayStatusChangeEvent.h"
+#include "lib/orderman/events/NewExecutionReportEvent.h"
+#include "lib/orderman/events/NewSecuritiesEvent.h"
+#include "lib/orderman/orderman_utils/fix_order_codec.h"
+#include "lib/api/services/AdminService.pb.h"
+#include "lib/orderman/events/NewAdminDataEvent.h"
+#include "lib/tracing/bts_trace.h"
+#include "lib/utils/TimeStamp.h"
+#include "lib/utils/helper_macros.h"
+#include "lib/utils/Dispatcher.h"
+#include "lib/utils/bts_chrono.h"
+
+#include <boost/pool/object_pool.hpp>
+#include <atomic>
+#include <set>
+
+namespace bts {
+namespace orderman {
+
+using namespace jetfix;
+
+using JFGw = JetFIXGateway<fix::ICEExecutionReportMsg, fix::BusinessMessageRejectMsg, fix::RejectMsg, 
+                           fix::ICECancelRejectMsg, fix::ICETraderLoginResponseMsg, fix::ICETraderLogoutResponseMsg>;
+
+
+
+struct ICEExecutionGateway::PImpl 
+{
+   PImpl(const config::JetFIX::Connection &cfg, trace::PerfLog &l,
+         md::SecurityOracle o)
+       : gw(cfg, l), exe_report_pool(), state(State::WAITING_FOR_SECURITIES),
+         securities(), securities_by_exchange_id(), gw_name(cfg.name()),
+         cfg(cfg), perf_log(l), admin_data(), nusers(-1), connection_time(), exec_id_to_memo(), 
+         sec_oracle(o)
+   {}
+
+   JFGw gw;
+   boost::object_pool<ExecutionReport> exe_report_pool;
+   State state;
+   std::unordered_map<SecurityKey::Value, api::Security const *> securities;
+   std::unordered_map<int64_t, api::Security const *> securities_by_exchange_id;
+   GatewayName gw_name;
+   config::JetFIX::Connection cfg;
+   OnExeReport on_exe_report;
+   OnGwStatusChange on_gw_status_change;
+   OnCancelReject on_cancel_reject;
+   trace::PerfLog &perf_log;
+   api::admin::AdminData admin_data;
+   std::atomic<int> nusers;
+   TimeStamp connection_time;
+   std::unordered_map<std::string, std::string> exec_id_to_memo; //for the bullshit adjusts
+   md::SecurityOracle sec_oracle;
+};
+
+ICEExecutionGateway::ICEExecutionGateway(const config::JetFIX::Connection &cfg, trace::PerfLog &log, md::SecurityOracle o)
+    : IExecutionGateway(), me(new PImpl(cfg, log, o)) 
+{
+   me->state = State::WAITING_FOR_ADMIN;
+}
+
+ICEExecutionGateway::~ICEExecutionGateway() {}
+
+void ICEExecutionGateway::on_admin_data(const mercury::Update<NewAdminDataEvent>& update) 
+{
+   if (update.events) 
+   {
+      for (auto const &ev : *(update.events)) 
+         me->admin_data.CopyFrom(*ev.admin_data);
+
+      if (me->state == State::WAITING_FOR_ADMIN) 
+      {
+         LOG(OrderMan, Warn, "Received admin data. Waiting for securities {}", me->gw_name.value());
+         me->state = State::WAITING_FOR_SECURITIES;
+      }
+   }   
+}
+
+void ICEExecutionGateway::on_securities(const mercury::Update<NewSecuritiesEvent> &update) 
+{
+   if (update.events) 
+   {
+      for (auto const &ev : *(update.events)) 
+      {
+         auto sec_ptr = ev.security;
+         me->securities[sec_ptr->id()] = sec_ptr;
+         if(sec_ptr->has_ice())
+            me->securities_by_exchange_id[sec_ptr->ice().exchange_id()] = sec_ptr;
+      }
+
+      if (me->state == State::WAITING_FOR_SECURITIES) 
+      {
+         LOG(OrderMan, Warn, "Received first set of {} securities. Beginning wait 2s for trader login {}",
+             update.events->size(), me->gw_name.value());         
+         // got our security synopsis, ready to start the GW
+         me->state = State::READY_TO_START_GW;
+         
+         // subscribe to gateway events
+         me->gw.on<fix::ICEExecutionReportMsg>() = [this](auto const &msg) 
+         {
+            this->on_exe_report(msg);
+         };
+         me->gw.on<fix::ICECancelRejectMsg>() = [this](auto const &msg) 
+         {
+            this->on_cancel_reject(msg);
+         };
+         me->gw.on_ready() = [this]() 
+         {
+            this->on_connection_status_change(true);
+         };
+         me->gw.on_disconnect() = [this]() 
+         {
+            this->on_connection_status_change(false);
+         };
+         me->gw.on<fix::BusinessMessageRejectMsg>() = [this](auto const &m) 
+         {
+            this->on_business_reject(m);
+         };
+         me->gw.on<fix::RejectMsg>() = [this](auto const &m) 
+         {
+            this->on_session_reject(m);
+         };            
+      }
+   }
+}
+
+void ICEExecutionGateway::start() 
+{ 
+   me->gw.start(); 
+   bts_assert_msg(me->state == State::READY_TO_START_GW, "ICEExecutionGateway statemachine failure");
+}
+
+void ICEExecutionGateway::on_connection_status_change(bool is_connected) 
+{
+   LOG(OrderMan, Info, "Connection state of {} changed to {}",
+       me->gw_name.value(), is_connected ? std::string("connected") : std::string("disconnected"));
+   
+   if(is_connected && me->state == State::READY_TO_START_GW)
+   {
+      me->state = State::TRADER_LOGIN_TIMEOUT;
+      me->connection_time = Timer::now();
+   }   
+   
+   if(me->state == State::READY)
+   {
+      me->on_gw_status_change(is_connected);      
+      if(!is_connected) //if we disconnect, set state so we log in again
+         me->state = State::READY_TO_START_GW;
+   }
+}
+
+
+void ICEExecutionGateway::on_trader_login_timeout_expired()
+{
+   if (me->state == State::TRADER_LOGIN_TIMEOUT) 
+   {
+      LOG(OrderMan, Warn, "Done waiting for login timeout, logging on traders {}", me->gw_name.value());
+
+      // got our security synopsis, ready to start the GW
+      me->state = State::LOGIN_TIMEOUT_COMPLETE;      
+      me->gw.on<fix::ICETraderLoginResponseMsg>() = [&](const fix::ICETraderLoginResponseMsg& lmsg) 
+      {
+         std::string u = lmsg.get<fix::UserName>();
+         std::string m;
+         if(lmsg.has<fix::Text>())
+            m = lmsg.get<fix::Text>();         
+         LOG(OrderMan, Error, "Succesffully logging trader {} into ICE {} {}", u, m, me->gw_name.value());
+         if(--me->nusers == 0)
+            this->on_login();
+      };
+      
+      me->gw.on<fix::ICETraderLogoutResponseMsg>() = [&](const fix::ICETraderLogoutResponseMsg& lmsg) 
+      {
+         std::string u = lmsg.get<fix::UserName>();
+         std::string m;
+         if(lmsg.has<fix::Text>())
+            m = lmsg.get<fix::Text>();
+         
+         LOG(OrderMan, Error, "Error logging trader {} into ICE {} {}", u, m, me->gw_name.value());
+         if(--me->nusers == 0)
+            this->on_login();
+      };    
+       
+      std::set<std::string> auth_traders;
+      for(const auto& u : me->admin_data.users())
+      {         
+         if(u.has_user_data() && u.user_data().has_exchange_user_data() && u.user_data().exchange_user_data().has_ice_data())
+         {
+            const auto& ice = u.user_data().exchange_user_data().ice_data();
+            if(auth_traders.find(ice.authorized_trader()) == auth_traders.end())
+            {
+               if(me->nusers == -1)
+                  me->nusers = 0;
+               me->nusers++;
+               
+               auto result = std::make_shared<OutFIX>(me->cfg);
+               auto &stream = result->fix.back();
+               auto &buffer = stream.new_msg(MessageTypes::ICETraderLogin);               
+               buffer.add<fix::CompanyID>(ice.company_id());      
+               buffer.add<fix::UserName>(ice.authorized_trader());      
+               buffer.add<fix::RawData>(ice.password());     
+               
+               if(ice.mifid_direct_electronic_access() >= 0)
+                  buffer.add<fix::MifidDirectElectronicAccess>(ice.mifid_direct_electronic_access());  
+               if(ice.mifid_trading_capacity() >= 0)   
+                  buffer.add<fix::MifidTradingCapacity>(ice.mifid_trading_capacity());     
+               if(ice.mifid_liquidity_provision() >= 0)
+                  buffer.add<fix::MifidLiquidityProvision>(ice.mifid_liquidity_provision());     
+               if(ice.mifid_commodity_deriv_indicator() >= 0)
+                  buffer.add<fix::MifidCommodityDerivIndicator>(ice.mifid_commodity_deriv_indicator());     
+               if(ice.mifid_investment_decision() > 0)
+                  buffer.add<fix::MifidInvestmentDecision>(ice.mifid_investment_decision());     
+               if(ice.mifid_execution_decision() > 0)
+                  buffer.add<fix::MifidExecutionDecision>(ice.mifid_execution_decision());     
+               if(ice.mifid_client_id_code() > 0)
+                  buffer.add<fix::MifidClientIDCode>(ice.mifid_client_id_code());                   
+               if(ice.mifid_id() > 0)
+                  buffer.add<fix::MifidId>(ice.mifid_id());                   
+               stream.close();
+               
+               trace::CorID trace;            
+               send_fix(result, trace);
+               auth_traders.insert(ice.authorized_trader());
+            }
+         }               
+      }
+      LOG(OrderMan, Warn, "{} traders identified as ICE traders {}", auth_traders.size(), me->gw_name.value());               
+   }
+}
+
+void ICEExecutionGateway::on_login()
+{
+   if (me->state == State::LOGIN_TIMEOUT_COMPLETE) 
+   {
+      LOG(OrderMan, Warn, "Successful login traders {}", me->gw_name.value());
+      me->state = State::READY;     
+      me->on_gw_status_change(true);                
+   }   
+}
+
+OnExeReport &ICEExecutionGateway::on_exe_report() { return me->on_exe_report; }
+
+OnGwStatusChange &ICEExecutionGateway::on_gw_status_change() 
+{
+   return me->on_gw_status_change;
+}
+
+OnCancelReject &ICEExecutionGateway::on_cancel_reject() 
+{
+   return me->on_cancel_reject;
+}
+
+bool ICEExecutionGateway::process_memo(const jetfix::fix::ICEExecutionReportMsg &msg, OrderKey* ok, SecurityKey* sk)
+{
+   if(msg.has<fix::Memo>())
+   {
+      decode_memo(msg.get<fix::Memo>(), ok, sk);   
+      return true;
+   }
+   else
+   {
+      if (msg.get<fix::ExecType>() == api::orderman::EXEC_TYPE_TRADE_CORRECT && msg.has<fix::ExecRefID>()) 
+      {
+         std::string ref_exec_id = msg.get<fix::ExecRefID>();
+         auto found = me->exec_id_to_memo.find(ref_exec_id);
+         if(found == me->exec_id_to_memo.end()) 
+         {
+            auto new_price = Price(0);
+            if(msg.has<fix::LastPx>())
+               new_price = Price(msg.get<fix::LastPx>());
+            
+            LOG(OrderMan, Error,
+               "Received unsupported trade correction msg.  This correction will not be forwarded. The trader should be alerted. "
+               "Partial original fill id: {}. New price: {}", ref_exec_id, new_price.value());
+            return false;
+         }
+         else
+         {
+            decode_memo(found->second, ok, sk);
+            return true;
+         }
+      }      
+   }
+   return false;      
+}
+
+void ICEExecutionGateway::on_exe_report(const jetfix::fix::ICEExecutionReportMsg &msg) 
+{
+   OrderKey ok;
+   SecurityKey sk;
+   if(!process_memo(msg, &ok, &sk))
+   {
+      std::string execid = "";
+      if(msg.has<fix::ExecID>())
+         execid = msg.get<fix::ExecID>();
+      LOG(OrderMan, Error,
+         "Received execution report with no memo execid: {}. Will not be able to process this report. This is likely a bug that could lead "
+         "to lost fills. Please investigate.", execid);
+      return;      
+   }
+   auto scoped_log = trace::log_scope(me->perf_log, msg.trace(), "ICEExecutionGateway::on_exe_report", ok.value());
+
+   LOG(Latency, Info, "Started handling of execution report for order {}", ok.value());
+   auto &sec = me->securities[sk.value()];
+   if (!sec) 
+   {
+      sec = me->sec_oracle.get_security(SecurityKey(sk.value()));
+   }   
+   if (!sec)
+      LOG(OrderMan, Error,
+          "Received execution report for an unknown security {}. Will not be able to process this report. This is likely a bug that could lead "
+          "to lost fills. Please investigate.",
+          sk.value());
+   else 
+   {
+      if(sec->has_ice())
+         me->securities_by_exchange_id[sec->ice().exchange_id()] = sec;
+      
+      api::Security const *leg_security = nullptr;
+      if (msg.has<fix::MultiLegReportingType>() && msg.get<fix::MultiLegReportingType>() == 2) 
+      {
+         auto it = me->securities_by_exchange_id.find(msg.get<fix::Int_Symbol>());
+         if (it == me->securities_by_exchange_id.end()) 
+         {
+            LOG(OrderMan, Error,
+                "Received execution report for an unknown spread leg"
+                "security {}. Will not be able to process this report. "
+                "This is likely a bug that could lead to "
+                "lost leg fills. Please investigate. Spread sec_key: {} leg "
+                "exchange id: {}",
+                sk.value(), msg.get<fix::Int_Symbol>());
+            return;
+         }
+         leg_security = it->second;
+      }      
+      std::string text_msg;
+      if (msg.has<fix::Text>())
+         text_msg = msg.get<fix::Text>();
+      if (msg.has<fix::OrdRejReason>())
+         text_msg += " Rejected OrdRejReason: " + std::to_string(msg.get<fix::OrdRejReason>());
+         
+      if (msg.has<fix::DeleteReason>())
+         text_msg += " Cancelled DeleteReason: " + std::to_string(msg.get<fix::DeleteReason>());
+
+      // allocate from our pool
+      auto data = me->exe_report_pool.construct();
+
+      if(msg.has<fix::Memo>())
+         data->memo = msg.get<fix::Memo>();
+      data->exec_type = api::orderman::ExecType(msg.get<fix::ExecType>());
+      data->msg_seq_num = msg.get<fix::MsgSeqNum>();
+
+      // copy data from fix msg to our structure
+      if (msg.has<fix::ClearingAccount>())
+         str_to(msg.get<fix::ClearingAccount>(), &data->account_name);
+      if (msg.has<fix::TargetSubID>())
+         str_to(msg.get<fix::TargetSubID>(), &data->operator_id);
+      if (msg.has<fix::CompanyID>())
+         data->ice_company_id = msg.get<fix::CompanyID>();         
+
+      data->order_key = ok;
+      data->security = sec;
+      data->leg_security = leg_security;
+      data->side = api::Side(msg.get<fix::Side>());
+      if (msg.has<fix::TimeInForce>())
+         data->time_in_force = api::orderman::TimeInForce(msg.get<fix::TimeInForce>());
+      else
+         data->time_in_force = api::orderman::TIME_IN_FORCE_UNDEFINED;
+
+      if (msg.has<fix::OrdType>())
+         data->order_type = api::orderman::NativeOrderType(msg.get<fix::OrdType>());
+      else
+         data->order_type = api::orderman::NATIVE_ORDER_TYPE_UNDEFINED;
+         
+      data->order_status = api::orderman::NativeOrderStatus(msg.get<fix::OrdStatus>());
+      data->manual_order_indicator = api::orderman::MANUAL_ORDER_INDICATOR_NO;
+      
+      std::string oid_string = get_ice_order_id_string(msg);
+      str_to(oid_string, &data->exchange_order_id);
+      data->ice_cl_ord_id = 0;
+      if(msg.has<fix::Int_ClOrdID>())
+         data->ice_cl_ord_id = msg.get<fix::Int_ClOrdID>();
+      
+      if (msg.has<fix::Price>())
+         data->price = securities::decode_exchange_price(*sec, Price(msg.get<fix::Price>()));
+      else
+         data->price = Price(os::nan());
+
+      if (msg.has<fix::StopPx>())
+         data->stop_price = securities::decode_exchange_price(*sec, Price(msg.get<fix::StopPx>()));
+      else
+         data->stop_price = Price(os::nan());
+         
+      if (msg.has<fix::MaxShow>())
+         data->max_show = Quantity(msg.get<fix::MaxShow>());
+      else
+         data->max_show = Quantity(0);         
+
+      if (msg.has<fix::OrderQty>())
+         data->quantity = Quantity(msg.get<fix::OrderQty>());
+      else
+         data->quantity = Quantity(0);
+
+      data->filled_quantity = Quantity(msg.get<fix::CumQty>());
+      
+      data->last_shares = msg.has<fix::LastShares>()
+                              ? Quantity(msg.get<fix::LastShares>())
+                              : Quantity(0);
+      auto last_px_sec = data->leg_security ? data->leg_security : sec;
+      data->last_price = msg.has<fix::LastPx>()
+                             ? securities::decode_exchange_price(*last_px_sec, Price(msg.get<fix::LastPx>()))
+                             : Price(0);
+      data->avg_price = msg.has<fix::AvgPx>()
+                            ? securities::decode_exchange_price(*last_px_sec, Price(msg.get<fix::AvgPx>()))
+                            : Price(0);
+      if (msg.has<fix::ExecID>())
+      {
+         auto execid = msg.get<fix::ExecID>();
+         str_to(execid, &data->exchange_fill_id);
+         if(msg.has<fix::Memo>())
+            me->exec_id_to_memo[execid] = msg.get<fix::Memo>();
+      }
+               
+      if (msg.has<fix::TransactTimeStr>())
+      {
+         std::string stamp = msg.get<fix::TransactTimeStr>();
+         stamp.resize(sizeof(mercury::TimeStampStr));
+         str_to(stamp, &data->transact_time);
+      }
+      data->cfg = &me->cfg;
+      data->msg = text_msg;
+      data->exchange = api::Venue::VENUE_ICE;
+      me->on_exe_report(*data, msg.trace());
+   }
+}
+
+void ICEExecutionGateway::on_cancel_reject(const jetfix::fix::ICECancelRejectMsg &msg) 
+{
+   OrderKey ok(msg.get<fix::Int_ClOrdID>());
+
+   std::string reject_msg;
+   if (msg.has<fix::Text>())
+      reject_msg = msg.get<fix::Text>();
+   else if (msg.has<fix::CXlRejReason>())
+      reject_msg = fmt::format("Order cancel was rejected with error code {}",
+                               msg.get<fix::CXlRejReason>());
+   me->on_cancel_reject(ok, reject_msg);
+}
+
+
+void ICEExecutionGateway::on_business_reject(const jetfix::fix::BusinessMessageRejectMsg& m) 
+{
+   LOG(OrderMan, Error,
+       "Unexpected business reject in {} gateway . Reason: {}. This should be investigated",
+       me->gw_name.value(),
+       m.has<fix::Text>() ? m.get<fix::Text>() : std::string("Unknown"));
+}
+
+void ICEExecutionGateway::on_session_reject(const jetfix::fix::RejectMsg& m) 
+{
+   LOG(OrderMan, Error,
+       "Unexpected reject message in {} gateway . Reason: {}. This should be investigated",
+       me->gw_name.value(),
+       m.has<fix::Text>() ? m.get<fix::Text>() : std::string("Unknown"));
+}
+
+void ICEExecutionGateway::dispatch_incoming_fix_messages() 
+{
+   me->gw.dispatch_events();   
+   if(me->state == State::TRADER_LOGIN_TIMEOUT && ms_diff(Timer::now(), me->connection_time) >= 2000)
+   {
+      on_trader_login_timeout_expired();
+   }
+}
+
+void ICEExecutionGateway::send_fix(const jetfix::OutFIXPtr& fix, trace::CorID trace) 
+{
+   auto scoped_log = trace::log_scope(me->perf_log,trace,"ICEExecutionGateway::send_fix");
+   me->gw.send(fix,trace);
+   me->gw.dispatch_events();
+}
+
+} // namespace orderman
+} // namespace bts

src/lib/orderman/actors/ICEExecutionGateway.h

@@ -0,0 +1,45 @@
+#pragma once
+
+#include "IExecutionGateway.h"
+
+namespace bts {
+namespace orderman {
+
+// fix execution gateway for a single FIX session
+class ICEExecutionGateway : public IExecutionGateway {
+ public:   
+   ICEExecutionGateway(const config::JetFIX::Connection&, trace::PerfLog &,
+                       md::SecurityOracle);
+   virtual ~ICEExecutionGateway();
+
+   void start() override;
+   void dispatch_incoming_fix_messages() override;
+   void on_securities(const mercury::Update<NewSecuritiesEvent> &) override;
+   void on_admin_data(const mercury::Update<NewAdminDataEvent> &) override;
+   void send_fix(const jetfix::OutFIXPtr& fix, trace::CorID trace) override;
+
+   OnExeReport& on_exe_report() override;
+   OnGwStatusChange& on_gw_status_change() override;
+   OnCancelReject &on_cancel_reject() override;
+   
+ private:
+   BTS_NO_COPY(ICEExecutionGateway);
+   enum class State { WAITING_FOR_ADMIN, WAITING_FOR_SECURITIES, READY_TO_START_GW, TRADER_LOGIN_TIMEOUT, LOGIN_TIMEOUT_COMPLETE, READY };
+   struct PImpl;
+   std::unique_ptr<PImpl> me;
+   
+   void on_trader_login_timeout_expired();
+   void on_login();
+
+   // fix gateway events
+   void on_exe_report(const jetfix::fix::ICEExecutionReportMsg&);
+   void on_cancel_reject(const jetfix::fix::ICECancelRejectMsg&);
+   void on_connection_status_change(bool connected);
+   void on_business_reject(const jetfix::fix::BusinessMessageRejectMsg&);
+   void on_session_reject(const jetfix::fix::RejectMsg&);
+   
+   bool process_memo(const jetfix::fix::ICEExecutionReportMsg &msg, OrderKey*, SecurityKey*);
+};
+
+} // namespace orderman
+} // namespace bts

src/lib/orderman/actors/IExecutionGateway.cpp

@@ -0,0 +1,20 @@
+
+#include "IExecutionGateway.h"
+#include "ICEExecutionGateway.h"
+#include "GLBXExecutionGateway.h"
+
+namespace bts {
+namespace orderman {
+
+std::unique_ptr<IExecutionGateway>
+ExecutionGatewayFactory::create_gateway(const config::JetFIX::Connection& conn, trace::PerfLog& pl, md::SecurityOracle o)
+{
+   if(conn.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_ICE || 
+      conn.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_SIMICE)
+      return std::make_unique<ICEExecutionGateway>(conn, pl, o);
+   else
+      return std::make_unique<GLBXExecutionGateway>(conn, pl, o);
+}
+
+} // namespace orderman
+} // namespace bts

src/lib/orderman/actors/IExecutionGateway.h

@@ -0,0 +1,52 @@
+#pragma once
+
+#include "lib/config/protos/jetfix.pb.h"
+#include "lib/jetfix/codec/MessageDefinitions.h"
+#include "lib/jetfix/common/OutFIXPtr.h"
+#include "lib/md/SecurityOracle.h"
+#include "lib/mercury/events/Update.h"
+#include "lib/orderman/orderman_utils/orderman_common.h"
+#include "lib/utils/bts_no_copy.h"
+
+#include <memory>
+
+namespace bts {
+namespace trace {struct CorID;class PerfLog;}
+namespace orderman {
+class NewSecuritiesEvent;
+class NewAdminDataEvent;
+class ExecutionReport;
+
+
+using OnGwStatusChange = std::function<void(bool)>;
+using OnExeReport = std::function<void(ExecutionReport const &, trace::CorID const &)>;
+using OnCancelReject = std::function<void(OrderKey, std::string const &)>;
+
+// fix execution gateway for a single FIX session
+class IExecutionGateway {
+ public:   
+   IExecutionGateway() = default;
+   virtual ~IExecutionGateway() {}
+
+   virtual void start() = 0;
+   virtual void dispatch_incoming_fix_messages() = 0;
+   virtual void on_securities(const mercury::Update<NewSecuritiesEvent> &) = 0;
+   virtual void on_admin_data(const mercury::Update<NewAdminDataEvent> &) = 0;
+   virtual void send_fix(const jetfix::OutFIXPtr&, trace::CorID) = 0;
+   
+   virtual OnExeReport& on_exe_report() = 0;
+   virtual OnGwStatusChange& on_gw_status_change() = 0;
+   virtual OnCancelReject &on_cancel_reject() = 0;
+private:
+   BTS_NO_COPY(IExecutionGateway);
+};
+
+class ExecutionGatewayFactory
+{
+public:
+   static std::unique_ptr<IExecutionGateway> create_gateway(const config::JetFIX::Connection&, trace::PerfLog&, md::SecurityOracle);
+   
+};
+
+} // namespace orderman
+} // namespace bts

src/lib/orderman/actors/order_manager/OrderManager.cpp

@@ -15,6 +15,7 @@
 #include "lib/orderman/orders/Order.h"
 #include "lib/orderman/securities/ISecurityDB.h"
 #include "lib/utils/inplace_function.h"
+#include "lib/utils/thread_manager.h"
 
 #include <algorithm>
 #include <boost/algorithm/string/trim.hpp>
@@ -44,12 +45,28 @@ struct OrderManager::PImpl {
    mercury::Actor::SubHandle filled_position_sub;
    OrderServicesData services_data;
    mercury::Stream<OrderManagerReadyEvent> pub_ready;
+   std::thread background_thread;
+   std::atomic<bool> shutdown = false;
 };
 
 OrderManager::OrderManager(DynamicOrderRegistry const &b, trace::PerfLog &l,
                            md::SecurityOracle o)
-    : me(new PImpl(*this, b, l, o)) {}
-OrderManager::~OrderManager() {}
+    : me(new PImpl(*this, b, l, o)) {
+   auto id_gen = &me->services_data.id_gen;
+   auto done = &me->shutdown;
+   me->background_thread =
+       threading::make_thread("oman_background", [id_gen, done]() {
+          while (!(*done)) {
+             id_gen->refill();
+             sleep_for(10ms);
+          }
+       });
+}
+
+OrderManager::~OrderManager() {
+   me->shutdown = true;
+   me->background_thread.join();
+}
 
 void OrderManager::init() {
    subscribe<NewAdminDataEvent>(
@@ -63,11 +80,13 @@ void OrderManager::init() {
 
 void OrderManager::on_admin_data(const mercury::Update<NewAdminDataEvent> &m) {
    if (m.events) {
-
       if (me->state == State::WAIT_ADMIN) {
          LOG(OrderMan, Info,
-             "Order manager received admin data. Subscribing "
-             "to security list now ");
+             "Order manager received admin data. Subscribing to security list "
+             "now ");
+         for (auto &osd : me->services_data.sessions)
+            osd->gw->on_admin_data(m);
+
          me->state = State::WAIT_SEC;
          subscribe<NewSecuritiesEvent>([this](auto const &sec_update) {
             this->on_securities(sec_update);
@@ -100,7 +119,7 @@ void OrderManager::on_securities(
 
    // forward to execution gateways
    for (auto &osd : me->services_data.sessions)
-      osd->gw.on_securities(update);
+      osd->gw->on_securities(update);
 
    if (update.events) {
 
@@ -440,19 +459,20 @@ void OrderManager::init_order_services() {
       // reject evets
       auto s_ptr = s.get();
       s->exe_report_pub = make_stream<NewExecutionReportEvent>(gw_name);
-      s->gw.on_exe_report() = [s_ptr, this](auto const &er, auto const &t) {
+      s->gw->on_exe_report() = [s_ptr, this](auto const &er, auto const &t) {
          publish(s_ptr->exe_report_pub, &er, t);
       };
       s->gw_status_change_pub = make_stream<GatewayStatusChangeEvent>(gw_name);
-      s->gw.on_gw_status_change() = [s_ptr, this, gw_name](bool b) {
+      s->gw->on_gw_status_change() = [s_ptr, this, gw_name](bool b) {
          publish(s_ptr->gw_status_change_pub, gw_name, b);
       };
       s->cancel_reject_pub = make_stream<CancelRejectEvent>(gw_name);
-      s->gw.on_cancel_reject() = [s_ptr, this](auto const &k, auto const &msg) {
+      s->gw->on_cancel_reject() = [s_ptr, this](auto const &k,
+                                                auto const &msg) {
          publish(s_ptr->cancel_reject_pub, k, msg);
       };
       // handlers are ready so we can start processing fix events
-      s->gw.start();
+      s->gw->start();
 
       reload_fix_buffer(*s);
    }
@@ -541,7 +561,7 @@ void OrderManager::after_event() {
             auto scoped_do_work = trace::log_scope(me->services_data.log, {},
                                                    "OrderManager::after_event");
             fix_stream.close();
-            osd->gw.send_fix(osd->current_buffer, osd->current_trace);
+            osd->gw->send_fix(osd->current_buffer, osd->current_trace);
             reload_fix_buffer(*osd);
          }
       }
@@ -566,7 +586,7 @@ void OrderManager::on_poll(trace::PerfLog &log) {
        trace::log_scope(log, {}, "OrderManager::dispatch_incoming_fix_messages",
                         utils::TimeStamp::Interval::from_microseconds(2));
    for (auto &osd : me->services_data.sessions)
-      osd->gw.dispatch_incoming_fix_messages();
+      osd->gw->dispatch_incoming_fix_messages();
 }
 
 void OrderManager::publish_order_def_update(

src/lib/orderman/actors/order_manager/OrderServices.h

@@ -47,6 +47,9 @@ class OrderServices {
       return ss.is_gateway_connected();
    }
 
+   uint64_t gen_id() {
+      return _data->id_gen.next();
+   }
    Governor &governor(SessionServices &ss) { return ss.governor(); }
 
    void queue_fix(SessionServices &ss, jetfix::MessageTypes mt,

src/lib/orderman/actors/order_manager/OrderServicesData.cpp

@@ -12,7 +12,9 @@ OrderServicesData::OrderServicesData(OrderManager &owner,
       order_factory(sec_db, order_router, b),
       last_timer_ev(
           Timer::now(),
-          to<mercury::TimeStampStr>(datetime::FIX_timestamp(datetime::now()))) {
+          to<mercury::TimeStampStr>(datetime::FIX_timestamp(datetime::now()))),
+      id_gen("orderman_ids",5000)
+{
 
    auto &all_connections = config::get<config::JetFIX>().connection();
    for (auto const &c : all_connections)

src/lib/orderman/actors/order_manager/OrderServicesData.h

@@ -9,7 +9,7 @@
 
 #include "lib/mercury/events/TimerEvent.h"
 #include "lib/md/SecurityOracle.h"
-#include "lib/orderman/actors/ExecutionGateway.h"
+#include "lib/orderman/actors/IExecutionGateway.h"
 #include "lib/orderman/events/ActivateOrderEvent.h"
 #include "lib/orderman/events/BookUpdateEvent.h"
 #include "lib/orderman/events/TickerUpdateEvent.h"
@@ -34,6 +34,7 @@
 #include "lib/orderman/limiter/Limiter.h"
 #include "lib/orderman/order_router/OrderRouter.h"
 #include "lib/utils/datetime.h"
+#include "lib/utils/Puid.h"
 
 #include <boost/date_time/posix_time/posix_time.hpp>
 
@@ -52,7 +53,10 @@ struct SessionServicesData {
                        trace::PerfLog &log, md::SecurityOracle o)
        : session(cfg.name()), gw_connected(false), cfg(cfg),
          governor(cfg.max_order_msgs_per_window(), cfg.window_duration_msecs()),
-         gw(cfg, log,o) {}
+         gw(ExecutionGatewayFactory::create_gateway(cfg, log, o)) 
+   {
+      
+   }
 
    std::string session;
    std::vector<jetfix::OutFIXPtr> free_buffers;
@@ -69,7 +73,7 @@ struct SessionServicesData {
    Governor governor;
    jetfix::OutFIXPtr current_buffer;
    trace::CorID current_trace;
-   ExecutionGateway gw;
+   std::unique_ptr<IExecutionGateway> gw;
    mercury::Stream<NewExecutionReportEvent> exe_report_pub;
    mercury::Stream<GatewayStatusChangeEvent> gw_status_change_pub;
    mercury::Stream<CancelRejectEvent> cancel_reject_pub;
@@ -107,7 +111,8 @@ class OrderServicesData {
    mercury::Stream<OrderOpResultEvent> report_order_op_result_pub;
    mercury::TimerEvent last_timer_ev;
    std::vector<std::unique_ptr<SessionServicesData>> sessions;
-
+   utils::IDGenerator id_gen ;
+   
    SessionServicesData &get_session(std::string const &session_name) {
       for (auto &s : sessions)
          if (s->session == session_name)

src/lib/orderman/actors/order_manager/SessionServices.h

@@ -57,14 +57,14 @@ class SessionServices {
       _gateway_status_sub = src._gateway_status_sub;
       return *this;
    }
+   
+   config::JetFIX::Connection const &session_cfg() const {
+      return _session_data->cfg;
+   }   
 
  protected:
    std::string const &session_name() const { return _session_data->session; }
 
-   config::JetFIX::Connection const &session_cfg() const {
-      return _session_data->cfg;
-   }
-
    void queue_fix(jetfix::MessageTypes mt, jetfix::FIXFragment const &f,
                   trace::CorID const &corid) {
       auto &stream = _session_data->current_buffer->first_stream();

src/lib/orderman/order_factory/OrderFactory.cpp

@@ -10,7 +10,8 @@
 // order types
 #include "lib/orderman/order_factory/DynamicOrderRegistry.h"
 #include "lib/orderman/orders/DynamicOrder.h"
-#include "lib/orderman/orders/NativeOrder.h"
+#include "lib/orderman/orders/GLBXNativeOrder.h"
+#include "lib/orderman/orders/ICENativeOrder.h"
 #include "lib/orderman/orders/TimeBoxOrder.h"
 #include "lib/orderman/orders/TriggerOrder.h"
 #include "lib/orderman/orders/algo/AlgoWrapperOrder.h"
@@ -88,7 +89,7 @@ Order &OrderFactory::create(OrderKey parent,
    // field correctly. Algos will just propagate the value of this
    // field down so eventually native orders launched by algos will
    // get the right value.
-   if (dynamic_cast<NativeOrder *>(&order) == nullptr &&
+   if (dynamic_cast<NativeOrderBase *>(&order) == nullptr &&
        def.manual_order_indicator() ==
            api::orderman::MANUAL_ORDER_INDICATOR_YES) {
       // An algo order incorrectly marked as a manual order
@@ -113,6 +114,24 @@ Order &OrderFactory::recover(api::orderman::OrderRecord const &rec) {
    return order;
 }
 
+NativeOrderBase *OrderFactory::make_order(const api::Venue exchange) {
+   if (exchange == api::Venue::VENUE_ICE ||
+       exchange == api::Venue::VENUE_SIMICE)
+      return new ICENativeOrder();
+   else
+      return new GLBXNativeOrder();
+}
+
+NativeOrderBase *
+OrderFactory::make_native_order(const api::orderman::OrderDefinition &def) {
+   auto secptr = me->sec_db.resolve(SecurityKey(def.security_key()));
+   if (secptr == nullptr)
+      return new GLBXNativeOrder(); // GLBX Native can handle unresolved
+                                    // securities
+   else
+      return make_order(secptr->exchange());
+}
+
 Order &OrderFactory::recreate_from_exe_report(ExecutionReport const &r) {
    using namespace api::orderman;
 
@@ -121,7 +140,7 @@ Order &OrderFactory::recreate_from_exe_report(ExecutionReport const &r) {
    case NATIVE_ORDER_TYPE_LIMIT:
    case NATIVE_ORDER_TYPE_STOP:
    case NATIVE_ORDER_TYPE_STOP_LIMIT: {
-      auto native_order = std::make_unique<NativeOrder>();
+      std::unique_ptr<NativeOrderBase> native_order(make_order(r.exchange));
       native_order->_key = r.order_key;
       native_order->_parent_key = null_order_key();
       auto def = native_order->on_init_from_exe_report(
@@ -151,7 +170,7 @@ Order &OrderFactory::allocate(OrderKey k, OrderKey pk,
    case CustomOrderDefinition::kStop:
    case CustomOrderDefinition::kStopLimit:
    case CustomOrderDefinition::kGlbxIceberg:
-      order_ptr.reset(new NativeOrder());
+      order_ptr.reset(make_native_order(def));
       break;
    case CustomOrderDefinition::kIceberg:
       order_ptr.reset(

src/lib/orderman/order_factory/OrderFactory.h

@@ -11,6 +11,7 @@ class ISecurityDB;
 }
 namespace orderman {
 class Order;
+class NativeOrderBase;
 class DynamicOrderRegistry;
 } // namespace orderman
 namespace orderman {
@@ -45,6 +46,8 @@ class OrderFactory : public IOrderFactory {
  private:
    Order &allocate(OrderKey k, OrderKey pk,
                    api::orderman::OrderDefinition const &);
+   NativeOrderBase* make_order(const api::Venue exchange);
+   NativeOrderBase* make_native_order(const api::orderman::OrderDefinition& def);   
    struct PImpl;
 
    std::unique_ptr<PImpl> me;

src/lib/orderman/orderman_utils/ExecutionReport.h

@@ -35,6 +35,10 @@ public:
    mercury::TimeStampStr transact_time;
    config::JetFIX::Connection const *cfg;
    std::string msg;
+   api::Venue exchange;
+   int64_t ice_cl_ord_id;
+   int32_t ice_company_id;
+   std::string memo;
 };
 
 }}

src/lib/orderman/orderman_utils/fix_order_codec.h

@@ -7,6 +7,7 @@
 #include "lib/jetfix/codec/i32_toa.h"
 #include "lib/utils/bts_assert.h"
 #include "orderman_common.h"
+#include "lib/utils/base62.h"
 
 namespace bts {
 namespace api {
@@ -19,6 +20,8 @@ constexpr int clordid_sz() { return 20; }
 constexpr int price_sz() { return 17; }
 constexpr int quantity_sz() { return 6; }
 constexpr int max_show_sz() { return 6; }
+constexpr int memo_sz() {return 12;}
+constexpr int ice_orderid_sz() {return 12;}
 constexpr uint32_t corid_sz() {
    return 16;
 } // 5 for channel, 1 separator, 10 sn
@@ -50,6 +53,49 @@ template <class MSG> OrderKey get_order_key(MSG const &msg) {
    return result;
 }
 
+inline void encode_memo(OrderKey ok, SecurityKey sk, std::string *out) 
+{
+   auto ok_string = utils::to_base_62(ok.value());
+   auto sk_string = utils::to_base_62(sk.value());
+   bts_assert_msg(ok_string.size() == 6 && sk_string.size() == 6, "base62 encoding didn't work");
+   *out = ok_string + sk_string;   
+}
+
+inline void decode_memo(const std::string& in, OrderKey* ok, SecurityKey* sk)
+{
+   bts_assert(in.size() == memo_sz());   
+   char buffer[memo_sz()/2+1];
+   buffer[memo_sz()/2] = 0;
+   std::memcpy(buffer,&in[0],memo_sz()/2);
+   *ok = OrderKey( utils::from_base_62(std::string(buffer)) );
+
+   std::memcpy(buffer,&in[memo_sz()/2],memo_sz()/2);
+   *sk = SecurityKey( utils::from_base_62(std::string(buffer)) );
+}
+
+template <class MSG>
+std::string get_ice_order_id_string(const MSG& msg)
+{   
+   bts_assert(msg.template has<jetfix::fix::Int_OrderID>());
+   std::string oid_string(ice_orderid_sz(), ' '); 
+   
+   int64_t id = msg.template get<jetfix::fix::Int_OrderID>();   
+   u64_toa(id, &(oid_string[0]), ice_orderid_sz() );
+   return oid_string;
+}
+
+template <class MSG>
+std::string get_ice_clord_id_string(const MSG& msg)
+{   
+   bts_assert(msg.template has<jetfix::fix::Int_ClOrdID>());
+   std::string oid_string(ice_orderid_sz(), ' '); 
+   
+   int64_t id = msg.template get<jetfix::fix::Int_ClOrdID>();   
+   u64_toa(id, &(oid_string[0]), ice_orderid_sz() );
+   return oid_string;
+}
+
+
 inline void encode_corid(const char *channel, uint32_t sn, std::array<char, 17> &out) {
    memset(&out[0], ' ', out.size());
    auto ch_sz = std::min(strlen(channel), (size_t)5);

src/lib/orderman/orders/GLBXNativeOrder.cpp

@@ -1,4 +1,4 @@
-#include "NativeOrder.h"
+#include "GLBXNativeOrder.h"
 
 #include "lib/api/services/OrderService.pb.h"
 #include "lib/config/config.h"

src/lib/orderman/orders/GLBXNativeOrder.h

@@ -1,7 +1,7 @@
 #pragma once
 
 #include "lib/api/services/OrderService.pb.h"
-#include "lib/orderman/orders/Order.h"
+#include "lib/orderman/orders/NativeOrderBase.h"
 #include "lib/tracing/PerfLog.h"
 #include <memory>
 
@@ -13,12 +13,12 @@ class CancelRejectEvent;
 class GatewayStatusChangeEvent;
 class ExecutionReport;
 
-class NativeOrder : public Order {
+class NativeOrder : public NativeOrderBase {
  public:
    NativeOrder();
    ~NativeOrder() override;
 
-   api::orderman::OrderDefinition const &definition() const;
+   api::orderman::OrderDefinition const &definition() const override;
    void on_init_new(api::orderman::OrderDefinition const &,
                     const PreActivationOrderServices &) override;
    void on_init_recovered(api::orderman::OrderRecord const &,
@@ -33,7 +33,7 @@ class NativeOrder : public Order {
 
    api::orderman::OrderDefinition
    on_init_from_exe_report(ExecutionReport const &,
-                           const PreActivationOrderServices &);
+                           const PreActivationOrderServices &) override;
 
  protected:
    void warm_up_activate(OrderServicesData &, OrderServices &) override;
@@ -73,5 +73,7 @@ class NativeOrder : public Order {
                          OrderOpResult *out) ;
 };
 
+typedef NativeOrder GLBXNativeOrder;
+
 } // namespace orderman
 } // namespace bts

src/lib/orderman/orders/ICENativeOrder.h

@@ -0,0 +1,82 @@
+#pragma once
+
+#include "lib/orderman/orders/NativeOrderBase.h"
+#include "lib/tracing/PerfLog.h"
+#include <memory>
+
+namespace bts {
+namespace orderman {
+struct RoutingRecord;
+class NewExecutionReportEvent;
+class CancelRejectEvent;
+class GatewayStatusChangeEvent;
+class ExecutionReport;
+
+class ICENativeOrder : public NativeOrderBase {
+ public:
+   ICENativeOrder();
+   ~ICENativeOrder() override;
+
+   const api::orderman::OrderDefinition& definition() const override;
+   void on_init_new(const api::orderman::OrderDefinition&,
+                    const PreActivationOrderServices&) override;
+   void on_init_recovered(const api::orderman::OrderRecord&,
+                          const PreActivationOrderServices &) override;
+
+   OrderOpResult on_activate(Price, Quantity, OrderServices &) override;
+   OrderOpResult on_update(Price, Quantity) override;
+   OrderOpResult on_update(const api::orderman::Update&) override;
+   OrderOpResult on_detach() override;
+   OrderOpResult on_cancel() override;
+   void on_attached(const api::orderman::OrderDefinition &new_parent) override;
+
+   api::orderman::OrderDefinition
+   on_init_from_exe_report(const ExecutionReport &, const PreActivationOrderServices &) override;
+   void ignore_or_cancel(const std::string&, api::orderman::OrderError, OrderOpResult* out);
+
+ protected:
+   void warm_up_activate(OrderServicesData &, OrderServices &) override;
+   void init_recovered_common(const api::orderman::OrderRecord &,
+                              const PreActivationOrderServices &);
+   
+
+ private:
+   struct PImpl;
+   std::unique_ptr<PImpl> me;
+   void on_exe_report(const NewExecutionReportEvent &);
+   void on_cancel_reject(const CancelRejectEvent &);
+   void on_gateway_status(const GatewayStatusChangeEvent &);
+   void report_state();
+   OrderOpResult _on_activate(log::Level reject_level,
+                              log::Level transition_level, Price p, Quantity q,
+                              bts::orderman::OrderServices &services);
+   void init_fix();
+   void add_exchange_order_id_to_fix();
+
+   void transition(log::Level,
+                   api::orderman::OrderStatus new_state,
+                   const std::string& msg);
+   void transition(api::orderman::OrderStatus new_state,
+                   const std::string& msg);
+   void transition(api::orderman::OrderStatus new_state);
+   void transition_to_error(api::orderman::NativeOrderState::SubError sub_error,
+                            const std::string& msg);
+
+   void update_working_position();
+   bool has_exchange_order_id();
+   Quantity working_position() ;
+   bool is_old_exe_report(api::orderman::NativeOrderState &,
+                          const NewExecutionReportEvent &) const;
+   bool terminal_state(api::orderman::OrderStatus status) const;
+   api::orderman::NativeOrderState &
+   get_native_state(api::orderman::OrderState &);
+      
+   int64_t get_new_clordid();
+   int64_t get_clordid(uint64_t);
+   void setup_ids(const std::string& sender_compid);
+   void generate_update_ids(const std::string&);
+   void generate_cancel_ids(const std::string&);
+};
+
+} // namespace orderman
+} // namespace bts

src/lib/orderman/orders/NativeOrderBase.h

@@ -0,0 +1,22 @@
+#pragma once
+
+#include "lib/api/services/OrderService.pb.h"
+#include "lib/orderman/orders/Order.h"
+
+namespace bts {
+namespace orderman {
+class ExecutionReport;
+
+class NativeOrderBase : public Order {
+ public:
+   NativeOrderBase() = default;
+   ~NativeOrderBase() override {};
+
+   virtual api::orderman::OrderDefinition const &definition() const = 0;
+   api::orderman::OrderDefinition
+   virtual on_init_from_exe_report(ExecutionReport const &, const PreActivationOrderServices &) = 0;
+};
+
+} // namespace orderman
+} // namespace bts
+

src/lib/utils/AuthSessionManager.cpp

@@ -80,7 +80,7 @@ bool AuthSessionManager::validate_ip_port(const bts::config::KeyCode& kc)
    return portgood && ipgood;
 }
 
-bool AuthSessionManager::add_or_replace_session(const api::user::TokenPayload& tok, const admin::BTSUser& user, std::string* msg)
+bool AuthSessionManager::add_or_replace_session(const api::user::TokenPayload& tok, const std::string& username, std::string* msg)
 {
    if(tok.wheel())
       return true;      
@@ -105,18 +105,18 @@ bool AuthSessionManager::add_or_replace_session(const api::user::TokenPayload& t
    if(it != _map.end())
    {
       _map[tok.id()] = Session{tok};
-      LOG(User, Warn, "KEYCODE: Replacing session {} user:{}:{} ", tok.id(), user.id(), user.name());
+      LOG(User, Warn, "KEYCODE: Replacing session {} user:{} ", tok.id(), username);
       return true;
    }
    else if(_map.size() < _nusers)
    {
       _map[tok.id()] = Session{tok};
-      LOG(User, Warn, "KEYCODE: Creating session {} user:{}:{}", tok.id(), user.id(), user.name());
+      LOG(User, Warn, "KEYCODE: Creating session {} user:{}", tok.id(), username);
       return true;         
    }            
    if(msg)
       *msg = "KEYCODES: Not Enough Logins available for session";
-   LOG(User, Error, "KEYCODES: Not Enough Logins available for session {} user:{}:{}", tok.id(), user.id(), user.name());
+   LOG(User, Error, "KEYCODES: Not Enough Logins available for session {} user:{}", tok.id(), username);
    return false;
 }
 
@@ -134,9 +134,18 @@ bool AuthSessionManager::check_session(const api::user::TokenPayload& tok)
          bool same_token = _concurrent || (tok.date() == it->second.token.date());
          return not_expired && same_token;
       }
+      else //not in map, try to login in case server booted
+      {
+         return add_or_replace_session(tok);
+      }
    }
    return false;
 }
+
+void AuthSessionManager::clear_sessions()
+{
+   _map.clear();
+}
    
 }}}
 #endif

src/lib/utils/AuthSessionManager.h

@@ -19,8 +19,9 @@ class AuthSessionManager
 public:
    AuthSessionManager();
 
-   bool add_or_replace_session(const api::user::TokenPayload& tok, const admin::BTSUser& user, std::string* msg = nullptr);
+   bool add_or_replace_session(const api::user::TokenPayload& tok, const std::string& username = "", std::string* msg = nullptr);
    bool check_session(const api::user::TokenPayload& tok);
+   void clear_sessions();
 
 private:
    bool validate_ip_port(const bts::config::KeyCode& kc);

src/lib/utils/AuthToken.cpp

@@ -51,6 +51,11 @@ AuthSessionManager& session_manager()
    return mgr;
 }
 
+void clear_all_sessions()
+{
+   session_manager().clear_sessions();
+}
+
 bool requires_auth_session(const admin::BTSUser& user)
 {
    //if the user can trade in any account or see market data, it needs a session
@@ -76,6 +81,14 @@ bool requires_auth_session(const admin::BTSUser& user)
                return true;
          }
       }
+      if(user.user_data().exchange_user_data().has_ice_data())
+      {
+         for(const auto& i: user.user_data().exchange_user_data().ice_data().md_permissions())
+         {
+            if(i.enabled())
+               return true;
+         }
+      }      
    }  
    return false; 
 }
@@ -89,7 +102,7 @@ AuthToken generate_token(const admin::BTSUser& user, std::string* msg)
       payload.set_wheel(true);
       
    if(requires_auth_session(user))
-      session_manager().add_or_replace_session(payload, user, msg);
+      session_manager().add_or_replace_session(payload, user.name(), msg);
    auto token = encrypt(payload.SerializeAsString()); 
    bts_assert_d(token.size() < 255-8);
    AuthToken result;

src/lib/utils/AuthToken.h

@@ -10,6 +10,8 @@ namespace auth {
 AuthToken generate_token(const admin::BTSUser&, std::string* msg = nullptr);
 AuthToken generate_wheel_token();
 bool decode_token(const api::AuthToken& token, user::TokenPayload* payload);
+bool requires_auth_session(const admin::BTSUser& user);
+void clear_all_sessions();
 
 }}}
 

src/lib/utils/Puid.cpp

@@ -1,3 +1,4 @@
+#ifndef _WIN32
 #include "Puid.h"
 
 #include "../os/os.h"
@@ -7,17 +8,17 @@
 #include <fstream>
 #include <vector>
 
-#include <boost/interprocess/sync/file_lock.hpp>
-#include <boost/interprocess/sync/scoped_lock.hpp>
+#include <boost/algorithm/string/classification.hpp>
 #include <boost/algorithm/string/predicate.hpp>
 #include <boost/algorithm/string/split.hpp>
-#include <boost/algorithm/string/classification.hpp>
 #include <boost/filesystem.hpp>
+#include <boost/interprocess/sync/file_lock.hpp>
+#include <boost/interprocess/sync/scoped_lock.hpp>
 
-#include <tbb/mutex.h>
 #include <iomanip>
-#include <time.h>
 #include <stdint.h>
+#include <tbb/mutex.h>
+#include <time.h>
 
 namespace bts {
 namespace utils {
@@ -35,93 +36,99 @@ const std::string counter_file_name = "counter_file";
 // relying on the fact that metadata changes (like renaming) are
 // atomic to provide transactional updates to a counter file which is
 // used to generate uuids.
-// 
-// Write process (on Foo.txt): 
-// - Step W1: Acquire "write lock" on the existing file. 
-// - Step W2: Copy the old file in a new unique temporary file. (copy Foo.txt Foo.TmpNNNN.txt)
-// - Step W3: Apply the writes to the new file (Foo.TmpNNNN.txt). 
-// - Step W4: Flush all the writes (for example those being remaining in the cache manager). 
-// - Step W5: Rename the old file in a new unique Alternate form (ren Foo.txt Foo.AltNNNN.txt)
-// - Step W6: Rename the new file into the old file (ren Foo.TmpNNNN.txt Foo.txt)
-// - Step W7: Delete the old Alternate file (del Foo.AltNNNN.txt). If this fails, simply ignore. The file will be deleted later during the next recovery. 
+//
+// Write process (on Foo.txt):
+// - Step W1: Acquire "write lock" on the existing file.
+// - Step W2: Copy the old file in a new unique temporary file. (copy Foo.txt
+// Foo.TmpNNNN.txt)
+// - Step W3: Apply the writes to the new file (Foo.TmpNNNN.txt).
+// - Step W4: Flush all the writes (for example those being remaining in the
+// cache manager).
+// - Step W5: Rename the old file in a new unique Alternate form (ren Foo.txt
+// Foo.AltNNNN.txt)
+// - Step W6: Rename the new file into the old file (ren Foo.TmpNNNN.txt
+// Foo.txt)
+// - Step W7: Delete the old Alternate file (del Foo.AltNNNN.txt). If this
+// fails, simply ignore. The file will be deleted later during the next
+// recovery.
 // - Step W8: Release "write lock" on the existing file.
 
-// Recovery from a crash during write (on Foo.txt): 
-// - Step R1: If Foo.txt is missing but we have both Foo.AltNNNN.txt and Foo.TmpNNNN.txt present, then we crashed between Step W5 and Step W6. Retry from Step W6. 
-// - Step R2: If Foo.txt is present but Foo.TmpNNNN.txt is also present, then we crashed before Step W5. Try to delete all Foo.TmpNNNN.txt files and ignore failures. 
-// - Step R3: If Foo.txt is present but Foo.AltNNNN.txt is also present, then we crashed between Step W6 and Step W7. Try to delete all Foo.AltNNNN.txt files and ignore failures.
+// Recovery from a crash during write (on Foo.txt):
+// - Step R1: If Foo.txt is missing but we have both Foo.AltNNNN.txt and
+// Foo.TmpNNNN.txt present, then we crashed between Step W5 and Step W6. Retry
+// from Step W6.
+// - Step R2: If Foo.txt is present but Foo.TmpNNNN.txt is also present, then we
+// crashed before Step W5. Try to delete all Foo.TmpNNNN.txt files and ignore
+// failures.
+// - Step R3: If Foo.txt is present but Foo.AltNNNN.txt is also present, then we
+// crashed between Step W6 and Step W7. Try to delete all Foo.AltNNNN.txt files
+// and ignore failures.
 
-void init_store (std::string const &path,
-                 std::string const &lock_path,
-                 std::string const &counter_path)
-{
-   if (!boost::filesystem::is_directory(path))
-   {
+void init_store(std::string const &path, std::string const &lock_path,
+                std::string const &counter_path) {
+   if (!boost::filesystem::is_directory(path)) {
       boost::filesystem::create_directories(path);
-      std::ofstream fs(lock_path.c_str ());
-      std::ofstream fc(counter_path.c_str ());
+      std::ofstream fs(lock_path.c_str());
+      std::ofstream fc(counter_path.c_str());
       fc << 0;
    }
 }
 
-std::string find_file (std::string const &path, std::string const &ending)
-{
+std::string find_file(std::string const &path, std::string const &ending) {
    for (boost::filesystem::directory_iterator p =
-           boost::filesystem::directory_iterator (path);
-        p != boost::filesystem::directory_iterator (); p++)
-   {
-      std::string fn (p->path ().string ());
+            boost::filesystem::directory_iterator(path);
+        p != boost::filesystem::directory_iterator(); p++) {
+      std::string fn(p->path().string());
       if (boost::algorithm::ends_with(fn, ending))
          return fn;
    }
-   return std::string ();
+   return std::string();
 }
 
 // boost::filesystem::copy_file is broken for C++ 11 unless one compiles boost
-// with c++ 11. (See http://boost.2283326.n4.nabble.com/Filesystem-problems-with-g-std-c-0x-td2639716.html)
+// with c++ 11. (See
+// http://boost.2283326.n4.nabble.com/Filesystem-problems-with-g-std-c-0x-td2639716.html)
 // so we provide our own.
-void copy_file(std::string const&in, std::string const &out)
-{
-   std::fstream f(in.c_str (), std::fstream::in|std::fstream::binary);
+void copy_file(std::string const &in, std::string const &out) {
+   std::fstream f(in.c_str(), std::fstream::in | std::fstream::binary);
    f << std::noskipws;
    std::istream_iterator<unsigned char> begin(f);
    std::istream_iterator<unsigned char> end;
- 
-   std::fstream f2(out.c_str (), std::fstream::out|std::fstream::trunc|std::fstream::binary);
+
+   std::fstream f2(out.c_str(), std::fstream::out | std::fstream::trunc |
+                                    std::fstream::binary);
    std::ostream_iterator<char> begin2(f2);
    std::copy(begin, end, begin2);
 }
 
-uint32_t get_host_ip_uint32 (std::string const &ip)
-{
+uint32_t get_host_ip_uint32(std::string const &ip) {
    std::vector<std::string> split_vec;
-   boost::algorithm::split( split_vec, ip,boost::algorithm::is_any_of(".") );
-   bts_assert (split_vec.size () == 4);
+   boost::algorithm::split(split_vec, ip, boost::algorithm::is_any_of("."));
+   bts_assert(split_vec.size() == 4);
    uint32_t result = 0;
    int idx = 0;
    for (std::string &byte : split_vec) {
-      uint32_t this_byte = atoi (byte.c_str ());
+      uint32_t this_byte = atoi(byte.c_str());
       this_byte = (this_byte << (idx++ * 8));
       result += this_byte;
    }
    return result;
 }
 
-std::string &get_host_ip_string (std::string const &ip)
-{
+std::string &get_host_ip_string(std::string const &ip) {
    static std::string result;
-   if (result.size () == 0) {
+   if (result.size() == 0) {
       std::ostringstream result_stream;
-      
-      bts_assert (ip.size ());
-      bts_assert (ip != "127.0.0.1");
-      std::istringstream ip_str (ip);
+
+      bts_assert(ip.size());
+      bts_assert(ip != "127.0.0.1");
+      std::istringstream ip_str(ip);
 
       for (int i = 0; i < 4; i++) {
          int b;
          ip_str >> b;
-         ip_str.get ();
-         
+         ip_str.get();
+
          char high_nibble = (b & 0xf0) >> 4;
          char low_nibble = b & 0x0f;
          if (high_nibble >= 10)
@@ -134,14 +141,13 @@ std::string &get_host_ip_string (std::string const &ip)
             low_nibble += '0';
          result_stream << high_nibble << low_nibble;
       }
-      result = result_stream.str ();
+      result = result_stream.str();
    }
    return result;
 }
 
-uint64_t reserve_ids (std::string const &scope, int n_ids)
-{
-   tbb::mutex::scoped_lock l (lock);
+uint64_t reserve_ids(std::string const &scope, int n_ids) {
+   tbb::mutex::scoped_lock l(lock);
 
    // we risk race conditions if the directory is not there.
    // but that's OK, this is rare and easily manageable.
@@ -149,83 +155,82 @@ uint64_t reserve_ids (std::string const &scope, int n_ids)
    std::string lock_path = path + "/" + lock_file_name;
    std::string counter_path = path + "/" + counter_file_name;
 
-   init_store (path,lock_path, counter_path);
+   init_store(path, lock_path, counter_path);
 
    // W1
-   boost::interprocess::file_lock f_lock(lock_path.c_str ());
-   boost::interprocess::scoped_lock<boost::interprocess::file_lock> scope_lock (f_lock);
+   boost::interprocess::file_lock f_lock(lock_path.c_str());
+   boost::interprocess::scoped_lock<boost::interprocess::file_lock> scope_lock(
+       f_lock);
 
-   std::string tmp_file = find_file (path, ".tmp");
-   std::string alt_file = find_file (path, ".alt");
-   std::string counter_file = find_file (path, counter_file_name);
+   std::string tmp_file = find_file(path, ".tmp");
+   std::string alt_file = find_file(path, ".alt");
+   std::string counter_file = find_file(path, counter_file_name);
 
    // r1?
-   if (counter_file.size () == 0 && tmp_file.size () && alt_file.size ()) {
-      boost::filesystem::rename (tmp_file,counter_path);
-      boost::filesystem::remove (alt_file);
-      f_lock.unlock ();
-      lock.unlock ();
-      return reserve_ids (scope,n_ids);
-      
+   if (counter_file.size() == 0 && tmp_file.size() && alt_file.size()) {
+      boost::filesystem::rename(tmp_file, counter_path);
+      boost::filesystem::remove(alt_file);
+      f_lock.unlock();
+      lock.unlock();
+      return reserve_ids(scope, n_ids);
+
    }
    // r2
-   else if (counter_file.size () && tmp_file.size ()) {
-      boost::filesystem::remove (tmp_file);
-      f_lock.unlock ();
-      lock.unlock ();
-      return reserve_ids (scope,n_ids);
+   else if (counter_file.size() && tmp_file.size()) {
+      boost::filesystem::remove(tmp_file);
+      f_lock.unlock();
+      lock.unlock();
+      return reserve_ids(scope, n_ids);
    }
    // r3
-   else if (counter_file.size () && alt_file.size ()) {
-      boost::filesystem::remove (alt_file);
-      f_lock.unlock ();
-      lock.unlock ();
-      return reserve_ids (scope,n_ids);
+   else if (counter_file.size() && alt_file.size()) {
+      boost::filesystem::remove(alt_file);
+      f_lock.unlock();
+      lock.unlock();
+      return reserve_ids(scope, n_ids);
    } else {
-      
+
       // W2, copy counter file to a temp file
-      tmp_file  = path + "/" + Uuid ().to_text_string () + ".tmp";
-      copy_file (counter_path, tmp_file);
+      tmp_file = path + "/" + Uuid().to_text_string() + ".tmp";
+      copy_file(counter_path, tmp_file);
 
       // W3, W4 update temp file, close and flush
       uint64_t counter_value;
-      std::ifstream (tmp_file.c_str ()) >> counter_value;
-      std::ofstream (tmp_file.c_str ()) << counter_value + n_ids;
-   
-      // W5, rename counter file to an alt file (renames are atomic)
-      alt_file  = path + "/" + Uuid ().to_text_string () + ".alt";
-      boost::filesystem::rename (counter_path, alt_file);
+      std::ifstream(tmp_file.c_str()) >> counter_value;
+      std::ofstream(tmp_file.c_str()) << counter_value + n_ids;
 
-      // W6, rename temp file to counter file 
-      boost::filesystem::rename (tmp_file,counter_path);
+      // W5, rename counter file to an alt file (renames are atomic)
+      alt_file = path + "/" + Uuid().to_text_string() + ".alt";
+      boost::filesystem::rename(counter_path, alt_file);
+
+      // W6, rename temp file to counter file
+      boost::filesystem::rename(tmp_file, counter_path);
 
       // W7 delete alt file
-      boost::filesystem::remove (alt_file);
+      boost::filesystem::remove(alt_file);
 
       return counter_value;
    }
 }
 
-std::vector<uint64_t> gen_ids (std::string const &scope, int n_ids)
-{
-   auto counter_value = reserve_ids (scope,n_ids);
+std::vector<uint64_t> gen_ids(std::string const &scope, int n_ids) {
+   auto counter_value = reserve_ids(scope, n_ids);
    std::vector<uint64_t> result;
    for (int i = 0; i < n_ids; i++)
-      result.push_back (counter_value + i);
+      result.push_back(counter_value + i);
    return result;
 }
 
-static char base_62_alphabet [] = {
-   '0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f',
-   'g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v',
-   'w','x','y','z','A','B','C','D','E','F','G','H','I','J','K','L',
-   'M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z'
-};
+static char base_62_alphabet[] = {
+    '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c',
+    'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p',
+    'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', 'A', 'B', 'C',
+    'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P',
+    'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'};
 
-std::string to_base_62 (uint32_t src)
-{
-   std::string out (6, ' ');
-   for (int i=5; i >= 0;i--) {
+std::string to_base_62(uint32_t src) {
+   std::string out(6, ' ');
+   for (int i = 5; i >= 0; i--) {
       int r = src % 62;
       out[i] = base_62_alphabet[r];
       src = src / 62;
@@ -233,47 +238,46 @@ std::string to_base_62 (uint32_t src)
    return out;
 }
 
-void to_base_62 (std::vector<uint32_t> const &id_vec, std::vector<char> *out)
-{
-   for (auto src : id_vec) 
-      for (int i=5; i >= 0;i--) {
+void to_base_62(std::vector<uint32_t> const &id_vec, std::vector<char> *out) {
+   for (auto src : id_vec)
+      for (int i = 5; i >= 0; i--) {
          int r = src % 62;
-         out->push_back (base_62_alphabet[r]);
+         out->push_back(base_62_alphabet[r]);
          src = src / 62;
       }
 }
 
-uint64_t gen_id (std::string const &scope)
-{
-   std::vector<uint64_t> result = gen_ids (scope, 1);
+uint64_t gen_id(std::string const &scope) {
+   std::vector<uint64_t> result = gen_ids(scope, 1);
    return result[0];
 }
 
-std::vector<Puid> gen_puids (std::string const &scope, int n_puids)
-{
+std::vector<Puid> gen_puids(std::string const &scope, int n_puids) {
    // first generate a series of unique uint32 numbers (For this machine)
-   std::vector<uint64_t> ids = gen_ids (scope, n_puids);
+   std::vector<uint64_t> ids = gen_ids(scope, n_puids);
 
    // now add the host ip and a timer string to the generated id to
    // guarantee uniqueness across machines
-   uint32_t ip = get_host_ip_uint32 (os::default_host_ip ());
-   std::string encoded_ip (to_base_62 (ip));
-   
-   uint32_t seconds = time (NULL);
-   std::string encoded_time (to_base_62 (seconds));
-   
+   uint32_t ip = get_host_ip_uint32(os::default_host_ip());
+   std::string encoded_ip(to_base_62(ip));
+
+   uint32_t seconds = time(NULL);
+   std::string encoded_time(to_base_62(seconds));
+
    std::vector<Puid> result;
    for (uint32_t id : ids) {
       std::ostringstream s;
-      s << encoded_ip << "-" << encoded_time << "-" << to_base_62 (id);
-      result.push_back (s.str ());
+      s << encoded_ip << "-" << encoded_time << "-" << to_base_62(id);
+      result.push_back(s.str());
    }
    return result;
 }
 
-Puid gen_puid (std::string const &scope)
-{
-   return *(gen_puids (scope,1).begin ());
+Puid gen_puid(std::string const &scope) {
+   return *(gen_puids(scope, 1).begin());
 }
 
-}}
+
+} // namespace utils
+} // namespace bts
+#endif

src/lib/utils/Puid.h

@@ -1,10 +1,12 @@
 #pragma once
 
 #include "bts_export.h"
+#include "lib/utils/bts_assert.h"
 
+#include <mutex>
+#include <stdint.h>
 #include <string>
 #include <vector>
-#include <stdint.h>
 
 namespace bts {
 namespace utils {
@@ -19,21 +21,77 @@ typedef std::string Puid;
 // machine's ip, the unix time (seconds since 1970) and a machine
 // unique counter. id's are 20 bytes long and can contain [a-z], [A-Z]
 // [0-9] and -
-BTS_EXPORT std::vector<Puid> gen_puids (std::string const &scope, int n_puids);
-BTS_EXPORT Puid gen_puid (std::string const &scope);
+BTS_EXPORT std::vector<Puid> gen_puids(std::string const &scope, int n_puids);
+BTS_EXPORT Puid gen_puid(std::string const &scope);
 
 // generate machine unique ids (no guarantees of uniqueness across
 // machines)
-BTS_EXPORT  std::vector<uint64_t> gen_ids (std::string const &scope, int n_ids);
+BTS_EXPORT std::vector<uint64_t> gen_ids(std::string const &scope, int n_ids);
 // Similar to the above but instead of copying the ids to a vector
 // this simply returns the first free id (x) and skips ahead by n_ids
 // so that the next gen_id call will will be x+n_ids
-BTS_EXPORT  uint64_t reserve_ids (std::string const &scope, int n_ids);
+BTS_EXPORT uint64_t reserve_ids(std::string const &scope, int n_ids);
 // generate a single id
-BTS_EXPORT  uint64_t gen_id (std::string const &scope);
+BTS_EXPORT uint64_t gen_id(std::string const &scope);
 
 // convert a uint32 to a 6 byte human readeable string containing only
 // alpha numerical characters.
-BTS_EXPORT std::string to_base_62 (uint32_t id);
-BTS_EXPORT void to_base_62 (std::vector<uint32_t> const &ids, std::vector<char> *out);
-}}
+BTS_EXPORT std::string to_base_62(uint32_t id);
+BTS_EXPORT void to_base_62(std::vector<uint32_t> const &ids,
+                           std::vector<char> *out);
+
+#ifndef _WIN32
+class BTS_EXPORT IDGenerator {
+ public:
+   IDGenerator(std::string const &scope, uint32_t batch_size)
+       : _batch_size(batch_size), _next(0), _end(0), _scope(scope), _lock() {
+      bts_assert((int)batch_size > 0);
+      refill();
+   }
+
+   // next should only be called from a single thread
+   uint64_t next() {
+      if (_remaning() == 0) {
+         std::lock_guard _(_lock);
+         if (!_next_block.has_value())
+            _get_next_block();
+
+         bts_assert(_next_block.has_value());
+         _next = _next_block.value().first;
+         _end = _next_block.value().second;
+         _next_block.reset();
+      }
+      bts_assert(_remaning() > 0);
+      return _next++;
+   }
+
+   // refill can be called from any thread
+   void refill() {
+      std::lock_guard _(_lock);
+      if (!_next_block.has_value())
+         _get_next_block();
+   }
+
+ private:
+   uint64_t _remaning() {
+      bts_assert(_end >= _next);
+      return _end - _next;
+   }
+
+   void _get_next_block() {
+      auto block_start = reserve_ids(_scope, _batch_size);
+      auto block_end = block_start + _batch_size;
+      _next_block = std::make_pair(block_start, block_end);
+   }
+
+   const uint32_t _batch_size;
+   uint64_t _next;
+   uint64_t _end;
+   const std::string _scope;
+   std::mutex _lock;
+   std::optional<std::pair<uint64_t, uint64_t>> _next_block;
+};
+#endif
+
+} // namespace utils
+} // namespace bts

src/lib/utils/base62.h

@@ -0,0 +1,46 @@
+#pragma once
+
+#include <utility>
+#include <memory>
+#include <algorithm>
+#include <cmath>
+
+#include "lib/utils/bts_assert.h"
+
+namespace bts {
+namespace utils {
+   
+static char base_62_alphabet [] = 
+{
+   '0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f',
+   'g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v',
+   'w','x','y','z','A','B','C','D','E','F','G','H','I','J','K','L',
+   'M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z'
+};
+
+inline std::string to_base_62 (uint32_t src)
+{
+   std::string out (6, ' ');
+   for (int i=5; i >= 0; i--) 
+   {
+      int r = src % 62;
+      out[i] = base_62_alphabet[r];
+      src = src / 62;
+   }
+   return out;
+}
+
+inline uint32_t from_base_62 (const std::string& src)
+{
+   bts_assert_msg(src.size() == 6, "bad base62 hash");
+   uint32_t out = 0;
+   for (int i=5; i >= 0; i--) 
+   {
+      uint32_t idx = std::distance(&base_62_alphabet[0], std::find(&base_62_alphabet[0], &base_62_alphabet[61], src[i]));
+      out += idx * pow(62, 5-i);      
+   }
+   return out;
+}
+
+
+}}

src/lib/utils/tests/AuthTokenTests.cpp

@@ -19,7 +19,9 @@ TEST_CASE("AuthToken", "[security]") {
       // We use 1 byte for api header size
       REQUIRE(token.ByteSize() <= 240);
       api::user::TokenPayload payload;
-      REQUIRE(!api::auth::decode_token(token, &payload));
+      api::auth::clear_all_sessions();
+      REQUIRE(!api::auth::requires_auth_session(admin_user));
+      REQUIRE(api::auth::decode_token(token, &payload));
       REQUIRE(payload.id() == admin_user.id());
       REQUIRE(payload.wheel() == false);
    }
@@ -28,7 +30,8 @@ TEST_CASE("AuthToken", "[security]") {
       auto token = api::auth::generate_token(trader_user);
       // We use 1 byte for api header size
       REQUIRE(token.ByteSize() <= 240);
-      api::user::TokenPayload payload;
+      api::user::TokenPayload payload;      
+      api::auth::clear_all_sessions();
       REQUIRE(api::auth::decode_token(token, &payload));
       REQUIRE(payload.id() == trader_user.id());
       REQUIRE(payload.wheel() == false);

src/lib/utils/tests/IDGeneratorTests.cpp

@@ -0,0 +1,55 @@
+#ifndef _WIN32
+#include "../Puid.h"
+#include "catch/catch.hpp"
+
+namespace bts {
+  
+TEST_CASE("IDGenerator", "[utils]") {
+   SECTION("1") {
+      utils::IDGenerator gen("test",1);
+      auto start = gen.next();
+      for (int i = 0; i < 10;i++) {
+         auto previous_start = start;
+         start = gen.next();
+         REQUIRE( start == previous_start + 1);
+      }
+   }
+   SECTION("10") {
+      utils::IDGenerator gen("test",4);
+      auto start = gen.next();
+      for (int i = 0; i < 10;i++) {
+         auto previous_start = start;
+         start = gen.next();
+         REQUIRE( start == previous_start + 1);
+      }
+   }
+   SECTION("10_interference") {
+      utils::IDGenerator gen("test",4);
+      auto start = gen.next();
+      auto previous_start = start;
+      auto interferance = utils::reserve_ids("test", 3);
+
+      previous_start = start;
+      start = gen.next();
+      REQUIRE( start == previous_start + 1);
+
+      previous_start = start;
+      start = gen.next();
+      REQUIRE( start == previous_start + 1);
+
+      previous_start = start;
+      start = gen.next();
+      REQUIRE( start == previous_start + 1);
+
+      previous_start = start;
+      start = gen.next();
+      REQUIRE( interferance + 3 == start);
+
+      previous_start = start;
+      start = gen.next();
+      REQUIRE( start == previous_start + 1);
+   }
+}
+
+}//ns
+#endif

src/server/sparkd/cmds/FIXSim.cpp

@@ -5,7 +5,7 @@ namespace bts {
 namespace cmd {
 
 int FIXSim::execute(std::vector<std::string> const &) {
-   bts::sim::start_exchange_sim();
+   bts::sim::start_exchange_sim(bts::api::Venue::VENUE_GLBX);
    return 0;
 }
 

src/server/sparkd/cmds/RunSim.cpp

@@ -6,7 +6,7 @@ namespace cmd {
 
 int RunSim::execute(std::vector<std::string> const &)
 {
-   sim::start_exchange_sim(nullptr);
+   sim::start_exchange_sim(api::Venue::VENUE_SIM, nullptr);
    return 0;
 }
 

src/server/sparkd/exchange_sim/ExchangeSim.cpp

@@ -10,6 +10,7 @@
 #include <fstream>
 
 #include "lib/os/os.h"
+#include "lib/domain_model/Security.pb.h"
 #include "lib/config/protos/Ports.pb.h"
 #include "lib/config/protos/ExchangeSimCfg.pb.h"
 #include "lib/config/config.h"
@@ -125,20 +126,31 @@ struct MDPlayer : md::IRelayWriter
 };
 
 
-int start_exchange_sim(md::MBOWriter* mbo)
+int start_exchange_sim(bts::api::Venue venue, md::MBOWriter* mbo)
 {
    try
    {
       auto str_settings = std::string(cfg);
-      boost::replace_all(str_settings,"$BTS_DATA_PATH", os::bts_data_path());
-      boost::replace_all(str_settings,"$EXCHANGE_SIM_FIX_PORT",
-                         std::to_string(
-                            config::get<config::Ports>().exchange_sim_fix_port())); 
+      auto venue_string = api::Venue_Name(venue);
+      boost::replace_all(str_settings,"$BTS_DATA_PATH", os::bts_data_path() + "_" + venue_string);
+      std::string port_string;
+      switch(venue)
+      {
+         case api::VENUE_SIM:
+            port_string = std::to_string(config::get<config::Ports>().glbx_sim_fix_port());
+            break;
+         case api::VENUE_SIMICE:
+            port_string = std::to_string(config::get<config::Ports>().ice_sim_fix_port());
+            break;
+         default:
+            bts_assert_msg(false, "Not a valid sim venue");
+      }
+      boost::replace_all(str_settings,"$EXCHANGE_SIM_FIX_PORT", port_string); 
 
       std::istringstream str{str_settings};
       FIX::SessionSettings settings( str );
 
-      ex_sim::ExchangeSimApplication application(mbo);
+      ex_sim::ExchangeSimApplication application( venue, mbo );
       FIX::FileStoreFactory storeFactory( settings );
       FIX::FileLogFactory logFactory( settings );
       FIX::SocketAcceptor acceptor( application, storeFactory, settings, logFactory );