buffer cancel and updates because ice sucks

RTM. Documented OrderMatcher better.

doc/user/content/bts_spark/Spreader/_index.md

@@ -154,7 +154,7 @@ insert_anchor_links = "right"
 
         -  Pay Less - If enabled by checking the checkbox, the spreader will
             attempt to pay less when the quantity at the level of the calculated
-            hedge price is more than the threshold defined.  The hedge order is
+            hedge price is less than the threshold defined.  The hedge order is
             paid less by the “Payless Ticks” as defined per leg.
         -  Pay Up - If enabled by checking the checkbox, the spreader will pay
             up when the quantity at the level of the calculated hedge price is

doc/user/content/release_notes/_index.md

@@ -4,17 +4,30 @@ weight = 3
 sort_by = "weight"
 insert_anchor_links = "right"
 +++
+## 2.11.0
+### User Visible Changes
+  - Fixed fills form time filter to also apply to SOD fills
+  - Addressed an issue that could prevent the web Admin UI from displaying over slow HTTP connections
+  - Removed the option to enter manual spread fills
+  - Fixed an issue that could cause erroneous fill account attribution in the event of a crash
+### Internal Changes
+  - Added support for tag 1031. Currently defaults to Y, can it be override through config.
+  - To facilitate support, allowed the bts user to view and edit algos from all other users 
+  - Improved system tests
+  - Improved log messages in UI to include dispatcher thread name
+  - Added support for Windows core dumps in the UI
+  
 ## 2.10.0
 ### User Visible Changes
   - Fills now include information about the algo that originated the fill
   - A new, more powerful PnL form is now available
 ### API Changes
-  - Added security state information (pre open, halted, normal, closed, etc) 
+  - Added security state information (pre open, halted, normal, closed, etc)
 ### Bug Fixes
   - Brought back the "select all" check box in the mini order book
   - Fixed an issue in the market data subsystem that on rare occasions could result in a startup crash
 ### Internal Changes
-  - Fixed an issue that was stopping us from interpreting SOD times as local rather than GMT 
+  - Fixed an issue that was stopping us from interpreting SOD times as local rather than GMT
 ## 2.9.3
 ### Internal Changes
   - Fixed system test scripts
@@ -30,7 +43,7 @@ insert_anchor_links = "right"
   - Avoided shipping production configuration defaults that lead to trying to download a position file for the development sftp server.
 ### Api
   - C# API sample now uses synchronous calls.
-  
+
 ## 2.9.0
 ### User Visible Changes
   - Improved Workspace management User Interface
@@ -38,11 +51,11 @@ insert_anchor_links = "right"
   - The Spark UI Window group now has a title in the Windows task bar
   - Added sound alerts for trade events, see Launcher -> Settings -> Alerts
   - Allowed composite algos that are attached to existing orders to identify fills that were already present prior to attachment
-  - Added import facilities for GMI position files 
+  - Added import facilities for GMI position files
   - Added the ability for risk managers to enter manual fills
   - Added "raw" P&L columns, that is columns that *do not* include spread legs in their accounting
 ### Bug Fixes
-  - Fixed the settings action menu (Load/Save/Reset to Factory) 
+  - Fixed the settings action menu (Load/Save/Reset to Factory)
   - Fixed the method used to determine if a security is expired to take into account the start of day time
 ### API Changes
   - Fixed an issue in the Client SDK that could lead to occasional failures to connect to the spark server as well as excessive memory usage.
@@ -79,10 +92,10 @@ insert_anchor_links = "right"
   - Made it easier to generate simple UI's for new order types
 ### API Changes
   - Dynamic orders can now use QuantityMeasure parameter types
-  
+
 ## 2.7.2
 ### User Visible Changes
-  - Changed the login dialog to avoid leaking workspace names from other users. 
+  - Changed the login dialog to avoid leaking workspace names from other users.
   - Added new Splitter algo. From the manual: The splitter is useful
     for pro-rata markets where you want to vary your exposure
     independent of the order quantity that you want to fill.  The
@@ -90,13 +103,13 @@ insert_anchor_links = "right"
     the order quantity but it will put exposed_qty total size on the
     market. Splitter favors order size over queue position.
 ### Internal Changes
-  - Made spark more resilient to lack of connectivity to the CME web site for downloading of templates. 
-  
+  - Made spark more resilient to lack of connectivity to the CME web site for downloading of templates.
+
 ## 2.7.1
 ### User Visible Changes
   - Fixed some issues with the Cover algo, where in some circumstances
     it would erroneously cancel launched hedges on completion.
-	
+
 ## 2.7.0
 ### User Visible Changes
   - Allowed algos to process "in-flight" fills while shutting down
@@ -115,25 +128,25 @@ insert_anchor_links = "right"
 ### User Visible Changes
   - Fixed an issue in the ladder where in some cases the mini order book could not be brought up from a badge
   - Fixed a rare race condition that could lead to erroneous limit enforcement
-  - Fixed an issue with the margin limits where an error could occur if an account name with more than 12 chars was used  
+  - Fixed an issue with the margin limits where an error could occur if an account name with more than 12 chars was used
 
 ## 2.6.1
 ### User Visible Changes
   - Fixed an issue in the ladder where an error could occur when trying to change ladder settings for the first time
-  
+
 ## 2.6.0
 ### User Visible Changes
   - In the ladder made last trade more distinctive when overlayed on the volumn column
-  - In the ladder, corrected display of volume tool tip 
+  - In the ladder, corrected display of volume tool tip
   - In the ladder, changed the consolidation of last trade to consider only the price ignoring aggressor changes
   - Added an option for the Top Algo to delete itself if within N ticks from top of book
   - Address a rare issue that could lead to an error report on UI exit
   - Added margin risk limits
   - Added the "Offset" algo. Offset prices a child algo at a parameterizable distance from the top of the book
-  - Avoided switching the current account and order type on open ladders when limits are changed 
+  - Avoided switching the current account and order type on open ladders when limits are changed
 ### Internal Changes
   - Ensure that admin logins are always allowed to login, regardless of keycode status.
-  
+
 ## 2.5.0
 ### User Visible Changes
   - Added loss limit risk check
@@ -163,13 +176,13 @@ insert_anchor_links = "right"
   - Made Cover order more independent from its hedging chidren:
      - Cover continues to operate even if an hedging child is canceled
 	 - When a cover is canceled it will detach from any running hedging children, instead of canceling them
-  - Changed Lean, Join and Cover algos to hide themselves in the ladder once they have completed their work 
+  - Changed Lean, Join and Cover algos to hide themselves in the ladder once they have completed their work
   - Changed manual fills to appear in day PnL rather than in the SOD records
   - Add buy/sell coloring to fill rows in fills window
   - Streamlined the mini order book:
     - Display of the mini order book popup is quicker
     - Only exchange orders are displayed
-	- Update buttons are only displayed for orders that are under control of an algo with the update applying to the entire algo, not just the exchange order 
+	- Update buttons are only displayed for orders that are under control of an algo with the update applying to the entire algo, not just the exchange order
     - We can now select multiple rows to attach, detach, update or cancel multiple orders in a batch
     - A checkbox column has been added to make clearer which rows are selected
 ### API Changes
@@ -179,7 +192,7 @@ insert_anchor_links = "right"
   - Added system requirements to Spark documentation
   - Implemented keycodes
   - Temporarily rejected stop and stop limit orders, pending bug fixes
-  
+
 ## 2.2.0
 ### User Visible Changes
   - Made our orders easier to spot in MBO display
@@ -190,30 +203,30 @@ insert_anchor_links = "right"
   - Improved handling of orders with undiscovered securities
   - Added feedback to the ladder when the user attempts to middle click on bid/ask columns using an order that does not have parameters
   - Fixed display of algo studio algo types in order book
-  - Hid net change setting in synthetic ladders 
+  - Hid net change setting in synthetic ladders
   - Fixed a number of problems with filtering on the Fills form
   - On the Lean algo, OnToward and Away are now calculated with respect to the opposite inside market at time of the order launch
   - Iceberg Spreaders with queue holders are not erroneously cancelled after the first fill
   - Added security symbol column to fills window
   - Re-organized ladder settings dialog to make it clearer and easier to use
-  
+
 ## 2.1.4
 ### User Visible Changes
   - Fixed handling of cancel button in security picker
-  
+
 ## 2.1.3
 ### User Visible Changes
-  - Fixed a simulator bug in the handling of 0 sized orders 
+  - Fixed a simulator bug in the handling of 0 sized orders
 
 ## 2.2.1
 ### User Visible Changes
   - Fixed order book form exception when display a spreader order for whichs the legs were not present yet
-  
+
 ## 2.1.1
 ### User Visible Changes
   - Added PIQ to GLBX Iceberg orders
   - Corrected handling of stop limit orders
-  
+
 ## 2.1.0
 ### User Visible Changes
   - Added support for the GLBX Iceberg order type
@@ -224,7 +237,7 @@ insert_anchor_links = "right"
   - Added PIQ to the mini order book
   - Fixed a rare bug that could lead to server failure when requesting market data
   - Fixed a bug that could cause incorrect price displays for certain types of synthetic securities
- 
+
 ## 2.0.5
 ### User Visible Changes
   - Fixed a bug in the handling of rejected QHO orders
@@ -234,19 +247,19 @@ insert_anchor_links = "right"
 ## 2.0.4
 ### User Visible Changes
   - Fixed performance regression in order manager
-  
+
 ### Internal changes
   - Improved market data diagnostics for bad configurations
   - Changed market data default buffer size
-  
+
 ## 2.0.3
 ### Internal changes
   - Improved backwards compatibility in UI settings
 
-  
+
 ## 2.0.2
 ### User Visible Changes
-  - Middle click the ladder price column to show the orders for that price level 
+  - Middle click the ladder price column to show the orders for that price level
   - Middle click the bid or ask column in the ladder no longer sends an order
 
 ### Internal changes
@@ -274,8 +287,8 @@ insert_anchor_links = "right"
   - Improved settings manager tool output
   - Added diagnostics in the event MD memory corruption is detected
   - Added systems tests for join,lean, iceberg and spreader orders
-  - Made exchange order recovery more resilient in the presence of database loss 
-  
+  - Made exchange order recovery more resilient in the presence of database loss
+
 ## 1.6.2
 ### User Visible Changes
 ### Exchange simulator
@@ -314,21 +327,21 @@ insert_anchor_links = "right"
     - These columns can be restored by toggling their visibility within the PnL's Settings form
     - A new Traded Today column has been added to the PnL
       - Filtering on this column will allow a user to restrict the report to only show today's position
-### Internal changes 
-  - Improved organization of the shell notification architecture 
+### Internal changes
+  - Improved organization of the shell notification architecture
   - Added logging to help troubleshoot shell exit hang
 
 ## 1.5.6
 ### User Visible Changes
-  - Fixed a bug that could cause limits to be erroneously enforced 
+  - Fixed a bug that could cause limits to be erroneously enforced
   - Improved algo documentation
   - Changed how the spreader uses net change
   - Simplified trigger algo user interface
-### Internal changes 
+### Internal changes
   - Defaulted to not recording sim market data
-  - Improved documentation build 
+  - Improved documentation build
   - Added additional performance tracing
-  
+
 ## 1.5.5
   - Fixed Market View 'stale working orders' bug
 
@@ -336,11 +349,11 @@ insert_anchor_links = "right"
   - Avoided logging when warming up cache
 
 ## 1.5.3
-  - Fixed sounds startup issue 
+  - Fixed sounds startup issue
 
 ## 1.5.2
   - Added better logging to AlgoDefinitionService
-  
+
 ## 1.5.1
   - Fixed missing securities in the Fills and P&L windows.
 
@@ -361,7 +374,7 @@ insert_anchor_links = "right"
 #### Bug Fixes
   - Fixed the "Since Open" time filter to use the exchange open time rather than 8:30 AM
   - Fixed a bug in the order book where sometimes parent orders would not update their last message column.
-  
+
 #### Algos
   - Added OCO Algo
     - An OCO order creates a relationship between a child order and a 'stop' or 'other' order such that when one of these is cancelled, both cancel, or when one is filled, the other order's size is debited in the amount of the fill.  Composing this order with Cover simulates scalping functionality
@@ -369,7 +382,7 @@ insert_anchor_links = "right"
   - Improved SDK documentation and usability
   - Dynamic orders are now properly identified in the order book
 #### Documentation
-  - Information on the Spark API has been added to the Spark manual 
+  - Information on the Spark API has been added to the Spark manual
   - The documentation has been updated and reorganized to make information easier to find
 ### Internal Changes
   - Improved robustness of limit system test

src/client/spark/Admin/AdminModel.cs

@@ -1,83 +1,74 @@
 using bts;
 using bts.api.admin;
+using bts.api.core;
+using bts.utils;
 using System;
 using System.Collections.Generic;
 using System.ComponentModel.Composition;
 using System.Linq;
 using System.Reactive.Linq;
-using System.Reactive.Subjects;
-using bts.api.core;
-using mbts.ui.shell;
 
 namespace mbts.spark {
    [Export]
    class AdminModel : IDisposable {
       public readonly UserTuple user;
-      
-      readonly BehaviorSubject<IReadOnlyDictionary<uint, string>>
-         _accounts_subject = new BehaviorSubject<IReadOnlyDictionary<uint, string>>(new Dictionary<uint, string>()),
-         _users_subject = new BehaviorSubject<IReadOnlyDictionary<uint, string>>(new Dictionary<uint, string>());               
-      IReadOnlyDictionary<uint, string>
-         _account_map = new Dictionary<uint, string>(),
-         _user_map = new Dictionary<uint, string>();
-      readonly BehaviorSubject<AdminModelWrapper> _wrapper_subject = new BehaviorSubject<AdminModelWrapper>(new AdminModelWrapper());         
-      IDisposable _dispose;      
+
+      IDisposable _dispose;
+      IReadOnlyDictionary<uint, string> _account_map = DictionaryHelper.empty<uint, string>();
 
       [ImportingConstructor]
-      AdminModel(AdminService service, [Import]UserTuple user) {
+      AdminModel(AdminService service, UserTuple user) {
          this.user = user;
-         _dispose = service.subscribei_admin_data().Select(update => {
-            _wrapper_subject.OnNext(new AdminModelWrapper(update.all.TryFirst(), user.name));
-            IEnumerable<BTSUserImmutable> active_users =
-               from admin_data in update.all
-               from bts_user in admin_data.Users
-               where bts_user.Name == user.name && bts_user.HasUserData && bts_user.UserData.HasAccountPermissions
-               select bts_user;
-
-            IEnumerable<BTSAccountImmutable> active_accounts =
-               from admin_data in update.all
-               from bts_account in admin_data.Accounts
-               where bts_account.IsPresent && _wrapper_subject.Value.can_view_account(bts_account.Name)
-               join account_id in (
-                  from bts_user in active_users
-                  from account_id in bts_user.UserData.AccountPermissions.AccountPermissions.Keys
-                  select account_id
-               ) on bts_account.Id equals account_id
-               select bts_account;
-
-            var users = active_users.ToDictionary(u => u.Id, u => u.Name);
-            var accounts = active_accounts.ToDictionary(a => a.Id, a => a.Name);
-            return (users, accounts);
-         }).Subscribe(t => {
-            _account_map = t.accounts;
-            _user_map = t.users;
-            _users_subject.OnNext(_user_map = t.users);
-            _accounts_subject.OnNext(_account_map = t.accounts);
-         });
+         // accounts
+         on_accounts = service.subscribei_admin_data()
+            .Select(up => up.all.TryGetFirst(out var ad) && ad.Users.Where(u => u.Name == user.name).TryGetSingle(out var usr) && usr.HasUserData && usr.UserData.HasAccountPermissions ?
+               usr.UserData.AccountPermissions.AccountPermissions
+                  .Where(kv => (kv.Value.TradingPermission != null && kv.Value.TradingPermission.HasPermissions) || kv.Value.AdminPermission.HasPermissions)
+                  .Select(kv => ad.Accounts.Where(a => a.Id == kv.Key).TryGetSingle(out var acc) ? acc : (BTSAccountImmutable?)null)
+                  .Where(acc => acc.HasValue)
+                  .Select(acc => acc.Value)
+                  .ToDictionary(acc => acc.Id, acc => acc.Name) :
+               new Dictionary<uint, string>(0))
+            .Replay(1)
+            .RefCount();
+         // users
+         on_users = Observable.Return(DictionaryHelper.wrap_value(user.id, user.name));
+         all_user_tuples = user.permissions == UserTuple.Permissions.Admin ?
+            service.subscribei_admin_data()
+               .Select(u => u.all.TryFirst())
+               .Where(ad => ad.HasValue)
+               .Select(ad => ad.Value.Users
+                  .Where(u => user.can_access_user_data(u.Id))
+                  .Select(u => new UserTuple(u.Id, u.Name))
+                  .ToDictionary(UserTuple.key_fn))
+               .Table(UserTuple.key_fn)
+               .Cache() :
+               Observable.Return(TableUpdate<UserTuple, uint>.mk_empty(UserTuple.key_fn).with_added(new[] { user }));
+         _dispose = on_accounts.Subscribe(map => _account_map = map);
       }
 
-      public IObservable<IReadOnlyDictionary<uint, string>> on_users => _users_subject.AsObservable();
-      public IObservable<IReadOnlyDictionary<uint, string>> on_accounts => _accounts_subject.AsObservable();      
-      public IObservable<IReadOnlyList<UserTuple>> on_user_tuples => on_users.Select(d => d.Select(kv => new UserTuple(kv.Key, kv.Value)).ToList());
-      public IObservable<IReadOnlyList<AccountTuple>> on_account_tuples => on_accounts.Select(d => d.Select(kv => new AccountTuple(kv.Key, kv.Value)).ToList());
-      public IObservable<AdminModelWrapper> on_admin_data => _wrapper_subject.AsObservable();
+      public IObservable<IReadOnlyDictionary<uint, string>> on_users { get; }
+      public IObservable<IReadOnlyDictionary<uint, string>> on_accounts { get; }
+      public IObservable<TableUpdate<UserTuple, uint>> all_user_tuples { get; }
+      public IObservable<IReadOnlyList<AccountTuple>> on_account_tuples => on_accounts.Select(d => d.Select(kv => new AccountTuple(kv.Key, kv.Value)).ToListSorted(a => a.name));
 
-      public AccountTuple? get_default_account() {
-         foreach (var kv in _account_map) {
-            return new AccountTuple(kv.Key, kv.Value);
-         }
-         return null;
-      }
-      public uint? get_user_id(string user) => _id_for_name(user, _user_map);
-      public uint? get_account_id(string account) => _id_for_name(account, _account_map);
-      public void Dispose() => Disposable.dispose(ref _dispose);
+      public IObservable<Update<T, TKey>> permission_filter<T, TKey>(IObservable<Update<T, TKey>> positions, Func<T, uint> get_account_id) where TKey : IEquatable<TKey> =>
+         user.permissions == UserTuple.Permissions.Admin ?
+            positions :
+            positions.Filter(on_accounts.Select(map => new Func<T, bool>(p => map.ContainsKey(get_account_id(p)))));
 
-      static uint? _id_for_name(string name, IReadOnlyDictionary<uint, string> map) {
-         foreach (var kv in map) {
-            if (kv.Value == name)
-               return kv.Key;
-         }
-         return null;
-      }
+      public IObservable<Update<T, TKey>> permission_filter<T, TKey>(IObservable<Update<T, TKey>> positions, Func<T, string> get_account_name) where TKey : IEquatable<TKey> =>
+         user.permissions == UserTuple.Permissions.Admin ?
+            positions :
+            positions.Filter(on_accounts.Select(map => {
+               var hash = new HashSet<string>(map.Values);
+               return new Func<T, bool>(v => hash.Contains(get_account_name(v)));
+            }));
+
+      public AccountTuple? get_default_account() =>
+         _account_map.Select(kv => new AccountTuple(kv.Key, kv.Value)).TryFirst();
+
+      public void Dispose() =>
+         Disposable.dispose(ref _dispose);
    }
 }

src/client/spark/Daemons/AboutDaemon.cs

@@ -3,15 +3,10 @@ using mbts;
 using mbts.config;
 using mbts.spark;
 using mbts.ui;
-using mbts.ui.shell;
 using System;
-using System.Collections.Generic;
 using System.ComponentModel.Composition;
 using System.Diagnostics;
-using System.Linq;
 using System.Reactive.Linq;
-using System.Text;
-using System.Threading.Tasks;
 using System.Windows;
 using System.Windows.Input;
 
@@ -23,8 +18,8 @@ namespace bts.ui {
       IDisposable _dispose;
 
       [ImportingConstructor]
-      public AboutDaemon(IBus bus, [Import(LoginContainer.Exports.UserName)]string username) {
-         _user = username;
+      public AboutDaemon(IBus bus, UserTuple user) {
+         _user = user.name;
          _bus = bus;
          _dispose = bus.subscribe(SparkTags.About).OnDispatcher().Subscribe(_ => show_about());
       }
@@ -63,4 +58,4 @@ namespace bts.ui {
          Disposable.dispose(ref _dispose);
       }
    }
-}
+}

src/client/spark/Daemons/ErrorReporter.cs

@@ -7,13 +7,11 @@ using mbts.ui;
 using System;
 using System.ComponentModel.Composition;
 using System.Reactive.Linq;
-using System.Windows.Threading;
-using System.Windows;
 using System.Threading.Tasks;
-using mbts.ui.shell;
+using System.Windows;
+using System.Windows.Threading;
 
-namespace bts.spark
-{
+namespace bts.spark {
    [Export, Export(typeof(IShellDaemon))]
    public class ErrorReporter : IShellDaemon, IDisposable {
       readonly ErrorReportService _svc;
@@ -21,9 +19,9 @@ namespace bts.spark
       IDisposable _dispose;
 
       [ImportingConstructor]
-      public ErrorReporter(ErrorReportService svc, IBus bus, [Import(LoginContainer.Exports.UserName)] string username) {
+      public ErrorReporter(ErrorReportService svc, IBus bus, UserTuple user) {
          _svc = svc;
-         _username = username;
+         _username = user.name;
          _dispose = bus.subscribe(SparkTags.Support).OnDispatcher().Subscribe(msg => show_support_window(msg, "Report a problem"));
       }
 
@@ -48,6 +46,45 @@ namespace bts.spark
             ErrorMessage = message,
          });
       }
+      public async Task report_silently(string contact, string description, string userdata) {
+         var request = append_logs(new ErrorReport {
+            User = _username,
+            ErrorMessage = description,
+            UserDescription = userdata,
+            Contact = contact
+         }.immutable());
+         var reply = await _svc.send_report_async(request);
+      }
+
+      public static ErrorReportImmutable append_logs(ErrorReportImmutable src) {
+         var result = src.CloneToMutable();
+
+         var windows_log = LogUtil.read_windows_event_log();
+         if (!windows_log.IsEmpty) {
+            result.Files.Add(new FileData {
+               Name = "Application.evtx.gz",
+               Contents = windows_log
+            });
+         }
+
+         var (data,date) = LogUtil.read_latest_crash_dump();
+         if (!data.IsEmpty) {
+            result.Files.Add(new FileData {
+               Name = $"spark-{date.ToString("s")}.dmp.gz",
+               Contents = data
+            });
+         }
+
+         var spark_log = LogUtil.read_logs_compressed();
+         if (!spark_log.IsEmpty) {
+            result.Files.Add(new FileData {
+               Name = "log.gz",
+               Contents = spark_log
+            });
+         }
+         return result.immutable();
+      }
+
    }
 
    static class ErrorReporterExtensions {

src/client/spark/Daemons/MDDaemon.cs

@@ -1,5 +1,6 @@
 using bts.api;
 using bts.api.synthetic;
+using bts.md;
 using mbts;
 using mbts.co2;
 using mbts.spark;
@@ -7,15 +8,10 @@ using mbts.ui;
 using System;
 using System.Collections.Generic;
 using System.ComponentModel.Composition;
+using System.Linq;
 using System.Reactive.Linq;
 using System.Threading.Tasks;
-using System.Linq;
 using SecurityKey = System.UInt32;
-using System.Collections.Concurrent;
-using System.Threading;
-using bts.api.user;
-using bts.md;
-using mbts.ui.shell;
 
 namespace bts {
    [Export(typeof(IInitializable))]
@@ -25,17 +21,21 @@ namespace bts {
       readonly Lazy<SyntheticSecurityService> _ss;
       readonly Dictionary<SecurityKey, SecurityMD> _deleted_synthetics = new Dictionary<SecurityKey, SecurityMD>();
 
-      static uint get_naggle() => mbts.config.Configuration.get<md.RelayOptions>().naggle_ms;
-
       [ImportingConstructor]
-      public MDDaemon(Lazy<IBus> bus, Lazy<AuthTokenImmutable> token, Lazy<SyntheticSecurityService> ss, [Import(LoginContainer.Exports.UserID)]uint user_id) : 
-         base(get_naggle(), MDSourceOptions.DeferInit, user_id) {
+      public MDDaemon(Lazy<IBus> bus, Lazy<AuthTokenImmutable> token, Lazy<SyntheticSecurityService> ss, UserTuple user)
+         : base(mbts.config.Configuration.get<md.RelayOptions>().naggle_ms, MDSourceOptions.DeferInit, user) {
          _bus = bus;
          _token = token;
-         _ss = ss;         
+         _ss = ss;
       }
 
-      override public void initialize() {
+      public int priority => 55;
+
+      [Export]
+      MDConsumer consumer => _consumer;
+      IBus bus => _bus.Value;
+
+      public override void initialize() {
          _relay.set_token(_token.Value);
          _dispose.Add(
             _ss.Value.subscribei_metadata()
@@ -60,16 +60,8 @@ namespace bts {
                // Post full dictionary of syntheics here
                bus.post(SparkTags.SyntheticstrategiesTag, t);
             }));
-         _dispose.Add( this.on_securities.Subscribe( (md) => {
-            bus.post(SparkTags.SecuritiesTag, md);
-         }));            
+         _dispose.Add(on_securities.Subscribe(md => bus.post(SparkTags.SecuritiesTag, md)));
          Task.Run(run);
       }
-
-      [Export]
-      MDConsumer consumer => _consumer;
-
-      public int priority => 55;
-      IBus bus => _bus.Value;
    }
 }

src/client/spark/Daemons/OopsDaemon.cs

@@ -0,0 +1,55 @@
+using System;
+using System.ComponentModel.Composition;
+using mbts.ui;
+using mbts;
+using Newtonsoft.Json;
+using System.Threading.Tasks;
+
+namespace bts.spark {
+   [Export(typeof(IShellDaemon))]
+   class OopsDaemon : IShellDaemon, IDisposable {
+      [ImportingConstructor]
+      public OopsDaemon(ErrorReporter e) {
+         var pid = get_pid();
+         if (pid.HasValue) {
+            handle_unclean_shutdown(e, pid.Value);
+         }
+         save_pid();
+      }
+
+      public void Dispose() {
+         clear_pid();
+      }
+
+      void handle_unclean_shutdown(ErrorReporter e, int oldpid) {
+         string extra = "";
+         try {
+            var conf = mbts.config.Configuration.get<bts.config.System>();
+            extra = JsonConvert.SerializeObject(conf, Formatting.Indented);
+         }
+         catch { }
+         Task.Run(() => e.report_silently(
+               $"Host: {Environment.MachineName}",
+               $"pid: {oldpid}\n{extra}",
+               "Automated crash report")
+         );
+      }
+
+      const string _subkey = @"Software\BTS";
+      private void save_pid() {
+         using (var key = Microsoft.Win32.Registry.CurrentUser.CreateSubKey(_subkey)) {
+            key.SetValue("pid", System.Diagnostics.Process.GetCurrentProcess().Id, Microsoft.Win32.RegistryValueKind.DWord);
+         }
+      }
+      private void clear_pid() {
+         using (var key = Microsoft.Win32.Registry.CurrentUser.CreateSubKey(_subkey)) {
+            key.DeleteValue("pid",false);
+         }
+      }
+      private int? get_pid() {
+         using (var key = Microsoft.Win32.Registry.CurrentUser.CreateSubKey(_subkey)) {
+            return key.GetValue("pid") as int?;
+         }
+      }
+   }
+}

src/client/spark/Daemons/ShellLatencyDaemon.cs

@@ -3,17 +3,13 @@ using bts.metrics;
 using mbts;
 using mbts.config;
 using mbts.ui;
-using mbts.ui.shell;
 using System;
-using System.Collections.Generic;
 using System.ComponentModel.Composition;
 using System.Linq;
 using System.Net.NetworkInformation;
-using System.Text;
 using System.Threading.Tasks;
 
 namespace bts.perf {
-
    [Export, Export(typeof(IShellDaemon))]
    class ShellLatencyDaemon : IShellDaemon, IDisposable {
       readonly ShellLatencyService _service;
@@ -24,10 +20,10 @@ namespace bts.perf {
       Ping _pinger = new Ping();
 
       [ImportingConstructor]
-      public ShellLatencyDaemon(ShellLatencyService service, [Import(LoginContainer.Exports.UserName)] string user) {
+      public ShellLatencyDaemon(ShellLatencyService service, UserTuple user) {
          _service = service;
          _version = Configuration.get<bts.config.System>().version;
-         _username = user;
+         _username = user.name;
          _hostname = Environment.MachineName;
 
          _forever_ping_host(_pinger);
@@ -105,4 +101,4 @@ namespace bts.perf {
          Disposable.dispose(ref _subs);
       }
    }
-}
+}

src/client/spark/Modules/Market/MarketShellModule.cs

@@ -5,7 +5,6 @@ using mbts.co2;
 using mbts.persist;
 using mbts.spark;
 using mbts.ui;
-using mbts.ui.shell;
 using System.Collections.Generic;
 using System.ComponentModel.Composition;
 using System.IO;
@@ -13,13 +12,11 @@ using System.Threading.Tasks;
 using System.Windows.Media;
 using static mbts.ui.shell.ProtoBufSerializationHelper;
 
-namespace bts.shell
-{
+namespace bts.shell {
    [Export(typeof(IShellModule))]
-   class MarketShellModule : IShellModule
-   {
+   class MarketShellModule : IShellModule {
       [ImportingConstructor]
-      public MarketShellModule(IBus bus, MDConsumer consumer, UserSettingsHub settings_hub, OrdersModel orders_model, [Import(LoginContainer.Exports.UserName)]string user_name) {
+      public MarketShellModule(IBus bus, MDConsumer consumer, UserSettingsHub settings_hub, OrdersModel orders_model) {
          _tsf = new TimeAndSalesWindowFactory(this, bus, consumer, settings_hub);
          _mvf = new MarketViewWindowFactory(this, bus, settings_hub, orders_model);
       }

src/client/spark/Modules/Market/MarketView/MarketViewRow.cs

@@ -26,6 +26,7 @@ namespace mbts.spark {
          readonly ObservableFilterCollection<OrderRecordImmutable> _working_orders, _working_bids, _working_asks;
          readonly string _security_text, _security_key_text;
          readonly Property<double> _bid_price, _ask_price, _last_price, _day_change, _settlement, _session_high, _session_low;
+         readonly Property<bool> _last_price_from_spread;
          readonly CellProperty<int> _position;
          readonly CellProperty<uint> _last_size;
          readonly CellProperty<ulong> _volume;
@@ -54,6 +55,7 @@ namespace mbts.spark {
             _day_change = backing(nameof(day_change), double.NaN);
             _session_low = backing(nameof(session_low), double.NaN);
             _session_high = backing(nameof(session_high), double.NaN);
+            _last_price_from_spread = backing(nameof(last_price_from_spread), false);
             _settlement = backing(nameof(settlement), double.NaN);
             _volume = new CellProperty<ulong>(this, nameof(volume), 0);
             _inside_bids = new CellProperty<bool>(this, nameof(inside_bids));
@@ -109,6 +111,7 @@ namespace mbts.spark {
          public int position => _position.value;
          public double raw_day_position => _raw_day_position;
          public double last_price => _last_price.Value;
+         public bool last_price_from_spread => _last_price_from_spread.Value;
          public uint last_size => _last_size.value;
          public double bid_price => _bid_price.Value;
          public uint bid_direct_size => _bid_direct_size.Value;
@@ -167,15 +170,17 @@ namespace mbts.spark {
                _inside_bids.on_next_distinct(_bid_price.Value >= _working_best_bid.value, ref dirty);
                _inside_asks.on_next_distinct(_ask_price.Value <= _working_best_ask.value, ref dirty);
             }
-            if (md.md.get_last_trade(ref _last_ref, out NativePriceSize ps, out DateTime ts, out ulong v)) {
+            if (md.md.get_last_trade(ref _last_ref, out NativePriceSize ps, out DateTime ts, out ulong v, out bool from_spread)) {               
                _last_price.set(ps.price);
+               _last_price_from_spread.set(from_spread);
                _last_size.on_next_distinct(ps.size, ref dirty);
                _volume.on_next_distinct(v, ref dirty);
             }
-            _settlement.set(md.md.settlement_price);
-            _session_low.set(md.md.session_low);
-            _session_high.set(md.md.session_high);
-            _day_change.set(last_price - settlement);
+            dirty |= _settlement.set_new(md.md.settlement_price);
+            dirty |= _session_low.set_new(md.md.session_low);
+            dirty |= _session_high.set_new(md.md.session_high);
+            dirty |= _day_change.set_new(last_price - settlement);
+            _volume.on_next_distinct(md.md.total_volume, ref dirty);
             if (dirty)
                invalidated?.Invoke();
 

src/client/spark/Modules/Market/MarketView/MarketViewVM.cs

@@ -38,7 +38,8 @@ namespace mbts.spark {
          IMultiValueConverter price_converter = new ui.spark.converters.TickDisplayConverter();
          add_col(Columns.Security, "Security", r => r.security.ToString(), nameof(Row.security), DefaultColumnWidths.Security, security_filter, DataGridFilterKnownTemplates.Regex);
          add_col(Columns.Volume, "Volume", r => r.volume.ToString(), nameof(Row.volume), DefaultColumnWidths.Volume, volume_filter, DataGridFilterKnownTemplates.ULongMinMax);
-         add_col(Columns.Last, "Last", r => r.format_price(r.last_price), nameof(Row.last_price), DefaultColumnWidths.Price, last_price_filter, DataGridFilterKnownTemplates.DoubleMinMax);
+         add_col(Columns.LastSpd, "Spd", r => r.last_price_from_spread ? "S" : "", nameof(Row.last_price_from_spread), DefaultColumnWidths.Side, last_price_from_spread_filter, DataGridFilterKnownTemplates.Regex);
+         add_col(Columns.Last, "Last", r => r.format_price(r.last_price), nameof(Row.last_price), DefaultColumnWidths.Price, last_price_filter, DataGridFilterKnownTemplates.DoubleMinMax);         
          add_col(Columns.LastSize, merged_size_col_header, r => r.last_size.ToString(), nameof(Row.last_size), DefaultColumnWidths.Size, last_size_filter, DataGridFilterKnownTemplates.UIntMinMax, h_alignment: System.Windows.HorizontalAlignment.Right);
          add_col(Columns.Position, "Pos", r => r.position.ToString(), nameof(Row.position), DefaultColumnWidths.Size, position_filter, DataGridFilterKnownTemplates.IntMinMax, true, h_alignment: System.Windows.HorizontalAlignment.Right);
          add_col(Columns.RawDayPosition, "Raw Day Pos", r => SparkUtils.fmt_position(r.raw_day_position), nameof(Row.raw_day_position), DefaultColumnWidths.Size, raw_day_position_filter, DataGridFilterKnownTemplates.DoubleMinMax, true, h_alignment: System.Windows.HorizontalAlignment.Right);
@@ -138,6 +139,7 @@ namespace mbts.spark {
                      ask_direct_size_filter.load_state(filters.ask_direct_size);
                      ask_implied_size_filter.load_state(filters.ask_implied_size);
                      last_price_filter.load_state(filters.last_price);
+                     last_price_from_spread_filter.load_state(filters.last_price_from_spread);
                      last_size_filter.load_state(filters.last_size);
                      day_change_filter.load_state(filters.day_change);
                      settlement_filter.load_state(filters.settlement);
@@ -155,7 +157,7 @@ namespace mbts.spark {
             }), DataGridFilterHelper.hook_filters(rows_view, security_filter,
             bid_price_filter, bid_merged_size_filter, bid_direct_size_filter, bid_implied_size_filter,
             ask_price_filter, ask_merged_size_filter, ask_direct_size_filter, ask_implied_size_filter,
-            last_price_filter, last_size_filter, day_change_filter, settlement_filter,
+            last_price_filter, last_price_from_spread_filter, last_size_filter, day_change_filter, settlement_filter,
             session_low_filter, session_high_filter, position_filter, raw_day_position_filter, working_bid_volume_filter, working_ask_volume_filter, volume_filter, security_key_filter),
             settings_manager.subscribe_current().OnDispatcher().Subscribe(grid_settings_manager),
             WorkLoop.Start("MarketView", _refresh)
@@ -218,6 +220,7 @@ namespace mbts.spark {
       public IDataGridColumnPersistentFilter<Row, UIntMinMaxState> ask_direct_size_filter { get; } = new UIntMinMax<Row>(r => r.ask_direct_size, nameof(Row.ask_direct_size));
       public IDataGridColumnPersistentFilter<Row, UIntMinMaxState> ask_implied_size_filter { get; } = new UIntMinMax<Row>(r => r.ask_implied_size, nameof(Row.ask_implied_size));
       public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> last_price_filter { get; } = new DoubleMinMax<Row>(r => r.last_price, nameof(Row.last_price));
+      public IDataGridColumnPersistentFilter<Row, string> last_price_from_spread_filter { get; } = new RegexFilterVM<Row>(r => r.last_price_from_spread.ToString(), nameof(Row.last_price_from_spread));
       public IDataGridColumnPersistentFilter<Row, UIntMinMaxState> last_size_filter { get; } = new UIntMinMax<Row>(r => r.last_size, nameof(Row.last_size));
       public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> day_change_filter { get; } = new DoubleMinMax<Row>(r => r.day_change, nameof(Row.day_change));
       public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> settlement_filter { get; } = new DoubleMinMax<Row>(r => r.settlement, nameof(Row.settlement));
@@ -250,6 +253,7 @@ namespace mbts.spark {
             ask_direct_size = ask_direct_size_filter.get_state(),
             ask_implied_size = ask_implied_size_filter.get_state(),
             last_price = last_price_filter.get_state(),
+            last_price_from_spread = last_price_from_spread_filter.get_state(),
             last_size = last_size_filter.get_state(),
             day_change = day_change_filter.get_state(),
             settlement = settlement_filter.get_state(),

src/client/spark/Modules/Market/MarketView/MarketViewView.xaml.cs

@@ -12,6 +12,7 @@ namespace mbts.ui.spark {
          Security,
          Volume,
          Last,
+         LastSpd,
          LastSize,
          Position,
          RawDayPosition,

src/client/spark/Modules/Market/MarketView/MarketViewWindowFactory.cs

@@ -100,6 +100,7 @@ namespace bts.shell {
             public static GridColumnSettings security => new GridColumnSettings(col_id(Columns.Security), DefaultColumnHeaders.Security, width: DefaultColumnWidths.Security);
             public static GridColumnSettings volume => new GridColumnSettings(col_id(Columns.Volume), DefaultColumnHeaders.Volume, width: DefaultColumnWidths.Volume);
             public static GridColumnSettings last => new GridColumnSettings(col_id(Columns.Last), DefaultColumnHeaders.Last, width: DefaultColumnWidths.Price);
+            public static GridColumnSettings last_from_spread => new GridColumnSettings(col_id(Columns.LastSpd), DefaultColumnHeaders.S, width: DefaultColumnWidths.Price, is_visible: false);
             public static GridColumnSettings last_size => new GridColumnSettings(col_id(Columns.LastSize), DefaultColumnHeaders.Size, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right);
             public static GridColumnSettings position => new GridColumnSettings(col_id(Columns.Position), DefaultColumnHeaders.Pos, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right);
             public static GridColumnSettings raw_day_position => new GridColumnSettings(col_id(Columns.RawDayPosition), DefaultColumnHeaders.RawDayPos, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right, is_visible: false);
@@ -138,6 +139,7 @@ namespace bts.shell {
                   yield return security;
                   yield return volume;
                   yield return last;
+                  yield return last_from_spread;
                   yield return last_size;
                   yield return position;
                   yield return raw_day_position;
@@ -224,6 +226,8 @@ namespace bts.shell {
          [ProtoMember(25)]
          public string strip_name;
          [ProtoMember(26)]
+         public string last_price_from_spread;
+         [ProtoMember(28)]         
          public string mic_code;
       }
 

src/client/spark/Modules/Spreader/ExecutionStrategyEditor.xaml.cs

@@ -227,7 +227,7 @@ namespace mbts.ui.spark {
       }
 
       DataGridColumn _column_factory(LegSpreaderParamsTuple l) {
-         var col = new DataGridColumnInternal(l.leg.Security, ParameterCellTemplateConverter, ParameterCellEditingTemplateConverter) { Width = 75, };
+         var col = new DataGridColumnInternal(l.leg.Security, ParameterCellTemplateConverter, ParameterCellEditingTemplateConverter);
          col.HeaderTemplate = (DataTemplate)FindResource(IsNew ? RemovableColumnHeaderTemplateKey : NonRemovableColumnHeaderTemplateKey);
          DataGridEx.SetRightBorder(col, new DataGridExVerticalLine(false, true, Colors.DarkGray, 1));
          return col;

src/client/spark/Modules/Spreader/SpreaderModel.cs

@@ -1,7 +1,9 @@
-using bts.api;
+using bts;
+using bts.api;
+using bts.api.core;
 using bts.api.synthetic;
+using bts.collections;
 using bts.spark;
-using mbts.ui.shell;
 using System;
 using System.Collections.Generic;
 using System.ComponentModel.Composition;
@@ -18,11 +20,36 @@ namespace mbts.spark {
       readonly uint _user_id;
 
       [ImportingConstructor]
-      public SpreaderModel(IBus bus, SyntheticSecurityService service, ErrorReporter error, [Import(LoginContainer.Exports.UserID)]uint user_id) {
+      public SpreaderModel(IBus bus, SyntheticSecurityService service, AdminModel admin_model, ErrorReporter error, UserTuple user) {
          _bus = bus;
          _service = service;
          _error = error;
-         _user_id = user_id;
+         _user_id = user.id;
+         // synthetic tuples
+         var strategy_source = _bus.subscribe(SparkTags.SyntheticstrategiesTag)
+            .Select(map => MappedDictionary.create<uint, ExecutionStrategyImmutable, (uint security_id, ExecutionStrategyImmutable strategy)>(map, (k, v) => (k, v)))
+            .Table(tup => tup.security_id);
+
+         var synthetic_source = _bus.subscribe(SparkTags.SecuritiesTag)
+            .Select(map => new FilteredDictionary<uint, SecurityMD>(map, smd => smd.security.ContractCase == Security.ContractOneofCase.Synthetic))
+            .Table(smd => smd.security.Id);
+
+         var synthetic_user_join = synthetic_source.LeftJoin(
+            admin_model.all_user_tuples,
+            SecurityMD.key_fn,
+            UserTuple.key_fn,
+            smd => smd.security.Synthetic.UserId,
+            (smd, sid, ut, uid) => new SyntheticTuple(smd.security, ut.name, null),
+            (smd, sid, uid) => new SyntheticTuple(smd.security, null, null),
+            SyntheticTuple.key_fn);
+
+         synthetic_tuples = synthetic_user_join.LeftJoin(
+            strategy_source,
+            SyntheticTuple.key_fn,
+            tup => tup.security_id,
+            (synth_tup, id, strat_tup) => new SyntheticTuple(synth_tup.security, synth_tup.user_name, strat_tup.strategy),
+            (synth_tup, id) => synth_tup,
+            SyntheticTuple.key_fn);
       }
 
       public IObservable<IReadOnlyDictionary<uint, ExecutionStrategyImmutable>> strategy => _bus.subscribe(SparkTags.SyntheticstrategiesTag);
@@ -36,6 +63,8 @@ namespace mbts.spark {
                .LeftJoin(strategies, (security, strategy) => (new ExecutionStrategyImmutable?(strategy), security), security => (new ExecutionStrategyImmutable?(), security))
          );
 
+      public IObservable<TableUpdate<SyntheticTuple, uint>> synthetic_tuples { get; }
+
       public IObservable<ExecutionStrategyImmutable> security_strategy(uint security_key) =>
          strategy.TrySelect(s => {
             bool success = s.TryGetValue(security_key, out ExecutionStrategyImmutable result);

src/client/spark/Modules/Spreader/SyntheticManagementView.xaml.cs

@@ -7,8 +7,15 @@ namespace mbts.ui.spark {
    /// Interaction logic for SyntheticManagementView.xaml
    /// </summary>
    partial class SyntheticManagementView : UserControl {
+      public enum Columns {
+         Name,
+         Description,
+         n_legs,
+         n_working_orders,
+         SecurityKey,
+         User
+      }
 
-      public enum Columns { Name, Description, n_legs, n_working_orders, SecurityKey}
       public SyntheticManagementView() =>
          InitializeComponent();
    }

src/client/spark/Modules/Spreader/SyntheticManagementWindow.cs

@@ -10,6 +10,7 @@ using mbts.ui.utility;
 using mbts.utils;
 using System;
 using System.Collections.Generic;
+using System.Collections.Immutable;
 using System.Linq;
 using System.Reactive.Linq;
 using System.Windows;
@@ -27,7 +28,7 @@ namespace mbts.spark {
       readonly IWindowSettingsManager<GridSettings> _settings_manager;
 
 
-      public SyntheticManagementWindow(SyntheticManagementWindowFactory factory, IBus bus, OrdersModel orders_model, StartupData state, WindowSettingsManager<GridSettings> settings_manager)
+      public SyntheticManagementWindow(SyntheticManagementWindowFactory factory, IBus bus, OrdersModel orders_model, StartupData state, IWindowSettingsManager<GridSettings> settings_manager)
          : base("Synthetic Management", bus, factory, SparkCommands.ShowSyntheticManagement.icon) {
          _orders_model = orders_model;
          _state = state;
@@ -66,8 +67,8 @@ namespace mbts.spark {
             ), SparkCommand.create("Clone", Icons.CloneSynthetic, SparkTags.ShowSyntheticCloneTag).complete(
                current_row_connectable_builder.create(observe_row_strategy),
                rs => rs.HasValue, select_row_strategy_tuple
-            ), SparkCommands.ShowLadder(SparkCommands.KeyGestures.ShowLadder).complete(
-               vm_connectable_builder.create(vm => vm.current_security_md), md => md.HasValue, ConvertUtils.ValueOrDefault
+            ), SparkCommands.ShowLadderAutoLookup(SparkCommands.KeyGestures.ShowLadder).complete(
+               vm_connectable_builder.create(vm => vm.current_security_observable.Select(s => s.HasValue ? ImmutableHashSet<uint>.Empty.Add(s.Value.Id) : null)), id => id != null
             ), SparkCommands.ShowOrders.complete(() => {
                var ob_settings = bts.shell.orderbook.OrderBookStartupData.Default;
                ob_settings.filters = new bts.shell.orderbook.FilterStates {

src/client/spark/Modules/Spreader/SyntheticManagementWindowFactory.cs

@@ -1,4 +1,5 @@
 using bts.api;
+using bts.settings;
 using mbts.persist;
 using mbts.spark;
 using mbts.spark.filters;
@@ -7,6 +8,7 @@ using mbts.ui.settings;
 using ProtoBuf;
 using System;
 using System.Collections.Generic;
+using System.ComponentModel.Composition;
 using System.Reactive.Linq;
 using System.Threading.Tasks;
 using static mbts.ui.controls.DIYGrid;
@@ -33,13 +35,8 @@ namespace bts.shell {
       public async void show(SparkTags.Nothing nothing) =>
          _bus.post(SparkTags.ShowWindow, await load(new StartupData()));
 
-      IShellWindow _create(StartupData state) {
-         var settings_manager = new WindowSettingsManager<GridSettings>(_settings_hub,
-            state.grid,
-            get: us => us.synthetic_management_grid,
-            set: (us, gs) => us.with(provided_synthetic_management_gridsettings: gs));
-         return new SyntheticManagementWindow(this, _bus, _orders_model, state, settings_manager);
-      }
+      IShellWindow _create(StartupData state) =>
+         new SyntheticManagementWindow(this, _bus, _orders_model, state, _settings_hub.create_synth_settings_manager(state.grid));
 
       [ProtoContract]
       public class StartupData {
@@ -67,6 +64,7 @@ namespace bts.shell {
             public static GridColumnSettings n_legs => new GridColumnSettings(col_id(Columns.n_legs), DefaultColumnHeaders.N_Legs, width: DefaultColumnWidths.N_Legs);
             public static GridColumnSettings n_working_orders => new GridColumnSettings(col_id(Columns.n_working_orders), DefaultColumnHeaders.N_Working_Orders, width: DefaultColumnWidths.N_Working_Orders);
             public static GridColumnSettings security_key => new GridColumnSettings(col_id(Columns.SecurityKey), DefaultColumnHeaders.SecurityKey, is_visible: false, width: DefaultColumnWidths.SecurityKey);
+            public static GridColumnSettings user => new GridColumnSettings(col_id(Columns.User), DefaultColumnHeaders.User, is_visible: false, width: DefaultColumnWidths.User);
 
             public static IEnumerable<SortingCriteria> default_sorting_criterion {
                get {
@@ -81,6 +79,7 @@ namespace bts.shell {
                   yield return n_legs;
                   yield return n_working_orders;
                   yield return security_key;
+                  yield return user;
                }
             }
 
@@ -101,6 +100,15 @@ namespace bts.shell {
          public UIntMinMaxState n_working_orders;
          [ProtoMember(6)]
          public string security_key;
+         [ProtoMember(7)]
+         public uint[] user;
+      }
+
+      // migration rules
+      [Export(typeof(ISyntheticManagementMigrationRule))]
+      class AddMissingColumnsMigrationRule : ISyntheticManagementMigrationRule {
+         public bool apply(GridSettings settings, out GridSettings new_settings) =>
+            settings.add_missing_columns(StartupData.DefaultGridSettings.columns, out new_settings);
       }
    }
-}
+}

src/client/spark/Modules/Spreader/VM/SyntheticManagementDialogVM.cs

@@ -1,12 +1,15 @@
-using bts.api;
+using bts;
+using bts.api;
 using bts.collections;
 using bts.spark;
+using bts.utils;
 using mbts.persist;
 using mbts.spark.filters;
 using mbts.ui;
 using mbts.ui.settings;
 using System;
 using System.Collections.Generic;
+using System.Collections.Immutable;
 using System.Collections.ObjectModel;
 using System.Linq;
 using System.Reactive.Linq;
@@ -17,43 +20,45 @@ using static mbts.ui.controls.DIYGrid;
 using static mbts.ui.spark.SyntheticManagementView;
 
 namespace mbts.spark {
-   class SyntheticManagementDialogVM : SecurityMDGrid<SyntheticManagementDialogVM.Row, (ExecutionStrategyImmutable? strategy, SecurityMD md)> {
+   class SyntheticManagementDialogVM : SecurityGrid<SyntheticManagementDialogVM.Row, SyntheticTuple> {
       readonly IBus _bus;
       readonly OrdersModel _orders_model;
-      IWindowSettingsManager<GridSettings> _settings_manager;
-
+      readonly IWindowSettingsManager<GridSettings> _settings_manager;
 
       public SyntheticManagementDialogVM(IBus bus, OrdersModel orders_model, IWindowSettingsManager<GridSettings> settings_manager) : this(bus, orders_model, settings_manager, null, null) { }
       public SyntheticManagementDialogVM(IBus bus, OrdersModel orders_model, IWindowSettingsManager<GridSettings> settings_manager, IReadOnlyCollection<SortingCriteria> sorting, FilterStates filters)
-         : base(orders_model.spreader.strategy_tuple.Select(map => map.Values), sorting) {
+         : base(orders_model.spreader.synthetic_tuples.Select(t => t.state.Values), sorting) {
          _bus = bus;
          _orders_model = orders_model;
+         _settings_manager = settings_manager;
+         // init filters
+         user_filter = DiscreteValuesFilterVM.CreateTableUpdate<Row, UserTuple, uint>(r => r.user_tuple, UserTuple.key_fn, out var user_filter_items_observer, out var user_filter_items_view, nameof(Row.user_tuple));
+         user_filter_items_view.CustomSort = Comparer<string>.Default.wrap<string, UserTuple>(ut => ut.name).to_non_generic();
          if (filters != null) {
             create_time_filter.load_state(filters.create_time);
             name_filter.load_state(filters.name);
             description_filter.load_state(filters.description);
             n_legs_filter.load_state(filters.n_legs);
             n_working_orders_filter.load_state(filters.n_working_orders);
-			security_key_filter.load_state(filters.security_key);
+            security_key_filter.load_state(filters.security_key);
+            user_filter.load_state(filters.user);
          }
-         _settings_manager = settings_manager;
          // columns
          add_col(Columns.Name, DefaultColumnHeaders.Name, r => r.name, nameof(Row.name), name_filter, DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.Security);
          add_col(Columns.Description, DefaultColumnHeaders.Description, r => r.description, nameof(Row.description), description_filter, DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.Description);
          add_col(Columns.n_legs, DefaultColumnHeaders.N_Legs, r => r.n_legs.ToString(), nameof(Row.n_legs), n_legs_filter, DataGridFilterKnownTemplates.IntMinMax, DefaultColumnWidths.N_Legs);
          columns.Add(new HyperlinkElementColumn<Row, Columns>(
-            Columns.n_working_orders, 
-            new Binding(nameof(Row.n_working_orders)), 
+            Columns.n_working_orders,
+            new Binding(nameof(Row.n_working_orders)),
             new Binding(nameof(Row.show_order_book)),
             n_working_orders_filter,
             DataGridFilterKnownTemplates.UIntMinMax) {
                id = col_id(Columns.n_working_orders),
                Header = DefaultColumnHeaders.N_Working_Orders,
                Width = DefaultColumnWidths.N_Working_Orders
-            }
-         );
+            });
          add_col(Columns.SecurityKey, DefaultColumnHeaders.SecurityKey, r => r.security_key.ToString(), nameof(Row.security_key), security_key_filter, DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.SecurityKey);
-
+         add_col(Columns.User, DefaultColumnHeaders.User, r => r.user_name, nameof(Row.user_name), user_filter, DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.User);
          //
          void add_col(Columns col_key, string header, Func<Row, string> get_text, string prop, IDataGridColumnFilter filter, string filter_template, double width) =>
             columns.Add(new RenderTextColumn<Row, Columns>(col_key, get_text, filter, filter_template) {
@@ -63,42 +68,39 @@ namespace mbts.spark {
                Width = width
             });
          //
-         grid_settings_manager = new SimpleGridSettingsManager<Row>(columns, new Dictionary<string, GridColumnSettingsFeatureFlags> { }, row_vms);
+         grid_settings_manager = new SimpleGridSettingsManager<Row>(columns, DictionaryHelper.empty<string, GridColumnSettingsFeatureFlags>(), row_vms);
          // this will initialize the immutable properties that correspond to the col_settings_flags above (e.g. Bid/Ask font_color_override, etc.)
          grid_settings_manager.load_from(settings_manager.current_settings());
 
          _resources.AddRange(
-            DataGridFilterHelper.hook_filters(rows_view, create_time_filter, name_filter, description_filter, n_legs_filter, n_working_orders_filter, security_key_filter),
+            orders_model.admin_model.all_user_tuples.OnDispatcher().Subscribe(user_filter_items_observer),
+            DataGridFilterHelper.hook_all<SyntheticManagementDialogVM, Row>(this, rows_view),
             settings_manager.subscribe_current().OnDispatcher().Subscribe(grid_settings_manager)
          );
       }
 
-      protected override Row row_factory((ExecutionStrategyImmutable? strategy, SecurityMD md) row_model) =>
-         new Row(_bus, row_model.md, row_model.strategy, _orders_model);
-
-      protected override void on_row_removed(Row row) => row.Dispose();
-
-      public ObservableCollection<ColumnBase> columns { get; } =  new ObservableCollection<ColumnBase>();
+      public ObservableCollection<ColumnBase> columns { get; } = new ObservableCollection<ColumnBase>();
       public SimpleGridSettingsManager<Row> grid_settings_manager { get; }
 
-      public GridSettings get_grid_save_state() {
-         if (_settings_manager.instance_settings == null)
-            return null;
-         else
-            return grid_settings_manager.create();
-      }
-
-      public StartupData create() =>
-         new StartupData(rows_view.get_sorting_criteria(columns), _get_filter_states(), get_grid_save_state());
-
-
-      #region Filters
       public IDataGridColumnPersistentFilter<Row, TimeFilterTuple> create_time_filter { get; } = new TimeFilterVM<Row>(r => r.create_time, nameof(Row.create_time));
       public IDataGridColumnPersistentFilter<Row, string> name_filter { get; } = new RegexFilterVM<Row>(r => r.name, nameof(Row.name));
       public IDataGridColumnPersistentFilter<Row, string> description_filter { get; } = new RegexFilterVM<Row>(r => r.description, nameof(Row.description));
       public IDataGridColumnPersistentFilter<Row, IntMinMaxState> n_legs_filter { get; } = new IntMinMax<Row>(r => r.n_legs, nameof(Row.n_legs));
       public IDataGridColumnPersistentFilter<Row, UIntMinMaxState> n_working_orders_filter { get; } = new UIntMinMax<Row>(r => r.n_working_orders, nameof(Row.n_legs));
       public IDataGridColumnPersistentFilter<Row, string> security_key_filter { get; } = new RegexFilterVM<Row>(r => r.security_key.ToString(), nameof(Row.security_key));
+      public IDataGridColumnPersistentFilter<Row, uint[]> user_filter { get; }
+
+      public GridSettings get_grid_save_state() =>
+         _settings_manager.instance_settings == null ? null : grid_settings_manager.create();
+
+      public StartupData create() =>
+         new StartupData(rows_view.get_sorting_criteria(columns), _get_filter_states(), get_grid_save_state());
+
+      protected override Row row_factory(SyntheticTuple row_model) =>
+         new Row(_bus, row_model, _orders_model);
+
+      protected override void on_row_removed(Row row) =>
+         row.Dispose();
 
       FilterStates _get_filter_states() =>
          new FilterStates {
@@ -107,19 +109,19 @@ namespace mbts.spark {
             name = name_filter.get_state(),
             n_legs = n_legs_filter.get_state(),
             n_working_orders = n_working_orders_filter.get_state(),
-            security_key = security_key_filter.get_state()
+            security_key = security_key_filter.get_state(),
+            user = user_filter.get_state()
          };
-      #endregion
 
-      public class Row : SecurityMDRow, IEquatable<Row>, IRowVM, IActionableRowVM {
-         public readonly ExecutionStrategyImmutable? strategy;
+      public class Row : SecurityRow, IEquatable<Row>, IRowVM, IActionableRowVM {
+         public readonly SyntheticTuple synthetic_tuple;
 
          readonly IBus _bus;
          readonly Property<uint> _n_working_orders;
 
-         public Row(IBus bus, SecurityMD security_md, ExecutionStrategyImmutable? strategy, OrdersModel orders_model) : base(security_md) {
+         public Row(IBus bus, SyntheticTuple synthetic_tuple, OrdersModel orders_model) : base(synthetic_tuple.security) {
             _bus = bus;
-            this.strategy = strategy;
+            this.synthetic_tuple = synthetic_tuple;
             _n_working_orders = backing(nameof(n_working_orders),
                orders_model.on_security_orders(security_key)
                .Select(t => (uint)(t.state
@@ -133,25 +135,23 @@ namespace mbts.spark {
             show_order_book = new DelegateCommand(_show_order_book);
          }
 
+         public event Action invalidated;
+
          public DateTime create_time => DateTime.MinValue; //TODO
          public string name { get; }
+         public string user_name => synthetic_tuple.user_name;
+         public UserTuple user_tuple => new UserTuple(synthetic_tuple.security.Synthetic.UserId, user_name);
          public int n_legs { get; }
          public string description { get; }
+         public ExecutionStrategyImmutable? strategy => synthetic_tuple.strategy;
          public bool strategy_present => strategy.HasValue;
          public uint n_working_orders => _n_working_orders.Value;
          public ICommand show_ladder { get; }
          public ICommand show_order_book { get; }
          public ICommand default_command => show_ladder;
 
-
-         public ObservableCollection<RowAnimation> row_animations => new ObservableCollection<RowAnimation>();
-
-         public IReadOnlyList<IGridCellStore> cell_stores => new IGridCellStore[0];
-
-
-         public event Action invalidated;
-
-         public void raise_invalidated() => invalidated?.Invoke();
+         public void raise_invalidated() =>
+            invalidated?.Invoke();
 
          public bool Equals(Row other) =>
             other != null &&
@@ -163,10 +163,8 @@ namespace mbts.spark {
          public override int GetHashCode() =>
             security_key.GetHashCode();
 
-
-
          void _show_ladder() =>
-            _bus.post(SparkTags.ShowLadderTag, md);
+            _bus.post(SparkTags.ShowLadderAutoLookupTag, ImmutableHashSet<uint>.Empty.Add(security.Id));
 
          void _show_order_book() {
             var ob_settings = bts.shell.orderbook.OrderBookStartupData.Default;
@@ -177,4 +175,4 @@ namespace mbts.spark {
          }
       }
    }
-}
+}

src/client/spark/Modules/Trading/AlgoStudio/Model/AlgoStudioFlatDefinitionModel.cs

@@ -27,7 +27,7 @@ namespace mbts.spark {
          _error_reporter = error_reporter;
          definitions =
             service.subscribei_algo_definitions()
-            .Filter(r => r.Metadata.IsActive && r.Metadata.CreatorUserId == _admin_model.user.id)
+            .Filter(r => r.Metadata.IsActive && _admin_model.user.can_access_user_data(r.Metadata.CreatorUserId))
             .SelectKeyValue(_definition_data, d => d.key)
             .Table()
             .Cache();

src/client/spark/Modules/Trading/Fills/FillsVM.cs

@@ -118,13 +118,14 @@ namespace mbts.spark {
          add_col_expl(Columns.Side, new SideColumn<Row, Columns>(Columns.Side, r => r.side, side_filter) { id = col_id(Columns.Side) }, side_filter);
          // all others (easily identified by the enum 'Columns.Foo')
          add_col(Columns.FillTime, DefaultColumnHeaders.FillTime, r => r.fill_time.ToString("M/d h:mm:ss tt"), new TimeFilterVM<FillTuple>(r => r.fill.Timestamp.ToDateTimeOffset().LocalDateTime), DataGridFilterKnownTemplates.TimeFilter, DefaultColumnWidths.Time);
-         add_col(Columns.User, DefaultColumnHeaders.User, r => r.user, dv_filter_rx(_model.factory.subscribe_users().Select(d => d.Values), r => r.order?.user() ?? string.Empty), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.User);
-         add_col(Columns.Account, DefaultColumnHeaders.Account, r => r.account, dv_filter_rx(_model.factory.subscribe_accounts().Select(d => d.Values), r => r.account), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.Account);
+         add_col(Columns.User, DefaultColumnHeaders.User, r => r.user, dv_filter_rx(_model.factory.subscribe_users().Select(d => d.Values.Ordered()), r => r.order?.user() ?? string.Empty), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.User);
+         add_col(Columns.Account, DefaultColumnHeaders.Account, r => r.account, dv_filter_rx(_model.factory.subscribe_accounts().Select(d => d.Values.Ordered()), r => r.account), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.Account);
          add_col(Columns.Exchange, DefaultColumnHeaders.Exchange, r => r.exchange.ToString(), dv_filter(EnumHelper.get_values<Venue>().ToArray(), r => r.security.exchange_or_default()), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.Exchange);
          add_col(Columns.TimeInForce, DefaultColumnHeaders.TimeInForce, r => r.time_in_force.ToString(), dv_filter(EnumHelper.get_values<TimeInForce>().ToArray(), r => r.order?.time_in_force() ?? TimeInForce.Undefined), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.TimeInForce);
          add_col(Columns.Size, DefaultColumnHeaders.Size, r => r.size.ToString(), new DoubleMinMax<FillTuple>(r => r.fill.LastShares), DataGridFilterKnownTemplates.UIntMinMax, DefaultColumnWidths.Size);
          add_col(Columns.Price, DefaultColumnHeaders.Price, r => r.format_price(r.price), new DoubleMinMax<FillTuple>(r => r.fill.LastPrice), DataGridFilterKnownTemplates.DoubleMinMax, DefaultColumnWidths.Price);
          add_col(Columns.FillKind, DefaultColumnHeaders.FillKind, r => r.fill_kind.ToString(), dv_filter(EnumHelper.get_values<FillKind>().ToArray(), r => r.fill.FillKind), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.FillKind);
+         add_col(Columns.FillStatus, DefaultColumnHeaders.FillStatus, r => r.fill_status.ToString(), dv_filter(EnumHelper.get_values<FillStatus>().ToArray(), r => r.fill.FillStatus), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.FillKind);
          add_col(Columns.FillKey, DefaultColumnHeaders.FillKey, r => r.fill_key.ToString(), new RegexFilterVM<FillTuple>(r => r.fill.FillKey), DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.FillKey);
          add_col(Columns.OrderKey, DefaultColumnHeaders.OrderKey, r => r.order_key.ToString(), new RegexFilterVM<FillTuple>(r => r.fill.OrderKey.ToString()), DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.OrderKey);
          add_col(Columns.SecurityKey, DefaultColumnHeaders.SecurityKey, r => r.security_key.ToString(), new RegexFilterVM<FillTuple>(r => r.fill.SecurityKey.ToString()), DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.SecurityKey);
@@ -146,6 +147,7 @@ namespace mbts.spark {
          });
          _dispose = new CompositeDisposable(
             ObservableAutoWrapper.connect(col_tups, columns, ct => ct.col),
+            ObservableAutoWrapper.connect_item_hooks(col_tups, ct => ct.filter as IDisposable),
             RXCollection.connect(rows, fills_buffer.Filter(filter_src, FillTuple.key_fn), comp_src, ft => new Row(ft, model), r => r.fill_tuple),
             fills_source.OnDispatcher().Subscribe(t => {
                last_unfiltered_fills = t;
@@ -227,6 +229,7 @@ namespace mbts.spark {
             case Columns.Exchange: return wrap_default(ft => ft.security.exchange_or_default());
             case Columns.FillKey: return wrap(ft => ft.fill.FillKey);
             case Columns.FillKind: return wrap_default(ft => ft.fill.FillKind);
+            case Columns.FillStatus: return wrap_default(ft => ft.fill.FillStatus);
             case Columns.FillTime: return wrap(ft => ft.fill.Timestamp.to_local_time());
             case Columns.OrderKey: return wrap(ft => ft.fill.OrderKey);
             case Columns.OrderType: return wrap_default(ft => ft.order?.order_type() ?? OrderType.None);
@@ -293,6 +296,7 @@ namespace mbts.spark {
          public double price => _fill.LastPrice;
          public double size => _fill.LastShares;
          public FillKind fill_kind => _fill.FillKind;
+         public FillStatus fill_status => _fill.FillStatus;
          public string fill_key => _fill.FillKey;
          public string order_key => _order.Key.ToString();
 

src/client/spark/Modules/Trading/Fills/FillsView.xaml.cs

@@ -21,6 +21,7 @@ namespace mbts.ui.spark {
          Price,
          Size,
          FillKind,
+         FillStatus,
          FillKey,
          OrderKey,
          SecurityKey

src/client/spark/Modules/Trading/Fills/FillsWindowFactory.cs

@@ -37,7 +37,8 @@ namespace bts.shell {
             { nameof(FillsVM.Row.side), Columns.Side },
             { nameof(FillsVM.Row.size), Columns.Size },
             { nameof(FillsVM.Row.time_in_force), Columns.TimeInForce },
-            { nameof(FillsVM.Row.user), Columns.User }
+            { nameof(FillsVM.Row.user), Columns.User },
+            { nameof(FillsVM.Row.fill_status ), Columns.FillStatus },
          };
 
       public FillsWindowFactory(TradingShellModule module, IBus bus, UserSettingsHub settings_hub, OrdersModel orders_model)

src/client/spark/Modules/Trading/Fills/Settings/FillsFilterStates.cs

@@ -74,6 +74,10 @@ namespace bts.shell.fills {
       [ProtoMember(15)]
       [DIYGridColumnID(nameof(Columns.Symbol))]
       public string[] symbol { get; set; }
+
+      [ProtoMember(17)]
+      [DIYGridColumnID(nameof(Columns.FillStatus))]
+      public Fill.Types.FillStatus[] fill_status { get; set; }
    }
 
    static class FillsFilterStatesExtensions {
@@ -96,6 +100,7 @@ namespace bts.shell.fills {
          pb.add_discrete_values(fs.side, ft => ft.fill.Side);
          pb.add_discrete_values(fs.fill_kind, ft => ft.fill.FillKind);
          pb.add_discrete_values(fs.symbol, ft => ft.security?.Symbol ?? string.Empty);
+         pb.add_discrete_values(fs.fill_status, ft => ft.fill.FillStatus);
          // numeric
          pb.add_numeric(fs.size, ft => ft.fill.LastShares);
          pb.add_numeric(fs.price, ft => ft.fill.LastPrice);

src/client/spark/Modules/Trading/Ladder/LadderWindowFactory.cs

@@ -1,4 +1,5 @@
 using bts.api;
+using bts.collections;
 using bts.md;
 using mbts;
 using mbts.config;
@@ -50,6 +51,7 @@ namespace bts.shell {
          _mbo = mbo;
          _dispose = Disposable.Create(
             bus.subscribe(SparkTags.SecuritiesTag).Subscribe(s => _securities = s),
+            bus.subscribe(SparkTags.ShowLadderAutoLookupTag).Subscribe(_show_windows),
             bus.subscribe(SparkTags.ShowLadderTag).OnDispatcher().Subscribe(s => show_windows(new[] { s })),
             bus.subscribe(SparkTags.ShowLaddersTag).OnDispatcher().Subscribe(show_windows)
          );
@@ -273,7 +275,21 @@ namespace bts.shell {
             });
       }
 
-      void show_windows(IReadOnlyCollection<SecurityMD> securities) {
+      async void _show_windows(IImmutableSet<uint> securities) {
+         if (securities == null)
+            return;
+         var start_time = DateTime.Now;
+         do {
+            var found = securities.KeyJoin(_securities).ToList();
+            securities = securities.RemoveRange(found.Select(SecurityMD.key_fn));
+            show_windows(found);
+            if (securities.Count == 0)
+               break;
+            await Task.Delay(500);
+         } while ((DateTime.Now - start_time) < _ladder_conf.security_timeout);
+      }
+
+      void show_windows(IEnumerable<SecurityMD> securities) {
          int n_allowed = (int)_ladder_conf.max_open_ladders - _wm.list_windows().Where(w => w.window_type == _window_type).Count();
          int n_opened = 0;
          foreach (var s in securities) {

src/client/spark/Modules/Trading/ManualFills/ManualFillsVM.cs

@@ -70,7 +70,7 @@ namespace bts.shell.manual_fills {
                mfd => new ManualFillRow(mfd),
                mfr => mfr.data),
             _bus.subscribe(SparkTags.SecuritiesTag).OnDispatcher().Subscribe(m => {
-               security_map = m;
+               security_map = new FilteredDictionary<uint, SecurityMD>(m, smd => smd.security.ContractCase == Security.ContractOneofCase.Future);
                RaisePropertyChanged(nameof(security_map));
             }),
             ObservableAutoWrapper.connect_item_hooks(_persisted_rows, r => Disposable.Create(dirty_mon.join(r), selection_mon.join(r))),

src/client/spark/Modules/Trading/OrderBook/OrderBookColumns.cs

@@ -121,7 +121,7 @@ namespace mbts.spark {
                r => r.user,
                DefaultColumnWidths.User,
                DiscreteValuesFilterVM.Create<OrderSecurityRootPair, string>(
-                  model.factory.subscribe_users().Select(d => d.Values),
+                  model.factory.subscribe_users().Select(d => d.Values.Ordered()),
                   ort => ort.order.order.user(),
                   nameof(OrderBookRowVM.user)),
                DataGridFilterKnownTemplates.DiscreteValues,
@@ -133,7 +133,7 @@ namespace mbts.spark {
                r => r.account,
                DefaultColumnWidths.Account,
                DiscreteValuesFilterVM.Create<OrderSecurityRootPair, string, string[]>(
-                  model.factory.subscribe_accounts().Select(d => d.Values),
+                  model.factory.subscribe_accounts().Select(d => d.Values.Ordered()),
                   ort => ort.order.order.account(),
                   vals => vals.ToArray(),
                   vals => Observable.Return(vals),

src/client/spark/Modules/Trading/OrderBook/OrderBookVM.cs

@@ -239,7 +239,7 @@ namespace mbts.spark {
             grid_settings_manager,
             order_mod_vm.algo_vm.connect_data(Observable.Return(model)),
             order_mod_vm.algo_vm.connect_data(_selected_params_mon.subscribe_spreader_security()),
-            order_mod_vm.algo_vm,
+            order_mod_vm,
             attach_vm,
             _show_fills.observe_current(true)
             .Subscribe(is_visible => {
@@ -735,17 +735,13 @@ namespace mbts.spark {
             _size_permissions = AlgoParamPermissions.ReadOnly | AlgoParamPermissions.Write;
          SecurityMD? _security_md;
 
-         public OrderParamsVM(OrderBookVM owner, AlgoFormats init_format, AlgoEditingPermissions init_permissions)
-            : this(owner,
-                 AlgoDefinitionVM.create(
-                    owner._model,
-                     AlgoData.create_unpersisted(OrderDataHelper.Defaults.CustomOrderDefinitionImmutable(CustomOrderDefinition.CustomOneofCase.Iceberg), CustomOrderDefinition.CustomOneofCase.Iceberg.display_name()),
-                     init_format: init_format,
-                     editing_permissions: init_permissions)) { }
-
-         OrderParamsVM(OrderBookVM owner, AlgoDefinitionVM algo_vm) {
+         public OrderParamsVM(OrderBookVM owner, AlgoFormats init_format, AlgoEditingPermissions init_permissions) {
             _owner = owner;
-            this.algo_vm = algo_vm;
+            algo_vm = AlgoDefinitionVM.create(
+               owner._model,
+               AlgoData.create_unpersisted(OrderDataHelper.Defaults.CustomOrderDefinitionImmutable(CustomOrderDefinition.CustomOneofCase.Iceberg), CustomOrderDefinition.CustomOneofCase.Iceberg.display_name()),
+               init_format: init_format,
+               editing_permissions: init_permissions);
             _accounts = backing(
                nameof(selected_account),
                _model.factory.subscribe_account_tuples(),
@@ -784,7 +780,8 @@ namespace mbts.spark {
                      appended_param_rows.insert_front(new AlgoParamRowVM(root, _price_param, null, AlgoParamTemplateKeys.Price));
                   }
                }),
-               ObservableAutoWrapper.connect_item_hooks(param_rows, dirty_params_mon.join));
+               ObservableAutoWrapper.connect_item_hooks(param_rows, dirty_params_mon.join),
+               algo_vm);
             // payup methods
             void payup_bid(int ticks) => payup(bbo.bid, -ticks);
             void payup_ask(int ticks) => payup(bbo.ask, ticks);

src/client/spark/Modules/Trading/PNL/Data/PNLData.cs

@@ -501,6 +501,12 @@ namespace bts.shell.pnl {
          account_name == other.account_name &&
          sod == other.sod &&
          day == other.day &&
+         bid_price.Equals(other.bid_price) &&
+         ask_price.Equals(other.ask_price) &&
+         last_price.Equals(other.last_price) &&
+         mid_price.Equals(other.mid_price) &&
+         settlement_price.Equals(other.settlement_price) &&
+         calc_type.Equals(other.calc_type) &&
          raw_day == other.raw_day &&
          working.summary == other.working.summary;
 

src/client/spark/Modules/Trading/PNL/VM/PNLRowVM.cs

@@ -97,22 +97,22 @@ namespace bts.shell.pnl {
                if (security.HasValue && security.Value.ContractCase != Security.ContractOneofCase.Future) {
                   day_long = day_short = sod_long = sod_short = 0;
                } else {
-                  day_long = (decimal)DoubleUtil.nan_to_zero(data.day.long_pos);
-                  day_short = (decimal)DoubleUtil.nan_to_zero(data.day.short_pos);
-                  sod_long = (decimal)DoubleUtil.nan_to_zero(data.sod.long_pos);
-                  sod_short = (decimal)DoubleUtil.nan_to_zero(data.sod.short_pos);
+                  day_long = DoubleUtil.round_to_decimal(data.day.long_pos);
+                  day_short = DoubleUtil.round_to_decimal(data.day.short_pos);
+                  sod_long = DoubleUtil.round_to_decimal(data.sod.long_pos);
+                  sod_short = DoubleUtil.round_to_decimal(data.sod.short_pos);
                }
                var nd = new PNLNormalizedData(
-                  (decimal)DoubleUtil.nan_to_zero(day_pnl.value),
+                  DoubleUtil.round_to_decimal(day_pnl.value),
                   day_long,
                   day_short,
-                  (decimal)DoubleUtil.nan_to_zero(data.raw_day.long_pos),
-                  (decimal)DoubleUtil.nan_to_zero(data.raw_day.short_pos),
-                  (decimal)DoubleUtil.nan_to_zero(sod_pnl.value),
+                  DoubleUtil.round_to_decimal(data.raw_day.long_pos),
+                  DoubleUtil.round_to_decimal(data.raw_day.short_pos),
+                  DoubleUtil.round_to_decimal(sod_pnl.value),
                   sod_long,
                   sod_short,
-                  (decimal)DoubleUtil.nan_to_zero(data.working_long_pos),
-                  (decimal)DoubleUtil.nan_to_zero(data.working_short_pos));
+                  DoubleUtil.round_to_decimal(data.working_long_pos),
+                  DoubleUtil.round_to_decimal(data.working_short_pos));
                if (_normalized_data != nd) {
                   _normalized_data = nd;
                   updated?.Invoke(nd);

src/client/spark/Modules/Trading/PNL/VM/PNLVM.cs

@@ -103,7 +103,7 @@ namespace bts.shell.pnl {
          load_from(settings);
          _dispose = Disposable.Create(
             pnl_source.Operate(
-               _calculation_type.observe_current(),
+               _calculation_type.observe_current(true),
                (data, calc) => data.with(calc_type: calc)
             ).FilterWithRaw(filter_src, PNLRowData.key_fn, null)
             .OnDispatcher()

src/client/spark/Orders/OrderDataHelper.cs

@@ -67,7 +67,6 @@ namespace mbts.spark.utilities {
 
       public IObservable<IReadOnlyDictionary<uint, string>> subscribe_accounts() => _admin_model.on_accounts;
       public IObservable<IReadOnlyDictionary<uint, string>> subscribe_users() => _admin_model.on_users;
-      public IObservable<IReadOnlyList<UserTuple>> subscribe_user_tuples() => _admin_model.on_user_tuples;
       public IObservable<IReadOnlyList<AccountTuple>> subscribe_account_tuples() => _admin_model.on_account_tuples;
 
       #region Defaults

src/client/spark/Orders/OrdersModel.cs

@@ -59,23 +59,30 @@ namespace mbts.spark {
          this.md_auxiliary_model = md_auxiliary_model;
          var scheduler = ThreadPoolScheduler.Instance;
          var interval = TimeSpan.FromMilliseconds(100);
-         on_orders =
-            _service
-            .subscribei_orders()
-            .Naggle(interval, scheduler)
-            .PermissionFilter(admin_model.on_admin_data)
-            .join(sstore, order => order.security_key())
+         on_orders = admin_model
+            .permission_filter(
+               _service.subscribei_orders()
+               .Naggle(interval, scheduler),
+               o => o.account_id()
+            ).join(sstore, order => order.security_key())
             .Table()
             .Cache();
 
-         on_positions = _position_service.subscribei_position_data().Naggle(interval, scheduler).PermissionFilter(admin_model.on_admin_data).join(sstore, pos => pos.SecurityId).Table().Cache();
-         on_pnl = _position_service.subscribei_pnl_data()
-            .Naggle(interval, scheduler)
+         on_positions = admin_model
+            .permission_filter(
+               _position_service.subscribei_position_data().Naggle(interval, scheduler),
+               p => p.AccountId)
+            .join(sstore, pos => pos.SecurityId)
+            .Table()
+            .Cache();
+         on_pnl = admin_model
+            .permission_filter(
+               _position_service.subscribei_pnl_data().Naggle(interval, scheduler),
+               p => p.Pos.AccountId)
             .join(sstore, p => p.Pos.SecurityId)
             .TableCheckedAutoCorrect(EqualityComparer<(SecurityImmutable? sec, PNLUnitImmutable data)>.Default)
             .Where(t => t.has_changes())
             .Cache();
-         on_account_pnl = _position_service.subscribei_pnl_account_data().Naggle(interval, scheduler).Table().Cache();
          var working_orders = on_orders.Filter(tup => tup.order.is_active());
          this.working_orders =
             on_orders.Scan(
@@ -174,10 +181,11 @@ namespace mbts.spark {
                .Where(t => t.has_changes())
                .Cache();
          // fill tuples
-         var on_fills =
-            _service.subscribei_fills()
-            .Naggle(interval, scheduler)
-            .PermissionFilter(admin_model.on_admin_data)
+         var on_fills = admin_model
+            .permission_filter(
+               _service.subscribei_fills()
+               .Naggle(interval, scheduler),
+               f => f.Account)
             .join(sstore, f => f.SecurityKey)
             .Table().Cache();
          var fill_order_tuples_join = on_fills.LeftJoin(
@@ -265,7 +273,6 @@ namespace mbts.spark {
       public IObservable<IReadOnlyDictionary<uint, OrderSecurity>> working_orders { get; }
       public IObservable<TableUpdate<(SecurityImmutable?, PNLUnitImmutable), int>> on_pnl { get; }
       public IObservable<TableUpdate<PNLRowData, PNLKey>> security_account_pnl { get; }
-      public IObservable<TableUpdate<AccountPNLImmutable, uint>> on_account_pnl { get; }
       public IObservable<TableUpdate<(SecurityImmutable?, PositionUnitImmutable), int>> on_positions { get; }
       public IObservable<TableUpdate<(AlgoData algo, AlgoDefinitionMetadata? meta), AlgoKey>> studio_algos_with_meta { get; }
       public IObservable<TableUpdate<(SecurityImmutable?, ManualFillImmutable), int>> on_manual_fills_securities { get; }

src/client/spark/core/Bus/SparkCommands.cs

@@ -8,6 +8,7 @@ using mbts.spark;
 using mbts.ui;
 using System;
 using System.Collections.Generic;
+using System.Collections.Immutable;
 using System.Windows.Input;
 
 namespace bts.spark {
@@ -41,6 +42,9 @@ namespace bts.spark {
          SparkCommand.create("P&L (2.0)", Icons.Accounting, SparkTags.ShowPNLTag);
 
       // Ladder
+      public static PartialCommand<IImmutableSet<uint>> ShowLadderAutoLookup(KeyGesture key_gesture = null) =>
+         SparkCommand.create("Ladder", Icons.Ladder, SparkTags.ShowLadderAutoLookupTag, key_gesture);
+
       public static PartialCommand<SecurityMD> ShowLadder(KeyGesture key_gesture = null) =>
          SparkCommand.create("Ladder", Icons.Ladder, SparkTags.ShowLadderTag, key_gesture);
 

src/client/spark/core/Bus/SparkTags.cs

@@ -7,6 +7,7 @@ using bts.spark;
 using mbts.ui;
 using System;
 using System.Collections.Generic;
+using System.Collections.Immutable;
 using System.Threading.Tasks;
 
 namespace mbts.spark {
@@ -55,6 +56,8 @@ namespace mbts.spark {
       // The owners of the various SparkWindows may remove dependencies on the former at their leasure.
       // For now, both tags go through the same logic in the LadderWindowFactory anyways.
       public static ITag<SecurityMD> ShowLadderTag { get; } = new SparkTag<SecurityMD>("Show Ladder");
+      // TODO: get rid of the SecurityMD ShowLadderTag.  This has created burdensome dependency issues.
+      public static ITag<IImmutableSet<uint>> ShowLadderAutoLookupTag { get; } = new SparkTag<IImmutableSet<uint>>("Show Ladder Auto Lookup");
       public static ITag<IReadOnlyList<SecurityMD>> ShowLaddersTag { get; } = new SparkTag<IReadOnlyList<SecurityMD>>(nameof(ShowLaddersTag));
       public static ITag<IObservable<SecurityMD>> ShowLinkedLadder { get; } = new SparkTag<IObservable<SecurityMD>>("Linked Ladder");
       public static ITag<SecurityCriteria> ShowMarketViewTag { get; } = new SparkTag<SecurityCriteria>("Show MarketView");

src/client/spark/core/ShellInterface/UISettings.cs

@@ -11,6 +11,7 @@ using bts.api.persistence;
 using System.Text.RegularExpressions;
 using System.Reactive.Linq;
 using bts.spark;
+using bts;
 
 namespace mbts.persist
 {
@@ -146,16 +147,18 @@ namespace mbts.persist
    }
 
    [Export(typeof(IUserPersist))]
-   public class UserUISettings : UISettingsBase
-   {
+   public class UserUISettings : UISettingsBase {
       string _user;
 
-      [ImportingConstructor]
-      public UserUISettings([Import("userid")]uint user, [Import("settings")]PersistenceService persist, ErrorReporter err)
+      public UserUISettings(uint user_id, [Import("settings")]PersistenceService persist, ErrorReporter err)
          : base(persist,err) {
-         _user = $"User {user}";
+         _user = $"User {user_id}";
       }
 
+      [ImportingConstructor]
+      UserUISettings(UserTuple user, [Import("settings")]PersistenceService persist, ErrorReporter err)
+         : this(user.id, persist, err) { }
+
       protected override string user() { return _user; }
    }
 }

src/client/spark/core/ShellInterface/UserSettings/ISettingsMigrationRule.cs

@@ -17,6 +17,7 @@ namespace bts.settings {
    interface IMarketViewGridSettingsMigrationRule : ISettingsMigrationRule<GridSettings> { }
    interface IPAndLGridSettingsMigrationRule : ISettingsMigrationRule<GridSettings> { }
    interface IPNLSettingsMigrationRule : ISettingsMigrationRule<PNLSettings> { }
+   interface ISyntheticManagementMigrationRule : ISettingsMigrationRule<GridSettings> { }
 
    static class SettingsMigrationHelper {
       public static bool apply<T>(T settings, IEnumerable<ISettingsMigrationRule<T>> migration_rules, out T new_settings) {

src/client/spark/core/ShellInterface/UserSettings/UserSettingsHub.cs

@@ -25,7 +25,8 @@ namespace mbts.persist {
       static readonly IImmutableProperty<UserSettings, GridSettings>
          _fills_propi = ImmutableProperty.create<UserSettings, GridSettings>(us => us.fills_grid, (us, gs) => us.with(provided_fills_gridsettings: gs)),
          _mview_propi = ImmutableProperty.create<UserSettings, GridSettings>(us => us.market_view_grid, (us, gs) => us.with(provided_market_view_gridsettings: gs)),
-         _pandl_propi = ImmutableProperty.create<UserSettings, GridSettings>(us => us.pnl_grid, (us, gs) => us.with(provided_pnl_gridsettings: gs));
+         _pandl_propi = ImmutableProperty.create<UserSettings, GridSettings>(us => us.pnl_grid, (us, gs) => us.with(provided_pnl_gridsettings: gs)),
+         _synth_propi = ImmutableProperty.create<UserSettings, GridSettings>(us => us.synthetic_management_grid, (us, gs) => us.with(provided_synthetic_management_gridsettings: gs));
 
       static readonly IImmutableProperty<UserSettings, PNLSettings>
          _pnl_propi = ImmutableProperty.create<UserSettings, PNLSettings>(us => us.pnl, (us, pnl) => us.with(pnl: pnl));
@@ -36,6 +37,7 @@ namespace mbts.persist {
       readonly IEnumerable<IMarketViewGridSettingsMigrationRule> _mview_migration_rules;
       readonly IEnumerable<IPAndLGridSettingsMigrationRule> _pandl_migration_rules;
       readonly IEnumerable<IPNLSettingsMigrationRule> _pnl_migration_rules;
+      readonly IEnumerable<ISyntheticManagementMigrationRule> _synth_migration_rules;
       readonly IUserPersist _persist;
 
       UserSettings _default_settings;
@@ -46,13 +48,15 @@ namespace mbts.persist {
          [ImportMany]IEnumerable<IFillsGridSettingsMigrationRule> fills_migration_rules,
          [ImportMany]IEnumerable<IMarketViewGridSettingsMigrationRule> mview_migration_rules,
          [ImportMany]IEnumerable<IPAndLGridSettingsMigrationRule> pandl_migration_rules,
-         [ImportMany]IEnumerable<IPNLSettingsMigrationRule> pnl_migration_rules) {
+         [ImportMany]IEnumerable<IPNLSettingsMigrationRule> pnl_migration_rules,
+         [ImportMany]IEnumerable<ISyntheticManagementMigrationRule> synth_migration_rules) {
          //
          _persist = persist;
          _fills_migration_rules = fills_migration_rules ?? Enumerable.Empty<IFillsGridSettingsMigrationRule>();
          _mview_migration_rules = mview_migration_rules ?? Enumerable.Empty<IMarketViewGridSettingsMigrationRule>();
          _pandl_migration_rules = pandl_migration_rules ?? Enumerable.Empty<IPAndLGridSettingsMigrationRule>();
          _pnl_migration_rules = pnl_migration_rules ?? Enumerable.Empty<IPNLSettingsMigrationRule>();
+         _synth_migration_rules = synth_migration_rules ?? Enumerable.Empty<ISyntheticManagementMigrationRule>();
          init();
          async void init() {
             UserSettings s;
@@ -62,7 +66,9 @@ namespace mbts.persist {
                   var wrapped_rules = new[] {
                      _fills_migration_rules.merge().wrap(_fills_propi),
                      _mview_migration_rules.merge().wrap(_mview_propi),
-                     _pandl_migration_rules.merge().wrap(_pandl_propi)
+                     _pandl_migration_rules.merge().wrap(_pandl_propi),
+                     _pnl_migration_rules.merge().wrap(_pnl_propi),
+                     _synth_migration_rules.merge().wrap(_synth_propi)
                   };
                   SettingsMigrationHelper.apply_quiet(ref s, wrapped_rules);
                   // TODO: move ladder sanitization logic into ISettingsMigrationRule
@@ -136,6 +142,12 @@ namespace mbts.persist {
             _pnl_migration_rules,
             _pnl_propi);
 
+      public IWindowSettingsManager<GridSettings> create_synth_settings_manager(GridSettings instance_settings) =>
+         _create_settings_manager(
+            instance_settings,
+            _synth_migration_rules,
+            _synth_propi);
+
       public async Task save_default(UserSettings value) {
          // notify observers before persisting so the changes are reflected immediately in the ui
          lock (_default_settings_lock) {

src/client/spark/core/Utility/ObjectSnapshot.cs

@@ -15,12 +15,8 @@ namespace bts.utils {
    }
 
    static class ObjectSnapshot {
-      public static IEnumerable<Func<TSource, TValue>> property_delegates_of_type<TSource, TValue>() where TSource : class where TValue : class {
-         var tval = typeof(TValue);
-         return typeof(TSource).GetProperties(BindingFlags.Public | BindingFlags.Instance)
-            .Where(p => tval.IsAssignableFrom(p.PropertyType))
-            .Select(_prop_getter<TSource, TValue>);
-      }
+      public static IReadOnlyList<Func<TSource, TValue>> property_delegates_of_type<TSource, TValue>() =>
+         PropertyGetterStore<TSource, TValue>.value;
 
       public static IObservable<TableUpdate<(string name, object current, object original), string>> property_table<T>(T instance, params string[] prop_names) where T : INotifyPropertyChanged =>
          _property_table(instance, Array.ConvertAll(prop_names, PropertyInfoStore<T>.type.GetProperty));
@@ -162,6 +158,18 @@ namespace bts.utils {
          }
       }
 
+      static class PropertyGetterStore<TSource, TValue> {
+         public static readonly IReadOnlyList<Func<TSource, TValue>> value;
+
+         static PropertyGetterStore() {
+            var tval = typeof(TValue);
+            value = typeof(TSource).GetProperties(BindingFlags.Public | BindingFlags.Instance)
+               .Where(p => tval.IsAssignableFrom(p.PropertyType))
+               .Select(_prop_getter<TSource, TValue>)
+               .ToReadOnlyList();
+         }
+      }
+
       class PropNode {
          public readonly PropertyInfo prop;
 

src/client/spark/core/data/SyntheticTuple.cs

@@ -0,0 +1,46 @@
+using bts.api;
+using bts.spark;
+using bts.utils;
+using System;
+
+namespace bts {
+   struct SyntheticTuple : IEquatable<SyntheticTuple>, IComparable<SyntheticTuple>, IComparable {
+      public static readonly Func<SyntheticTuple, uint> key_fn = a => a.security.Id;
+      public static readonly Func<SyntheticTuple?, uint> nullable_key_fn = a => a.HasValue ? a.Value.security.Id : 0u;
+
+      public readonly SecurityImmutable security;
+      public readonly string user_name;
+      public readonly ExecutionStrategyImmutable? strategy;
+      public readonly int? num_working_orders;
+
+      public SyntheticTuple(SecurityImmutable security, string user_name, ExecutionStrategyImmutable? strategy, int? num_working_orders = null) {
+         this.security = security;
+         this.user_name = user_name;
+         this.strategy = strategy;
+         this.num_working_orders = num_working_orders;
+      }
+
+      public int CompareTo(SyntheticTuple other) =>
+         SecurityHelper.Comparer.Compare(security, other.security);
+
+      int IComparable.CompareTo(object obj) =>
+         obj is SyntheticTuple other ? CompareTo(other) : 1;
+
+      public bool Equals(SyntheticTuple other) =>
+         security == other.security &&
+         user_name == other.user_name &&
+         EqualityComparer.nullable_equals(strategy, other.strategy);
+
+      public override bool Equals(object obj) =>
+         EqualityComparer.equals_obj(this, obj);
+
+      public override int GetHashCode() =>
+         security.Id.GetHashCode();
+
+      public override string ToString() =>
+         user_name == null ? security.ToString() : $"{user_name}:{security}";
+
+      public static bool operator ==(SyntheticTuple a, SyntheticTuple b) => a.Equals(b);
+      public static bool operator !=(SyntheticTuple a, SyntheticTuple b) => !a.Equals(b);
+   }
+}

src/client/spark/view/Controls/DIYGrid/Columns/ColumnBase.cs

@@ -114,6 +114,7 @@ namespace mbts.ui.controls {
             Aggressor = "Aggressor",
             Volume = "Volume",
             Last = "Last",
+            S = "S",
             Pos = "Pos",
             Working = "Working",
             Bid = "Bid",
@@ -124,6 +125,7 @@ namespace mbts.ui.controls {
             SessionLow = "Low",
             SessionHigh = "High",
             FillKind = "Fill Kind",
+            FillStatus = "Fill Status",
             Name = "Name",
             Description = "Description",
             N_Legs = "# Legs",

src/client/spark/view/Controls/DIYGrid/DoubleUtil.cs

@@ -1,10 +1,9 @@
 using System;
 using System.Linq;
 
-namespace mbts.ui.controls
-{
-   internal class DoubleUtil
-   {
+namespace mbts.ui.controls {
+   static class DoubleUtil {
+      public const int API_DECIMAL_PRECISION = 9;
       /* smallest such that 1.0+DBL_EPSILON != 1.0 */
       private const double EPSILON  =   2.2204460492503131e-016;
       private const double BIG_EPSILON = EPSILON * 10;
@@ -36,6 +35,9 @@ namespace mbts.ui.controls
             (double.IsNaN(right) ? double.NaN : -right) :
             (double.IsNaN(right) ? left : left - right);
 
+      public static decimal round_to_decimal(double value, int decimals = API_DECIMAL_PRECISION) =>
+         double.IsNaN(value) ? 0 : (decimal)Math.Round(value, decimals, MidpointRounding.AwayFromZero);
+
       internal static bool AreClose(double v1, double v2) {
          if (v1 == v2) return true;
          double delta = Math.Abs(v1 - v2);

src/client/spark/view/Controls/DataGridEx/Filters/DataGridFilterHelper.cs

@@ -1,11 +1,22 @@
-using System;
+using bts.utils;
+using System;
 using System.Collections.Generic;
 using System.ComponentModel;
 using System.Linq;
 
 namespace mbts.spark.filters {
    static class DataGridFilterHelper {
-      public static IDisposable hook_filters<TRow>(ICollectionView rows, params IDataGridColumnFilter<TRow>[] filters) {
+      public static IDisposable hook_all<TSource, TRow>(TSource source, ICollectionView rows) =>
+         hook_filters(rows,
+            ObjectSnapshot.property_delegates_of_type<TSource, IDataGridColumnFilter<TRow>>()
+            .Select(f => f(source))
+            .ExceptNull()
+            .ToList());
+
+      public static IDisposable hook_filters<TRow>(ICollectionView rows, params IDataGridColumnFilter<TRow>[] filters) =>
+         hook_filters(rows, filters.AsEnumerable());
+
+      public static IDisposable hook_filters<TRow>(ICollectionView rows, IEnumerable<IDataGridColumnFilter<TRow>> filters) {
          var icvls = (ICollectionViewLiveShaping)rows;
          icvls.IsLiveFiltering = true;
          return _hook_filters(set_filter, filters);
@@ -22,7 +33,7 @@ namespace mbts.spark.filters {
          }
       }
 
-      static IDisposable _hook_filters<TRow>(Action set_filter, params IDataGridColumnFilter<TRow>[] filters) {
+      static IDisposable _hook_filters<TRow>(Action set_filter, IEnumerable<IDataGridColumnFilter<TRow>> filters) {
          foreach (var f in filters)
             f.filter_changed += set_filter;
          set_filter();
@@ -34,7 +45,7 @@ namespace mbts.spark.filters {
          });
       }
 
-      static List<IDataGridColumnFilter<TRow>> _get_active_filters<TRow>(IDataGridColumnFilter<TRow>[] filters) =>
+      static List<IDataGridColumnFilter<TRow>> _get_active_filters<TRow>(IEnumerable<IDataGridColumnFilter<TRow>> filters) =>
          filters.Where(f => f.active.HasValue && f.active.Value).ToList();
    }
 

src/client/spark/view/Controls/DataGridEx/Filters/VM/DiscreteValuesFilterVM.cs

@@ -68,10 +68,11 @@ namespace mbts.spark.filters {
          Func<TRow, TValue> factory,
          Func<TValue, TKey> key,
          out IObserver<TableUpdate<TValue, TKey>> items_observer,
-         out ListCollectionView<TValue> items_view)
+         out ListCollectionView<TValue> items_view,
+         string prop_name = null)
          where TKey : struct, IEquatable<TKey> =>
          //
-         CreateTableUpdate(factory, key, keys => keys, keys => keys.ToArray(), out items_observer, out items_view);
+         CreateTableUpdate(factory, key, keys => keys, keys => keys.ToArray(), out items_observer, out items_view, prop_name);
 
       public static IDataGridColumnPersistentFilter<TRow, TState> CreateTableUpdate<TRow, TValue, TKey, TState>(
          Func<TRow, TValue> factory,
@@ -79,10 +80,11 @@ namespace mbts.spark.filters {
          Func<TState, IEnumerable<TKey>> load_state,
          Func<IEnumerable<TKey>, TState> get_state,
          out IObserver<TableUpdate<TValue, TKey>> items_observer,
-         out ListCollectionView<TValue> items_view)
+         out ListCollectionView<TValue> items_view,
+         string prop_name = null)
          where TKey : struct, IEquatable<TKey> {
          //
-         var filter = new TableUpdateItemsFilter<TRow, TValue, TKey, TState>(factory, key, load_state, get_state);
+         var filter = new TableUpdateItemsFilter<TRow, TValue, TKey, TState>(factory, key, load_state, get_state, prop_name);
          items_observer = filter;
          items_view = filter.items_view;
          return filter;
@@ -291,8 +293,8 @@ namespace mbts.spark.filters {
          readonly HashSet<TKey> _selected_keys = new HashSet<TKey>(), _desired_keys = new HashSet<TKey>();
          ListCollectionView<TValue> _items_view;
 
-         public TableUpdateItemsFilter(Func<TRow, TValue> factory, Func<TValue, TKey> key, Func<TState, IEnumerable<TKey>> load_state, Func<IEnumerable<TKey>, TState> get_state)
-            : base(factory, null) {
+         public TableUpdateItemsFilter(Func<TRow, TValue> factory, Func<TValue, TKey> key, Func<TState, IEnumerable<TKey>> load_state, Func<IEnumerable<TKey>, TState> get_state, string prop_name)
+            : base(factory, prop_name) {
             _factory = factory;
             _key = key;
             _load_state = load_state;

src/client/spark/view/Controls/DataGridEx/Filters/Views/TimeFilter.cs

@@ -24,6 +24,7 @@ namespace mbts.ui {
       public TimeTextTuple end => _end;
       public bool is_active =>
          _mode == TimeFilterMode.Today ||
+         _mode == TimeFilterMode.CMEOpen ||
          (_mode == TimeFilterMode.Period && (_begin.result.HasValue || _end.result.HasValue));
 
 

src/client/spark/view/Controls/Dialogs/SupportRequestWindow.xaml.cs

@@ -1,10 +1,4 @@
-using bts.support;
-using Google.Protobuf;
-using mbts;
-using mbts.ui;
-using mbts.ui.shell;
-using mbts.ui.utility;
-using System;
+using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Text;
@@ -18,6 +12,13 @@ using System.Windows.Media;
 using System.Windows.Media.Imaging;
 using System.Windows.Navigation;
 using System.Windows.Shapes;
+using bts.support;
+using Google.Protobuf;
+using mbts;
+using mbts.ui;
+using mbts.ui.controls;
+using mbts.ui.shell;
+using mbts.ui.utility;
 
 namespace bts.view.controls {
    partial class SupportRequestWindow : Window {
@@ -50,28 +51,40 @@ namespace bts.view.controls {
          _base = report_base;
          send_command = new DelegateCommand(send_report, _ => !string.IsNullOrWhiteSpace(_text.Value));
          _text = backing(nameof(user_text), "");
-         _text.Changed += (_,__) => send_command.TryRaiseCanExecuteChanged();
+         _text.Changed += (_, __) => send_command.TryRaiseCanExecuteChanged();
          _contact = backing(nameof(user_contact), "");
-         _contact.Changed += (_,__) => send_command.TryRaiseCanExecuteChanged();
+         _contact.Changed += (_, __) => send_command.TryRaiseCanExecuteChanged();
       }
 
       async void send_report(object _) {
-         ByteString screenshot = null;
-         if (include_screenshot) {
-            var stream = ScreenCapture.capture_encoded("jpeg");
-            screenshot = await ByteString.FromStreamAsync(stream);
-         }
-         var reply = await _service.send_report_async(_base.with(
-            user_description: _text.Value,
-            contact: user_contact,
-            log: include_logs ? LogUtil.read_logs() : "",
-            image: screenshot
-            ));
-         if (reply.success) {
-            BTSMessageBox.Show("Your message has been received.", "Thank You", BTSMessageBoxButtons.OK, MessageBoxImage.Information, is_modal: false, top_most:true);
-         }
-         else {
-            BTSMessageBox.Show($"There was an error submitting your message. '{reply.error_message}'", "Error", BTSMessageBoxButtons.OK, MessageBoxImage.Error, is_modal: false, top_most: true);
+         using (var d = new BusyDialog { Text = "Sending your error report...", Topmost = true }) {
+            d.Show();
+            ByteString screenshot = null;
+            if (include_screenshot) {
+               var stream = ScreenCapture.capture_encoded("jpeg");
+               screenshot = await ByteString.FromStreamAsync(stream);
+            }
+            var request = await Task.Run(() => {
+               var result = _base.with(
+                  user_description: _text.Value,
+                  contact: user_contact,
+                  image: screenshot);
+               if (include_logs) {
+                  result = spark.ErrorReporter.append_logs(result);
+               }
+               return result;
+            });
+
+            var reply = await _service.send_report_async(request);
+            if (reply.success) {
+               Log.Ui.info("Support request message has been received.");
+               BTSMessageBox.Show("Your message has been received.", "Thank You", BTSMessageBoxButtons.OK, MessageBoxImage.Information, is_modal: false, top_most: true);
+            }
+            else {
+               var errmsg = $"There was an error submitting your message. '{reply.error_message}'";
+               Log.Ui.error(errmsg);
+               BTSMessageBox.Show(errmsg, "Error", BTSMessageBoxButtons.OK, MessageBoxImage.Error, is_modal: false, top_most: true);
+            }
          }
       }
 

src/client/spark/view/Controls/TickUpDown/TickUpDown.xaml

@@ -457,15 +457,15 @@
               Name="PART_TextBox"
               u:TextBoxBehavior.EnableArrowKeyTraverse="True"
               BorderThickness="0">
-              <TextBox.Style>
+              <TextBox.Style>
                 <Style TargetType="{x:Type TextBox}">
                   <Style.Triggers>
                     <Trigger Property="IsMouseOver" Value="True">
                       <Setter Property="FocusManager.FocusedElement" Value="{Binding RelativeSource={RelativeSource Self}}"></Setter>
                     </Trigger>
                   </Style.Triggers>
-                </Style>
-              </TextBox.Style>
+                </Style>
+              </TextBox.Style>
             </TextBox>
           </c:TickUpDown>
         </ControlTemplate>

src/client/spark/view/Controls/Time/DateTimePicker.xaml

@@ -93,7 +93,6 @@
                 u:TextBoxBehavior.EnableArrowKeyTraverse="True"
                 u:Behavior.DisablePaste="True"
                 Height="20"
-                BorderThickness="0"
               >
                 <TextBox.InputBindings>
                   <KeyBinding Key="Up" Command="{x:Static c:DateTimePicker.TickTimeComponentCommand}" CommandParameter="{u:Int 1}"/>

src/client/spark/view/Shell/Login/LoginModel.cs

@@ -15,8 +15,7 @@ namespace mbts.ui.shell {
    // The LoginContainer participates in the startup Import/Export dependency system
    // Since the login form should not depend on *anything* it will be shown before this is created
    // Once this is created it access the global LoginModel and retrieves the username
-   internal class LoginContainer
-   {
+   class LoginContainer {
       [ImportingConstructor]
       public LoginContainer(ServiceContext context) {
          var model = LoginModel.instance;
@@ -28,10 +27,8 @@ namespace mbts.ui.shell {
          model.Dispose();
       }
 
-      [Export(Exports.UserName)]
       public string UserName { get; private set; }
 
-      [Export(Exports.UserID)]
       public uint UserId { get; private set; }
 
       [Export]
@@ -42,13 +39,6 @@ namespace mbts.ui.shell {
 
       [Export("workspace")]
       public string Workspace { get; private set; }
-
-      public static class Exports {
-         public const string
-            UserName = "username",
-            UserID = "userid",
-            UserTuple = "usertuple";
-      }
    }
 
    public struct User {

src/client/spark/view/Shell/ShellModel.cs

@@ -114,10 +114,16 @@ namespace mbts.ui
       }
 
       public void Dispose() {
-         foreach (IShellModule module in _modules.OrderBy(m => m.priority).Reverse()) {
+         foreach (var module in _modules.OrderBy(m => m.priority).Reverse()) {
             module.Dispose();
          }
          _modules.Clear();
+         foreach (var daemon in _services) {
+            if (daemon is IDisposable d) {
+               d.Dispose();
+            }
+         }
+         _services.Clear();
       }
    }
 }

src/client/spark/view/Shell/ViewModels/LauncherVM.cs

@@ -12,8 +12,7 @@ using System.Windows.Input;
 
 namespace mbts.ui.shell {
    [Export(typeof(ILauncher))]
-   public class LauncherVM : ViewModelBase, ILauncher
-   {
+   public class LauncherVM : ViewModelBase, ILauncher {
       Window _window;
       string _user;
       readonly IBus _bus;
@@ -23,9 +22,9 @@ namespace mbts.ui.shell {
       [ImportingConstructor]
       public LauncherVM(
          ObservableCollection<IShellModule> modules, 
-         [Import(LoginContainer.Exports.UserName)]string username, 
+         UserTuple user, 
          ErrorReporter errors, IBus bus) {
-         _user = username;
+         _user = user.name;
          _modules = new ObservableAutoWrapper<ShellModuleVM, IShellModule>(modules, m => new ShellModuleVM(m, bus));
          _error_service = errors;
          _bus = bus;
@@ -123,7 +122,7 @@ namespace mbts.ui.shell {
          _shown = true;
          if (_window != null) {
             Application.Current.ShutdownMode = ShutdownMode.OnMainWindowClose;
-            if (shown != null) shown();
+            shown?.Invoke();
             _window.Show();
          }
       }
@@ -140,4 +139,4 @@ namespace mbts.ui.shell {
 
       #endregion
    }
-}
+}

src/client/spark/view/Shell/ViewModels/ShellThreadFactory.cs

@@ -42,8 +42,8 @@ namespace mbts.ui.shell
             try {
                Dispatcher.Run();
             }
-            catch (Exception e) {
-               Log.assert(false, e.ToString());
+            catch (Exception ex) {
+               Log.assert(false, $"Exception on dispatcher thread '{name}': {ex}.");
             }
             finally {
                Log.Ui.info("Disposing slow {0} and cleaning up background thread.".fmt(name));

src/client/spark_api/Co2/MDConsumerHandle.cs

@@ -133,6 +133,12 @@ namespace mbts.co2 {
          total_volume = _cold->total_volume;
          return (_cold->last_trade.load_if_new(ref time, out ps));
       }
+      public bool get_last_trade(ref ulong time, out NativePriceSize ps, out DateTime timestamp, out ulong total_volume, out bool from_spread) {
+         timestamp = _cold->last_trade_time;
+         total_volume = _cold->total_volume;
+         from_spread = (_cold->from_spread != 0);
+         return (_cold->last_trade.load_if_new(ref time, out ps));
+      }      
       public ulong misc_timestamp { get { return _cold->misc_timestamp; } }
 
       public ulong total_volume { get { return _cold->total_volume; } }
@@ -259,7 +265,7 @@ namespace mbts.co2 {
          // Last trade
          public AtomicPriceSize last_trade;
          public ulong total_volume;
-         public utils.LLDateTime last_trade_time;
+         public utils.LLDateTime last_trade_time;         
 
          //// Settlement
          public UInt64 misc_timestamp;
@@ -280,6 +286,7 @@ namespace mbts.co2 {
          public double session_low_offer;
          public double fixing_price;
 
+         public int from_spread;
          public int deleted;
          // FixedSecurity will not be directly addressable.  Instead the security is available through the callback
          public Int64 security_size;

src/client/spark_api/api/Update.cs

@@ -8,7 +8,6 @@ using System.Linq;
 using System.Reactive.Concurrency;
 using System.Reactive.Disposables;
 using System.Reactive.Linq;
-using System.Reactive.Subjects;
 using System.Text;
 
 namespace bts.api.core {
@@ -94,38 +93,38 @@ namespace bts.api.core {
          new Update<T, TKey>(new T[0], new T[0], new Change<T>[0], new T[0], key_extractor);
 
       public Update<T,TKey> merge(Update<T, TKey> next) {
-         var comp = new RowEqualityComparer<T, TKey>(_key);
-         var added = this.added as HashSet<T> ?? new HashSet<T>(this.added, comp);
+         var added = this.added as KeyedValueSet<TKey, T> ?? new KeyedValueSet<TKey, T>(this.added, _key);
          var changes = this.changed as ChangeDict<T,TKey> ?? new ChangeDict<T,TKey>(this.changes, _key);
-         var removed = this.removed as HashSet<T> ?? new HashSet<T>(this.removed, comp);
+         var removed = this.removed as KeyedValueSet<TKey, T> ?? new KeyedValueSet<TKey, T>(this.removed, _key);
 
          // Add
-         added.UnionWith(next.added);
+         foreach (var v in next.added) {
+            var k = _key(v);
+            if (removed.Remove(k, out var oldval)) {
+               changes.AddOrMerge(new Change<T>(oldval, v));
+            } else {
+               added[k] = v;
+            }
+         }
 
          // Change
          foreach (var c in next.changes) {
-            if (added.Contains(c.Old)) {
-               added.Remove(c.Old);
-               added.Add(c.New);
-            }
-            else {
+            var k = _key(c.Old);
+            if (!added.Replace(k, c.New))
                changes.AddOrMerge(c);
-            }
          }
 
          // Remove
          foreach (var r in next.removed) {
-            if (!added.Remove(r)) {
-               removed.Add(r);
-            }
-            changes.Remove(_key(r));
+            var k = _key(r);
+            if (!added.Remove(k))
+               removed[k] = changes.TryRemove(k, out var ch) ? ch.Old : r;
          }
 
          return new Update<T, TKey>(
             next.all,
             added, changes, removed, 
-            _key
-            );
+            _key);
       }
 
       public IEnumerable<T> all { get; }
@@ -148,6 +147,39 @@ namespace bts.api.core {
 #endif
    }
 
+   class KeyedValueSet<TKey, TValue> : IEnumerable<TValue> {
+      readonly Dictionary<TKey, TValue> _cache = new Dictionary<TKey, TValue>();
+
+      public KeyedValueSet(IEnumerable<TValue> value, Func<TValue, TKey> key) {
+         foreach (var v in value)
+            _cache[key(v)] = v;
+      }
+
+      public TValue this[TKey key] {
+         set => _cache[key] = value;
+      }
+
+      public bool Remove(TKey key) =>
+         _cache.Remove(key);
+
+      public bool Remove(TKey key, out TValue value) =>
+         _cache.TryGetValue(key, out value) && _cache.Remove(key);
+
+      public bool Replace(TKey key, TValue value) {
+         if (_cache.ContainsKey(key)) {
+            _cache[key] = value;
+            return true;
+         }
+         return false;
+      }
+
+      public IEnumerator<TValue> GetEnumerator() =>
+         _cache.Values.GetEnumerator();
+
+      IEnumerator IEnumerable.GetEnumerator() =>
+         GetEnumerator();
+   }
+
    class ChangeDict<T, TKey> : IEnumerable<Change<T>> {
       Dictionary<TKey, Change<T>> _changes;
       Func<Change<T>, TKey> _key;
@@ -167,9 +199,11 @@ namespace bts.api.core {
          return null;
       }
 
-      public void Remove(TKey key) {
+      public bool Remove(TKey key) =>
          _changes.Remove(key);
-      }
+
+      public bool TryRemove(TKey key, out Change<T> result) =>
+         _changes.TryGetValue(key, out result) && _changes.Remove(key);
 
       #region IEnumerable
       public IEnumerator<Change<T>> GetEnumerator() {
@@ -181,31 +215,6 @@ namespace bts.api.core {
       #endregion
    }
 
-   struct RowEqualityComparer<T,TKey> : IEqualityComparer<T>, IEqualityComparer<Change<T>>
-      where TKey : IEquatable<TKey> {
-      readonly Func<T,TKey> _key;
-
-      public RowEqualityComparer(Func<T,TKey> key) {
-         _key = key;
-      }
-
-      public bool Equals(T x, T y) {
-         return _key(x).Equals(_key(y));
-      }
-
-      public bool Equals(Change<T> x, Change<T> y) {
-         return Equals(x.New, y.New);
-      }
-
-      public int GetHashCode(T obj) {
-         return _key(obj).GetHashCode();
-      }
-
-      public int GetHashCode(Change<T> c) {
-         return GetHashCode(c.New);
-      }
-   }
-
    public class TableUpdate<T, TKey> where TKey : IEquatable<TKey> {
       // warning: this does not have a key_extractor
       public static readonly TableUpdate<T, TKey> empty = new TableUpdate<T, TKey>(ImmutableDictionary<TKey, T>.Empty, Array.Empty<T>(), Array.Empty<Change<T>>(), Array.Empty<T>(), null);
@@ -736,13 +745,19 @@ namespace bts.api.core {
             return new CompositeDisposable(source.Subscribe(aggregator.on_left), right.Subscribe(aggregator.on_right));
          });
 
-      public static IObservable<TResult> AggregateLatest<TLeft, TRight, TAggregate, TResult>(this IObservable<TLeft> source, IObservable<TRight> right,
-         Func<TAggregate, Change<TLeft>, TRight, TAggregate> update_left, Func<TAggregate, TLeft, Change<TRight>, TAggregate> update_right, TAggregate seed,
+      public static IObservable<TResult> AggregateLatest<TLeft, TRight, TAggregate, TResult>(
+         this IObservable<TLeft> source,
+         IObservable<TRight> right,
+         Func<TAggregate, Change<TLeft>, TRight, TAggregate> update_left,
+         Func<TAggregate, TLeft, Change<TRight>, TAggregate> update_right,
+         Func<TAggregate> seed_factory,
          Func<TAggregate, TResult> result_selector,
-         TLeft init_left = default, TRight init_right = default) =>
+         TLeft init_left = default,
+         TRight init_right = default) =>
+         //
          Observable.Create<TResult>(o => {
             Log.DataFlow.diag($"Create AggregateLatest (Left: {typeof(TLeft)}, Right: {typeof(TRight)}).");
-            var aggregator = new ConcurrentAggregator<TLeft, TRight, TAggregate>(update_left, update_right, agg => o.OnNext(result_selector(agg)), seed, init_left, init_right);
+            var aggregator = new ConcurrentAggregator<TLeft, TRight, TAggregate>(update_left, update_right, agg => o.OnNext(result_selector(agg)), seed_factory(), init_left, init_right);
             return new CompositeDisposable(source.Subscribe(aggregator.on_left), right.Subscribe(aggregator.on_right));
          });
 
@@ -922,4 +937,4 @@ namespace bts.api.core {
             $"T is '{typeof(T)}', TKey is '{typeof(TKey)}'";
       }
    }
-}
+}

src/client/spark_api/api/Update/TableUpdateFullJoin.cs

@@ -18,7 +18,7 @@ namespace bts.api.core {
          return left_source.AggregateLatest<TableUpdate<TLeft, TKey>, TableUpdate<TRight, TKey>, (TableUpdate<TResult, TKey> result, IReadOnlyDictionary<TKey, TLeft> left_state, IReadOnlyDictionary<TKey, TRight> right_state), TableUpdate<TResult, TKey>>(right_source,
             process_left,
             process_right,
-            (TableUpdate<TResult, TKey>.mk_empty(result_key), new Dictionary<TKey, TLeft>(0), new Dictionary<TKey, TRight>(0)),
+            () => (TableUpdate<TResult, TKey>.mk_empty(result_key), new Dictionary<TKey, TLeft>(0), new Dictionary<TKey, TRight>(0)),
             t => t.result,
             TableUpdate<TLeft, TKey>.mk_empty(left_key), TableUpdate<TRight, TKey>.mk_empty(right_key));
          // left

src/client/spark_api/api/Update/TableUpdateLeftJoin.cs

@@ -20,7 +20,7 @@ namespace bts.api.core {
             right_source,
             process_left,
             process_right,
-            (TableUpdate<TResult, TLeftKey>.mk_empty(result_key), new Dictionary<TRightKey, HashSet<TLeftKey>>()),
+            () => (TableUpdate<TResult, TLeftKey>.mk_empty(result_key), new Dictionary<TRightKey, HashSet<TLeftKey>>()),
             state => state.result,
             TableUpdate<TLeft, TLeftKey>.mk_empty(left_key),
             TableUpdate<TRight, TRightKey>.mk_empty(right_key));

src/client/spark_api/data/UserTuple.cs

@@ -1,15 +1,28 @@
 using System;
 
 namespace bts {
-   struct UserTuple : IEquatable<UserTuple>, IComparable<UserTuple>, IComparable {
+   public readonly struct UserTuple : IEquatable<UserTuple>, IComparable<UserTuple>, IComparable {
+      const string _admin_name = "bts";
+
       public static readonly Func<UserTuple, uint> key_fn = u => u.id;
+      public static UserTuple admin => new UserTuple(0, _admin_name, Permissions.Admin);
 
       public readonly uint id;
       public readonly string name;
+      public readonly Permissions permissions;
 
-      public UserTuple(uint id, string name) {
+      public UserTuple(uint id, string name)
+         : this(id, name, name == _admin_name ? Permissions.Admin : Permissions.Regular) { }
+
+      public UserTuple(uint id, string name, Permissions permissions) {
          this.id = id;
          this.name = name;
+         this.permissions = permissions;
+      }
+
+      public enum Permissions {
+         Regular,
+         Admin
       }
 
       public int CompareTo(UserTuple other) {
@@ -28,4 +41,9 @@ namespace bts {
       public static bool operator ==(UserTuple a, UserTuple b) => a.Equals(b);
       public static bool operator !=(UserTuple a, UserTuple b) => !a.Equals(b);
    }
-}
+
+   public static class UserTupleExtensions {
+      public static bool can_access_user_data(this UserTuple user, uint user_id) =>
+         user.id == user_id || user.permissions == UserTuple.Permissions.Admin;
+   }
+}

src/client/spark_api/md/MDSource.cs

@@ -13,84 +13,78 @@ namespace bts.md {
    enum MDSourceOptions {
       DeferInit = 1,
    }
-   class MDSource : IObserver<List<SecurityAndHandle>>, IDisposable
-   {
+
+   class MDSource : IObserver<List<SecurityAndHandle>>, IDisposable {
       const uint _no_user_id = 0;
+
       protected readonly MDConsumer _consumer = new MDConsumer();
       protected readonly Co2RelayControl _relay;
-      protected DateTime _version = DateTime.MinValue;
       protected readonly ConcurrentDictionary<SecurityKey, SecurityMD> _exchange_sec_map = new ConcurrentDictionary<SecurityKey, SecurityMD>();
-      protected List<IDisposable> _dispose = new List<IDisposable>();
       protected readonly MDSourceOptions _options;
-      readonly uint _user_id;
+      protected DateTime _version = DateTime.MinValue;
+      protected List<IDisposable> _dispose = new List<IDisposable>();
 
-      public MDSource(uint naggle_ms, MDSourceOptions options = 0, uint user_id = _no_user_id)
-      {
+      readonly UserTuple _user;
+      readonly BehaviorSubject<ConcurrentDictionary<SecurityKey, SecurityMD>> _on_securities =
+         new BehaviorSubject<ConcurrentDictionary<SecurityKey, SecurityMD>>(new ConcurrentDictionary<SecurityKey, SecurityMD>());
+
+      public MDSource(uint naggle_ms, MDSourceOptions options = 0)
+         : this(naggle_ms, options, UserTuple.admin) { }
+
+      public MDSource(uint naggle_ms, MDSourceOptions options, UserTuple user) {
          _options = options;
-         _user_id = user_id;
+         _user = user;
          _relay = new Co2RelayControl(_consumer, naggle_ms);
          _relay.preload_securities((co2.MDRequest.Types.Flags)0xff).ignore();
          _dispose.Add(_consumer.on_security.Subscribe(this));
-         if(!options.HasFlag(MDSourceOptions.DeferInit))
+         if (!options.HasFlag(MDSourceOptions.DeferInit))
             initialize();
       }
 
-      readonly BehaviorSubject<ConcurrentDictionary<SecurityKey, SecurityMD>> _on_securities = 
-         new BehaviorSubject<ConcurrentDictionary<SecurityKey, SecurityMD>>( new ConcurrentDictionary<SecurityKey, SecurityMD>() );
       public IObservable<ConcurrentDictionary<SecurityKey, SecurityMD>> on_securities => _on_securities;
 
-      virtual public void initialize()
-      {
+      public virtual void initialize() =>
          Task.Run(run);
-      }
 
-      public void set_token(AuthTokenImmutable token)
-      {
+      public void set_token(AuthTokenImmutable token) =>
          _relay.set_token(token);
-      }
 
       public void OnCompleted() { }
-      public void OnError(Exception error) => Log.Md.error(error.ToString());
+
+      public void OnError(Exception error) =>
+         Log.Md.error(error.ToString());
+
       public void OnNext(List<SecurityAndHandle> values) {
          if (values.Count > 0) {
             foreach (var v in values) {
-               if (_user_id != _no_user_id &&
-                  v.security.ContractCase == Security.ContractOneofCase.Synthetic &&
-                  v.security.Synthetic.UserId != _user_id &&
-                  v.security.Synthetic.UserId != _no_user_id) {
-                  continue;
+               if (v.security.ContractCase != Security.ContractOneofCase.Synthetic || _user.can_access_user_data(v.security.Synthetic.UserId)) {
+                  var smd = new SecurityMD(new SecurityImmutable(v.security), v.handle);
+                  _exchange_sec_map.AddOrUpdate(smd.security.Id, smd, (_, __) => smd);
                }
-               var smd = new SecurityMD(new SecurityImmutable(v.security), v.handle);
-               _exchange_sec_map.AddOrUpdate(smd.security.Id, smd, (_, __) => smd);
             }
             _version = DateTime.UtcNow;
          }
       }
 
-      public void Dispose() => Disposable.dispose(_dispose);
-      async protected Task run()
-      {
+      public void Dispose() =>
+         Disposable.dispose(_dispose);
+
+      protected async Task run() {
          DateTime run_version = DateTime.MinValue;
-         while (true)
-         {
+         while (true) {
             var exch = _exchange_sec_map;
             var current_version = _version;
-            if (current_version > run_version)
-            {
+            if (current_version > run_version) {
                run_version = current_version;
-               try
-               {
+               try {
                   Log.Md.info($"MDSource publishing {exch.Count} securities");
                   _on_securities.OnNext(exch);
-               }
-               catch (Exception e)
-               {
+               } catch (Exception e) {
                   Log.Md.error($"Error in publish_securities: {e}");
                }
             }
             await Task.Delay(TimeSpan.FromSeconds(1));
          }
       }
-
    }
-}
+}

src/client/spark_api/md/SecurityMD.cs

@@ -1,13 +1,11 @@
 using bts.api;
 using mbts.co2;
 using System;
-using System.Collections.Generic;
-using System.Linq;
-using System.Text;
-using System.Threading.Tasks;
 
 namespace mbts {
-   struct SecurityMD : IEquatable<SecurityMD> {
+   readonly struct SecurityMD : IEquatable<SecurityMD> {
+      public static readonly Func<SecurityMD, uint> key_fn = smd => smd.security.Id;
+
       public SecurityImmutable security { get; }
       public MDConsumerHandle md { get; }
 
@@ -36,4 +34,4 @@ namespace mbts {
          return !(left == right);
       }
    }
-}
+}

src/client/spark_core/Collections/Extensions/DictionaryExtensions.cs

@@ -278,6 +278,9 @@ namespace System.Collections.Generic {
       public static IComparer<KeyValuePair<TKey, TValue>> key_comparer<TKey, TValue>() =>
          Impl<TKey, TValue>.key_comparer;
 
+      public static IReadOnlyDictionary<TKey, TValue> wrap_value<TKey, TValue>(TKey key, TValue value) =>
+         new Impl<TKey, TValue>.ValueWrapper(key, value);
+
       static class Impl<TKey, TValue> {
          public static readonly IReadOnlyDictionary<TKey, TValue> empty =
             new ReadOnlyDictionary<TKey, TValue>(new Dictionary<TKey, TValue>(0));
@@ -297,6 +300,57 @@ namespace System.Collections.Generic {
             public int Compare(object x, object y) =>
                x is KeyValuePair<TKey, TValue> xkv ? (y is KeyValuePair<TKey, TValue> ykv ? Compare(xkv, ykv) : 1) : -1;
          }
+
+         public class ValueWrapper : IReadOnlyDictionary<TKey, TValue> {
+            readonly TKey _key;
+            readonly TValue _value;
+            readonly IEqualityComparer<TKey> _key_comparer;
+
+            public ValueWrapper(TKey key, TValue value)
+               : this(key, value, EqualityComparer<TKey>.Default) { }
+
+            public ValueWrapper(TKey key, TValue value, IEqualityComparer<TKey> key_comparer) {
+               _key = key;
+               _value = value;
+               _key_comparer = key_comparer;
+            }
+
+            public TValue this[TKey key] =>
+               _key_comparer.Equals(key, _key) ? _value : throw new KeyNotFoundException();
+
+            public int Count => 1;
+
+            public IEnumerable<TKey> Keys {
+               get {
+                  yield return _key;
+               }
+            }
+
+            public IEnumerable<TValue> Values {
+               get {
+                  yield return _value;
+               }
+            }
+
+            public bool ContainsKey(TKey key) => _key_comparer.Equals(key, _key);
+
+            public IEnumerator<KeyValuePair<TKey, TValue>> GetEnumerator() {
+               yield return new KeyValuePair<TKey, TValue>(_key, _value);
+            }
+
+            IEnumerator IEnumerable.GetEnumerator() {
+               yield return new KeyValuePair<TKey, TValue>(_key, _value);
+            }
+
+            public bool TryGetValue(TKey key, out TValue value) {
+               if (_key_comparer.Equals(key, _key)) {
+                  value = _value;
+                  return true;
+               }
+               value = default;
+               return false;
+            }
+         }
       }
    }
 }

src/client/spark_core/Collections/Extensions/EnumerableExtensions.cs

@@ -343,6 +343,18 @@ namespace System.Collections.Generic {
          return false;
       }
 
+      public static List<T> ToListSorted<T, TKey>(this IEnumerable<T> enumerable, Func<T, TKey> key_selector) =>
+         enumerable.ToListSorted(key_selector, Comparer<TKey>.Default);
+
+      public static List<T> ToListSorted<T, TKey>(this IEnumerable<T> enumerable, Func<T, TKey> key_selector, IComparer<TKey> key_comparer) =>
+         enumerable.ToListSorted(key_comparer.wrap(key_selector));
+
+      public static List<T> ToListSorted<T>(this IEnumerable<T> enumerable, IComparer<T> comparer) {
+         var list = new List<T>(enumerable);
+         list.Sort(comparer);
+         return list;
+      }
+
       /// <summary>
       /// Simply returns enumerable.ToList(), under the guise ofo IReadOnlyList&lt;&gt;
       /// </summary>

src/client/spark_core/Collections/Extensions/ImmutableSetExtensions.cs

@@ -0,0 +1,29 @@
+using System;
+using System.Collections.Generic;
+using System.Collections.Immutable;
+
+namespace bts.collections {
+   public static class ImmutableSetExtensions {
+      public static IImmutableSet<T> RemoveRange<T>(this IImmutableSet<T> source, IEnumerable<T> values) {
+         /// Prefer the <see cref="ImmutableHashSet{T}"/> implementation.
+         /// It uses an <see cref="ImmutableHashSet{T}.Builder"/> to remove values, which is optimized for bulk changes (less memory allocations, etc.).
+         if (source is ImmutableHashSet<T> ihs) {
+            return ihs.RemoveRange(values);
+         }
+         if (source == null) throw new ArgumentNullException(nameof(source));
+         if (values == null) throw new ArgumentNullException(nameof(values));
+         foreach (var v in values)
+            source = source.Remove(v);
+         return source;
+      }
+
+      public static ImmutableHashSet<T> RemoveRange<T>(this ImmutableHashSet<T> source, IEnumerable<T> values) {
+         if (source == null) throw new ArgumentNullException(nameof(source));
+         if (values == null) throw new ArgumentNullException(nameof(values));
+         var bd = source.ToBuilder();
+         foreach (var v in values)
+            bd.Remove(v);
+         return bd.ToImmutable();
+      }
+   }
+}

src/client/spark_core/TestRunner/InProcessTestRunner.cs

@@ -10,9 +10,9 @@ using System.Diagnostics;
 namespace bts.test {
    public static class InProcessTestRunner {
 
-      public static void run_test_synchronously(string v, TestEnumerator enumerator, Action<TestResult> report) {
-         foreach (var t in enumerator.find(v)) {
-                        run_test_synchronously(t, report);
+      public static void run_test_synchronously(string v, TestType type, TestEnumerator enumerator, Action<TestResult> report) {
+         foreach (var t in enumerator.find(v,type)) {
+            run_test_synchronously(t, report);
          }
       }
       public static void run_test_synchronously(TestMethod test, Action<TestResult> report) {

src/client/spark_core/TestRunner/TestEnumerator.cs

@@ -36,18 +36,19 @@ namespace bts.test {
       }
 
 
-      public IEnumerable<TestMethod> find(string name) {
+      public IEnumerable<TestMethod> find(string name, TestType type) {
          foreach (var g in TestGroups) {
             foreach (var t in g.methods) {
-               if (g.name == name || t.name == name || t.full_name == name) {
+               if (g.test_type == type &&
+                  (g.name == name || t.name == name || t.full_name == name)) {
                   yield return t;
                }
             }
          }
       }
 
-      public IEnumerable<TestMethod> match(IEnumerable<string> names) {
-         return names.SelectMany(find).Distinct();
+      public IEnumerable<TestMethod> match(TestType type, IEnumerable<string> names) {
+         return names.SelectMany(name => find(name,type)).Distinct();
       }
    }
 

src/client/spark_core/admin/AdminModelWrapper.cs

@@ -1,64 +0,0 @@
-using bts;
-using bts.api.admin;
-using System;
-using System.Collections.Generic;
-using System.Linq;
-
-namespace bts.api.core
-{
-   public class AdminModelWrapper
-   {
-      readonly AdminDataImmutable? _admin_data;
-      public AdminDataImmutable? admin_data => _admin_data;
-      readonly string _user_name;
-      readonly List<BTSAccountImmutable> _allowed_accounts;
-      readonly Dictionary<string, string> _allowed_account_dict = new Dictionary<string, string>();
-      public AdminModelWrapper(AdminDataImmutable? admin_data = null, string user_name = null)
-      {
-         _admin_data = admin_data;
-         _user_name = user_name ?? string.Empty;
-         _allowed_accounts = search_admin(u => u.Name == _user_name).Where(x =>
-         {
-            return (x.Item2 != null && x.Item2.HasPermissions) || (x.Item3 != null && x.Item3.HasPermissions);
-         }).Select(xx => xx.Item1).ToList();
-         foreach (var val in _allowed_accounts)
-            _allowed_account_dict[val.Name] = val.Name;
-      }
-      public IEnumerable<BTSAccountImmutable> viewable_accounts()
-      {
-         return _allowed_accounts;
-      }
-      public bool can_view_account(string account)
-      {
-         return _allowed_account_dict.ContainsKey(account) || _user_name.ToLower().Equals("bts");
-      }
-
-      IEnumerable<Tuple<BTSAccountImmutable, TradingPermissionUnit, AdminPermissionUnit>> 
-         search_admin(Func<BTSUserImmutable, bool> predicate)
-      {
-         var rv = new List<Tuple<BTSAccountImmutable, TradingPermissionUnit, AdminPermissionUnit>>();
-         if (_admin_data.HasValue)
-         {
-            var users = _admin_data.Value.Users.Where(predicate);
-            if (users != null && users.Count() == 1)
-            {
-               var user = users.ElementAt(0);
-               if (user.HasUserData && user.UserData.HasAccountPermissions)
-               {
-                  foreach (var kv in user.UserData.AccountPermissions.AccountPermissions)
-                  {
-                     var account_id = kv.Key;
-                     var accounts = _admin_data.Value.Accounts.Where(a => a.Id == account_id);
-                     if (accounts != null && accounts.Count() == 1)
-                     {
-                        var account = accounts.ElementAt(0);
-                        rv.Add(Tuple.Create(account, kv.Value.TradingPermission, kv.Value.AdminPermission));
-                     }
-                  }
-               }
-            }
-         }
-         return rv;
-      }
-   }
-}

src/client/spark_core/admin/PermissionFilters.cs

@@ -1,64 +0,0 @@
-using bts;
-using bts.api;
-using bts.api.core;
-using bts.api.orderman;
-using bts.api.position;
-using bts.spark;
-using System;
-using System.Collections.Generic;
-using System.Collections.Immutable;
-using System.Collections.Specialized;
-using System.Linq;
-using System.Reactive.Concurrency;
-using System.Reactive.Linq;
-
-namespace bts.api.core
-{
-   public static class PermissionFilters
-   {
-      public static IObservable<Update<PositionUnitImmutable, int>> 
-         PermissionFilter(this IObservable<Update<PositionUnitImmutable, int>> source, IObservable<AdminModelWrapper> am)
-      {
-         var pred = Observable.Create<Func<PositionUnitImmutable, bool>>( observer =>
-         {
-            return am.Subscribe(x =>
-            {
-               Func<PositionUnitImmutable, bool> filter = (z => x.can_view_account(z.Account));
-               observer.OnNext(filter);
-            });
-         });
-
-         return source.Filter(pred);
-      }
-
-      public static IObservable<Update<OrderRecordImmutable, uint>> 
-         PermissionFilter(this IObservable<Update<OrderRecordImmutable, uint>> source, IObservable<AdminModelWrapper> am)
-      {
-         var pred = Observable.Create<Func<OrderRecordImmutable, bool>>(observer =>
-        {
-           return am.Subscribe(x =>
-           {
-              Func<OrderRecordImmutable, bool> filter = (z => x.can_view_account(z.Def.Account ?? string.Empty));
-              observer.OnNext(filter);
-           });
-        });
-
-         return source.Filter(pred);
-      }
-
-      public static IObservable<Update<FillImmutable, string>> 
-         PermissionFilter(this IObservable<Update<FillImmutable, string>> source, IObservable<AdminModelWrapper> am)
-      {
-         var pred = Observable.Create<Func<FillImmutable, bool>>(observer =>
-        {
-           return am.Subscribe(x =>
-           {
-              Func<FillImmutable, bool> filter = (z => x.can_view_account(z.Account));
-              observer.OnNext(filter);
-           });
-        });
-
-         return source.Filter(pred);
-      }
-   }
-}

src/client/spark_core/logging/LogUtil.cs

@@ -0,0 +1,96 @@
+using System;
+using System.IO;
+using mbts.config;
+using System.IO.Compression;
+using System.Runtime.InteropServices;
+using Google.Protobuf;
+using bts;
+using System.Linq;
+using bts.support;
+
+namespace mbts
+{
+   public static class LogUtil {
+      private static string log_path = Path.Combine(Configuration.get<bts.config.System>().data_path, "logging", "log");
+
+
+
+      public static ByteString read_logs_compressed() {
+         return FileUtils.read_and_gzip(log_path);
+      }
+
+      public static (ByteString,DateTime) read_latest_crash_dump() {
+         // Find the latest crash dump
+         // If it is < 5 days old
+         // gzip it in place (delete original)
+         // upload the gziped dump
+
+         var crash_dump_folder = "c:\\programdata\\bts\\crash_dumps";
+         FileUtils.keep_n_matching_files(crash_dump_folder, "*.gz", 4);
+         try {
+            if (Directory.Exists(crash_dump_folder)) {
+               var latest =
+                  Directory.GetFiles(crash_dump_folder, "*.dmp")
+                  .Select(f => new { name = f, modified = File.GetLastWriteTimeUtc(f) })
+                  .OrderBy(f => f.modified)
+                  .Reverse()
+                  .FirstOrDefault();
+               if (latest != null) {
+                  if (DateTime.Now - latest.modified <= TimeSpan.FromDays(5)) {
+                     var newname = FileUtils.gzip_file_in_place(latest.name);
+                     if (!string.IsNullOrEmpty(newname)) {
+                        return (ByteString.FromStream(File.OpenRead(newname)), latest.modified);
+                     }
+                  }
+               }
+            }
+         }
+         catch (Exception e) {
+            Log.Gen.error($"Failed to read crash dumps. Exception: {e.Message}");
+         }
+         return (ByteString.Empty, DateTime.MinValue);
+      }
+
+      public static ByteString read_windows_event_log() {
+         // try to export the events to temp
+         var temp = Path.Combine(Path.GetTempPath(), Path.GetTempFileName() + ".evtx");
+         bool success = NativeMethods.EvtExportLog(
+            IntPtr.Zero, 
+            "Application",
+            "*", 
+            temp, 
+            NativeMethods.EventExportLogFlags.ChannelPath);
+         if (success) {
+            var result = FileUtils.read_and_gzip(temp);
+            FileUtils.try_delete(temp);
+            return result;
+         }
+         else {
+            string msg = new System.ComponentModel.Win32Exception(Marshal.GetLastWin32Error()).Message;
+            Log.Gen.error($"Error exporting windows event log. {msg}");
+         }
+         return ByteString.Empty;
+      }
+
+
+      static class NativeMethods {
+         [Flags]
+         public enum EventExportLogFlags {
+            ChannelPath = 1,
+            LogfilePath = 2,
+            TolerateQueryErrors = 0x1000
+         }
+
+         [DllImport("wevtapi.dll", CallingConvention=CallingConvention.Winapi, CharSet = CharSet.Auto, SetLastError =true)]
+         public static extern bool EvtExportLog(
+            IntPtr session, 
+            string path, 
+            string query, 
+            string targetPath,
+            [MarshalAs(UnmanagedType.I4)]
+            EventExportLogFlags flags);
+
+      }
+
+   }
+}

src/client/spark_core/logging/Logger.cs

@@ -1,32 +0,0 @@
-using System;
-using System.Diagnostics;
-using System.IO;
-using System.Net.Mail;
-using System.Reflection;
-using System.Runtime.CompilerServices;
-using System.Runtime.InteropServices;
-using mbts.config;
-using System.Collections.Generic;
-
-
-namespace mbts
-{
-   public static class LogUtil {
-      public static string read_logs() {
-         var log_path = Path.Combine(Configuration.get<bts.config.System>().data_path, "logging", "log");
-         if (File.Exists(log_path)) {
-            try {
-               using (var f = File.Open(log_path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)) {
-                  var reader = new StreamReader(f);
-                  return reader.ReadToEnd();
-               }
-            }
-            catch (Exception e) {
-               return e.ToString();
-            }
-         }
-         return "Log file does not exist";
-      }
-
-   }
-}

src/client/spark_core/utils/FileUtils.cs

@@ -0,0 +1,76 @@
+using System;
+using System.IO;
+using System.IO.Compression;
+using System.Linq;
+using Google.Protobuf;
+
+namespace bts.support {
+   static class FileUtils {
+      public static bool try_delete(string path) {
+         try {
+            if (File.Exists(path)) {
+               File.Delete(path);
+            }
+            else if (Directory.Exists(path)) {
+               Directory.Delete(path, true);
+            }
+         }
+         catch {
+            return false;
+         }
+         return true;
+      }
+
+      public static ByteString read_and_gzip(string path) {
+         if (File.Exists(path)) {
+            try {
+               using (var fs = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite /*Note: ReadWrite is more permissive than Read*/))
+               using (var ms = new MemoryStream())
+               using (var gz = new GZipStream(ms, CompressionMode.Compress, true)) {
+                  fs.CopyTo(gz);
+                  gz.Close();
+                  return ByteString.CopyFrom(ms.GetBuffer(), 0, (int)ms.Length);
+               }
+            }
+            catch (Exception e) {
+               Log.Gen.error($"Failed to read file '{path}' -- {e.Message}");
+               return ByteString.Empty;
+            }
+         }
+         return ByteString.Empty;
+      }
+
+
+      // If input exists compress it and add a .gz extension
+      // original file is deleted
+      public static string gzip_file_in_place(string file) {
+         if (File.Exists(file)) {
+            string destination = file + ".gz";
+            using (var input = File.OpenRead(file)) {
+               using (var output = File.OpenWrite(destination)) {
+                  using (var writer = new GZipStream(output, CompressionMode.Compress)) {
+                     input.CopyTo(writer);
+                  }
+               }
+            }
+            try_delete(file);
+            return destination;
+         }
+         return null;
+      }
+
+      // Delete the oldest files matching pattern (*.txt) keeping {count} files
+      public static void keep_n_matching_files(string directory, string pattern, int count) {
+         var to_delete =
+            Directory.GetFiles(directory, pattern)
+            .Select(f => new { name = f, modified = File.GetLastWriteTimeUtc(f) })
+            .OrderBy(f => f.modified)
+            .Reverse()
+            .Skip(count);
+
+         foreach (var f in to_delete) {
+            try_delete(f.name);
+         }
+      }
+   }
+}

src/client/spark_test/Co2/Co2Tests.cs

@@ -142,7 +142,8 @@ namespace mbts.co2.tests
          Assert.IsTrue(entry.session_low_offer == 5.5);
          Assert.IsTrue(entry.fixing_price == 6.6);
 
-         Assert.IsTrue(entry.deleted == 1);
+         Assert.IsTrue(entry.from_spread == 0);
+         Assert.IsTrue(entry.deleted == 1);         
       }
 
       [Test]

src/client/spark_test/Utils/MarketDataUtilities.cs

@@ -2,6 +2,7 @@
 using bts.co2;
 using System;
 using System.Collections.Generic;
+using System.Linq;
 using System.Text;
 using System.Threading.Tasks;
 
@@ -39,8 +40,29 @@ namespace bts.tests {
          return null;
       }
 
+public static async Task<IEnumerable<SecurityImmutable>> get_multiple_securities(MarketDataRelay relay, IEnumerable<Product> products, SecurityType security_type, MDRequest.Types.Flags flags = MDRequest.Types.Flags.Bbo, Venue v = Venue.Sim) {
+         var request = new MDRequest() { Host = _MD_HOST };
+         request.Channels.AddRange(products.Select(z =>
+                                       new SecurityChannel() {
+                                          Time = 0,
+                                          Flags = (uint)flags,
+                                          Type = security_type,
+                                          Exchange = v,
+                                          Symbol = z.Symbol,
+                                          TradeCursor = 0,
+                                          RfqCursor = 0
+                                       }));
+
+         var result = await relay.requestmd_async(request);
+
+         if (result.success) {
+            return result.value.Channels.Where(s => s.Updates.Count > 0).Select(x => (SecurityImmutable) x.Updates.First().Security);
+         }
+         return Enumerable.Empty<SecurityImmutable>();
+      }
+
       public static async Task<IEnumerable<FutureContract>> get_expirations(MarketDataRelay relay, string symbol, SecurityType security_type, MDRequest.Types.Flags flags = MDRequest.Types.Flags.Bbo, Venue v = Venue.Sim) {
-          var request = new MDRequest() {
+         var request = new MDRequest() {
             Host = _MD_HOST,
             Channels = {
                new SecurityChannel() {

src/client/spark_test/Utils/TestUtils.cs

@@ -85,6 +85,8 @@ namespace bts.tests.utils {
          if(timeout == null) {
             timeout = TimeSpan.FromMinutes(2);
          }
+         Console.WriteLine("Please Wait... this is going to take some time...");
+         await Task.Delay(180000);
 
          System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch();
          sw.Start();
@@ -131,6 +133,13 @@ namespace bts.tests.utils {
          System.Diagnostics.Stopwatch watch = new System.Diagnostics.Stopwatch();
          watch.Start();
 
+         var prts = mbts.config.Configuration.get<bts.config.Ports>();
+         Console.WriteLine("\n\n\n\n\n\n\n\n\n\n");
+         Console.WriteLine("config_port: " + prts.config_port);
+         Console.WriteLine("qbus_port: " + prts.qbus_port);
+         Console.WriteLine("glbx_sim_fix_port: " + prts.glbx_sim_fix_port);
+         Console.WriteLine("\n\n\n\n\n\n\n\n\n\n");
+
          var ctx = new ServiceContext();
          var mdrelay = ctx.locate<bts.co2.MarketDataRelay>();
 
@@ -143,20 +152,28 @@ namespace bts.tests.utils {
                products = tmpp.value;
                break;
             }
+            else {
+               Console.WriteLine("mdrelay.request_products_async Error: ");
+               Console.WriteLine(tmpp.error_message);
+            }
             await Task.Delay(10000);
          }
+         //UserSymbol.get_all() in MDConsumer.cs
 
          bts.test.Assert.IsTrue(products != null, "No products found");
-         var filtered_products = products.Products.Where(z => z.Exchange == Venue.Sim && z.Type == SecurityType.Future);
-         bts.test.Assert.IsTrue(filtered_products.Count() > 0, "No suitable products found");
+         IEnumerable<Product> filtered_products = products.Products.Where(z => z.Exchange == Venue.Sim && z.Type == SecurityType.Future);
+         bts.test.Assert.IsTrue(filtered_products.Count() > 0, "No suitable products found"); //products are not guaranteed to have securities, so I need to get the product from the list of securities
 
          uint? sec_id = null;
 
+         /* get securities if the list is not empty use the top one */
+
          for(int i = 0; i < 10; i++) {
-            var test_security = await MarketDataUtilities.get_security(mdrelay, filtered_products.First().Symbol, SecurityType.Future, f => true);
-            if(test_security.HasValue) {
+            var test_security = await MarketDataUtilities.get_multiple_securities(mdrelay, filtered_products, SecurityType.Future);
+
+            if(test_security.Count() > 0) {
                Console.Write("," + i);
-               sec_id = test_security.Value.Id;
+               sec_id = test_security.FirstOrDefault().Id;
                break;
             }
             await Task.Delay(10000);
@@ -237,7 +254,7 @@ namespace bts.tests.utils {
          Result<co2.MDReply>? md_result = await await_with_timeout(
                                                                md_relay.requestmd_async(md_request),
                                                                timeout);
-
+         Console.Write(md_result);
          return md_result != null;
       }
    }

src/client/spark_test/api/ApiUpdateTests.cs

@@ -347,6 +347,12 @@ namespace mbts.api.Test {
          var a3 = new Update<string, int>(new[] { "a3" }, empty, new[] { new Change<string>("a2", "a3") }, empty, key);
          var merged = a2.merge(a3);
          Assert.SequenceEqual(new[] { new Change<string>("a1", "a3") }, merged.changes);
+         // clear the state, then add a2 back
+         // (this validates the fix for the Update.naggle bug where interim changes were converted into added/removed).
+         var cleared = new Update<string, int>(empty, empty, Array.Empty<Change<string>>(), merged.all, key);
+         a2 = new Update<string, int>(new[] { "a2" }, new[] { "a2" }, Array.Empty<Change<string>>(), empty, key);
+         merged = merged.merge(cleared).merge(a2);
+         Assert.SequenceEqual(new[] { new Change<string>("a1", "a2") }, merged.changes);
       }
 
       T id<T>(T t) => t;

src/client/spark_test/system_test_scripts/test_position_limits.sh

@@ -30,6 +30,8 @@ cat << EOF > $sparkdata_dir/conf/spark.conf
    }
 EOF
 
+echo "Port"
+./sparkd configcheck Ports
 
 nohup ./sparkd run &
 pid=$!

src/client/sparkcmd/Cmd/TestCommand.cs

@@ -48,7 +48,7 @@ namespace sparkcmd {
          }
          else Configuration.initialize();
 
-         Console.WriteLine(JsonConvert.SerializeObject(o));
+         ConsoleEx.WriteLine(JsonConvert.SerializeObject(o), ConsoleColor.DarkGray);
 
          if (o.list) {
             list_tests((o.type));
@@ -58,13 +58,19 @@ namespace sparkcmd {
          if (o.timeout == 0 && o.tests.Count() == 1 && !o.self_contained) {
             //System.Threading.Thread.Sleep(5000);
             using (var trr = new TestResultReceiver(true)) {
-               InProcessTestRunner.run_test_synchronously(o.tests.First(), Enumerator, trr.get_reporter());
+               InProcessTestRunner.run_test_synchronously(o.tests.First(), o.type, Enumerator, trr.get_reporter());
                trr.print_summary();
+               if (trr.ResultCount == 0) {
+                  ConsoleEx.Write("No ", ConsoleColor.Red);
+                  ConsoleEx.Write(o.type.ToString(), ConsoleColor.Yellow);
+                  ConsoleEx.WriteLine($" tests matched filter '{o.tests.First()}'", ConsoleColor.Red);
+                  return -1;
+               }
                return trr.ErrorCount;
             }
          }
          else if (o.tests.Count() > 0) {
-            return run_tests(o, Enumerator.match(o.tests));
+            return run_tests(o, Enumerator.match(o.type, o.tests));
          }
          else {
             return run_tests(o, Enumerator.get_of_type(o.type));

src/client/sparkcmd/Program.cs

@@ -29,8 +29,13 @@ namespace sparkcmd {
          // initialized from c++ instead of c#
          BTSNativeMethods.provide_command_line(args, args.Length);
 
+         var parser = new Parser(ps => {
+            ps.CaseInsensitiveEnumValues = true;
+            ps.HelpWriter = Console.Out;
+         });
+
          // note: this can be made to infer the options type list from the list of delegates
-         return Parser.Default
+         return parser
          .ParseArguments<
             ConfigcheckOptions,
             ScriptClientOptions,

src/client/sparkcmd/sparkcmd.csproj

@@ -22,7 +22,7 @@
   </ItemGroup>
 
   <ItemGroup>
-    <PackageReference Include="CommandLineParser" Version="2.4.3" />
+    <PackageReference Include="CommandLineParser" Version="2.6.0" />
     <PackageReference Include="Microsoft.CSharp" Version="4.5.0" />
     <PackageReference Include="Newtonsoft.Json" Version="10.0.3" />
   </ItemGroup>

src/data_files/files/onixs_ice/ConnectivityConfiguration.APITest.xml

@@ -11,7 +11,6 @@
 -->
 <connectivityConfiguration>
     <!-- TCP Connection Information -->
-
     <tcp>
         <server name="default">
             <ip>63.247.113.163</ip>
@@ -19,7 +18,7 @@
             <userName>bluetrd-ps</userName>
             <password>Starts123</password>
             <getStripInfoMessages>N</getStripInfoMessages>
-            <strategyPreference>Legacy</strategyPreference>
+            <strategyPreference>New</strategyPreference>
             <localNetworkInterface></localNetworkInterface>
         </server>
         <server name="endex">
@@ -28,7 +27,7 @@
             <userName>Value from ICE Support</userName>
             <password>Value from ICE Support</password>
             <getStripInfoMessages>N</getStripInfoMessages>
-            <strategyPreference>Legacy</strategyPreference>
+            <strategyPreference>New</strategyPreference>
             <localNetworkInterface></localNetworkInterface>
         </server>
         <server name="liffe-eu">
@@ -37,7 +36,7 @@
             <userName>bluetrdl-ps</userName>
             <password>Starts123</password>
             <getStripInfoMessages>N</getStripInfoMessages>
-            <strategyPreference>Legacy</strategyPreference>
+            <strategyPreference>New</strategyPreference>
             <localNetworkInterface></localNetworkInterface>
         </server>
         <settings>

src/data_files/files/onixs_ice/ConnectivityConfiguration.PerfTest.xml

@@ -17,10 +17,10 @@
         <server name="default">
             <ip>63.247.113.214</ip>
             <port>3000</port>
-            <userName>Value from ICE Support</userName>
-            <password>Value from ICE Support</password>
+            <userName>bt_ps</userName>
+            <password>Starts123</password>
             <getStripInfoMessages>N</getStripInfoMessages>
-            <strategyPreference>Legacy</strategyPreference>
+            <strategyPreference>New</strategyPreference>
             <localNetworkInterface></localNetworkInterface>
         </server>
         <settings>

src/lib/CMakeDeps_win.cmake

@@ -1 +1 @@
-set(sources api/apicore.proto;api/apigateway.proto;api/apioptions.proto;api/core/ApiClientInterop.cpp;api/core/ApiServerInterop.cpp;api/public/ApiServer.cpp;api/public/ClientChannel.cpp;api/public/QuickbusClientChannel.cpp;api/public/ServerChannel.cpp;api/public/ServiceContext.cpp;api/service_helpers/AdminServiceUtils.cpp;api/service_helpers/AlgoServiceUtils.cpp;api/service_helpers/PositionUtilities.cpp;api/services/AdminService.proto;api/services/AlgoDefinitionService.proto;api/services/AlgoService.proto;api/services/ApiUnitTestService.proto;api/services/AuthService.proto;api/services/BasePositionService.proto;api/services/ErrorReportService.proto;api/services/HelloService.proto;api/services/MBOService.proto;api/services/MarginService.proto;api/services/MarketDataAuxiliaryService.proto;api/services/MarketDataRelay.proto;api/services/MoneyFileService.proto;api/services/OrderService.proto;api/services/OutOfProcessURLListener.proto;api/services/PersistenceService.proto;api/services/PositionService.proto;api/services/ShellDiagnosticService.proto;api/services/ShellLatencyService.proto;api/services/SyntheticSecurityService.proto;api/test/ApiTests.cpp;api/test/HelloServiceServer.cpp;config/ConfigCheck.cpp;config/ConfigDownloader.cpp;config/ConfigErrorCollector.cpp;config/ConfigLog.cpp;config/ConfigPropertyEvaluator.cpp;config/ConfigValueQueue.cpp;config/ConfigurationException.cpp;config/ConfigurationState.cpp;config/PreprocessInputStream.cpp;config/config.cpp;config/defaults/MarketData.conf;config/defaults/Ports.conf;config/defaults/Symbols.conf;config/defaults/cme_reference_data.conf;config/defaults/edge.conf;config/defaults/jetfix.conf;config/defaults/keycode.conf;config/defaults/logging.conf;config/defaults/orderman_cfg.conf;config/defaults/orders_fills_cleanup.conf;config/defaults/position_cfg.conf;config/defaults/span.conf;config/defaults/spreader.conf;config/defaults/testkeys.conf;config/defaults/threads.conf;config/defaults/toggles.conf;config/extensions/extensions.proto;config/invalid/invalid.proto;config/protos/ExchangeSimCfg.proto;config/protos/MarketData.proto;config/protos/Ports.proto;config/protos/Symbols.proto;config/protos/UI.proto;config/protos/cme_reference_data.proto;config/protos/jetfix.proto;config/protos/keycode.proto;config/protos/logging.proto;config/protos/mercury_cfg.proto;config/protos/orderman_cfg.proto;config/protos/orders_fills_cleanup.proto;config/protos/perflog_cfg.proto;config/protos/position_cfg.proto;config/protos/quickbus_cfg.proto;config/protos/span.proto;config/protos/spreader.proto;config/protos/system.proto;config/protos/testkeys.proto;config/protos/threads.proto;config/protos/toggles.proto;config/tests/HTTPServer.cpp;config/tests/config_ref_tests.cpp;config/tests/config_tests.cpp;config/text_format/text_format.cpp;domain_model/Security.cpp;domain_model/Security.proto;domain_model/SecurityTests.cpp;domain_model/sod.cpp;domain_model/tests/domain_tests.cpp;expression/BooleanGrammar.cpp;expression/ExpressionEvaluator.cpp;expression/ParserExceptions.cpp;expression/Value.cpp;expression/tests/expression_parser_tests.cpp;logging/LoggerHub.cpp;logging/Sink.cpp;logging/logging.cpp;logging/tests/logging_tests.cpp;md/Diag/MDPeriodicDumper.cpp;md/Diag/ShmDumper.cpp;md/MBOProcessor.cpp;md/MBOServiceServer.cpp;md/MDClient.cpp;md/MDConsumer.cpp;md/MDConsumerHandle.cpp;md/MDConsumerInterop.cpp;md/MDProducer.cpp;md/MDProducerHandle.cpp;md/MemorySegmentUtils.cpp;md/PlatformLoader.cpp;md/PriceSize.cpp;md/SecurityCache.cpp;md/SecurityDB.cpp;md/SecurityIDRecorder.cpp;md/SecurityMapper.cpp;md/SecurityOracle.cpp;md/SettlementRecorder.cpp;md/Shm/FixedSecurity.cpp;md/SyntheticSecurityService.cpp;md/cme/MDP3Utils.cpp;md/common.cpp;md/ice/ICEUtils.cpp;md/protos/co2.proto;md/relay/QueryServer.cpp;md/relay/RelayClient.cpp;md/relay/RelayRecorder.cpp;md/test/Co2ManagedTestHelper.cpp;md/test/Co2TestInstrument.cpp;md/test/mbo_tests.cpp;md/test/md2_tests.cpp;md/test/md_test_utils.cpp;md/test/synthetic_book_merge_tests.cpp;md/vendor/OnixsUtils.cpp;os/os.cpp;os/tests/os_tests.cpp;quickbus/core/LocalNode.cpp;quickbus/core/Quickbus.cpp;quickbus/core/SubscriptionID.cpp;quickbus/core/TableID.cpp;quickbus/message_model/Update.cpp;quickbus/message_model/merge_updates.cpp;quickbus/network/QBConnection.cpp;quickbus/network/QBConnectionManager.cpp;quickbus/network/endpoint_description.cpp;quickbus/network/quickbus_msgs.proto;quickbus/pubsub/TablePublisher.cpp;quickbus/pubsub/TableSubscriber.cpp;quickbus/pubsub/tests/qb_pubsub_tests.cpp;quickbus/routing/INode.cpp;quickbus/routing/NodeID.cpp;quickbus/routing/Router.cpp;quickbus/rpc/RPCClient.cpp;quickbus/rpc/RPCServer.cpp;quickbus/rpc/RPCSessionID.cpp;quickbus/rpc/rpc_utils.cpp;quickbus/rpc/tests/qb_rpc_tests.cpp;quickbus/tests/quickbus_tests.cpp;sparkproto/SparkProtos.proto;testing/catch_test_runner.cpp;utils/AuthSessionManager.cpp;utils/AuthToken.cpp;utils/Dispatcher.cpp;utils/GrowableSPSCQueue.cpp;utils/HTTPStream.cpp;utils/PBUtils.cpp;utils/Puid.cpp;utils/SecureLogoLine truncated
+set(sources api/apicore.proto;api/apigateway.proto;api/apioptions.proto;api/core/ApiClientInterop.cpp;api/core/ApiServerInterop.cpp;api/public/ApiServer.cpp;api/public/ClientChannel.cpp;api/public/QuickbusClientChannel.cpp;api/public/ServerChannel.cpp;api/public/ServiceContext.cpp;api/service_helpers/AdminServiceUtils.cpp;api/service_helpers/AlgoServiceUtils.cpp;api/service_helpers/PositionUtilities.cpp;api/services/AdminService.proto;api/services/AlgoDefinitionService.proto;api/services/AlgoService.proto;api/services/ApiUnitTestService.proto;api/services/AuthService.proto;api/services/BasePositionService.proto;api/services/ErrorReportService.proto;api/services/HelloService.proto;api/services/MBOService.proto;api/services/MarginService.proto;api/services/MarketDataAuxiliaryService.proto;api/services/MarketDataRelay.proto;api/services/MoneyFileService.proto;api/services/OrderService.proto;api/services/OutOfProcessURLListener.proto;api/services/PersistenceService.proto;api/services/PositionService.proto;api/services/ShellDiagnosticService.proto;api/services/ShellLatencyService.proto;api/services/SyntheticSecurityService.proto;api/test/ApiTests.cpp;api/test/HelloServiceServer.cpp;config/ConfigCheck.cpp;config/ConfigDownloader.cpp;config/ConfigErrorCollector.cpp;config/ConfigLog.cpp;config/ConfigPropertyEvaluator.cpp;config/ConfigValueQueue.cpp;config/ConfigurationException.cpp;config/ConfigurationState.cpp;config/PreprocessInputStream.cpp;config/config.cpp;config/defaults/MarketData.conf;config/defaults/Ports.conf;config/defaults/Symbols.conf;config/defaults/cme_reference_data.conf;config/defaults/edge.conf;config/defaults/jetfix.conf;config/defaults/keycode.conf;config/defaults/logging.conf;config/defaults/orderman_cfg.conf;config/defaults/orders_fills_cleanup.conf;config/defaults/position_cfg.conf;config/defaults/span.conf;config/defaults/spreader.conf;config/defaults/testkeys.conf;config/defaults/threads.conf;config/defaults/toggles.conf;config/extensions/extensions.proto;config/invalid/invalid.proto;config/protos/ExchangeSimCfg.proto;config/protos/MarketData.proto;config/protos/Ports.proto;config/protos/Symbols.proto;config/protos/UI.proto;config/protos/cme_reference_data.proto;config/protos/jetfix.proto;config/protos/keycode.proto;config/protos/logging.proto;config/protos/mercury_cfg.proto;config/protos/orderman_cfg.proto;config/protos/orders_fills_cleanup.proto;config/protos/perflog_cfg.proto;config/protos/position_cfg.proto;config/protos/quickbus_cfg.proto;config/protos/span.proto;config/protos/spreader.proto;config/protos/system.proto;config/protos/testkeys.proto;config/protos/threads.proto;config/protos/toggles.proto;config/tests/HTTPServer.cpp;config/tests/config_ref_tests.cpp;config/tests/config_tests.cpp;config/text_format/text_format.cpp;domain_model/Security.cpp;domain_model/Security.proto;domain_model/SecurityTests.cpp;domain_model/sod.cpp;domain_model/tests/domain_tests.cpp;expression/BooleanGrammar.cpp;expression/ExpressionEvaluator.cpp;expression/ParserExceptions.cpp;expression/Value.cpp;expression/tests/expression_parser_tests.cpp;logging/LoggerHub.cpp;logging/Sink.cpp;logging/logging.cpp;logging/tests/logging_tests.cpp;md/Diag/MDPeriodicDumper.cpp;md/Diag/ShmDumper.cpp;md/MBOProcessor.cpp;md/MBOServiceServer.cpp;md/MDClient.cpp;md/MDConsumer.cpp;md/MDConsumerHandle.cpp;md/MDConsumerInterop.cpp;md/MDProducer.cpp;md/MDProducerHandle.cpp;md/MemorySegmentUtils.cpp;md/PlatformLoader.cpp;md/PriceSize.cpp;md/SecurityCache.cpp;md/SecurityDB.cpp;md/SecurityIDRecorder.cpp;md/SecurityMapper.cpp;md/SecurityOracle.cpp;md/SettlementRecorder.cpp;md/Shm/FixedSecurity.cpp;md/SyntheticSecurityService.cpp;md/cme/MDP3Utils.cpp;md/common.cpp;md/ice/ICEUtils.cpp;md/protos/co2.proto;md/relay/QueryServer.cpp;md/relay/RelayClient.cpp;md/relay/RelayRecorder.cpp;md/test/Co2ManagedTestHelper.cpp;md/test/Co2TestInstrument.cpp;md/test/mbo_tests.cpp;md/test/md2_tests.cpp;md/test/md_test_utils.cpp;md/test/synthetic_book_merge_tests.cpp;md/vendor/OnixsUtils.cpp;os/os.cpp;os/tests/os_tests.cpp;quickbus/core/LocalNode.cpp;quickbus/core/Quickbus.cpp;quickbus/core/SubscriptionID.cpp;quickbus/core/TableID.cpp;quickbus/message_model/Update.cpp;quickbus/message_model/merge_updates.cpp;quickbus/network/QBConnection.cpp;quickbus/network/QBConnectionManager.cpp;quickbus/network/endpoint_description.cpp;quickbus/network/quickbus_msgs.proto;quickbus/pubsub/TablePublisher.cpp;quickbus/pubsub/TableSubscriber.cpp;quickbus/pubsub/tests/qb_pubsub_tests.cpp;quickbus/routing/INode.cpp;quickbus/routing/NodeID.cpp;quickbus/routing/Router.cpp;quickbus/rpc/RPCClient.cpp;quickbus/rpc/RPCServer.cpp;quickbus/rpc/RPCSessionID.cpp;quickbus/rpc/rpc_utils.cpp;quickbus/rpc/tests/qb_rpc_tests.cpp;quickbus/tests/quickbus_tests.cpp;sparkproto/SparkProtos.proto;testing/catch_test_runner.cpp;utils/AuthSessionManager.cpp;utils/AuthToken.cpp;utils/Dispatcher.cpp;utils/GrowableSPSCQueue.cpp;utils/HTTPStream.cpp;utils/PBUtils.cpp;utils/Puid.cpp;utils/SecureLogoLine truncated

src/lib/api/service_compiler/templates/c#/immutable.pb.cs

@@ -10,6 +10,7 @@ namespace {{package}} {
     {{#each message_type}}
     partial class {{name}} : bts.api.IImmutableMessage {
         MessageDescriptor bts.api.IImmutableMessage.Descriptor => {{name}}.Descriptor;
+        public {{name}}Immutable immutable() => new {{name}}Immutable(this);
     }
     public readonly partial struct {{name}}Immutable : bts.api.IImmutableMessage {
         readonly {{name}} _impl;

src/lib/api/service_helpers/AdminServiceUtils.cpp

@@ -46,12 +46,15 @@ bool AdminServiceUtils::has_exchange(const ExchangeConnection& conn,
       break;
    case api::Venue::VENUE_UNSET:
    case api::Venue::VENUE_SIM:
+   case api::Venue::VENUE_SIMICE:
       rv = conn.has_sim_connection();
+      break;      
+   case api::Venue::VENUE_ICE:
+      rv = conn.has_ice_connection();
       break;
    case api::Venue::VENUE_BTS:
    case api::Venue::VENUE_CBOE:
    case api::Venue::VENUE_CBOECFE:
-   case api::Venue::VENUE_ICE:
    case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
    case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:
       rv = false;
@@ -89,6 +92,23 @@ const ExchangeConnection* AdminServiceUtils::resolve_cme_connection(
    return nullptr;                                                               
 }
 
+const bts::api::admin::ExchangeConnection*
+AdminServiceUtils::resolve_ice_connection(const bts::api::admin::AdminData& data,
+                                          const bts::api::admin::BTSAccount& account,
+                                          const bts::api::Security&)
+{
+   if(account.has_connections())
+   {
+      for (const std::string& conn_name : account.connections().exchange_connection_names())
+      {
+         const bts::api::admin::ExchangeConnection* conn = find_connection(data, conn_name);
+         if (conn!=nullptr && conn->has_ice_connection())
+            return conn;  
+      }
+   }
+   return nullptr;                                                               
+}
+
 const bts::api::admin::ExchangeConnection*
 AdminServiceUtils::resolve_sim_connection(const bts::api::admin::AdminData& data,
                                           const bts::api::admin::BTSAccount& account,
@@ -117,14 +137,22 @@ AdminServiceUtils::resolve_connection(const bts::api::admin::AdminData& data,
          rv = resolve_sim_connection(data, account, sec);
       break;
    }
+   case api::Venue::VENUE_ICE: 
+   {      
+      rv = resolve_ice_connection(data, account, sec);
+      auto deployment_type = boost::to_lower_copy(config::get<config::DeploymentType>().type());
+      if (rv == nullptr && (deployment_type == "dev" || deployment_type == "demo"))
+         rv = resolve_sim_connection(data, account, sec);
+      break;
+   }   
    case api::Venue::VENUE_UNSET:
    case api::Venue::VENUE_BTS:
    case api::Venue::VENUE_SIM:
+   case api::Venue::VENUE_SIMICE:
       rv = resolve_sim_connection(data, account, sec);
       break;
    case api::Venue::VENUE_CBOE:
    case api::Venue::VENUE_CBOECFE:      
-   case api::Venue::VENUE_ICE:
    case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
       case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:
       rv = nullptr;
@@ -161,9 +189,15 @@ void AdminServiceUtils::fill_connections_from_config(bts::api::admin::AdminData&
       case config::JetFIX::Connection::CBOE:
          break;
       case config::JetFIX::Connection::SIM:
+      case config::JetFIX::Connection::SIMICE:
          connection.mutable_sim_connection();
          add_connection(admin_data,connection);
          break;
+      case config::JetFIX::Connection::ICE:
+         connection.mutable_ice_connection()->set_sender_compid(c.sender_compid());
+         add_connection(admin_data,connection);
+         break;
+         
       }
    }
    for(int i=0;i<admin_data.accounts_size();i++)
@@ -350,13 +384,16 @@ std::string AdminServiceUtils::find_account_exchange_account (
          rv = cme_account_data->clearing_account();
       break;
    }
-   case api::Venue::VENUE_ICE:{
+   case api::Venue::VENUE_ICE:
+   {
       auto ice_account_data = get_account_ice_data(account, data);
       if (ice_account_data != nullptr)
-         rv = ice_account_data->clearing_account_override();
-      break;
+         rv = ice_account_data->clearing_account();
+      break;      
    }
-   case api::Venue::VENUE_SIM: {
+   case api::Venue::VENUE_SIM: 
+   case api::Venue::VENUE_SIMICE:
+   {
       auto sim_account_data = get_account_sim_data(account, data);
       if (sim_account_data != nullptr)
          rv = sim_account_data->clearing_account();
@@ -466,6 +503,46 @@ bts::api::orderman::RoutingRecord AdminServiceUtils::get_routing_record(const bt
       return rv;
    }
    switch (sec.exchange()) {
+   case api::Venue::VENUE_ICE:
+   {
+      const bts::api::admin::ICEUserData* ice_user_data = get_user_ice_data(*user_ptr, data);
+      if(ice_user_data==nullptr)
+      {
+        out_result.set_is_success(false);
+        out_result.set_error_text("Could not find ICE data for user");
+        return rv;
+      }     
+      const bts::api::admin::ICEAccountData* ice_account_data = get_account_ice_data(*account_ptr, data);
+      if(ice_account_data==nullptr)
+      {
+         out_result.set_is_success(false);
+         out_result.set_error_text("Could not find ICE data for account");
+         return rv;
+      }        
+      // check for data corectness
+      if (ice_user_data->operator_id().length() == 0)
+      {
+         out_result.set_is_success(false);
+         out_result.set_error_text("Operator ID is not set");
+         return rv;
+      }
+
+      rv.set_session(conn_ptr->name());
+      rv.set_sender_compid(conn_ptr->ice_connection().sender_compid());
+      rv.set_account(ice_account_data->clearing_account());
+      rv.set_operator_id(ice_user_data->operator_id());   
+      rv.set_cti_code(ice_account_data->cti());
+      rv.set_customer_or_firm(ice_account_data->origin());
+      rv.set_clearing_firm(ice_account_data->clearing_firm());
+      rv.set_customer_account_ref_id(ice_account_data->customer_account_ref_id());
+      rv.set_account_code(ice_account_data->account_code());
+      rv.set_company_id(std::to_string(ice_user_data->company_id()));
+      rv.set_authorized_trader(ice_user_data->authorized_trader());
+      rv.set_authorized_group(ice_user_data->authorized_group());      
+      rv.set_cust_order_handing_inst(ice_user_data->cust_order_handing_inst());
+      
+      return rv;             
+   }
    case bts::api::Venue::VENUE_GLBX: {
      const bts::api::admin::CMEUserData* cme_user_data = get_user_cme_data(*user_ptr, data);
      if(cme_user_data==nullptr){
@@ -516,7 +593,9 @@ bts::api::orderman::RoutingRecord AdminServiceUtils::get_routing_record(const bt
      rv.set_customer_or_firm(cme_account_data->origin());
      return rv;
    }
-   case api::Venue::VENUE_SIM: {
+   case api::Venue::VENUE_SIM: 
+   case api::Venue::VENUE_SIMICE: 
+   {
      const bts::api::admin::SIMUserData* sim_user_data = get_user_sim_data(*user_ptr, data);
      if(sim_user_data==nullptr){
         out_result.set_is_success(false);
@@ -562,8 +641,7 @@ bts::api::orderman::RoutingRecord AdminServiceUtils::get_routing_record(const bt
    case api::Venue::VENUE_UNSET:
    case api::Venue::VENUE_BTS:
    case api::Venue::VENUE_CBOE:
-   case api::Venue::VENUE_CBOECFE:
-   case api::Venue::VENUE_ICE:
+   case api::Venue::VENUE_CBOECFE:   
    case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
    case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:
       out_result.set_is_success(false);
@@ -710,6 +788,18 @@ const bts::api::admin::CMEUserData* AdminServiceUtils::get_group_cme_data(const
    return nullptr;
 }
 
+const bts::api::admin::ICEUserData* AdminServiceUtils::get_group_ice_data(const bts::api::admin::BTSUserGroup& group,
+                                                                          const bts::api::admin::AdminData& data){
+   if (group.has_user_data() && group.user_data().has_exchange_user_data() && group.user_data().exchange_user_data().has_ice_data())
+      return &(group.user_data().exchange_user_data().ice_data());
+   else if (has_id(group.parent_group_id())){
+      const bts::api::admin::BTSUserGroup* parent_group = find_group(data, group.parent_group_id());
+      if (parent_group != nullptr)
+         return get_group_ice_data(*parent_group, data);
+   }
+   return nullptr;
+}
+
 const bts::api::admin::SIMUserData* AdminServiceUtils::get_group_sim_data(const bts::api::admin::BTSUserGroup& group,
                                                                           const bts::api::admin::AdminData& data){
    if (group.has_user_data() && group.user_data().has_exchange_user_data() && group.user_data().exchange_user_data().has_sim_data())
@@ -840,6 +930,18 @@ const bts::api::admin::CMEUserData* AdminServiceUtils::get_user_cme_data(const b
    return nullptr;
 }
 
+const bts::api::admin::ICEUserData* AdminServiceUtils::get_user_ice_data(const bts::api::admin::BTSUser& user, 
+                                                                         const bts::api::admin::AdminData& data){
+   if (user.has_user_data() && user.user_data().has_exchange_user_data() && user.user_data().exchange_user_data().has_ice_data())
+      return &(user.user_data().exchange_user_data().ice_data());
+   else{
+      const bts::api::admin::BTSUserGroup* parent_group = find_group(data, user.parent_group_id());
+         if (parent_group != nullptr)
+            return get_group_ice_data(*parent_group, data);
+   }
+   return nullptr;
+}
+
 const bts::api::admin::SIMUserData* AdminServiceUtils::get_user_sim_data(const bts::api::admin::BTSUser& user, 
                                                                          const bts::api::admin::AdminData& data){
    if (user.has_user_data() && user.user_data().has_exchange_user_data() && user.user_data().exchange_user_data().has_sim_data())
@@ -867,7 +969,16 @@ std::string AdminServiceUtils::find_user_exchange_id (const bts::api::admin::Adm
          rv = cme_user_data->operator_id();
       break;
    }
-   case api::Venue::VENUE_SIM: {
+   case api::Venue::VENUE_ICE:
+   {
+      auto ice_user_data = get_user_ice_data(user, data);
+      if (ice_user_data != nullptr)
+         rv = ice_user_data->operator_id();
+      break;      
+   }
+   case api::Venue::VENUE_SIM: 
+   case api::Venue::VENUE_SIMICE:
+   {
       auto sim_user_data = get_user_sim_data(user, data);
       if (sim_user_data != nullptr)
          rv = sim_user_data->operator_id();
@@ -877,8 +988,7 @@ std::string AdminServiceUtils::find_user_exchange_id (const bts::api::admin::Adm
    case api::Venue::VENUE_UNSET:
    case api::Venue::VENUE_BTS:
    case api::Venue::VENUE_CBOE:
-   case api::Venue::VENUE_CBOECFE:      
-   case api::Venue::VENUE_ICE:
+   case api::Venue::VENUE_CBOECFE:         
    case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
    case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:      
       out_result.set_is_success(false); out_result.set_error_text("Exchange not supported");
@@ -886,30 +996,92 @@ std::string AdminServiceUtils::find_user_exchange_id (const bts::api::admin::Adm
    }
    return rv;
 }
-std::pair<const bts::api::admin::BTSAccount*, const bts::api::admin::BTSUser*>
-AdminServiceUtils::find_user_account_matches (const bts::api::admin::AdminData& admin_data,
-                                              const std::string& connection, const std::string& exchange_account,
+
+std::string AdminServiceUtils::find_user_company_id (const bts::api::admin::AdminData& data,
+                                                     const bts::api::admin::BTSUser& user,
+                                                     api::Venue exchange,
+                                                     bts::api::admin::AdminServiceOperationResult& out_result)
+{
+   out_result.set_is_success(true);
+   std::string rv = "";
+   switch(exchange)
+   {
+   case api::Venue::VENUE_ICE:
+   {
+      auto ice_user_data = get_user_ice_data(user, data);
+      if (ice_user_data != nullptr)
+         rv = std::to_string(ice_user_data->company_id());
+      break;      
+   }
+   case api::Venue::VENUE_SIM: 
+   case api::Venue::VENUE_SIMICE:
+   case api::Venue::VENUE_GLBX:
+   case api::Venue::VENUE_UNSET:
+   case api::Venue::VENUE_BTS:
+   case api::Venue::VENUE_CBOE:
+   case api::Venue::VENUE_CBOECFE:         
+   case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_:
+   case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_:      
+      out_result.set_is_success(false); out_result.set_error_text("Exchange not supported");
+      break;
+   }
+   return rv;
+}
+
+std::pair<const BTSAccount*, const BTSUser*>
+AdminServiceUtils::find_user_account_matches (const AdminData& admin_data,
+                                              const std::string& connection,
+                                              const std::string& exchange_account,
                                               const std::string& exchange_user,
                                               api::Venue exchange){      
-   bts::api::admin::AdminServiceOperationResult find_account_result, find_user_result;
-   auto account_ptrs = accounts(admin_data, false);
-   auto user_ptrs = users(admin_data, false);
-   for(const bts::api::admin::BTSAccount* account_ptr : account_ptrs){
-      if (account_has_connection(*account_ptr, connection)){
-         if (boost::algorithm::iequals(exchange_account,
-                                       find_account_exchange_account(admin_data,*account_ptr,exchange,find_account_result)) &&
-                            find_account_result.is_success()){
-            for(const bts::api::admin::BTSUser* user_ptr : user_ptrs){
-               if (boost::algorithm::iequals(exchange_user,
-                                             find_user_exchange_id(admin_data,*user_ptr,exchange,find_user_result)) &&
-                   find_user_result.is_success())
-                  return std::pair<const bts::api::admin::BTSAccount*, const bts::api::admin::BTSUser*>(account_ptr,user_ptr);
-            }
-         }
-      }
+   AdminServiceOperationResult find_account_result, find_user_result;
+   // i. uses the exchange and exchange_user to reverse lookup the user(s)
+   auto user_ptrs = users(admin_data, false);   
+   std::vector<BTSUser const*> matching_users;
+   for (BTSUser const* u : user_ptrs){
+      auto exchange_u = find_user_exchange_id(admin_data, *u, 
+                                              exchange, find_user_result);
+      if (find_user_result.is_success() && boost::algorithm::iequals (
+                                                   exchange_u,
+                                                   exchange_user))
+          matching_users.push_back(u);
    }
-   return std::pair<const bts::api::admin::BTSAccount*, const bts::api::admin::BTSUser*>(nullptr, nullptr);
+   // i.5 
+   //At the ice, they send back the login level tag50 in exec reports which doesn't match any user level tag50s in the exchange_user
+   //so instead we look for the CompanyID (or client_id, tag 109) field here.  If this field matches we have a good bet that 
+   //the user we find is enough to fulfill the routing record enough to cancel or modify orphaned orders, etc   
+   for (BTSUser const* u : user_ptrs)
+   {
+      auto exchange_u = find_user_company_id(admin_data, *u, exchange, find_user_result);
+      if (find_user_result.is_success() && boost::algorithm::iequals ( exchange_u, exchange_user))
+         matching_users.push_back(u);
+   }   
+   // ii.  uses the exchange, connection and exchange_account to reverse lookup the account(s)
+   auto account_ptrs = accounts(admin_data, false);
+   std::vector<BTSAccount const*> matching_accounts;
+   for (BTSAccount const* a :account_ptrs)
+      if (account_has_connection(*a, connection)){
+         auto exchange_a = find_account_exchange_account(admin_data, *a,
+                                                         exchange,
+                                                         find_account_result);
+         if (find_account_result.is_success() &&  boost::algorithm::iequals (
+                                                      exchange_a,
+                                                      exchange_account))
+            matching_accounts.push_back(a);
+      }
+   // iii. finds the first account from ii that the user from i is authorized to trade
+   for (BTSUser const* u : matching_users){
+      std::vector<BTSAccount const*> user_auth_accounts;
+      find_user_trading_accounts (admin_data, u->id(), user_auth_accounts);
+      for (BTSAccount const* i : user_auth_accounts)
+         for (BTSAccount const* j : matching_accounts)
+            if(i->id() == j->id())
+               return std::pair<BTSAccount const*, BTSUser const*>(i,u);
+   }
+   return std::pair<const BTSAccount*, const BTSUser*>(nullptr, nullptr);
 }
+
+
 // for a given user finds the account permissions (if permissions are
 // derived from the group, it traverses the group hierarcy to find the
 // correct permissions.
@@ -995,16 +1167,26 @@ void AdminServiceUtils::validate_user_exchange_data (const bts::api::admin::BTSU
 }
 void AdminServiceUtils::get_md_entitlement_report(const AdminData& admin_data,
                           std::vector<MDEntitlementReportUnit>& entitlements){
-   for (const auto& user : admin_data.users()){
-      const CMEUserData* cme_data=AdminServiceUtils::get_user_cme_data(user,
-                                                                  admin_data);
-      if(cme_data!=nullptr){
-         for (const auto& cme_mic_data : cme_data->md_permissions()){
-            MDEntitlementReportUnit rv=get_cme_md_entitlement_unit(user,
-                                                                 cme_mic_data);
+   for (const auto& user : admin_data.users())
+   {
+      const CMEUserData* cme_data=AdminServiceUtils::get_user_cme_data(user, admin_data);
+      if(cme_data!=nullptr)
+      {
+         for (const auto& cme_mic_data : cme_data->md_permissions())
+         {
+            MDEntitlementReportUnit rv=get_cme_md_entitlement_unit(user, cme_mic_data);
             entitlements.push_back(rv);
          }
       }
+      const ICEUserData* ice_data=AdminServiceUtils::get_user_ice_data(user, admin_data);
+      if(ice_data!=nullptr)
+      {
+         for (const auto& ice_mic_data : ice_data->md_permissions())
+         {
+            MDEntitlementReportUnit rv=get_ice_md_entitlement_unit(user, ice_mic_data);
+            entitlements.push_back(rv);
+         }
+      }      
    }   
 }
 MDEntitlementReportUnit AdminServiceUtils::get_cme_md_entitlement_unit(
@@ -1052,6 +1234,22 @@ MDEntitlementReportUnit AdminServiceUtils::get_cme_md_entitlement_unit(
    rv.set_record_id(key.str());
    return rv;
 }
+MDEntitlementReportUnit AdminServiceUtils::get_ice_md_entitlement_unit(
+                                             const BTSUser& user,
+                                             const ICEMicData& ice_mic_data){
+   MDEntitlementReportUnit rv;
+   rv.set_user(user.name());
+   rv.set_user_id(user.id());
+   rv.set_exchange( bts::api::Venue_Name(bts::api::Venue::VENUE_ICE) );
+   rv.set_mic( bts::api::ICEMics_Name( ice_mic_data.mic()) );
+
+   rv.set_enabled(ice_mic_data.enabled());
+   std::ostringstream key;
+   key << rv.user() << rv.exchange() << rv.mic();
+   rv.set_record_id(key.str());
+   return rv;
+}
+
 void AdminServiceUtils::get_user_symbol_permissions(const AdminData& data,
                                                     const BTSUser& user,
                               std::vector<UserSymbolMDPermissionUnit>& rv){
@@ -1126,13 +1324,13 @@ void AdminServiceUtils::set_user_exchange_data(bts::api::admin::BTSUser* user,
    if (!user->user_data().has_exchange_user_data())
       user->mutable_user_data()->mutable_exchange_user_data()->CopyFrom(exchange_user_data);
 }
-void AdminServiceUtils::set_user_md_entitlement_data(bts::api::admin::BTSUser* user,
-                                                     const bts::api::admin::ExchangeUserMarketDataEntitlementData& exchange_user_md_entitelemnt_data){
-   if (!user->has_user_data())
-      user->mutable_user_data();
-   if (!user->user_data().has_exchange_md_entitlement_data())
-      user->mutable_user_data()->mutable_exchange_md_entitlement_data()->CopyFrom(exchange_user_md_entitelemnt_data);
-}
+// void AdminServiceUtils::set_user_md_entitlement_data(bts::api::admin::BTSUser* user,
+//                                                      const bts::api::admin::ExchangeUserMarketDataEntitlementData& exchange_user_md_entitelemnt_data){
+//    if (!user->has_user_data())
+//       user->mutable_user_data();
+//    if (!user->user_data().has_exchange_md_entitlement_data())
+//       user->mutable_user_data()->mutable_exchange_md_entitlement_data()->CopyFrom(exchange_user_md_entitelemnt_data);
+// }
 void AdminServiceUtils::clear_user_accounts (bts::api::admin::BTSUser* user){
    if(user->has_user_data() && user->user_data().has_account_permissions())
       user->mutable_user_data()->clear_account_permissions();
@@ -1377,6 +1575,7 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    auto account_1 = create_account(data, parent_account->id(), std::string("ACC A1"), std::string("description"), true, account_result); account_1->set_is_active(true);      
    auto account_2 = create_account(data, parent_account->id(), std::string("ACC A2"), std::string("description"), true, account_result); account_2->set_is_active(true);
    auto account_3 = create_account(data, parent_account->id(), std::string("ACC A3"), std::string("description"), true, account_result); account_3->set_is_active(true);
+   auto account_4 = create_account(data, parent_account->id(), std::string("BLUETST"), std::string("description"), true, account_result); account_4->set_is_active(true);
    create_account(data, parent_account->id(), std::string("ACC A4"), std::string("description"), true, account_result);
    // add group
    auto def_group = create_group(data, 0, std::string("default"), result);
@@ -1387,8 +1586,7 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    auto user_2 = create_user(data, def_group->id(), std::string("bts_2"), std::string("bts_2"), empty, empty,  false, license_data, license_entitlements, user_result); user_2->set_is_active(true);
    auto user_3 = create_user(data, def_group->id(), std::string("bts_3"), std::string("bts_3"), empty, empty, false, license_data, license_entitlements, user_result); user_3->set_is_active(true);
    //associate users
-   bts::api::admin::UserAccountPermissionUnit permission_unit;
-   permission_unit.mutable_trading_permission()->set_submit_market_orders(true);
+   bts::api::admin::UserAccountPermissionUnit permission_unit;   
    permission_unit.mutable_trading_permission()->set_has_permissions(true);
    permission_unit.mutable_admin_permission()->set_has_permissions(true);
    bts::api::admin::UserAccountPermissions account_permissions;
@@ -1396,6 +1594,7 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair<uint32_t, bts::api::admin::UserAccountPermissionUnit>(account_1->id(), permission_unit));
    account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair<uint32_t, bts::api::admin::UserAccountPermissionUnit>(account_2->id(), permission_unit));
    account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair<uint32_t, bts::api::admin::UserAccountPermissionUnit>(account_3->id(), permission_unit));      
+   account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair<uint32_t, bts::api::admin::UserAccountPermissionUnit>(account_4->id(), permission_unit));      
    set_user_accounts(user_1, account_permissions);
    set_user_accounts(user_2, account_permissions);
    set_user_accounts(user_3, account_permissions);
@@ -1405,6 +1604,12 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    exchange_account_data.mutable_cme_account_data()->set_cti(bts::api::orderman::CTICode::CTI_CODE_CTI_CODE1);
    exchange_account_data.mutable_cme_account_data()->set_origin(bts::api::orderman::CustomerOrFirm::CUSTOMER_OR_FIRM_FIRM);
    exchange_account_data.mutable_cme_account_data()->set_clearing_account(parent_account->name());
+   
+   exchange_account_data.mutable_ice_account_data()->set_cti(bts::api::orderman::CTICode::CTI_CODE_UNDEFINED);
+   exchange_account_data.mutable_ice_account_data()->set_origin(bts::api::orderman::CustomerOrFirm::CUSTOMER_OR_FIRM_FIRM);
+   exchange_account_data.mutable_ice_account_data()->set_clearing_account(parent_account->name());
+   exchange_account_data.mutable_ice_account_data()->set_clearing_firm("649");
+   
    exchange_account_data.mutable_sim_account_data()->set_cti(bts::api::orderman::CTICode::CTI_CODE_CTI_CODE1);
    exchange_account_data.mutable_sim_account_data()->set_origin(bts::api::orderman::CustomerOrFirm::CUSTOMER_OR_FIRM_FIRM);
    exchange_account_data.mutable_sim_account_data()->set_clearing_account(parent_account->name());
@@ -1412,33 +1617,87 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
 
 
    exchange_account_data.mutable_cme_account_data()->set_clearing_account(account_1->name());
+   exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_1->name());
    exchange_account_data.mutable_sim_account_data()->set_clearing_account(account_1->name());
    set_account_exchange_data(account_1, exchange_account_data);
 
    exchange_account_data.mutable_cme_account_data()->set_clearing_account(account_2->name());
+   exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_2->name());
    exchange_account_data.mutable_sim_account_data()->set_clearing_account(account_2->name());
    set_account_exchange_data(account_2, exchange_account_data);
    
    exchange_account_data.mutable_cme_account_data()->set_clearing_account(account_3->name());
+   exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_3->name());
    exchange_account_data.mutable_sim_account_data()->set_clearing_account(account_3->name());
    set_account_exchange_data(account_3, exchange_account_data);
 
+   exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_4->name());
+   set_account_exchange_data(account_4, exchange_account_data);
+   
+
    // add cme and sim user data
    bts::api::admin::ExchangeUserData exchange_user_data;
    exchange_user_data.mutable_cme_data()->set_operator_id("op_id_" + user_1->name());
    exchange_user_data.mutable_cme_data()->set_location("us,nc");
+   
+   exchange_user_data.mutable_ice_data()->set_operator_id("op_id_" + user_1->name());
+   exchange_user_data.mutable_ice_data()->set_company_id(51251);
+   exchange_user_data.mutable_ice_data()->set_authorized_trader("bluetrd-fx");
+   exchange_user_data.mutable_ice_data()->set_authorized_group("Group");
+   exchange_user_data.mutable_ice_data()->set_password("Starts123");
+   exchange_user_data.mutable_ice_data()->set_mifid_direct_electronic_access(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_trading_capacity(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_liquidity_provision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_commodity_deriv_indicator(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_investment_decision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_execution_decision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_client_id_code(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_id(1);
+      
    exchange_user_data.mutable_sim_data()->set_operator_id("op_id_" + user_1->name());
    exchange_user_data.mutable_sim_data()->set_location("us,nc");
    set_user_exchange_data(user_1, exchange_user_data);
 
    exchange_user_data.mutable_cme_data()->set_operator_id("op_id_" + user_2->name());
    exchange_user_data.mutable_cme_data()->set_location("us,nc");
+   
+   exchange_user_data.mutable_ice_data()->set_operator_id("op_id_" + user_2->name());
+   exchange_user_data.mutable_ice_data()->set_company_id(51251);
+   exchange_user_data.mutable_ice_data()->set_authorized_trader("bluetrd-fx");
+   exchange_user_data.mutable_ice_data()->set_authorized_group("Group");
+   exchange_user_data.mutable_ice_data()->set_password("Starts123");
+   exchange_user_data.mutable_ice_data()->set_mifid_direct_electronic_access(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_trading_capacity(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_liquidity_provision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_commodity_deriv_indicator(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_investment_decision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_execution_decision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_client_id_code(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_id(1);
+   
+   
    exchange_user_data.mutable_sim_data()->set_operator_id("op_id_" + user_2->name());
    exchange_user_data.mutable_sim_data()->set_location("us,nc");
    set_user_exchange_data(user_2, exchange_user_data);
 
    exchange_user_data.mutable_cme_data()->set_operator_id("op_id_" + user_3->name());
    exchange_user_data.mutable_cme_data()->set_location("us,nc");
+   
+   exchange_user_data.mutable_ice_data()->set_operator_id("op_id_" + user_3->name());
+   exchange_user_data.mutable_ice_data()->set_company_id(51251);
+   exchange_user_data.mutable_ice_data()->set_authorized_trader("bluetrd-fx");
+   exchange_user_data.mutable_ice_data()->set_authorized_group("Group");
+   exchange_user_data.mutable_ice_data()->set_password("Starts123");
+   exchange_user_data.mutable_ice_data()->set_mifid_direct_electronic_access(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_trading_capacity(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_liquidity_provision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_commodity_deriv_indicator(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_investment_decision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_execution_decision(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_client_id_code(-1);
+   exchange_user_data.mutable_ice_data()->set_mifid_id(1);
+   
+   
    exchange_user_data.mutable_sim_data()->set_operator_id("op_id_" + user_3->name());
    exchange_user_data.mutable_sim_data()->set_location("us,nc");
    set_user_exchange_data(user_3, exchange_user_data);
@@ -1480,6 +1739,13 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){
    add_security_clip_sizes_to_account(account_1, api::Venue::VENUE_GLBX, api::SecurityType::SECURITYTYPE_SPREAD,100);
    add_security_clip_sizes_to_account(account_1, api::Venue::VENUE_SIM, api::SecurityType::SECURITYTYPE_FUTURE,100);
    add_security_clip_sizes_to_account(account_1, api::Venue::VENUE_SIM, api::SecurityType::SECURITYTYPE_SPREAD,100);
+
+   //ICE limits   
+   add_product_position_limits(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_FUTURE,50000);
+   add_product_position_limits(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_SPREAD,50000);
+   add_security_clip_sizes_to_account(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_FUTURE,50);
+   add_security_clip_sizes_to_account(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_SPREAD,50);
+   
 }
 bool AdminServiceUtils::has_id (uint32_t id){return id>0;}
 bool AdminServiceUtils::is_symbol_set (const std::string& symbol){

src/lib/api/service_helpers/AdminServiceUtils.h

@@ -35,6 +35,10 @@ public:
                                                    const AdminData& data,
                                                    const BTSAccount& account,
                                                    const api::Security& sec);
+   static const ExchangeConnection* resolve_ice_connection(
+                                                   const AdminData& data,
+                                                   const BTSAccount& account,
+                                                   const api::Security& sec);                                                   
    static const ExchangeConnection* resolve_sim_connection(
                                                    const AdminData& data,
                                                    const BTSAccount& account,
@@ -164,6 +168,8 @@ public:
    // travels up the group tree until it finds a populated cme data 
    static const CMEUserData* get_group_cme_data(const BTSUserGroup& group,
                                                 const AdminData& data);
+   static const ICEUserData* get_group_ice_data(const BTSUserGroup& group,
+                                                const AdminData& data);                                                
    static const SIMUserData* get_group_sim_data(const BTSUserGroup& group, 
                                                 const AdminData& data);
    static const UserAccountPermissions* const get_group_user_permissions (
@@ -198,6 +204,8 @@ public:
    static uint32_t find_max_user_id (const AdminData& data);
    static const CMEUserData* get_user_cme_data(const BTSUser& user, 
                                                const AdminData& data);
+   static const ICEUserData* get_user_ice_data(const BTSUser& user, 
+                                               const AdminData& data);                                               
    static const SIMUserData* get_user_sim_data(const BTSUser& user, 
                                                const AdminData& data);
    // finds the "exchange user" (ex: "operator_id" at CME or "authorized_trader" 
@@ -207,6 +215,15 @@ public:
                                     const BTSUser& user,
                                     api::Venue exchange,
                                     AdminServiceOperationResult& out_result);
+   // i.   uses the exchange and exchange_user to reverse lookup the user(s)
+   // ii.  uses the exchange, connection and exchange_account to reverse lookup the account(s)
+   // iii. finds the first account from ii that a user from i is authorized to trade
+   static std::string find_user_company_id (
+                                    const AdminData& data,
+                                    const BTSUser& user,
+                                    api::Venue exchange,
+                                    AdminServiceOperationResult& out_result);
+                                    
    static std::pair<const BTSAccount*, const BTSUser*>
    find_user_account_matches (const AdminData& admin_data,
                               const std::string& connection,
@@ -246,6 +263,9 @@ public:
    static MDEntitlementReportUnit get_cme_md_entitlement_unit(
                                              const BTSUser& user,
                                              const CMEMicData& cme_mic_data);
+   static MDEntitlementReportUnit get_ice_md_entitlement_unit(
+                                             const BTSUser& user,
+                                             const ICEMicData& ice_mic_data);                                             
    static void get_user_symbol_permissions(
                                  const AdminData& data,
                                  const BTSUser& user,
@@ -272,8 +292,6 @@ public:
    static void set_user_exchange_data(
                                  BTSUser* user,
                                  const ExchangeUserData& exchange_user_data);
-   static void set_user_md_entitlement_data(BTSUser* user,
-                                 const ExchangeUserMarketDataEntitlementData&);
    static void clear_user_accounts (BTSUser* user);
    static void set_user_accounts (BTSUser* user,
                      const UserAccountPermissions& user_account_permissions);

src/lib/api/services/AdminService.proto

@@ -55,11 +55,12 @@ message BTSUser{
    string name = 30;
    string password = 40;
    string full_name = 42;
-   string company = 44;   
+   string company = 44;
    bool is_active = 50;
    bool can_create = 55;
-   LicenseData license_data = 60; //unused
-   LicenseEntitlements license_entitlements = 70; //unused
+   LicenseData license_data = 60;
+   LicenseEntitlements license_entitlements = 70;
+
    BTSUserDerivableData user_data = 80;
 }
 
@@ -76,7 +77,7 @@ message BTSUserDerivableData{
    // b. else its overridden at the user level
    UserAccountPermissions account_permissions = 70;
    ExchangeUserData exchange_user_data = 80;
-   ExchangeUserMarketDataEntitlementData exchange_md_entitlement_data = 90;
+   reserved 90; reserved "exchange_md_entitlement_data"; //ExchangeUserMarketDataEntitlementData exchange_md_entitlement_data = 90;
 }
 
 message LicenseData{
@@ -91,21 +92,22 @@ message LicenseEntitlements{
    bool spreader = 30;
 }
 
-message AlgoTradingPermissionsUnit{
-   bool submit_iceberg_orders = 10; //unused
-   bool submit_time_box_orders = 20; //unused
-   bool submit_spreader_orders = 30; //unused
-}
+// message AlgoTradingPermissionsUnit{
+//    bool submit_iceberg_orders = 10; //unused
+//    bool submit_time_box_orders = 20; //unused
+//    bool submit_spreader_orders = 30; //unused
+// }
 
 message TradingPermissionUnit{
+   reserved 10; reserved "submit_market_orders"; //bool submit_market_orders = 10; //unused
+   reserved 20; reserved "submit_gtc_orders"; //bool submit_gtc_orders = 20; //unused
+   reserved 30; reserved "algo_trading_permission"; //AlgoTradingPermissionsUnit algo_trading_permission = 30; //unused
+
    bool has_permissions = 5;
-   bool submit_market_orders = 10; //unused
-   bool submit_gtc_orders = 20; //unused  
-   AlgoTradingPermissionsUnit algo_trading_permission = 30; //unused
 }
 
 message AdminPermissionUnit{
-   // TODO: needs to be more ellaborate (account writing, 
+   // TODO: needs to be more ellaborate (account writing,
    // reading etc, routing etc)
    bool has_permissions = 10;
    bool has_md_permissions = 20;
@@ -119,15 +121,15 @@ message UserAccountPermissionUnit{
 
 message UserAccountPermissions{
    //uint32 = account_id
-   map<uint32, UserAccountPermissionUnit> account_permissions = 10;   
+   map<uint32, UserAccountPermissionUnit> account_permissions = 10;
 }
 
-// unused now
-message UserSecurityTradingLimit{
-   SecurityScope security = 10;
-   uint32 clip_size = 20;
-   bool can_trade = 30;   
-}
+// // unused now
+// message UserSecurityTradingLimit{
+//    SecurityScope security = 10;
+//    uint32 clip_size = 20;
+//    bool can_trade = 30;
+// }
 
 /************************************************************************/
 /********************************BTS Account*****************************/
@@ -152,7 +154,7 @@ message BTSAccount{
    AccountLossLimit account_loss_limit = 130;
 }
 
-message RiskLimitEnforcement{   
+message RiskLimitEnforcement{
    bool product_limit_in_net_terms = 10;
    bool ignore_spread_position_limits = 20;
 }
@@ -204,23 +206,22 @@ message AccountLossLimit{
 message CMEMicData{
    api.CMEMics mic=10;
    bool enabled=20;
-   // there are other things here   
+   // there are other things here
 }
 
 message ICEMicData{
-   api.CMEMics mic=10;
+   api.ICEMics mic=10;
    bool enabled=20;
 }
 
-message ExchangeUserMarketDataEntitlementData{
-   CMEUserMarketDataEntitlementData cme_data = 10;
-   ICEUserMarketDataEntitlementData ice_data = 20;
-}
 
 message ExchangeUserData{
    CMEUserData cme_data = 10;
    ICEUserData ice_data = 20;
    SIMUserData sim_data = 30;
+
+   enum AdminDisplay { All = 0; MD = 1; Routing = 2; None = 100; }
+   AdminDisplay admin_display_flag = 120;
 }
 
 message ExchangeAccountData{
@@ -229,9 +230,6 @@ message ExchangeAccountData{
    SIMAccountData sim_account_data = 30;
 }
 
-message MarketDataEntitlement{
-   bool can_view = 10;
-}
 
 message ExchangeConnection{
    uint32 id = 10;
@@ -274,19 +272,19 @@ message CMEMSGWConnection{
    uint32 market_segment_id = 60;
 }
 
-message CMEAccountData{
-   // should perhaps be an enum
+message CMEAccountData {
+   reserved 30; reserved "takeup_member_code"; //string takeup_member_code = 30;
+   reserved 50; reserved "direct_electronic_access"; //string direct_electronic_access = 50;
+   reserved 60; reserved "trading_capacity"; //string trading_capacity = 60;
+   reserved 70; reserved "liquidity_provision"; //string liquidity_provision = 70;
+   reserved 80; reserved "commodity_derivative_indicator"; //string commodity_derivative_indicator = 80;
+   reserved 90; reserved "investment_decision_id"; //string investment_decision_id = 90;
+   reserved 100; reserved "execution_decision_id"; //string execution_decision_id = 100;
+   reserved 110; reserved "client_id"; //string client_id = 110;
+
    api.orderman.CTICode cti = 10;
    api.orderman.CustomerOrFirm origin = 20;
-   string takeup_member_code = 30;
    string clearing_account = 40;
-   string direct_electronic_access = 50;
-   string trading_capacity = 60;
-   string liquidity_provision = 70;
-   string commodity_derivative_indicator = 80;
-   string investment_decision_id = 90;
-   string execution_decision_id = 100;
-   string client_id = 110;
    map<string, string> misc_tags = 120;
 }
 
@@ -309,21 +307,22 @@ enum DisplayDesignation
 }
 
 message CMEUserData{
+   reserved 20; reserved "ats_operator_id"; //string ats_operator_id = 20;
+   reserved 40; reserved "text_a"; //string text_a = 40;
+   reserved 50; reserved "text_b"; //string text_b = 50;
+   reserved 80; reserved "max_order_rate_per_connection"; //string max_order_rate_per_connection = 80;
+   //enum AdminDisplay { All = 0; MD = 1; Routing = 2; None = 100; }
+   reserved 120; reserved "admin_display_flag"; //AdminDisplay admin_display_flag = 120;
+   repeated CMEMicData md_permissions = 1000;
+
    string operator_id = 10;
-   string ats_operator_id = 20;
    string smp_id = 30;
-   string text_a = 40;
-   string text_b = 50;
    map<string, string> misc_tags = 60;
    string smp_instruction = 70;
-   string max_order_rate_per_connection = 80;
+
    string location = 90;
    ProfessionalDesignation pro_nonpro = 100;
    DisplayDesignation display_nondisplay = 110;
-   
-   enum AdminDisplay { All = 0; MD = 1; Routing = 2; None = 100; } 
-   AdminDisplay admin_display_flag = 120; 
-   repeated CMEMicData md_permissions = 1000;
 }
 
 message SIMUserData {
@@ -331,58 +330,67 @@ message SIMUserData {
    string location = 90;
 }
 
-message CMEUserMarketDataEntitlementData{
-   MarketDataEntitlement cbot_entitlement = 10;
-   MarketDataEntitlement cme_entitlement = 20;
-   MarketDataEntitlement nymex_entitlement = 30;
-   MarketDataEntitlement bmandf_entitlement = 40;
-   MarketDataEntitlement comex_entitlement = 50;
-   MarketDataEntitlement cme_europe_entitlement = 60;
-}
-
 /********************************ICE Data*********************************/
 
-message ICEConnection{
-   string authorized_member_id = 10;
-   string sender_company_id = 20;
-   string sender_sub_id = 30;
-   string session_id = 40;
-   string session_password = 50;
-   string host_ip = 60;
-   string port = 70;
-   string default_routing_trader = 80;
-   string routing_trader_password = 90;
-   int32 max_order_rate = 100;
+message ICEConnection {
+   string sender_compid = 10;
 }
 
-message ICEAccountData{
-   string nybit_house = 10;
-   int32 wce_house = 20;
-   string clearing_account_override = 30;
-   bool direct_electronic_access = 40;
-   string trading_capacity = 50;
-   bool liquidity_provision = 60;
-   bool commodity_derivative_indicator = 70;
-   string investment_decision_id = 80;
-   string execution_decision_id = 90;
-   string client_id = 100;
-   string mfid_id = 110;
+message ICEAccountData {
+   reserved 10; reserved "nybit_house"; //string nybit_house = 10;
+   reserved 20; reserved "wce_house"; //int32 wce_house = 20;
+   reserved 30; reserved "clearing_account_override"; //string clearing_account_override = 30;
+   reserved 40; reserved "direct_electronic_access"; //bool direct_electronic_access = 40;
+   reserved 50; reserved "trading_capacity"; //string trading_capacity = 50;
+   reserved 60; reserved "liquidity_provision"; //bool liquidity_provision = 60;
+   reserved 70; reserved "commodity_derivative_indicator"; //bool commodity_derivative_indicator = 70;
+   reserved 80; reserved "investment_decision_id"; //string investment_decision_id = 80;
+   reserved 90; reserved "execution_decision_id"; //string execution_decision_id = 90;
+   reserved 100; reserved "client_id"; //string client_id = 100;
+   reserved 110; reserved "mfid_id"; //string mfid_id = 110;
+
+   api.orderman.CTICode cti = 3;
+   api.orderman.CustomerOrFirm origin = 5;
+   string clearing_firm = 200;
+   string clearing_account = 210;
+   string customer_account_ref_id = 220;
+   string account_code = 230;
+   map<string, string> misc_tags = 500;
 }
 
-message ICEUserData{
-   string authorized_trader = 10;
-   string memo_field = 20;
-   string text_b = 30;
-   uint32 max_order_rate = 40;
-}
+message ICEUserData {
+   reserved 20; reserved "memo_field"; //string memo_field = 20;
+   reserved 30; reserved "text_b"; //string text_b = 30;
+   reserved 40; reserved "max_order_rate"; //uint32 max_order_rate = 40;
 
-message ICEUserMarketDataEntitlementData{
+   int32 company_id = 3; //tag 109 clientid
+   string operator_id = 5; //tag 50
+   string authorized_trader = 10; //tag 553 user_name
+   string authorized_group = 15; //tag 553 user_name
+   string password = 21; //tag 96 raw data
+   string cust_order_handing_inst = 25;
+
+   //mifid
+   int32 mifid_direct_electronic_access = 500;
+   int32 mifid_trading_capacity = 510;
+   int32 mifid_liquidity_provision = 520;
+   int32 mifid_commodity_deriv_indicator = 530;
+   int64 mifid_investment_decision = 540;
+   int64 mifid_execution_decision = 550;
+   int64 mifid_client_id_code = 560;
+   int64 mifid_id = 570;
+
+   map<string, string> misc_tags = 60;
+   ProfessionalDesignation pro_nonpro = 100;
+   DisplayDesignation display_nondisplay = 110;
+
+   repeated ICEMicData md_permissions = 1000;
 }
 
 ///////////////////////////////////////////////////////////
 // Order Auxiliary Service ////////////////////////////////
 
-message OrdersSummaryUnit{   
+message OrdersSummaryUnit{
    string account = 10;
    string user = 20;
    uint32 total_orders = 30;
@@ -393,11 +401,11 @@ message OrdersSummary{
 }
 message OrderSummaryRequest{
 }
-// top level orders that have account 
+// top level orders that have account
 // and user that are present here, will
 // be cancelled. wild cards are not
 // supported.
-message OrderCancelRequest{   
+message OrderCancelRequest{
    repeated string accounts = 10;
    repeated string users = 20;
 }
@@ -433,7 +441,7 @@ message AccountCreateAPIArgument{
    string creator_user = 40;
 }
 message AccountUpdateAPIArgument{
-   BTSAccount complete_data=10;    
+   BTSAccount complete_data=10;
 }
 message UserCreateAPIArgument{
    uint32 parent_group_id = 20;
@@ -477,7 +485,7 @@ service AdminService {
    // users
    rpc create_user(UserCreateAPIArgument) returns (AdminServiceCreateEntityResult);
    rpc update_user(UserUpdateAPIArgument) returns (AdminServiceOperationResult);
-   rpc delete_user(IDAPIArgument) returns (AdminServiceOperationResult);   
+   rpc delete_user(IDAPIArgument) returns (AdminServiceOperationResult);
    // subscription
    rpc admin_data(bts.api.VoidMessage) returns (stream AdminData);
    rpc get_admin_data(bts.api.VoidMessage) returns (AdminData);

src/lib/api/services/AuthService.proto

@@ -57,4 +57,7 @@ service AuthService {
 
    rpc get_users(GetUsersRequest) returns (GetUsersResponse)
    {option (bts.api.method_description)="Returns list of valid users";}
+   
+   rpc clear_all_sessions(bts.api.VoidMessage) returns (bts.api.ResultMessage)
+   {option (bts.api.method_description)="Clears all reserved auth sessions";}   
 }

src/lib/api/services/ErrorReportService.proto

@@ -4,6 +4,11 @@ option csharp_namespace="bts.support";
 import "lib/api/apioptions.proto";
 import "google/protobuf/timestamp.proto";
 
+message FileData {
+    string name = 1;
+    bytes contents = 2;
+}
+
 message ErrorReport {
     google.protobuf.Timestamp timestamp = 1; 
     string user = 2;
@@ -13,6 +18,7 @@ message ErrorReport {
     string log = 6;
     bytes  image = 7;
     string contact = 8;
+    repeated FileData files = 9;
 }
 message ErrorResponse {
 }

src/lib/api/services/OrderService.proto

@@ -310,6 +310,13 @@ message Fill {
     FILL_KIND_SOD = 2;
   }
   FillKind fill_kind = 10;
+  
+  enum FillStatus { 
+     FILL_STATUS_ACTIVE = 0;
+     FILL_STATUS_CORRECTED = 1;
+     FILL_STATUS_BUSTED = 2;
+  }
+  FillStatus fill_status = 20;
 }
 
 enum OrderSubType {
@@ -327,6 +334,17 @@ message RoutingRecord {
   string smp_id = 6 ;
   SelfMatchPreventionInstruction smp_instruction = 7;
   string location = 8;
+  string cust_order_handling_inst = 9;
+  
+  string sender_compid = 15;
+  string clearing_firm = 20;
+  string customer_account_ref_id = 25;
+  string account_code = 30;    
+  string company_id = 45;
+  string authorized_trader = 50;  
+  string authorized_group = 60;
+  string cust_order_handing_inst = 70;
+  string order_server_id = 200;
 }
 
 message NativeOrderState {

src/lib/config/ConfigurationState.cpp

@@ -24,6 +24,7 @@
 #include <map>
 #include <tuple>
 
+
 #define URDL_HEADER_ONLY 1
 #define URDL_DISALE_SSL 1
 #include <urdl/url.hpp>
@@ -908,3 +909,4 @@ std::string get_fallback(const proto::Message& message)
 }
 
 }}}//end namespaces
+

src/lib/config/defaults/MarketData.conf

@@ -62,15 +62,44 @@ ICEOptions {
    value {
       channel {
          name: "oil" #these names are arbitrary and are for debugging
+         market_type: 3 
+         uds_market_type: 3
+         market_type: 4 #ICE IPE Gas Oil Futures
+         uds_market_type: 4         
          market_type: 5 #ICE Futures Brent, IPE Brent Futures
-         market_type: 9 #ICE Futures WTI, ICE WTI Crude Futures
-         #udp_buffer_size: 33554432 #32mb
-      }
-      channel {
-         name: "juice"
-         market_type: 23 #ICE Futures US Agricultures, FCOJ A
+         uds_market_type: 5
+         market_type: 9 #ICE Futures WTI, ICE WTI Crude Futures                  
+         uds_market_type: 9
+         market_type: 13 #ICE Heating Oil Futures         
+         uds_market_type: 13
+         market_type: 15 #ICE NYH (RBOB) Gasoline Futures
+         uds_market_type: 15
+         ##market_type: 21 #ICE Futures US Agricultures, Coffee C
+         ##uds_market_type: 21
+         ##market_type: 23 #ICE Futures US Agricultures, FCOJ A
+         ##uds_market_type: 23               
+         
+         ##market_type: 26 #ICE Futures US Financials DX
+         ##uds_market_type: 26
+         
+         uds_market_type: 11 #ICE WTI-Brent spreads
+         uds_market_type: 14 #ICE Heating Oil/WTI Futures Crack
+         uds_market_type: 16 #ICE NYH (RBOB) Gasoline/WTI Futures Crack         
+         uds_market_type: 58 #ICE Futures UI Heating Oil / Brent
+         uds_market_type: 59 #ICE NYH (RBOB) Gasoline/Brent Futures Crack 
+         uds_market_type: 66 #ICE NYH (RBOB) Gasoline/Heating Oil Spread
+         uds_market_type: 70 #ICE Heating Oil/LS Gasoil Futures Spread
+         uds_market_type: 71 #ICE LS Gasoil/Brent Futures Crack
+         uds_market_type: 72 #ICE NYH (RBOB) Gasoline/LS Gasoil Futures Spread         
+         
+         #onixs_log_mode: Debug
       }
+      # channel {
+      #    name: "juice"
+      #    market_type: 26 #ICE Futures US Agricultures, FCOJ A
+      # }
       ice_config_file: "ConnectivityConfiguration.APITest.xml"      
+      #ice_config_file: "ConnectivityConfiguration.PerfTest.xml"      
    }
 }
 

src/lib/config/defaults/Ports.conf

@@ -3,7 +3,8 @@ Ports {
    value {
       config_port: 8000
       qbus_port: 8001
-      exchange_sim_fix_port: 8002
+      glbx_sim_fix_port: 8002
+      ice_sim_fix_port: 8003
       replicator_port: 8080
    }
 }

src/lib/config/defaults/Symbols.conf

@@ -19,7 +19,6 @@ SymbolList {
         symbol: "FOB"
         symbol: "GC"
         symbol: "GE"
-        symbol: "GLI"
         symbol: "HE"
         symbol: "HG"
         symbol: "HO"
@@ -66,9 +65,16 @@ SymbolLists
       list
       {
          key: "ICE"
-         # symbol: "BRN"
-         # symbol: "WBS"
-         # symbol: "OJ"
+         symbol: "BRN"
+         symbol: "WBS"
+         symbol: "OJ"
+         symbol: "G"
+         symbol: "KC"
+         symbol: "UHO"
+         symbol: "UHU"
+         symbol: "DX"
+         
+         symbol: "*"
       }
       list
       {
@@ -112,5 +118,9 @@ Symbols {
             future: @SymbolLists.list[ICE].symbol
             spread: @SymbolLists.list[ICE].symbol
         }
+        simice {
+            future: @SymbolLists.list[ICE].symbol
+            spread: @SymbolLists.list[ICE].symbol            
+        }
     }
 }

src/lib/config/defaults/cme_reference_data.conf

@@ -3,9 +3,9 @@
 # source: tools/cpp_helper/generate-cpps.ps1
 #
 
-CppConfig
-{
-	value
+CppConfig
+{
+	value
 	{
 		cpp_entry { key: "098" value: 2000 }
 		cpp_entry { key: "0A" value: 100 }
@@ -354,6 +354,7 @@ CppConfig
 		cpp_entry { key: "BOB" value: 1000 }
 		cpp_entry { key: "BOO" value: 6350 }
 		cpp_entry { key: "BOS" value: 250 }
+		cpp_entry { key: "BPU" value: 1000 }
 		cpp_entry { key: "BR7" value: 1000 }
 		cpp_entry { key: "BSF" value: 25 }
 		cpp_entry { key: "BTC" value: 5 }
@@ -442,12 +443,12 @@ CppConfig
 		cpp_entry { key: "D3L" value: 80 }
 		cpp_entry { key: "D4L" value: 5 }
 		cpp_entry { key: "D7L" value: 80 }
+		cpp_entry { key: "DAB" value: 1000 }
 		cpp_entry { key: "DAX" value: 5 }
 		cpp_entry { key: "DAZ" value: 80 }
 		cpp_entry { key: "DBB" value: 1000 }
 		cpp_entry { key: "DBI" value: 1000 }
 		cpp_entry { key: "DBL" value: 100 }
-		cpp_entry { key: "DBR" value: 1000 }
 		cpp_entry { key: "DBT" value: 100 }
 		cpp_entry { key: "DC" value: 2000 }
 		cpp_entry { key: "DCB" value: 1000 }
@@ -458,14 +459,13 @@ CppConfig
 		cpp_entry { key: "DDE" value: 1000 }
 		cpp_entry { key: "DDI" value: 1000 }
 		cpp_entry { key: "DDO" value: 1000 }
-		cpp_entry { key: "DEB" value: 744 }
+		cpp_entry { key: "DEB" value: 720 }
 		cpp_entry { key: "DEN" value: 250 }
-		cpp_entry { key: "DEP" value: 264 }
+		cpp_entry { key: "DEP" value: 252 }
 		cpp_entry { key: "DFL" value: 100 }
 		cpp_entry { key: "DFN" value: 100 }
 		cpp_entry { key: "DG2" value: 100 }
 		cpp_entry { key: "DG3" value: 100 }
-		cpp_entry { key: "DGH" value: 1000 }
 		cpp_entry { key: "DGI" value: 100 }
 		cpp_entry { key: "DGS" value: 100 }
 		cpp_entry { key: "DGT" value: 100 }
@@ -473,19 +473,13 @@ CppConfig
 		cpp_entry { key: "DHI" value: 1000 }
 		cpp_entry { key: "DJR" value: 100 }
 		cpp_entry { key: "DKR" value: 1000 }
-		cpp_entry { key: "DMB" value: 100 }
-		cpp_entry { key: "DOB" value: 1000 }
-		cpp_entry { key: "DOP" value: 1000 }
-		cpp_entry { key: "DRB" value: 1000 }
 		cpp_entry { key: "DRI" value: 100 }
 		cpp_entry { key: "DRS" value: 100 }
 		cpp_entry { key: "DRT" value: 100 }
 		cpp_entry { key: "DSD" value: 1000 }
 		cpp_entry { key: "DSE" value: 1000 }
-		cpp_entry { key: "DSG" value: 1000 }
 		cpp_entry { key: "DTR" value: 2 }
 		cpp_entry { key: "DTT" value: 2 }
-		cpp_entry { key: "DUA" value: 6350 }
 		cpp_entry { key: "DUP" value: 1000 }
 		cpp_entry { key: "DVE" value: 200 }
 		cpp_entry { key: "DVT" value: 200 }
@@ -530,8 +524,11 @@ CppConfig
 		cpp_entry { key: "EP1" value: 1000 }
 		cpp_entry { key: "EPN" value: 1000 }
 		cpp_entry { key: "EPZ" value: 500000 }
+		cpp_entry { key: "EQ1" value: 50 }
 		cpp_entry { key: "ERL" value: 5 }
 		cpp_entry { key: "ES" value: 50 }
+		cpp_entry { key: "ES1" value: 50 }
+		cpp_entry { key: "ES2" value: 50 }
 		cpp_entry { key: "ESB" value: 1000 }
 		cpp_entry { key: "ESK" value: 125000 }
 		cpp_entry { key: "ESQ" value: 50 }
@@ -733,8 +730,8 @@ CppConfig
 		cpp_entry { key: "IPO" value: 10 }
 		cpp_entry { key: "IPT" value: 10 }
 		cpp_entry { key: "IS" value: 50 }
-		cpp_entry { key: "ITB" value: 744 }
-		cpp_entry { key: "ITP" value: 264 }
+		cpp_entry { key: "ITB" value: 720 }
+		cpp_entry { key: "ITP" value: 252 }
 		cpp_entry { key: "IW" value: 50 }
 		cpp_entry { key: "J4L" value: 80 }
 		cpp_entry { key: "J7" value: 6250000 }
@@ -748,7 +745,7 @@ CppConfig
 		cpp_entry { key: "JKB" value: 10000 }
 		cpp_entry { key: "JKF" value: 10000 }
 		cpp_entry { key: "JKM" value: 10000 }
-		cpp_entry { key: "JML" value: 480 }
+		cpp_entry { key: "JML" value: 240 }
 		cpp_entry { key: "JNC" value: 1000 }
 		cpp_entry { key: "JNG" value: 10000 }
 		cpp_entry { key: "JNL" value: 80 }
@@ -776,6 +773,7 @@ CppConfig
 		cpp_entry { key: "K5" value: 20 }
 		cpp_entry { key: "K5K" value: 20 }
 		cpp_entry { key: "K5N" value: 20 }
+		cpp_entry { key: "KC" value: 375 }
 		cpp_entry { key: "KE" value: 50 }
 		cpp_entry { key: "KET" value: 50 }
 		cpp_entry { key: "KQ" value: 20 }
@@ -1005,6 +1003,7 @@ CppConfig
 		cpp_entry { key: "OAD" value: 1 }
 		cpp_entry { key: "ODO" value: 1000 }
 		cpp_entry { key: "OFF" value: 5 }
+		cpp_entry { key: "OJ" value: 150 }
 		cpp_entry { key: "OML" value: 5 }
 		cpp_entry { key: "OMM" value: 80 }
 		cpp_entry { key: "OOD" value: 5 }
@@ -1147,7 +1146,7 @@ CppConfig
 		cpp_entry { key: "SBT" value: 50 }
 		cpp_entry { key: "SD" value: 1000 }
 		cpp_entry { key: "SDA" value: 250 }
-		cpp_entry { key: "SDD" value: 1760 }
+		cpp_entry { key: "SDD" value: 1600 }
 		cpp_entry { key: "SDG" value: 250 }
 		cpp_entry { key: "SDI" value: 1000 }
 		cpp_entry { key: "SDM" value: 100 }
@@ -1186,7 +1185,7 @@ CppConfig
 		cpp_entry { key: "SR3" value: 2500 }
 		cpp_entry { key: "SR5" value: 1000 }
 		cpp_entry { key: "SRB" value: 1000 }
-		cpp_entry { key: "SRT" value: 1760 }
+		cpp_entry { key: "SRT" value: 1600 }
 		cpp_entry { key: "SSI" value: 1000 }
 		cpp_entry { key: "SSP" value: 5000 }
 		cpp_entry { key: "SSW" value: 1000 }
@@ -1197,27 +1196,28 @@ CppConfig
 		cpp_entry { key: "STY" value: 1000 }
 		cpp_entry { key: "SU" value: 250 }
 		cpp_entry { key: "SUT" value: 250 }
+		cpp_entry { key: "T01" value: 800 }
+		cpp_entry { key: "T02" value: 800 }
 		cpp_entry { key: "T03" value: 800 }
 		cpp_entry { key: "T04" value: 800 }
-		cpp_entry { key: "T06" value: 800 }
 		cpp_entry { key: "T07" value: 800 }
 		cpp_entry { key: "T08" value: 800 }
 		cpp_entry { key: "T09" value: 800 }
+		cpp_entry { key: "T10" value: 800 }
+		cpp_entry { key: "T11" value: 800 }
 		cpp_entry { key: "T12" value: 800 }
 		cpp_entry { key: "T13" value: 800 }
-		cpp_entry { key: "T14" value: 800 }
-		cpp_entry { key: "T15" value: 800 }
 		cpp_entry { key: "T16" value: 800 }
+		cpp_entry { key: "T17" value: 800 }
+		cpp_entry { key: "T18" value: 800 }
 		cpp_entry { key: "T19" value: 800 }
 		cpp_entry { key: "T1U" value: 1000 }
 		cpp_entry { key: "T20" value: 800 }
-		cpp_entry { key: "T21" value: 800 }
-		cpp_entry { key: "T22" value: 800 }
 		cpp_entry { key: "T23" value: 800 }
+		cpp_entry { key: "T24" value: 800 }
+		cpp_entry { key: "T25" value: 800 }
 		cpp_entry { key: "T26" value: 800 }
 		cpp_entry { key: "T27" value: 800 }
-		cpp_entry { key: "T28" value: 800 }
-		cpp_entry { key: "T29" value: 800 }
 		cpp_entry { key: "T2B" value: 1000 }
 		cpp_entry { key: "T2D" value: 1000 }
 		cpp_entry { key: "T2M" value: 100 }
@@ -1276,7 +1276,7 @@ CppConfig
 		cpp_entry { key: "TT" value: 50000 }
 		cpp_entry { key: "TTD" value: 24 }
 		cpp_entry { key: "TTE" value: 10000 }
-		cpp_entry { key: "TTF" value: 744 }
+		cpp_entry { key: "TTF" value: 720 }
 		cpp_entry { key: "TUB" value: 1000 }
 		cpp_entry { key: "TUF" value: 1000 }
 		cpp_entry { key: "TUL" value: 1000 }
@@ -1295,12 +1295,11 @@ CppConfig
 		cpp_entry { key: "UDL" value: 80 }
 		cpp_entry { key: "UFB" value: 100 }
 		cpp_entry { key: "UFE" value: 100 }
-		cpp_entry { key: "UFG" value: 100 }
 		cpp_entry { key: "UFN" value: 100 }
 		cpp_entry { key: "UFU" value: 100 }
 		cpp_entry { key: "UFV" value: 100 }
 		cpp_entry { key: "UGG" value: 42000 }
-		cpp_entry { key: "UKG" value: 310 }
+		cpp_entry { key: "UKG" value: 300 }
 		cpp_entry { key: "ULB" value: 42000 }
 		cpp_entry { key: "UMB" value: 1000 }
 		cpp_entry { key: "UMD" value: 1000 }
@@ -1308,6 +1307,9 @@ CppConfig
 		cpp_entry { key: "UN" value: 1000 }
 		cpp_entry { key: "UNB" value: 1000 }
 		cpp_entry { key: "UNS" value: 1000 }
+		cpp_entry { key: "UP5" value: 1000 }
+		cpp_entry { key: "UPB" value: 1000 }
+		cpp_entry { key: "UPM" value: 1000 }
 		cpp_entry { key: "UR" value: 1000 }
 		cpp_entry { key: "USC" value: 1000 }
 		cpp_entry { key: "USDBRL" value: 1 }
@@ -1455,12 +1457,12 @@ CppConfig
 		cpp_entry { key: "ZT" value: 2000 }
 		cpp_entry { key: "ZW" value: 50 }
 		cpp_entry { key: "ZWT" value: 50 }
-	}
-}
+	}
+}
 
-CmeSymbolMics
-{
-	value
+CmeSymbolMics
+{
+	value
 	{
 		cme_mic { key: "098" value: CBT }
 		cme_mic { key: "0A" value: NYMEX }
@@ -1809,6 +1811,7 @@ CmeSymbolMics
 		cme_mic { key: "BOB" value: CBT }
 		cme_mic { key: "BOO" value: NYMEX }
 		cme_mic { key: "BOS" value: CME }
+		cme_mic { key: "BPU" value: NYMEX }
 		cme_mic { key: "BR7" value: NYMEX }
 		cme_mic { key: "BSF" value: CBT }
 		cme_mic { key: "BTC" value: CME }
@@ -1897,12 +1900,12 @@ CmeSymbolMics
 		cme_mic { key: "D3L" value: NYMEX }
 		cme_mic { key: "D4L" value: NYMEX }
 		cme_mic { key: "D7L" value: NYMEX }
+		cme_mic { key: "DAB" value: NYMEX }
 		cme_mic { key: "DAX" value: NYMEX }
 		cme_mic { key: "DAZ" value: NYMEX }
 		cme_mic { key: "DBB" value: NYMEX }
 		cme_mic { key: "DBI" value: DME }
 		cme_mic { key: "DBL" value: NYMEX }
-		cme_mic { key: "DBR" value: DME }
 		cme_mic { key: "DBT" value: NYMEX }
 		cme_mic { key: "DC" value: CME }
 		cme_mic { key: "DCB" value: NYMEX }
@@ -1920,7 +1923,6 @@ CmeSymbolMics
 		cme_mic { key: "DFN" value: CBT }
 		cme_mic { key: "DG2" value: CBT }
 		cme_mic { key: "DG3" value: CBT }
-		cme_mic { key: "DGH" value: DME }
 		cme_mic { key: "DGI" value: CBT }
 		cme_mic { key: "DGS" value: CBT }
 		cme_mic { key: "DGT" value: CBT }
@@ -1928,19 +1930,13 @@ CmeSymbolMics
 		cme_mic { key: "DHI" value: DME }
 		cme_mic { key: "DJR" value: CBT }
 		cme_mic { key: "DKR" value: DME }
-		cme_mic { key: "DMB" value: DME }
-		cme_mic { key: "DOB" value: DME }
-		cme_mic { key: "DOP" value: DME }
-		cme_mic { key: "DRB" value: DME }
 		cme_mic { key: "DRI" value: CBT }
 		cme_mic { key: "DRS" value: CBT }
 		cme_mic { key: "DRT" value: CBT }
 		cme_mic { key: "DSD" value: DME }
 		cme_mic { key: "DSE" value: DME }
-		cme_mic { key: "DSG" value: DME }
 		cme_mic { key: "DTR" value: CBT }
 		cme_mic { key: "DTT" value: CBT }
-		cme_mic { key: "DUA" value: DME }
 		cme_mic { key: "DUP" value: DME }
 		cme_mic { key: "DVE" value: CME }
 		cme_mic { key: "DVT" value: CME }
@@ -1985,8 +1981,11 @@ CmeSymbolMics
 		cme_mic { key: "EP1" value: NYMEX }
 		cme_mic { key: "EPN" value: NYMEX }
 		cme_mic { key: "EPZ" value: CME }
+		cme_mic { key: "EQ1" value: CME }
 		cme_mic { key: "ERL" value: NYMEX }
 		cme_mic { key: "ES" value: CME }
+		cme_mic { key: "ES1" value: CME }
+		cme_mic { key: "ES2" value: CME }
 		cme_mic { key: "ESB" value: NYMEX }
 		cme_mic { key: "ESK" value: CME }
 		cme_mic { key: "ESQ" value: CME }
@@ -2652,27 +2651,28 @@ CmeSymbolMics
 		cme_mic { key: "STY" value: NYMEX }
 		cme_mic { key: "SU" value: CME }
 		cme_mic { key: "SUT" value: CME }
+		cme_mic { key: "T01" value: NYMEX }
+		cme_mic { key: "T02" value: NYMEX }
 		cme_mic { key: "T03" value: NYMEX }
 		cme_mic { key: "T04" value: NYMEX }
-		cme_mic { key: "T06" value: NYMEX }
 		cme_mic { key: "T07" value: NYMEX }
 		cme_mic { key: "T08" value: NYMEX }
 		cme_mic { key: "T09" value: NYMEX }
+		cme_mic { key: "T10" value: NYMEX }
+		cme_mic { key: "T11" value: NYMEX }
 		cme_mic { key: "T12" value: NYMEX }
 		cme_mic { key: "T13" value: NYMEX }
-		cme_mic { key: "T14" value: NYMEX }
-		cme_mic { key: "T15" value: NYMEX }
 		cme_mic { key: "T16" value: NYMEX }
+		cme_mic { key: "T17" value: NYMEX }
+		cme_mic { key: "T18" value: NYMEX }
 		cme_mic { key: "T19" value: NYMEX }
 		cme_mic { key: "T1U" value: CBT }
 		cme_mic { key: "T20" value: NYMEX }
-		cme_mic { key: "T21" value: NYMEX }
-		cme_mic { key: "T22" value: NYMEX }
 		cme_mic { key: "T23" value: NYMEX }
+		cme_mic { key: "T24" value: NYMEX }
+		cme_mic { key: "T25" value: NYMEX }
 		cme_mic { key: "T26" value: NYMEX }
 		cme_mic { key: "T27" value: NYMEX }
-		cme_mic { key: "T28" value: NYMEX }
-		cme_mic { key: "T29" value: NYMEX }
 		cme_mic { key: "T2B" value: NYMEX }
 		cme_mic { key: "T2D" value: NYMEX }
 		cme_mic { key: "T2M" value: NYMEX }
@@ -2750,7 +2750,6 @@ CmeSymbolMics
 		cme_mic { key: "UDL" value: NYMEX }
 		cme_mic { key: "UFB" value: CBT }
 		cme_mic { key: "UFE" value: CBT }
-		cme_mic { key: "UFG" value: CBT }
 		cme_mic { key: "UFN" value: CBT }
 		cme_mic { key: "UFU" value: CBT }
 		cme_mic { key: "UFV" value: CBT }
@@ -2763,6 +2762,9 @@ CmeSymbolMics
 		cme_mic { key: "UN" value: NYMEX }
 		cme_mic { key: "UNB" value: NYMEX }
 		cme_mic { key: "UNS" value: NYMEX }
+		cme_mic { key: "UP5" value: NYMEX }
+		cme_mic { key: "UPB" value: NYMEX }
+		cme_mic { key: "UPM" value: NYMEX }
 		cme_mic { key: "UR" value: NYMEX }
 		cme_mic { key: "USC" value: NYMEX }
 		cme_mic { key: "USDBRL" value: CME }
@@ -2910,5 +2912,5 @@ CmeSymbolMics
 		cme_mic { key: "ZT" value: CBT }
 		cme_mic { key: "ZW" value: CBT }
 		cme_mic { key: "ZWT" value: CBT }
-	}
+	}
 }

src/lib/config/defaults/jetfix.conf

@@ -5,7 +5,7 @@ JetFIXSimConnection {
         name: "sim"
         platform: SIM
         socket_connect_host_ip: "127.0.0.1"
-        socket_connect_port: @Ports.exchange_sim_fix_port
+        socket_connect_port: @Ports.glbx_sim_fix_port
         sender_compid: "SPARK" # this is the FIX session name
         sender_subid: "BTS"
         target_compid: "BTS"
@@ -18,9 +18,506 @@ JetFIXSimConnection {
         seconds_to_pause : 1
 	max_connection_attempts: 9999999
       }
+      sim_ice {
+        name: "sim_ice"
+        platform: SIMICE
+        socket_connect_host_ip: "127.0.0.1"
+        socket_connect_port: @Ports.ice_sim_fix_port
+        sender_compid: "51239" # this is the FIX session name
+        sender_subid: "BTS"
+        target_compid: "BTS"
+        target_subid: "BTS"
+        cme_market_segment_id: 0
+        end_time: "02:00:00"
+        start_time: "02:05:00"
+        raw_data: "1JE9X"
+        use_cme_secure_logon: false 
+        seconds_to_pause : 1
+      }      
    }
 }
 
+JetFIX {
+   rule: "CannedCertSession.name='ice_cert'"
+   value {
+      #connection: @JetFIXSimConnection.sim_ice
+      connection {
+         name: "ice_cert"
+         platform: ICE
+         use_cme_secure_logon: false
+         log_fix_stream: true
+         socket_connect_host_ip: "63.247.113.154"
+         socket_connect_port: 80
+         sender_compid: "51251"
+         target_compid: "ICE"         
+         start_time: "22:55:00"
+         end_time: "22:05:00"
+         sender_subid: "BTS"
+         user_name: "bluetrd-os" #this is the order_server_id
+         raw_data: "Starts123"
+         max_order_msgs_per_window: 300
+         auto_fix_seq_num: true
+         
+         seconds_to_replay: 6000
+      }
+      connection: @JetFIXSimConnection.sim
+      #connection: @JetFIXSimConnection.sim_ice
+      
+      connection {
+         name: "cme_mc5_99"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.199"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "99"
+         cme_market_segment_id: 99
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_58"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.172"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "58"
+         cme_market_segment_id: 58
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_54"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.191"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "54"
+         cme_market_segment_id: 54
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_82"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.179"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "82"
+         cme_market_segment_id: 82
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_56"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.174"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "56"
+         cme_market_segment_id: 56
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_50"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.202"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "50"
+         cme_market_segment_id: 50
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc5_52"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.181"
+         socket_connect_port: 61922
+         sender_compid: "MC5004N"
+         target_compid: "CME"
+         target_subid: "52"
+         cme_market_segment_id: 52
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }      
+   }
+}   
+
+JetFIX {
+   rule: "CannedCertSession.name='gt_cert'"
+   value {
+      connection {
+         name: "ice_cert"
+         platform: ICE
+         use_cme_secure_logon: false
+         log_fix_stream: true
+         socket_connect_host_ip: "63.247.113.154"
+         socket_connect_port: 80
+         sender_compid: "51251"
+         target_compid: "ICE"         
+         start_time: "22:55:00"
+         end_time: "22:05:00"
+         sender_subid: "BTS"
+         user_name: "bluetrd-os" #this is the order_server_id
+         raw_data: "Starts123"
+         max_order_msgs_per_window: 300
+         auto_fix_seq_num: true
+         
+         seconds_to_replay: 6000
+      }
+      connection: @JetFIXSimConnection.sim
+      connection {
+         name: "cme_mc7_99"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.199"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "99"
+         cme_market_segment_id: 99
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_58"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.172"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "58"
+         cme_market_segment_id: 58
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_54"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.191"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "54"
+         cme_market_segment_id: 54
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_82"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.179"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "82"
+         cme_market_segment_id: 82
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_56"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.174"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "56"
+         cme_market_segment_id: 56
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_50"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.202"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "50"
+         cme_market_segment_id: 50
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_52"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.181"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "52"
+         cme_market_segment_id: 52
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_60"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.195"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "60"
+         cme_market_segment_id: 60
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_76"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.204"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "76"
+         cme_market_segment_id: 76
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_88"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.169"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "88"
+         cme_market_segment_id: 88
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_74"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.197"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "74"
+         cme_market_segment_id: 74
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_64"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.177"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "64"
+         cme_market_segment_id: 64
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_72"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.189"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "72"
+         cme_market_segment_id: 72
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_70"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.183"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "70"
+         cme_market_segment_id: 70
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_68"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.168"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "68"
+         cme_market_segment_id: 68
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_80"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.187"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "80"
+         cme_market_segment_id: 80
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_84"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.198"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "84"
+         cme_market_segment_id: 84
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+      connection {
+         name: "cme_mc7_78"
+         platform: GLBX
+         socket_connect_host_ip: "69.50.112.167"
+         socket_connect_port: 61923
+         sender_compid: "MC7004N"
+         target_compid: "CME"
+         target_subid: "78"
+         cme_market_segment_id: 78
+         start_time: "00:30:00"
+         end_time: "23:30:00"
+         start_day: "sunday"
+         end_day: "friday"
+         sender_subid: "BTS"
+         raw_data: "S4PMA"
+         auto_fix_seq_num: true
+      }
+   }
+}   
+
+
 JetFIX {
    rule: "CannedCertSession.name='mc7'"
    value {
@@ -456,193 +953,193 @@ JetFIX {
          raw_data: "S4PMA"
          auto_fix_seq_num: true
       }
-      connection {
-         name: "cme_mc5_60"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.195"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "60"
-         cme_market_segment_id: 60
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_76"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.204"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "76"
-         cme_market_segment_id: 76
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_88"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.169"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "88"
-         cme_market_segment_id: 88
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_74"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.197"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "74"
-         cme_market_segment_id: 74
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_64"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.177"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "64"
-         cme_market_segment_id: 64
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_72"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.189"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "72"
-         cme_market_segment_id: 72
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_70"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.183"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "70"
-         cme_market_segment_id: 70
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_68"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.168"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "68"
-         cme_market_segment_id: 68
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_80"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.187"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "80"
-         cme_market_segment_id: 80
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_84"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.198"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "84"
-         cme_market_segment_id: 84
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
-      connection {
-         name: "cme_mc5_78"
-         platform: GLBX
-         socket_connect_host_ip: "69.50.112.167"
-         socket_connect_port: 61922
-         sender_compid: "MC5004N"
-         target_compid: "CME"
-         target_subid: "78"
-         cme_market_segment_id: 78
-         start_time: "00:30:00"
-         end_time: "23:30:00"
-         start_day: "sunday"
-         end_day: "friday"
-         sender_subid: "BTS"
-         raw_data: "S4PMA"
-         auto_fix_seq_num: true
-      }
+      # connection {
+      #    name: "cme_mc5_60"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.195"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "60"
+      #    cme_market_segment_id: 60
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_76"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.204"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "76"
+      #    cme_market_segment_id: 76
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_88"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.169"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "88"
+      #    cme_market_segment_id: 88
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_74"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.197"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "74"
+      #    cme_market_segment_id: 74
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_64"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.177"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "64"
+      #    cme_market_segment_id: 64
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_72"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.189"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "72"
+      #    cme_market_segment_id: 72
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_70"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.183"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "70"
+      #    cme_market_segment_id: 70
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_68"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.168"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "68"
+      #    cme_market_segment_id: 68
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_80"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.187"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "80"
+      #    cme_market_segment_id: 80
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_84"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.198"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "84"
+      #    cme_market_segment_id: 84
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
+      # connection {
+      #    name: "cme_mc5_78"
+      #    platform: GLBX
+      #    socket_connect_host_ip: "69.50.112.167"
+      #    socket_connect_port: 61922
+      #    sender_compid: "MC5004N"
+      #    target_compid: "CME"
+      #    target_subid: "78"
+      #    cme_market_segment_id: 78
+      #    start_time: "00:30:00"
+      #    end_time: "23:30:00"
+      #    start_day: "sunday"
+      #    end_day: "friday"
+      #    sender_subid: "BTS"
+      #    raw_data: "S4PMA"
+      #    auto_fix_seq_num: true
+      # }
    }
 }
 JetFIX {

src/lib/config/defaults/keycode.conf

@@ -1,9 +1,13 @@
+
+
 #nusers: 1
-#allow concurrent : false
-#expires: 2019-Sep-30 00:00:00   637053984000000000
-#generation_time: 2019-Jun-18 18:25:44.333280   5248650809870720704
+#allow concurrent : true
+#expires: 2019-Dec-31 00:00:00   637133472000000000
+#generation_time: 2019-Sep-30 15:52:10.434845   5248740573731736354
 EncryptedKeyCode {
    value {
-      keycode: "pgtv/OmievnpmNlgxTYuN801Hbetgu7ZGkHnlGWtK+DBrKSEgKxr5lGvCW1ld/jsztkWhKHDh0iediMDjrPm4+C5qmXlt4JJlAFvKolMN8HVC59m82fECvakcqKq28WSEjf6/DuPJNP1ZwxjcmKouxx1LZ15VlNkECtNrC9pkCnifazbfQZGV3kD08zMEyw/RPmOaVLFFeav8lwI5UpcjPYBIK1rJowXdGSvm1CJViwG5Z2JbdHR39QbKjVx112d5VbVtBq9nO3i0FEKvobjN/Gzcz/5x4TgQtEShKsbmgEN+MBVL3AItQlcFzZvE/05JLmfACUFmk5Eu0a3DlLs3A=="
+      keycode: "fVZlub92ulkXME+uZSyO9+WmyAXYKD735Qzxxfdj4G+SqEDTuwfNxOZVPS3TCFQqWPDmgjKTbbvfqIrGe3icxkVObUcUoS7lBXRXFrSvzOfs9eTz9RqHRvquDidnOzKxmSXsxV2bMb21UqBCJFeTzb4Q6Agp8EtjXJRc57UMVVygvdXg528S3ocBIq1V4W5NYraFn5WpqrTuBhpYbnsZAHi77s/EEwUT+6la3J2OHPtbiQnB6JFb0zsExMuzE4gwp79ojuQ5pnV2DXBZFNwxYvFyBUU/ac6oU89saZ/ay1DbOjz76TNqnNFewZvzdXZl3y6+X1eblX2streH2pn7gg=="
    }
-}
+}
+
+

src/lib/config/defaults/orders_fills_cleanup.conf

@@ -2,4 +2,14 @@ OrdersSoftCleanupCutoff {
    value {
       extra_days: 1
    }
+}
+
+SODTime 
+{
+   rule: "DeploymentType.type == 'dev'"
+   value
+   {
+      sod_hh: 20
+      sod_mm: 30
+   }
 }

src/lib/config/protos/MarketData.proto

@@ -155,6 +155,7 @@ message ICEOptions
    message ICEChannel {
       required string name = 1;
       repeated uint32 market_type = 2;
+      repeated uint32 uds_market_type = 7;
       optional uint32 udp_buffer_size = 3 [default = 4194304]; //4mb      
       optional uint32 heartbeat_interval = 4 [default = 36000];
       optional uint32 sequence_gap_margin = 5 [default = 10000];

src/lib/config/protos/Ports.proto

@@ -5,6 +5,7 @@ message Ports {
   // see ports.conf for actual defaults
   optional int32 config_port = 1 ;
   optional int32 qbus_port = 2 ;
-  optional int32 exchange_sim_fix_port =  3 ;
+  optional int32 glbx_sim_fix_port =  3 ;
+  optional int32 ice_sim_fix_port =  5 ;
   optional int32 replicator_port = 4 ;
 }

src/lib/config/protos/Symbols.proto

@@ -20,4 +20,5 @@ message Symbols {
   optional SecurityExchangeList glbx = 10;
   optional SecurityExchangeList sim = 20;
   optional SecurityExchangeList ice = 30;
+  optional SecurityExchangeList simice = 40;
 }