diff --git a/.gitignore b/.gitignore index 6896b131c..b8b69fa2d 100644 --- a/.gitignore +++ b/.gitignore @@ -33,3 +33,4 @@ trace_data/ /src/lib/orderman/perf_analysis/log.csv /doc/user/public/* .ionide/* +/.ignore diff --git a/doc/user/content/bts_spark/Spreader/_index.md b/doc/user/content/bts_spark/Spreader/_index.md index a69a3731e..79a1c6168 100644 --- a/doc/user/content/bts_spark/Spreader/_index.md +++ b/doc/user/content/bts_spark/Spreader/_index.md @@ -154,7 +154,7 @@ insert_anchor_links = "right" - Pay Less - If enabled by checking the checkbox, the spreader will attempt to pay less when the quantity at the level of the calculated - hedge price is more than the threshold defined.  The hedge order is + hedge price is less than the threshold defined.  The hedge order is paid less by the “Payless Ticks” as defined per leg. - Pay Up - If enabled by checking the checkbox, the spreader will pay up when the quantity at the level of the calculated hedge price is diff --git a/doc/user/content/release_notes/_index.md b/doc/user/content/release_notes/_index.md index 7746bd20f..e581d77a0 100644 --- a/doc/user/content/release_notes/_index.md +++ b/doc/user/content/release_notes/_index.md @@ -4,17 +4,36 @@ weight = 3 sort_by = "weight" insert_anchor_links = "right" +++ +## 2.12.0 +### User Visible Changes + - Added support for ICE market data and order execution +### Internal Changes + - Lots of permission / admin changes to facilitate ICE + +## 2.11.0 +### User Visible Changes + - Fixed fills form time filter to also apply to SOD fills + - Addressed an issue that could prevent the web Admin UI from displaying over slow HTTP connections + - Removed the option to enter manual spread fills + - Fixed an issue that could cause erroneous fill account attribution in the event of a crash +### Internal Changes + - Added support for tag 1031. Currently defaults to Y, can it be override through config. + - To facilitate support, allowed the bts user to view and edit algos from all other users + - Improved system tests + - Improved log messages in UI to include dispatcher thread name + - Added support for Windows core dumps in the UI + ## 2.10.0 ### User Visible Changes - Fills now include information about the algo that originated the fill - A new, more powerful PnL form is now available ### API Changes - - Added security state information (pre open, halted, normal, closed, etc) + - Added security state information (pre open, halted, normal, closed, etc) ### Bug Fixes - Brought back the "select all" check box in the mini order book - Fixed an issue in the market data subsystem that on rare occasions could result in a startup crash ### Internal Changes - - Fixed an issue that was stopping us from interpreting SOD times as local rather than GMT + - Fixed an issue that was stopping us from interpreting SOD times as local rather than GMT ## 2.9.3 ### Internal Changes - Fixed system test scripts @@ -30,7 +49,7 @@ insert_anchor_links = "right" - Avoided shipping production configuration defaults that lead to trying to download a position file for the development sftp server. ### Api - C# API sample now uses synchronous calls. - + ## 2.9.0 ### User Visible Changes - Improved Workspace management User Interface @@ -38,11 +57,11 @@ insert_anchor_links = "right" - The Spark UI Window group now has a title in the Windows task bar - Added sound alerts for trade events, see Launcher -> Settings -> Alerts - Allowed composite algos that are attached to existing orders to identify fills that were already present prior to attachment - - Added import facilities for GMI position files + - Added import facilities for GMI position files - Added the ability for risk managers to enter manual fills - Added "raw" P&L columns, that is columns that *do not* include spread legs in their accounting ### Bug Fixes - - Fixed the settings action menu (Load/Save/Reset to Factory) + - Fixed the settings action menu (Load/Save/Reset to Factory) - Fixed the method used to determine if a security is expired to take into account the start of day time ### API Changes - Fixed an issue in the Client SDK that could lead to occasional failures to connect to the spark server as well as excessive memory usage. @@ -79,10 +98,10 @@ insert_anchor_links = "right" - Made it easier to generate simple UI's for new order types ### API Changes - Dynamic orders can now use QuantityMeasure parameter types - + ## 2.7.2 ### User Visible Changes - - Changed the login dialog to avoid leaking workspace names from other users. + - Changed the login dialog to avoid leaking workspace names from other users. - Added new Splitter algo. From the manual: The splitter is useful for pro-rata markets where you want to vary your exposure independent of the order quantity that you want to fill. The @@ -90,13 +109,13 @@ insert_anchor_links = "right" the order quantity but it will put exposed_qty total size on the market. Splitter favors order size over queue position. ### Internal Changes - - Made spark more resilient to lack of connectivity to the CME web site for downloading of templates. - + - Made spark more resilient to lack of connectivity to the CME web site for downloading of templates. + ## 2.7.1 ### User Visible Changes - Fixed some issues with the Cover algo, where in some circumstances it would erroneously cancel launched hedges on completion. - + ## 2.7.0 ### User Visible Changes - Allowed algos to process "in-flight" fills while shutting down @@ -115,25 +134,25 @@ insert_anchor_links = "right" ### User Visible Changes - Fixed an issue in the ladder where in some cases the mini order book could not be brought up from a badge - Fixed a rare race condition that could lead to erroneous limit enforcement - - Fixed an issue with the margin limits where an error could occur if an account name with more than 12 chars was used + - Fixed an issue with the margin limits where an error could occur if an account name with more than 12 chars was used ## 2.6.1 ### User Visible Changes - Fixed an issue in the ladder where an error could occur when trying to change ladder settings for the first time - + ## 2.6.0 ### User Visible Changes - In the ladder made last trade more distinctive when overlayed on the volumn column - - In the ladder, corrected display of volume tool tip + - In the ladder, corrected display of volume tool tip - In the ladder, changed the consolidation of last trade to consider only the price ignoring aggressor changes - Added an option for the Top Algo to delete itself if within N ticks from top of book - Address a rare issue that could lead to an error report on UI exit - Added margin risk limits - Added the "Offset" algo. Offset prices a child algo at a parameterizable distance from the top of the book - - Avoided switching the current account and order type on open ladders when limits are changed + - Avoided switching the current account and order type on open ladders when limits are changed ### Internal Changes - Ensure that admin logins are always allowed to login, regardless of keycode status. - + ## 2.5.0 ### User Visible Changes - Added loss limit risk check @@ -163,13 +182,13 @@ insert_anchor_links = "right" - Made Cover order more independent from its hedging chidren: - Cover continues to operate even if an hedging child is canceled - When a cover is canceled it will detach from any running hedging children, instead of canceling them - - Changed Lean, Join and Cover algos to hide themselves in the ladder once they have completed their work + - Changed Lean, Join and Cover algos to hide themselves in the ladder once they have completed their work - Changed manual fills to appear in day PnL rather than in the SOD records - Add buy/sell coloring to fill rows in fills window - Streamlined the mini order book: - Display of the mini order book popup is quicker - Only exchange orders are displayed - - Update buttons are only displayed for orders that are under control of an algo with the update applying to the entire algo, not just the exchange order + - Update buttons are only displayed for orders that are under control of an algo with the update applying to the entire algo, not just the exchange order - We can now select multiple rows to attach, detach, update or cancel multiple orders in a batch - A checkbox column has been added to make clearer which rows are selected ### API Changes @@ -179,7 +198,7 @@ insert_anchor_links = "right" - Added system requirements to Spark documentation - Implemented keycodes - Temporarily rejected stop and stop limit orders, pending bug fixes - + ## 2.2.0 ### User Visible Changes - Made our orders easier to spot in MBO display @@ -190,30 +209,30 @@ insert_anchor_links = "right" - Improved handling of orders with undiscovered securities - Added feedback to the ladder when the user attempts to middle click on bid/ask columns using an order that does not have parameters - Fixed display of algo studio algo types in order book - - Hid net change setting in synthetic ladders + - Hid net change setting in synthetic ladders - Fixed a number of problems with filtering on the Fills form - On the Lean algo, OnToward and Away are now calculated with respect to the opposite inside market at time of the order launch - Iceberg Spreaders with queue holders are not erroneously cancelled after the first fill - Added security symbol column to fills window - Re-organized ladder settings dialog to make it clearer and easier to use - + ## 2.1.4 ### User Visible Changes - Fixed handling of cancel button in security picker - + ## 2.1.3 ### User Visible Changes - - Fixed a simulator bug in the handling of 0 sized orders + - Fixed a simulator bug in the handling of 0 sized orders ## 2.2.1 ### User Visible Changes - Fixed order book form exception when display a spreader order for whichs the legs were not present yet - + ## 2.1.1 ### User Visible Changes - Added PIQ to GLBX Iceberg orders - Corrected handling of stop limit orders - + ## 2.1.0 ### User Visible Changes - Added support for the GLBX Iceberg order type @@ -224,7 +243,7 @@ insert_anchor_links = "right" - Added PIQ to the mini order book - Fixed a rare bug that could lead to server failure when requesting market data - Fixed a bug that could cause incorrect price displays for certain types of synthetic securities - + ## 2.0.5 ### User Visible Changes - Fixed a bug in the handling of rejected QHO orders @@ -234,19 +253,19 @@ insert_anchor_links = "right" ## 2.0.4 ### User Visible Changes - Fixed performance regression in order manager - + ### Internal changes - Improved market data diagnostics for bad configurations - Changed market data default buffer size - + ## 2.0.3 ### Internal changes - Improved backwards compatibility in UI settings - + ## 2.0.2 ### User Visible Changes - - Middle click the ladder price column to show the orders for that price level + - Middle click the ladder price column to show the orders for that price level - Middle click the bid or ask column in the ladder no longer sends an order ### Internal changes @@ -274,8 +293,8 @@ insert_anchor_links = "right" - Improved settings manager tool output - Added diagnostics in the event MD memory corruption is detected - Added systems tests for join,lean, iceberg and spreader orders - - Made exchange order recovery more resilient in the presence of database loss - + - Made exchange order recovery more resilient in the presence of database loss + ## 1.6.2 ### User Visible Changes ### Exchange simulator @@ -314,21 +333,21 @@ insert_anchor_links = "right" - These columns can be restored by toggling their visibility within the PnL's Settings form - A new Traded Today column has been added to the PnL - Filtering on this column will allow a user to restrict the report to only show today's position -### Internal changes - - Improved organization of the shell notification architecture +### Internal changes + - Improved organization of the shell notification architecture - Added logging to help troubleshoot shell exit hang ## 1.5.6 ### User Visible Changes - - Fixed a bug that could cause limits to be erroneously enforced + - Fixed a bug that could cause limits to be erroneously enforced - Improved algo documentation - Changed how the spreader uses net change - Simplified trigger algo user interface -### Internal changes +### Internal changes - Defaulted to not recording sim market data - - Improved documentation build + - Improved documentation build - Added additional performance tracing - + ## 1.5.5 - Fixed Market View 'stale working orders' bug @@ -336,11 +355,11 @@ insert_anchor_links = "right" - Avoided logging when warming up cache ## 1.5.3 - - Fixed sounds startup issue + - Fixed sounds startup issue ## 1.5.2 - Added better logging to AlgoDefinitionService - + ## 1.5.1 - Fixed missing securities in the Fills and P&L windows. @@ -361,7 +380,7 @@ insert_anchor_links = "right" #### Bug Fixes - Fixed the "Since Open" time filter to use the exchange open time rather than 8:30 AM - Fixed a bug in the order book where sometimes parent orders would not update their last message column. - + #### Algos - Added OCO Algo - An OCO order creates a relationship between a child order and a 'stop' or 'other' order such that when one of these is cancelled, both cancel, or when one is filled, the other order's size is debited in the amount of the fill. Composing this order with Cover simulates scalping functionality @@ -369,7 +388,7 @@ insert_anchor_links = "right" - Improved SDK documentation and usability - Dynamic orders are now properly identified in the order book #### Documentation - - Information on the Spark API has been added to the Spark manual + - Information on the Spark API has been added to the Spark manual - The documentation has been updated and reorganized to make information easier to find ### Internal Changes - Improved robustness of limit system test diff --git a/docker/linux b/docker/linux index e13bb5d8d..c40e6eba4 160000 --- a/docker/linux +++ b/docker/linux @@ -1 +1 @@ -Subproject commit e13bb5d8df14171a9a5abfdb8b6b4ecbc519032a +Subproject commit c40e6eba4ba72480a68cf784b6ffbcacc3fbf4af diff --git a/externals b/externals index b5ca26428..6d8f64ca5 160000 --- a/externals +++ b/externals @@ -1 +1 @@ -Subproject commit b5ca264286e2a7033b7ce6ad0000254dea03dc8a +Subproject commit 6d8f64ca5cee3acbf9addb51428f25d54b9f906a diff --git a/src/client/spark/Admin/AdminModel.cs b/src/client/spark/Admin/AdminModel.cs index 9e8508b22..2bbf79243 100644 --- a/src/client/spark/Admin/AdminModel.cs +++ b/src/client/spark/Admin/AdminModel.cs @@ -1,83 +1,74 @@ using bts; using bts.api.admin; +using bts.api.core; +using bts.utils; using System; using System.Collections.Generic; using System.ComponentModel.Composition; using System.Linq; using System.Reactive.Linq; -using System.Reactive.Subjects; -using bts.api.core; -using mbts.ui.shell; namespace mbts.spark { [Export] class AdminModel : IDisposable { public readonly UserTuple user; - - readonly BehaviorSubject> - _accounts_subject = new BehaviorSubject>(new Dictionary()), - _users_subject = new BehaviorSubject>(new Dictionary()); - IReadOnlyDictionary - _account_map = new Dictionary(), - _user_map = new Dictionary(); - readonly BehaviorSubject _wrapper_subject = new BehaviorSubject(new AdminModelWrapper()); - IDisposable _dispose; + + IDisposable _dispose; + IReadOnlyDictionary _account_map = DictionaryHelper.empty(); [ImportingConstructor] - AdminModel(AdminService service, [Import]UserTuple user) { + AdminModel(AdminService service, UserTuple user) { this.user = user; - _dispose = service.subscribei_admin_data().Select(update => { - _wrapper_subject.OnNext(new AdminModelWrapper(update.all.TryFirst(), user.name)); - IEnumerable active_users = - from admin_data in update.all - from bts_user in admin_data.Users - where bts_user.Name == user.name && bts_user.HasUserData && bts_user.UserData.HasAccountPermissions - select bts_user; - - IEnumerable active_accounts = - from admin_data in update.all - from bts_account in admin_data.Accounts - where bts_account.IsPresent && _wrapper_subject.Value.can_view_account(bts_account.Name) - join account_id in ( - from bts_user in active_users - from account_id in bts_user.UserData.AccountPermissions.AccountPermissions.Keys - select account_id - ) on bts_account.Id equals account_id - select bts_account; - - var users = active_users.ToDictionary(u => u.Id, u => u.Name); - var accounts = active_accounts.ToDictionary(a => a.Id, a => a.Name); - return (users, accounts); - }).Subscribe(t => { - _account_map = t.accounts; - _user_map = t.users; - _users_subject.OnNext(_user_map = t.users); - _accounts_subject.OnNext(_account_map = t.accounts); - }); + // accounts + on_accounts = service.subscribei_admin_data() + .Select(up => up.all.TryGetFirst(out var ad) && ad.Users.Where(u => u.Name == user.name).TryGetSingle(out var usr) && usr.HasUserData && usr.UserData.HasAccountPermissions ? + usr.UserData.AccountPermissions.AccountPermissions + .Where(kv => (kv.Value.TradingPermission != null && kv.Value.TradingPermission.HasPermissions) || kv.Value.AdminPermission.HasPermissions) + .Select(kv => ad.Accounts.Where(a => a.Id == kv.Key).TryGetSingle(out var acc) ? acc : (BTSAccountImmutable?)null) + .Where(acc => acc.HasValue) + .Select(acc => acc.Value) + .ToDictionary(acc => acc.Id, acc => acc.Name) : + new Dictionary(0)) + .Replay(1) + .RefCount(); + // users + on_users = Observable.Return(DictionaryHelper.wrap_value(user.id, user.name)); + all_user_tuples = user.permissions == UserTuple.Permissions.Admin ? + service.subscribei_admin_data() + .Select(u => u.all.TryFirst()) + .Where(ad => ad.HasValue) + .Select(ad => ad.Value.Users + .Where(u => user.can_access_user_data(u.Id)) + .Select(u => new UserTuple(u.Id, u.Name)) + .ToDictionary(UserTuple.key_fn)) + .Table(UserTuple.key_fn) + .Cache() : + Observable.Return(TableUpdate.mk_empty(UserTuple.key_fn).with_added(new[] { user })); + _dispose = on_accounts.Subscribe(map => _account_map = map); } - public IObservable> on_users => _users_subject.AsObservable(); - public IObservable> on_accounts => _accounts_subject.AsObservable(); - public IObservable> on_user_tuples => on_users.Select(d => d.Select(kv => new UserTuple(kv.Key, kv.Value)).ToList()); - public IObservable> on_account_tuples => on_accounts.Select(d => d.Select(kv => new AccountTuple(kv.Key, kv.Value)).ToList()); - public IObservable on_admin_data => _wrapper_subject.AsObservable(); + public IObservable> on_users { get; } + public IObservable> on_accounts { get; } + public IObservable> all_user_tuples { get; } + public IObservable> on_account_tuples => on_accounts.Select(d => d.Select(kv => new AccountTuple(kv.Key, kv.Value)).ToListSorted(a => a.name)); - public AccountTuple? get_default_account() { - foreach (var kv in _account_map) { - return new AccountTuple(kv.Key, kv.Value); - } - return null; - } - public uint? get_user_id(string user) => _id_for_name(user, _user_map); - public uint? get_account_id(string account) => _id_for_name(account, _account_map); - public void Dispose() => Disposable.dispose(ref _dispose); + public IObservable> permission_filter(IObservable> positions, Func get_account_id) where TKey : IEquatable => + user.permissions == UserTuple.Permissions.Admin ? + positions : + positions.Filter(on_accounts.Select(map => new Func(p => map.ContainsKey(get_account_id(p))))); - static uint? _id_for_name(string name, IReadOnlyDictionary map) { - foreach (var kv in map) { - if (kv.Value == name) - return kv.Key; - } - return null; - } + public IObservable> permission_filter(IObservable> positions, Func get_account_name) where TKey : IEquatable => + user.permissions == UserTuple.Permissions.Admin ? + positions : + positions.Filter(on_accounts.Select(map => { + var hash = new HashSet(map.Values); + return new Func(v => hash.Contains(get_account_name(v))); + })); + + public AccountTuple? get_default_account() => + _account_map.Select(kv => new AccountTuple(kv.Key, kv.Value)).TryFirst(); + + public void Dispose() => + Disposable.dispose(ref _dispose); } } \ No newline at end of file diff --git a/src/client/spark/Daemons/AboutDaemon.cs b/src/client/spark/Daemons/AboutDaemon.cs index 8a8ad2f06..1777477cb 100644 --- a/src/client/spark/Daemons/AboutDaemon.cs +++ b/src/client/spark/Daemons/AboutDaemon.cs @@ -3,15 +3,10 @@ using mbts; using mbts.config; using mbts.spark; using mbts.ui; -using mbts.ui.shell; using System; -using System.Collections.Generic; using System.ComponentModel.Composition; using System.Diagnostics; -using System.Linq; using System.Reactive.Linq; -using System.Text; -using System.Threading.Tasks; using System.Windows; using System.Windows.Input; @@ -23,8 +18,8 @@ namespace bts.ui { IDisposable _dispose; [ImportingConstructor] - public AboutDaemon(IBus bus, [Import(LoginContainer.Exports.UserName)]string username) { - _user = username; + public AboutDaemon(IBus bus, UserTuple user) { + _user = user.name; _bus = bus; _dispose = bus.subscribe(SparkTags.About).OnDispatcher().Subscribe(_ => show_about()); } @@ -63,4 +58,4 @@ namespace bts.ui { Disposable.dispose(ref _dispose); } } -} +} \ No newline at end of file diff --git a/src/client/spark/Daemons/ErrorReporter.cs b/src/client/spark/Daemons/ErrorReporter.cs index fa8ccc6ad..e2a7b462b 100644 --- a/src/client/spark/Daemons/ErrorReporter.cs +++ b/src/client/spark/Daemons/ErrorReporter.cs @@ -7,13 +7,11 @@ using mbts.ui; using System; using System.ComponentModel.Composition; using System.Reactive.Linq; -using System.Windows.Threading; -using System.Windows; using System.Threading.Tasks; -using mbts.ui.shell; +using System.Windows; +using System.Windows.Threading; -namespace bts.spark -{ +namespace bts.spark { [Export, Export(typeof(IShellDaemon))] public class ErrorReporter : IShellDaemon, IDisposable { readonly ErrorReportService _svc; @@ -21,9 +19,9 @@ namespace bts.spark IDisposable _dispose; [ImportingConstructor] - public ErrorReporter(ErrorReportService svc, IBus bus, [Import(LoginContainer.Exports.UserName)] string username) { + public ErrorReporter(ErrorReportService svc, IBus bus, UserTuple user) { _svc = svc; - _username = username; + _username = user.name; _dispose = bus.subscribe(SparkTags.Support).OnDispatcher().Subscribe(msg => show_support_window(msg, "Report a problem")); } @@ -48,6 +46,45 @@ namespace bts.spark ErrorMessage = message, }); } + public async Task report_silently(string contact, string description, string userdata) { + var request = append_logs(new ErrorReport { + User = _username, + ErrorMessage = description, + UserDescription = userdata, + Contact = contact + }.immutable()); + var reply = await _svc.send_report_async(request); + } + + public static ErrorReportImmutable append_logs(ErrorReportImmutable src) { + var result = src.CloneToMutable(); + + var windows_log = LogUtil.read_windows_event_log(); + if (!windows_log.IsEmpty) { + result.Files.Add(new FileData { + Name = "Application.evtx.gz", + Contents = windows_log + }); + } + + var (data,date) = LogUtil.read_latest_crash_dump(); + if (!data.IsEmpty) { + result.Files.Add(new FileData { + Name = $"spark-{date.ToString("s")}.dmp.gz", + Contents = data + }); + } + + var spark_log = LogUtil.read_logs_compressed(); + if (!spark_log.IsEmpty) { + result.Files.Add(new FileData { + Name = "log.gz", + Contents = spark_log + }); + } + return result.immutable(); + } + } static class ErrorReporterExtensions { diff --git a/src/client/spark/Daemons/MDDaemon.cs b/src/client/spark/Daemons/MDDaemon.cs index a2f866421..b6bcfa70b 100644 --- a/src/client/spark/Daemons/MDDaemon.cs +++ b/src/client/spark/Daemons/MDDaemon.cs @@ -1,5 +1,6 @@ using bts.api; using bts.api.synthetic; +using bts.md; using mbts; using mbts.co2; using mbts.spark; @@ -7,15 +8,10 @@ using mbts.ui; using System; using System.Collections.Generic; using System.ComponentModel.Composition; +using System.Linq; using System.Reactive.Linq; using System.Threading.Tasks; -using System.Linq; using SecurityKey = System.UInt32; -using System.Collections.Concurrent; -using System.Threading; -using bts.api.user; -using bts.md; -using mbts.ui.shell; namespace bts { [Export(typeof(IInitializable))] @@ -25,17 +21,21 @@ namespace bts { readonly Lazy _ss; readonly Dictionary _deleted_synthetics = new Dictionary(); - static uint get_naggle() => mbts.config.Configuration.get().naggle_ms; - [ImportingConstructor] - public MDDaemon(Lazy bus, Lazy token, Lazy ss, [Import(LoginContainer.Exports.UserID)]uint user_id) : - base(get_naggle(), MDSourceOptions.DeferInit, user_id) { + public MDDaemon(Lazy bus, Lazy token, Lazy ss, UserTuple user) + : base(mbts.config.Configuration.get().naggle_ms, MDSourceOptions.DeferInit, user) { _bus = bus; _token = token; - _ss = ss; + _ss = ss; } - override public void initialize() { + public int priority => 55; + + [Export] + MDConsumer consumer => _consumer; + IBus bus => _bus.Value; + + public override void initialize() { _relay.set_token(_token.Value); _dispose.Add( _ss.Value.subscribei_metadata() @@ -60,16 +60,8 @@ namespace bts { // Post full dictionary of syntheics here bus.post(SparkTags.SyntheticstrategiesTag, t); })); - _dispose.Add( this.on_securities.Subscribe( (md) => { - bus.post(SparkTags.SecuritiesTag, md); - })); + _dispose.Add(on_securities.Subscribe(md => bus.post(SparkTags.SecuritiesTag, md))); Task.Run(run); } - - [Export] - MDConsumer consumer => _consumer; - - public int priority => 55; - IBus bus => _bus.Value; } } \ No newline at end of file diff --git a/src/client/spark/Daemons/OopsDaemon.cs b/src/client/spark/Daemons/OopsDaemon.cs new file mode 100644 index 000000000..190f98fa2 --- /dev/null +++ b/src/client/spark/Daemons/OopsDaemon.cs @@ -0,0 +1,55 @@ +using System; +using System.ComponentModel.Composition; +using mbts.ui; +using mbts; +using Newtonsoft.Json; +using System.Threading.Tasks; + +namespace bts.spark { + [Export(typeof(IShellDaemon))] + class OopsDaemon : IShellDaemon, IDisposable { + [ImportingConstructor] + public OopsDaemon(ErrorReporter e) { + var pid = get_pid(); + if (pid.HasValue) { + handle_unclean_shutdown(e, pid.Value); + } + save_pid(); + } + + public void Dispose() { + clear_pid(); + } + + void handle_unclean_shutdown(ErrorReporter e, int oldpid) { + string extra = ""; + try { + var conf = mbts.config.Configuration.get(); + extra = JsonConvert.SerializeObject(conf, Formatting.Indented); + } + catch { } + Task.Run(() => e.report_silently( + $"Host: {Environment.MachineName}", + $"pid: {oldpid}\n{extra}", + "Automated crash report") + ); + } + + const string _subkey = @"Software\BTS"; + private void save_pid() { + using (var key = Microsoft.Win32.Registry.CurrentUser.CreateSubKey(_subkey)) { + key.SetValue("pid", System.Diagnostics.Process.GetCurrentProcess().Id, Microsoft.Win32.RegistryValueKind.DWord); + } + } + private void clear_pid() { + using (var key = Microsoft.Win32.Registry.CurrentUser.CreateSubKey(_subkey)) { + key.DeleteValue("pid",false); + } + } + private int? get_pid() { + using (var key = Microsoft.Win32.Registry.CurrentUser.CreateSubKey(_subkey)) { + return key.GetValue("pid") as int?; + } + } + } +} diff --git a/src/client/spark/Daemons/OrderTicketDaemon.cs b/src/client/spark/Daemons/OrderTicketDaemon.cs index cf18c0835..b825740e9 100644 --- a/src/client/spark/Daemons/OrderTicketDaemon.cs +++ b/src/client/spark/Daemons/OrderTicketDaemon.cs @@ -43,6 +43,11 @@ namespace bts.shell { void _show_order_window(Window window) { foreach (var kv in AlgoStudioModelUtils.get_dynamic_resources(_model.algo_model.ui_configuration)) window.Resources[kv.Key] = kv.Value; + // the dynamic algo 'SecurityKey' template has a ContractPickerPopup that expects the security map to be available as a DynamicResource. + var sub = _bus.subscribe(SparkTags.SecuritiesTag) + .OnDispatcher() + .Subscribe(sec_map => window.Resources[ResourceKeys.Securities] = sec_map); + window.Closed += (s, e) => Disposable.dispose(ref sub); window.Show(-200, -160, true); } diff --git a/src/client/spark/Daemons/ShellLatencyDaemon.cs b/src/client/spark/Daemons/ShellLatencyDaemon.cs index 0f4e10fd3..723174cb6 100644 --- a/src/client/spark/Daemons/ShellLatencyDaemon.cs +++ b/src/client/spark/Daemons/ShellLatencyDaemon.cs @@ -3,17 +3,13 @@ using bts.metrics; using mbts; using mbts.config; using mbts.ui; -using mbts.ui.shell; using System; -using System.Collections.Generic; using System.ComponentModel.Composition; using System.Linq; using System.Net.NetworkInformation; -using System.Text; using System.Threading.Tasks; namespace bts.perf { - [Export, Export(typeof(IShellDaemon))] class ShellLatencyDaemon : IShellDaemon, IDisposable { readonly ShellLatencyService _service; @@ -24,10 +20,10 @@ namespace bts.perf { Ping _pinger = new Ping(); [ImportingConstructor] - public ShellLatencyDaemon(ShellLatencyService service, [Import(LoginContainer.Exports.UserName)] string user) { + public ShellLatencyDaemon(ShellLatencyService service, UserTuple user) { _service = service; _version = Configuration.get().version; - _username = user; + _username = user.name; _hostname = Environment.MachineName; _forever_ping_host(_pinger); @@ -105,4 +101,4 @@ namespace bts.perf { Disposable.dispose(ref _subs); } } -} +} \ No newline at end of file diff --git a/src/client/spark/Modules/Market/MarketShellModule.cs b/src/client/spark/Modules/Market/MarketShellModule.cs index f2db00136..391801886 100644 --- a/src/client/spark/Modules/Market/MarketShellModule.cs +++ b/src/client/spark/Modules/Market/MarketShellModule.cs @@ -5,7 +5,6 @@ using mbts.co2; using mbts.persist; using mbts.spark; using mbts.ui; -using mbts.ui.shell; using System.Collections.Generic; using System.ComponentModel.Composition; using System.IO; @@ -13,13 +12,11 @@ using System.Threading.Tasks; using System.Windows.Media; using static mbts.ui.shell.ProtoBufSerializationHelper; -namespace bts.shell -{ +namespace bts.shell { [Export(typeof(IShellModule))] - class MarketShellModule : IShellModule - { + class MarketShellModule : IShellModule { [ImportingConstructor] - public MarketShellModule(IBus bus, MDConsumer consumer, UserSettingsHub settings_hub, OrdersModel orders_model, [Import(LoginContainer.Exports.UserName)]string user_name) { + public MarketShellModule(IBus bus, MDConsumer consumer, UserSettingsHub settings_hub, OrdersModel orders_model) { _tsf = new TimeAndSalesWindowFactory(this, bus, consumer, settings_hub); _mvf = new MarketViewWindowFactory(this, bus, settings_hub, orders_model); } diff --git a/src/client/spark/Modules/Market/MarketView/MarketViewRow.cs b/src/client/spark/Modules/Market/MarketView/MarketViewRow.cs index 947a02440..6f3688e77 100644 --- a/src/client/spark/Modules/Market/MarketView/MarketViewRow.cs +++ b/src/client/spark/Modules/Market/MarketView/MarketViewRow.cs @@ -26,6 +26,7 @@ namespace mbts.spark { readonly ObservableFilterCollection _working_orders, _working_bids, _working_asks; readonly string _security_text, _security_key_text; readonly Property _bid_price, _ask_price, _last_price, _day_change, _settlement, _session_high, _session_low; + readonly Property _last_price_from_spread; readonly CellProperty _position; readonly CellProperty _last_size; readonly CellProperty _volume; @@ -54,6 +55,7 @@ namespace mbts.spark { _day_change = backing(nameof(day_change), double.NaN); _session_low = backing(nameof(session_low), double.NaN); _session_high = backing(nameof(session_high), double.NaN); + _last_price_from_spread = backing(nameof(last_price_from_spread), false); _settlement = backing(nameof(settlement), double.NaN); _volume = new CellProperty(this, nameof(volume), 0); _inside_bids = new CellProperty(this, nameof(inside_bids)); @@ -109,6 +111,7 @@ namespace mbts.spark { public int position => _position.value; public double raw_day_position => _raw_day_position; public double last_price => _last_price.Value; + public bool last_price_from_spread => _last_price_from_spread.Value; public uint last_size => _last_size.value; public double bid_price => _bid_price.Value; public uint bid_direct_size => _bid_direct_size.Value; @@ -167,15 +170,17 @@ namespace mbts.spark { _inside_bids.on_next_distinct(_bid_price.Value >= _working_best_bid.value, ref dirty); _inside_asks.on_next_distinct(_ask_price.Value <= _working_best_ask.value, ref dirty); } - if (md.md.get_last_trade(ref _last_ref, out NativePriceSize ps, out DateTime ts, out ulong v)) { + if (md.md.get_last_trade(ref _last_ref, out NativePriceSize ps, out DateTime ts, out ulong v, out bool from_spread)) { _last_price.set(ps.price); + _last_price_from_spread.set(from_spread); _last_size.on_next_distinct(ps.size, ref dirty); _volume.on_next_distinct(v, ref dirty); } - _settlement.set(md.md.settlement_price); - _session_low.set(md.md.session_low); - _session_high.set(md.md.session_high); - _day_change.set(last_price - settlement); + dirty |= _settlement.set_new(md.md.settlement_price); + dirty |= _session_low.set_new(md.md.session_low); + dirty |= _session_high.set_new(md.md.session_high); + dirty |= _day_change.set_new(last_price - settlement); + _volume.on_next_distinct(md.md.total_volume, ref dirty); if (dirty) invalidated?.Invoke(); diff --git a/src/client/spark/Modules/Market/MarketView/MarketViewVM.cs b/src/client/spark/Modules/Market/MarketView/MarketViewVM.cs index b57e17d33..d2c8ca2b5 100644 --- a/src/client/spark/Modules/Market/MarketView/MarketViewVM.cs +++ b/src/client/spark/Modules/Market/MarketView/MarketViewVM.cs @@ -38,7 +38,8 @@ namespace mbts.spark { IMultiValueConverter price_converter = new ui.spark.converters.TickDisplayConverter(); add_col(Columns.Security, "Security", r => r.security.ToString(), nameof(Row.security), DefaultColumnWidths.Security, security_filter, DataGridFilterKnownTemplates.Regex); add_col(Columns.Volume, "Volume", r => r.volume.ToString(), nameof(Row.volume), DefaultColumnWidths.Volume, volume_filter, DataGridFilterKnownTemplates.ULongMinMax); - add_col(Columns.Last, "Last", r => r.format_price(r.last_price), nameof(Row.last_price), DefaultColumnWidths.Price, last_price_filter, DataGridFilterKnownTemplates.DoubleMinMax); + add_col(Columns.LastSpd, "Spd", r => r.last_price_from_spread ? "S" : "", nameof(Row.last_price_from_spread), DefaultColumnWidths.Side, last_price_from_spread_filter, DataGridFilterKnownTemplates.Regex); + add_col(Columns.Last, "Last", r => r.format_price(r.last_price), nameof(Row.last_price), DefaultColumnWidths.Price, last_price_filter, DataGridFilterKnownTemplates.DoubleMinMax); add_col(Columns.LastSize, merged_size_col_header, r => r.last_size.ToString(), nameof(Row.last_size), DefaultColumnWidths.Size, last_size_filter, DataGridFilterKnownTemplates.UIntMinMax, h_alignment: System.Windows.HorizontalAlignment.Right); add_col(Columns.Position, "Pos", r => r.position.ToString(), nameof(Row.position), DefaultColumnWidths.Size, position_filter, DataGridFilterKnownTemplates.IntMinMax, true, h_alignment: System.Windows.HorizontalAlignment.Right); add_col(Columns.RawDayPosition, "Raw Day Pos", r => SparkUtils.fmt_position(r.raw_day_position), nameof(Row.raw_day_position), DefaultColumnWidths.Size, raw_day_position_filter, DataGridFilterKnownTemplates.DoubleMinMax, true, h_alignment: System.Windows.HorizontalAlignment.Right); @@ -138,6 +139,7 @@ namespace mbts.spark { ask_direct_size_filter.load_state(filters.ask_direct_size); ask_implied_size_filter.load_state(filters.ask_implied_size); last_price_filter.load_state(filters.last_price); + last_price_from_spread_filter.load_state(filters.last_price_from_spread); last_size_filter.load_state(filters.last_size); day_change_filter.load_state(filters.day_change); settlement_filter.load_state(filters.settlement); @@ -155,7 +157,7 @@ namespace mbts.spark { }), DataGridFilterHelper.hook_filters(rows_view, security_filter, bid_price_filter, bid_merged_size_filter, bid_direct_size_filter, bid_implied_size_filter, ask_price_filter, ask_merged_size_filter, ask_direct_size_filter, ask_implied_size_filter, - last_price_filter, last_size_filter, day_change_filter, settlement_filter, + last_price_filter, last_price_from_spread_filter, last_size_filter, day_change_filter, settlement_filter, session_low_filter, session_high_filter, position_filter, raw_day_position_filter, working_bid_volume_filter, working_ask_volume_filter, volume_filter, security_key_filter), settings_manager.subscribe_current().OnDispatcher().Subscribe(grid_settings_manager), WorkLoop.Start("MarketView", _refresh) @@ -218,6 +220,7 @@ namespace mbts.spark { public IDataGridColumnPersistentFilter ask_direct_size_filter { get; } = new UIntMinMax(r => r.ask_direct_size, nameof(Row.ask_direct_size)); public IDataGridColumnPersistentFilter ask_implied_size_filter { get; } = new UIntMinMax(r => r.ask_implied_size, nameof(Row.ask_implied_size)); public IDataGridColumnPersistentFilter last_price_filter { get; } = new DoubleMinMax(r => r.last_price, nameof(Row.last_price)); + public IDataGridColumnPersistentFilter last_price_from_spread_filter { get; } = new RegexFilterVM(r => r.last_price_from_spread.ToString(), nameof(Row.last_price_from_spread)); public IDataGridColumnPersistentFilter last_size_filter { get; } = new UIntMinMax(r => r.last_size, nameof(Row.last_size)); public IDataGridColumnPersistentFilter day_change_filter { get; } = new DoubleMinMax(r => r.day_change, nameof(Row.day_change)); public IDataGridColumnPersistentFilter settlement_filter { get; } = new DoubleMinMax(r => r.settlement, nameof(Row.settlement)); @@ -250,6 +253,7 @@ namespace mbts.spark { ask_direct_size = ask_direct_size_filter.get_state(), ask_implied_size = ask_implied_size_filter.get_state(), last_price = last_price_filter.get_state(), + last_price_from_spread = last_price_from_spread_filter.get_state(), last_size = last_size_filter.get_state(), day_change = day_change_filter.get_state(), settlement = settlement_filter.get_state(), diff --git a/src/client/spark/Modules/Market/MarketView/MarketViewView.xaml.cs b/src/client/spark/Modules/Market/MarketView/MarketViewView.xaml.cs index 86f9d5593..e42bf0e89 100644 --- a/src/client/spark/Modules/Market/MarketView/MarketViewView.xaml.cs +++ b/src/client/spark/Modules/Market/MarketView/MarketViewView.xaml.cs @@ -12,6 +12,7 @@ namespace mbts.ui.spark { Security, Volume, Last, + LastSpd, LastSize, Position, RawDayPosition, diff --git a/src/client/spark/Modules/Market/MarketView/MarketViewWindowFactory.cs b/src/client/spark/Modules/Market/MarketView/MarketViewWindowFactory.cs index 9c49672a9..d3ae03818 100644 --- a/src/client/spark/Modules/Market/MarketView/MarketViewWindowFactory.cs +++ b/src/client/spark/Modules/Market/MarketView/MarketViewWindowFactory.cs @@ -100,6 +100,7 @@ namespace bts.shell { public static GridColumnSettings security => new GridColumnSettings(col_id(Columns.Security), DefaultColumnHeaders.Security, width: DefaultColumnWidths.Security); public static GridColumnSettings volume => new GridColumnSettings(col_id(Columns.Volume), DefaultColumnHeaders.Volume, width: DefaultColumnWidths.Volume); public static GridColumnSettings last => new GridColumnSettings(col_id(Columns.Last), DefaultColumnHeaders.Last, width: DefaultColumnWidths.Price); + public static GridColumnSettings last_from_spread => new GridColumnSettings(col_id(Columns.LastSpd), DefaultColumnHeaders.S, width: DefaultColumnWidths.Price, is_visible: false); public static GridColumnSettings last_size => new GridColumnSettings(col_id(Columns.LastSize), DefaultColumnHeaders.Size, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right); public static GridColumnSettings position => new GridColumnSettings(col_id(Columns.Position), DefaultColumnHeaders.Pos, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right); public static GridColumnSettings raw_day_position => new GridColumnSettings(col_id(Columns.RawDayPosition), DefaultColumnHeaders.RawDayPos, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right, is_visible: false); @@ -138,6 +139,7 @@ namespace bts.shell { yield return security; yield return volume; yield return last; + yield return last_from_spread; yield return last_size; yield return position; yield return raw_day_position; @@ -224,6 +226,8 @@ namespace bts.shell { [ProtoMember(25)] public string strip_name; [ProtoMember(26)] + public string last_price_from_spread; + [ProtoMember(28)] public string mic_code; } diff --git a/src/client/spark/Modules/Spreader/ExecutionStrategyEditor.xaml.cs b/src/client/spark/Modules/Spreader/ExecutionStrategyEditor.xaml.cs index 83d205f2d..ef8137e25 100644 --- a/src/client/spark/Modules/Spreader/ExecutionStrategyEditor.xaml.cs +++ b/src/client/spark/Modules/Spreader/ExecutionStrategyEditor.xaml.cs @@ -227,7 +227,7 @@ namespace mbts.ui.spark { } DataGridColumn _column_factory(LegSpreaderParamsTuple l) { - var col = new DataGridColumnInternal(l.leg.Security, ParameterCellTemplateConverter, ParameterCellEditingTemplateConverter) { Width = 75, }; + var col = new DataGridColumnInternal(l.leg.Security, ParameterCellTemplateConverter, ParameterCellEditingTemplateConverter); col.HeaderTemplate = (DataTemplate)FindResource(IsNew ? RemovableColumnHeaderTemplateKey : NonRemovableColumnHeaderTemplateKey); DataGridEx.SetRightBorder(col, new DataGridExVerticalLine(false, true, Colors.DarkGray, 1)); return col; diff --git a/src/client/spark/Modules/Spreader/SpreaderModel.cs b/src/client/spark/Modules/Spreader/SpreaderModel.cs index f34af26ab..916e7f078 100644 --- a/src/client/spark/Modules/Spreader/SpreaderModel.cs +++ b/src/client/spark/Modules/Spreader/SpreaderModel.cs @@ -1,7 +1,9 @@ -using bts.api; +using bts; +using bts.api; +using bts.api.core; using bts.api.synthetic; +using bts.collections; using bts.spark; -using mbts.ui.shell; using System; using System.Collections.Generic; using System.ComponentModel.Composition; @@ -18,11 +20,36 @@ namespace mbts.spark { readonly uint _user_id; [ImportingConstructor] - public SpreaderModel(IBus bus, SyntheticSecurityService service, ErrorReporter error, [Import(LoginContainer.Exports.UserID)]uint user_id) { + public SpreaderModel(IBus bus, SyntheticSecurityService service, AdminModel admin_model, ErrorReporter error, UserTuple user) { _bus = bus; _service = service; _error = error; - _user_id = user_id; + _user_id = user.id; + // synthetic tuples + var strategy_source = _bus.subscribe(SparkTags.SyntheticstrategiesTag) + .Select(map => MappedDictionary.create(map, (k, v) => (k, v))) + .Table(tup => tup.security_id); + + var synthetic_source = _bus.subscribe(SparkTags.SecuritiesTag) + .Select(map => new FilteredDictionary(map, smd => smd.security.ContractCase == Security.ContractOneofCase.Synthetic)) + .Table(smd => smd.security.Id); + + var synthetic_user_join = synthetic_source.LeftJoin( + admin_model.all_user_tuples, + SecurityMD.key_fn, + UserTuple.key_fn, + smd => smd.security.Synthetic.UserId, + (smd, sid, ut, uid) => new SyntheticTuple(smd.security, ut.name, null), + (smd, sid, uid) => new SyntheticTuple(smd.security, null, null), + SyntheticTuple.key_fn); + + synthetic_tuples = synthetic_user_join.LeftJoin( + strategy_source, + SyntheticTuple.key_fn, + tup => tup.security_id, + (synth_tup, id, strat_tup) => new SyntheticTuple(synth_tup.security, synth_tup.user_name, strat_tup.strategy), + (synth_tup, id) => synth_tup, + SyntheticTuple.key_fn); } public IObservable> strategy => _bus.subscribe(SparkTags.SyntheticstrategiesTag); @@ -36,6 +63,8 @@ namespace mbts.spark { .LeftJoin(strategies, (security, strategy) => (new ExecutionStrategyImmutable?(strategy), security), security => (new ExecutionStrategyImmutable?(), security)) ); + public IObservable> synthetic_tuples { get; } + public IObservable security_strategy(uint security_key) => strategy.TrySelect(s => { bool success = s.TryGetValue(security_key, out ExecutionStrategyImmutable result); diff --git a/src/client/spark/Modules/Spreader/SyntheticManagementView.xaml.cs b/src/client/spark/Modules/Spreader/SyntheticManagementView.xaml.cs index c3b7e6bea..e74b52c18 100644 --- a/src/client/spark/Modules/Spreader/SyntheticManagementView.xaml.cs +++ b/src/client/spark/Modules/Spreader/SyntheticManagementView.xaml.cs @@ -7,8 +7,15 @@ namespace mbts.ui.spark { /// Interaction logic for SyntheticManagementView.xaml /// partial class SyntheticManagementView : UserControl { + public enum Columns { + Name, + Description, + n_legs, + n_working_orders, + SecurityKey, + User + } - public enum Columns { Name, Description, n_legs, n_working_orders, SecurityKey} public SyntheticManagementView() => InitializeComponent(); } diff --git a/src/client/spark/Modules/Spreader/SyntheticManagementWindow.cs b/src/client/spark/Modules/Spreader/SyntheticManagementWindow.cs index 52458556b..cfbe2b7e3 100644 --- a/src/client/spark/Modules/Spreader/SyntheticManagementWindow.cs +++ b/src/client/spark/Modules/Spreader/SyntheticManagementWindow.cs @@ -10,6 +10,7 @@ using mbts.ui.utility; using mbts.utils; using System; using System.Collections.Generic; +using System.Collections.Immutable; using System.Linq; using System.Reactive.Linq; using System.Windows; @@ -27,7 +28,7 @@ namespace mbts.spark { readonly IWindowSettingsManager _settings_manager; - public SyntheticManagementWindow(SyntheticManagementWindowFactory factory, IBus bus, OrdersModel orders_model, StartupData state, WindowSettingsManager settings_manager) + public SyntheticManagementWindow(SyntheticManagementWindowFactory factory, IBus bus, OrdersModel orders_model, StartupData state, IWindowSettingsManager settings_manager) : base("Synthetic Management", bus, factory, SparkCommands.ShowSyntheticManagement.icon) { _orders_model = orders_model; _state = state; @@ -66,8 +67,8 @@ namespace mbts.spark { ), SparkCommand.create("Clone", Icons.CloneSynthetic, SparkTags.ShowSyntheticCloneTag).complete( current_row_connectable_builder.create(observe_row_strategy), rs => rs.HasValue, select_row_strategy_tuple - ), SparkCommands.ShowLadder(SparkCommands.KeyGestures.ShowLadder).complete( - vm_connectable_builder.create(vm => vm.current_security_md), md => md.HasValue, ConvertUtils.ValueOrDefault + ), SparkCommands.ShowLadderAutoLookup(SparkCommands.KeyGestures.ShowLadder).complete( + vm_connectable_builder.create(vm => vm.current_security_observable.Select(s => s.HasValue ? ImmutableHashSet.Empty.Add(s.Value.Id) : null)), id => id != null ), SparkCommands.ShowOrders.complete(() => { var ob_settings = bts.shell.orderbook.OrderBookStartupData.Default; ob_settings.filters = new bts.shell.orderbook.FilterStates { diff --git a/src/client/spark/Modules/Spreader/SyntheticManagementWindowFactory.cs b/src/client/spark/Modules/Spreader/SyntheticManagementWindowFactory.cs index e676dd306..29125b873 100644 --- a/src/client/spark/Modules/Spreader/SyntheticManagementWindowFactory.cs +++ b/src/client/spark/Modules/Spreader/SyntheticManagementWindowFactory.cs @@ -1,4 +1,5 @@ using bts.api; +using bts.settings; using mbts.persist; using mbts.spark; using mbts.spark.filters; @@ -7,6 +8,7 @@ using mbts.ui.settings; using ProtoBuf; using System; using System.Collections.Generic; +using System.ComponentModel.Composition; using System.Reactive.Linq; using System.Threading.Tasks; using static mbts.ui.controls.DIYGrid; @@ -33,13 +35,8 @@ namespace bts.shell { public async void show(SparkTags.Nothing nothing) => _bus.post(SparkTags.ShowWindow, await load(new StartupData())); - IShellWindow _create(StartupData state) { - var settings_manager = new WindowSettingsManager(_settings_hub, - state.grid, - get: us => us.synthetic_management_grid, - set: (us, gs) => us.with(provided_synthetic_management_gridsettings: gs)); - return new SyntheticManagementWindow(this, _bus, _orders_model, state, settings_manager); - } + IShellWindow _create(StartupData state) => + new SyntheticManagementWindow(this, _bus, _orders_model, state, _settings_hub.create_synth_settings_manager(state.grid)); [ProtoContract] public class StartupData { @@ -67,6 +64,7 @@ namespace bts.shell { public static GridColumnSettings n_legs => new GridColumnSettings(col_id(Columns.n_legs), DefaultColumnHeaders.N_Legs, width: DefaultColumnWidths.N_Legs); public static GridColumnSettings n_working_orders => new GridColumnSettings(col_id(Columns.n_working_orders), DefaultColumnHeaders.N_Working_Orders, width: DefaultColumnWidths.N_Working_Orders); public static GridColumnSettings security_key => new GridColumnSettings(col_id(Columns.SecurityKey), DefaultColumnHeaders.SecurityKey, is_visible: false, width: DefaultColumnWidths.SecurityKey); + public static GridColumnSettings user => new GridColumnSettings(col_id(Columns.User), DefaultColumnHeaders.User, is_visible: false, width: DefaultColumnWidths.User); public static IEnumerable default_sorting_criterion { get { @@ -81,6 +79,7 @@ namespace bts.shell { yield return n_legs; yield return n_working_orders; yield return security_key; + yield return user; } } @@ -101,6 +100,15 @@ namespace bts.shell { public UIntMinMaxState n_working_orders; [ProtoMember(6)] public string security_key; + [ProtoMember(7)] + public uint[] user; + } + + // migration rules + [Export(typeof(ISyntheticManagementMigrationRule))] + class AddMissingColumnsMigrationRule : ISyntheticManagementMigrationRule { + public bool apply(GridSettings settings, out GridSettings new_settings) => + settings.add_missing_columns(StartupData.DefaultGridSettings.columns, out new_settings); } } -} +} \ No newline at end of file diff --git a/src/client/spark/Modules/Spreader/VM/SyntheticManagementDialogVM.cs b/src/client/spark/Modules/Spreader/VM/SyntheticManagementDialogVM.cs index c6dd59102..1f63cb046 100644 --- a/src/client/spark/Modules/Spreader/VM/SyntheticManagementDialogVM.cs +++ b/src/client/spark/Modules/Spreader/VM/SyntheticManagementDialogVM.cs @@ -1,12 +1,15 @@ -using bts.api; +using bts; +using bts.api; using bts.collections; using bts.spark; +using bts.utils; using mbts.persist; using mbts.spark.filters; using mbts.ui; using mbts.ui.settings; using System; using System.Collections.Generic; +using System.Collections.Immutable; using System.Collections.ObjectModel; using System.Linq; using System.Reactive.Linq; @@ -17,43 +20,45 @@ using static mbts.ui.controls.DIYGrid; using static mbts.ui.spark.SyntheticManagementView; namespace mbts.spark { - class SyntheticManagementDialogVM : SecurityMDGrid { + class SyntheticManagementDialogVM : SecurityGrid { readonly IBus _bus; readonly OrdersModel _orders_model; - IWindowSettingsManager _settings_manager; - + readonly IWindowSettingsManager _settings_manager; public SyntheticManagementDialogVM(IBus bus, OrdersModel orders_model, IWindowSettingsManager settings_manager) : this(bus, orders_model, settings_manager, null, null) { } public SyntheticManagementDialogVM(IBus bus, OrdersModel orders_model, IWindowSettingsManager settings_manager, IReadOnlyCollection sorting, FilterStates filters) - : base(orders_model.spreader.strategy_tuple.Select(map => map.Values), sorting) { + : base(orders_model.spreader.synthetic_tuples.Select(t => t.state.Values), sorting) { _bus = bus; _orders_model = orders_model; + _settings_manager = settings_manager; + // init filters + user_filter = DiscreteValuesFilterVM.CreateTableUpdate(r => r.user_tuple, UserTuple.key_fn, out var user_filter_items_observer, out var user_filter_items_view, nameof(Row.user_tuple)); + user_filter_items_view.CustomSort = Comparer.Default.wrap(ut => ut.name).to_non_generic(); if (filters != null) { create_time_filter.load_state(filters.create_time); name_filter.load_state(filters.name); description_filter.load_state(filters.description); n_legs_filter.load_state(filters.n_legs); n_working_orders_filter.load_state(filters.n_working_orders); - security_key_filter.load_state(filters.security_key); + security_key_filter.load_state(filters.security_key); + user_filter.load_state(filters.user); } - _settings_manager = settings_manager; // columns add_col(Columns.Name, DefaultColumnHeaders.Name, r => r.name, nameof(Row.name), name_filter, DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.Security); add_col(Columns.Description, DefaultColumnHeaders.Description, r => r.description, nameof(Row.description), description_filter, DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.Description); add_col(Columns.n_legs, DefaultColumnHeaders.N_Legs, r => r.n_legs.ToString(), nameof(Row.n_legs), n_legs_filter, DataGridFilterKnownTemplates.IntMinMax, DefaultColumnWidths.N_Legs); columns.Add(new HyperlinkElementColumn( - Columns.n_working_orders, - new Binding(nameof(Row.n_working_orders)), + Columns.n_working_orders, + new Binding(nameof(Row.n_working_orders)), new Binding(nameof(Row.show_order_book)), n_working_orders_filter, DataGridFilterKnownTemplates.UIntMinMax) { id = col_id(Columns.n_working_orders), Header = DefaultColumnHeaders.N_Working_Orders, Width = DefaultColumnWidths.N_Working_Orders - } - ); + }); add_col(Columns.SecurityKey, DefaultColumnHeaders.SecurityKey, r => r.security_key.ToString(), nameof(Row.security_key), security_key_filter, DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.SecurityKey); - + add_col(Columns.User, DefaultColumnHeaders.User, r => r.user_name, nameof(Row.user_name), user_filter, DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.User); // void add_col(Columns col_key, string header, Func get_text, string prop, IDataGridColumnFilter filter, string filter_template, double width) => columns.Add(new RenderTextColumn(col_key, get_text, filter, filter_template) { @@ -63,42 +68,39 @@ namespace mbts.spark { Width = width }); // - grid_settings_manager = new SimpleGridSettingsManager(columns, new Dictionary { }, row_vms); + grid_settings_manager = new SimpleGridSettingsManager(columns, DictionaryHelper.empty(), row_vms); // this will initialize the immutable properties that correspond to the col_settings_flags above (e.g. Bid/Ask font_color_override, etc.) grid_settings_manager.load_from(settings_manager.current_settings()); _resources.AddRange( - DataGridFilterHelper.hook_filters(rows_view, create_time_filter, name_filter, description_filter, n_legs_filter, n_working_orders_filter, security_key_filter), + orders_model.admin_model.all_user_tuples.OnDispatcher().Subscribe(user_filter_items_observer), + DataGridFilterHelper.hook_all(this, rows_view), settings_manager.subscribe_current().OnDispatcher().Subscribe(grid_settings_manager) ); } - protected override Row row_factory((ExecutionStrategyImmutable? strategy, SecurityMD md) row_model) => - new Row(_bus, row_model.md, row_model.strategy, _orders_model); - - protected override void on_row_removed(Row row) => row.Dispose(); - - public ObservableCollection columns { get; } = new ObservableCollection(); + public ObservableCollection columns { get; } = new ObservableCollection(); public SimpleGridSettingsManager grid_settings_manager { get; } - public GridSettings get_grid_save_state() { - if (_settings_manager.instance_settings == null) - return null; - else - return grid_settings_manager.create(); - } - - public StartupData create() => - new StartupData(rows_view.get_sorting_criteria(columns), _get_filter_states(), get_grid_save_state()); - - - #region Filters public IDataGridColumnPersistentFilter create_time_filter { get; } = new TimeFilterVM(r => r.create_time, nameof(Row.create_time)); public IDataGridColumnPersistentFilter name_filter { get; } = new RegexFilterVM(r => r.name, nameof(Row.name)); public IDataGridColumnPersistentFilter description_filter { get; } = new RegexFilterVM(r => r.description, nameof(Row.description)); public IDataGridColumnPersistentFilter n_legs_filter { get; } = new IntMinMax(r => r.n_legs, nameof(Row.n_legs)); public IDataGridColumnPersistentFilter n_working_orders_filter { get; } = new UIntMinMax(r => r.n_working_orders, nameof(Row.n_legs)); public IDataGridColumnPersistentFilter security_key_filter { get; } = new RegexFilterVM(r => r.security_key.ToString(), nameof(Row.security_key)); + public IDataGridColumnPersistentFilter user_filter { get; } + + public GridSettings get_grid_save_state() => + _settings_manager.instance_settings == null ? null : grid_settings_manager.create(); + + public StartupData create() => + new StartupData(rows_view.get_sorting_criteria(columns), _get_filter_states(), get_grid_save_state()); + + protected override Row row_factory(SyntheticTuple row_model) => + new Row(_bus, row_model, _orders_model); + + protected override void on_row_removed(Row row) => + row.Dispose(); FilterStates _get_filter_states() => new FilterStates { @@ -107,19 +109,19 @@ namespace mbts.spark { name = name_filter.get_state(), n_legs = n_legs_filter.get_state(), n_working_orders = n_working_orders_filter.get_state(), - security_key = security_key_filter.get_state() + security_key = security_key_filter.get_state(), + user = user_filter.get_state() }; - #endregion - public class Row : SecurityMDRow, IEquatable, IRowVM, IActionableRowVM { - public readonly ExecutionStrategyImmutable? strategy; + public class Row : SecurityRow, IEquatable, IRowVM, IActionableRowVM { + public readonly SyntheticTuple synthetic_tuple; readonly IBus _bus; readonly Property _n_working_orders; - public Row(IBus bus, SecurityMD security_md, ExecutionStrategyImmutable? strategy, OrdersModel orders_model) : base(security_md) { + public Row(IBus bus, SyntheticTuple synthetic_tuple, OrdersModel orders_model) : base(synthetic_tuple.security) { _bus = bus; - this.strategy = strategy; + this.synthetic_tuple = synthetic_tuple; _n_working_orders = backing(nameof(n_working_orders), orders_model.on_security_orders(security_key) .Select(t => (uint)(t.state @@ -133,25 +135,23 @@ namespace mbts.spark { show_order_book = new DelegateCommand(_show_order_book); } + public event Action invalidated; + public DateTime create_time => DateTime.MinValue; //TODO public string name { get; } + public string user_name => synthetic_tuple.user_name; + public UserTuple user_tuple => new UserTuple(synthetic_tuple.security.Synthetic.UserId, user_name); public int n_legs { get; } public string description { get; } + public ExecutionStrategyImmutable? strategy => synthetic_tuple.strategy; public bool strategy_present => strategy.HasValue; public uint n_working_orders => _n_working_orders.Value; public ICommand show_ladder { get; } public ICommand show_order_book { get; } public ICommand default_command => show_ladder; - - public ObservableCollection row_animations => new ObservableCollection(); - - public IReadOnlyList cell_stores => new IGridCellStore[0]; - - - public event Action invalidated; - - public void raise_invalidated() => invalidated?.Invoke(); + public void raise_invalidated() => + invalidated?.Invoke(); public bool Equals(Row other) => other != null && @@ -163,10 +163,8 @@ namespace mbts.spark { public override int GetHashCode() => security_key.GetHashCode(); - - void _show_ladder() => - _bus.post(SparkTags.ShowLadderTag, md); + _bus.post(SparkTags.ShowLadderAutoLookupTag, ImmutableHashSet.Empty.Add(security.Id)); void _show_order_book() { var ob_settings = bts.shell.orderbook.OrderBookStartupData.Default; @@ -177,4 +175,4 @@ namespace mbts.spark { } } } -} +} \ No newline at end of file diff --git a/src/client/spark/Modules/Trading/AlgoStudio/AlgoStudioWindow.cs b/src/client/spark/Modules/Trading/AlgoStudio/AlgoStudioWindow.cs index 5d2b975e4..3edb390a9 100644 --- a/src/client/spark/Modules/Trading/AlgoStudio/AlgoStudioWindow.cs +++ b/src/client/spark/Modules/Trading/AlgoStudio/AlgoStudioWindow.cs @@ -62,7 +62,10 @@ namespace mbts.spark { _vm = new AlgoStudioVM(_model); _vm.load_from(_state); _view = new AlgoStudioView { DataContext = _vm }; - _resources = new CompositeDisposable(); + _resources = new CompositeDisposable( + _bus.subscribe(SparkTags.SecuritiesTag) + .OnDispatcher() + .Subscribe(sec_map => _view.Resources[ResourceKeys.Securities] = sec_map)); // add definition _commands.Add(new SparkCommandCommand("Add Algo", Icons.AddBoxes, _vm.add_definition_cmd)); // clone definition diff --git a/src/client/spark/Modules/Trading/AlgoStudio/Model/AlgoStudioFlatDefinitionModel.cs b/src/client/spark/Modules/Trading/AlgoStudio/Model/AlgoStudioFlatDefinitionModel.cs index a955270d7..aec98c8cd 100644 --- a/src/client/spark/Modules/Trading/AlgoStudio/Model/AlgoStudioFlatDefinitionModel.cs +++ b/src/client/spark/Modules/Trading/AlgoStudio/Model/AlgoStudioFlatDefinitionModel.cs @@ -27,7 +27,7 @@ namespace mbts.spark { _error_reporter = error_reporter; definitions = service.subscribei_algo_definitions() - .Filter(r => r.Metadata.IsActive && r.Metadata.CreatorUserId == _admin_model.user.id) + .Filter(r => r.Metadata.IsActive && _admin_model.user.can_access_user_data(r.Metadata.CreatorUserId)) .SelectKeyValue(_definition_data, d => d.key) .Table() .Cache(); diff --git a/src/client/spark/Modules/Trading/AlgoStudio/VM/AlgoParamRowVM.cs b/src/client/spark/Modules/Trading/AlgoStudio/VM/AlgoParamRowVM.cs index e7ebaa288..8d1297899 100644 --- a/src/client/spark/Modules/Trading/AlgoStudio/VM/AlgoParamRowVM.cs +++ b/src/client/spark/Modules/Trading/AlgoStudio/VM/AlgoParamRowVM.cs @@ -13,6 +13,9 @@ namespace mbts.spark { string _template_key; ICommand _set_value_cmd; + public AlgoParamRowVM(IAlgoVM algo, IAlgoParamVM param, AlgoParamRowVM parent) + : this(algo, param, parent, (param as ITemplateSelectorKey)?.key) { } + public AlgoParamRowVM(IAlgoVM algo, IAlgoParamVM param, AlgoParamRowVM parent, string template_key = null) { this.param = param; this.algo = algo; diff --git a/src/client/spark/Modules/Trading/AlgoStudio/View/AlgoEditorGridView/AlgoEditorGridView.xaml b/src/client/spark/Modules/Trading/AlgoStudio/View/AlgoEditorGridView/AlgoEditorGridView.xaml index ce29fd798..cabe66d50 100644 --- a/src/client/spark/Modules/Trading/AlgoStudio/View/AlgoEditorGridView/AlgoEditorGridView.xaml +++ b/src/client/spark/Modules/Trading/AlgoStudio/View/AlgoEditorGridView/AlgoEditorGridView.xaml @@ -247,6 +247,31 @@ + + + + + + + + + + + + + + + + + + + + + + + @@ -320,6 +345,21 @@ + + + + + diff --git a/src/client/spark/Modules/Trading/Fills/FillsVM.cs b/src/client/spark/Modules/Trading/Fills/FillsVM.cs index d600aef4f..75a876716 100644 --- a/src/client/spark/Modules/Trading/Fills/FillsVM.cs +++ b/src/client/spark/Modules/Trading/Fills/FillsVM.cs @@ -118,13 +118,14 @@ namespace mbts.spark { add_col_expl(Columns.Side, new SideColumn(Columns.Side, r => r.side, side_filter) { id = col_id(Columns.Side) }, side_filter); // all others (easily identified by the enum 'Columns.Foo') add_col(Columns.FillTime, DefaultColumnHeaders.FillTime, r => r.fill_time.ToString("M/d h:mm:ss tt"), new TimeFilterVM(r => r.fill.Timestamp.ToDateTimeOffset().LocalDateTime), DataGridFilterKnownTemplates.TimeFilter, DefaultColumnWidths.Time); - add_col(Columns.User, DefaultColumnHeaders.User, r => r.user, dv_filter_rx(_model.factory.subscribe_users().Select(d => d.Values), r => r.order?.user() ?? string.Empty), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.User); - add_col(Columns.Account, DefaultColumnHeaders.Account, r => r.account, dv_filter_rx(_model.factory.subscribe_accounts().Select(d => d.Values), r => r.account), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.Account); + add_col(Columns.User, DefaultColumnHeaders.User, r => r.user, dv_filter_rx(_model.factory.subscribe_users().Select(d => d.Values.Ordered()), r => r.order?.user() ?? string.Empty), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.User); + add_col(Columns.Account, DefaultColumnHeaders.Account, r => r.account, dv_filter_rx(_model.factory.subscribe_accounts().Select(d => d.Values.Ordered()), r => r.account), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.Account); add_col(Columns.Exchange, DefaultColumnHeaders.Exchange, r => r.exchange.ToString(), dv_filter(EnumHelper.get_values().ToArray(), r => r.security.exchange_or_default()), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.Exchange); add_col(Columns.TimeInForce, DefaultColumnHeaders.TimeInForce, r => r.time_in_force.ToString(), dv_filter(EnumHelper.get_values().ToArray(), r => r.order?.time_in_force() ?? TimeInForce.Undefined), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.TimeInForce); add_col(Columns.Size, DefaultColumnHeaders.Size, r => r.size.ToString(), new DoubleMinMax(r => r.fill.LastShares), DataGridFilterKnownTemplates.UIntMinMax, DefaultColumnWidths.Size); add_col(Columns.Price, DefaultColumnHeaders.Price, r => r.format_price(r.price), new DoubleMinMax(r => r.fill.LastPrice), DataGridFilterKnownTemplates.DoubleMinMax, DefaultColumnWidths.Price); add_col(Columns.FillKind, DefaultColumnHeaders.FillKind, r => r.fill_kind.ToString(), dv_filter(EnumHelper.get_values().ToArray(), r => r.fill.FillKind), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.FillKind); + add_col(Columns.FillStatus, DefaultColumnHeaders.FillStatus, r => r.fill_status.ToString(), dv_filter(EnumHelper.get_values().ToArray(), r => r.fill.FillStatus), DataGridFilterKnownTemplates.DiscreteValues, DefaultColumnWidths.FillKind); add_col(Columns.FillKey, DefaultColumnHeaders.FillKey, r => r.fill_key.ToString(), new RegexFilterVM(r => r.fill.FillKey), DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.FillKey); add_col(Columns.OrderKey, DefaultColumnHeaders.OrderKey, r => r.order_key.ToString(), new RegexFilterVM(r => r.fill.OrderKey.ToString()), DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.OrderKey); add_col(Columns.SecurityKey, DefaultColumnHeaders.SecurityKey, r => r.security_key.ToString(), new RegexFilterVM(r => r.fill.SecurityKey.ToString()), DataGridFilterKnownTemplates.Regex, DefaultColumnWidths.SecurityKey); @@ -146,6 +147,7 @@ namespace mbts.spark { }); _dispose = new CompositeDisposable( ObservableAutoWrapper.connect(col_tups, columns, ct => ct.col), + ObservableAutoWrapper.connect_item_hooks(col_tups, ct => ct.filter as IDisposable), RXCollection.connect(rows, fills_buffer.Filter(filter_src, FillTuple.key_fn), comp_src, ft => new Row(ft, model), r => r.fill_tuple), fills_source.OnDispatcher().Subscribe(t => { last_unfiltered_fills = t; @@ -227,6 +229,7 @@ namespace mbts.spark { case Columns.Exchange: return wrap_default(ft => ft.security.exchange_or_default()); case Columns.FillKey: return wrap(ft => ft.fill.FillKey); case Columns.FillKind: return wrap_default(ft => ft.fill.FillKind); + case Columns.FillStatus: return wrap_default(ft => ft.fill.FillStatus); case Columns.FillTime: return wrap(ft => ft.fill.Timestamp.to_local_time()); case Columns.OrderKey: return wrap(ft => ft.fill.OrderKey); case Columns.OrderType: return wrap_default(ft => ft.order?.order_type() ?? OrderType.None); @@ -293,6 +296,7 @@ namespace mbts.spark { public double price => _fill.LastPrice; public double size => _fill.LastShares; public FillKind fill_kind => _fill.FillKind; + public FillStatus fill_status => _fill.FillStatus; public string fill_key => _fill.FillKey; public string order_key => _order.Key.ToString(); diff --git a/src/client/spark/Modules/Trading/Fills/FillsView.xaml.cs b/src/client/spark/Modules/Trading/Fills/FillsView.xaml.cs index 197e97ea5..9e34bc3e9 100644 --- a/src/client/spark/Modules/Trading/Fills/FillsView.xaml.cs +++ b/src/client/spark/Modules/Trading/Fills/FillsView.xaml.cs @@ -21,6 +21,7 @@ namespace mbts.ui.spark { Price, Size, FillKind, + FillStatus, FillKey, OrderKey, SecurityKey diff --git a/src/client/spark/Modules/Trading/Fills/FillsWindowFactory.cs b/src/client/spark/Modules/Trading/Fills/FillsWindowFactory.cs index e89aa70a0..1f3c73ce7 100644 --- a/src/client/spark/Modules/Trading/Fills/FillsWindowFactory.cs +++ b/src/client/spark/Modules/Trading/Fills/FillsWindowFactory.cs @@ -37,7 +37,8 @@ namespace bts.shell { { nameof(FillsVM.Row.side), Columns.Side }, { nameof(FillsVM.Row.size), Columns.Size }, { nameof(FillsVM.Row.time_in_force), Columns.TimeInForce }, - { nameof(FillsVM.Row.user), Columns.User } + { nameof(FillsVM.Row.user), Columns.User }, + { nameof(FillsVM.Row.fill_status ), Columns.FillStatus }, }; public FillsWindowFactory(TradingShellModule module, IBus bus, UserSettingsHub settings_hub, OrdersModel orders_model) diff --git a/src/client/spark/Modules/Trading/Fills/Settings/FillsFilterStates.cs b/src/client/spark/Modules/Trading/Fills/Settings/FillsFilterStates.cs index b09603871..45077ef1d 100644 --- a/src/client/spark/Modules/Trading/Fills/Settings/FillsFilterStates.cs +++ b/src/client/spark/Modules/Trading/Fills/Settings/FillsFilterStates.cs @@ -74,6 +74,10 @@ namespace bts.shell.fills { [ProtoMember(15)] [DIYGridColumnID(nameof(Columns.Symbol))] public string[] symbol { get; set; } + + [ProtoMember(17)] + [DIYGridColumnID(nameof(Columns.FillStatus))] + public Fill.Types.FillStatus[] fill_status { get; set; } } static class FillsFilterStatesExtensions { @@ -96,6 +100,7 @@ namespace bts.shell.fills { pb.add_discrete_values(fs.side, ft => ft.fill.Side); pb.add_discrete_values(fs.fill_kind, ft => ft.fill.FillKind); pb.add_discrete_values(fs.symbol, ft => ft.security?.Symbol ?? string.Empty); + pb.add_discrete_values(fs.fill_status, ft => ft.fill.FillStatus); // numeric pb.add_numeric(fs.size, ft => ft.fill.LastShares); pb.add_numeric(fs.price, ft => ft.fill.LastPrice); diff --git a/src/client/spark/Modules/Trading/Ladder/LadderWindowFactory.cs b/src/client/spark/Modules/Trading/Ladder/LadderWindowFactory.cs index e7a16fdce..0d1b3ac40 100644 --- a/src/client/spark/Modules/Trading/Ladder/LadderWindowFactory.cs +++ b/src/client/spark/Modules/Trading/Ladder/LadderWindowFactory.cs @@ -1,4 +1,5 @@ using bts.api; +using bts.collections; using bts.md; using mbts; using mbts.config; @@ -50,6 +51,7 @@ namespace bts.shell { _mbo = mbo; _dispose = Disposable.Create( bus.subscribe(SparkTags.SecuritiesTag).Subscribe(s => _securities = s), + bus.subscribe(SparkTags.ShowLadderAutoLookupTag).Subscribe(_show_windows), bus.subscribe(SparkTags.ShowLadderTag).OnDispatcher().Subscribe(s => show_windows(new[] { s })), bus.subscribe(SparkTags.ShowLaddersTag).OnDispatcher().Subscribe(show_windows) ); @@ -273,7 +275,21 @@ namespace bts.shell { }); } - void show_windows(IReadOnlyCollection securities) { + async void _show_windows(IImmutableSet securities) { + if (securities == null) + return; + var start_time = DateTime.Now; + do { + var found = securities.KeyJoin(_securities).ToList(); + securities = securities.RemoveRange(found.Select(SecurityMD.key_fn)); + show_windows(found); + if (securities.Count == 0) + break; + await Task.Delay(500); + } while ((DateTime.Now - start_time) < _ladder_conf.security_timeout); + } + + void show_windows(IEnumerable securities) { int n_allowed = (int)_ladder_conf.max_open_ladders - _wm.list_windows().Where(w => w.window_type == _window_type).Count(); int n_opened = 0; foreach (var s in securities) { diff --git a/src/client/spark/Modules/Trading/Ladder/VM/LadderVM.cs b/src/client/spark/Modules/Trading/Ladder/VM/LadderVM.cs index f72d2cc1c..d5d1c38da 100644 --- a/src/client/spark/Modules/Trading/Ladder/VM/LadderVM.cs +++ b/src/client/spark/Modules/Trading/Ladder/VM/LadderVM.cs @@ -451,7 +451,7 @@ namespace mbts.spark { price = temp_price; } } - await _model.order_gateway.send_order(order, price, size); + await _model.order_gateway.send_order(security, order, price, size); if (auto_clear) size_calculator.value = 0; var fav_algo = order_properties_vm.selected_favorite_algo_definition; diff --git a/src/client/spark/Modules/Trading/Ladder/View/LadderView.xaml.cs b/src/client/spark/Modules/Trading/Ladder/View/LadderView.xaml.cs index 015680d5c..d3696d053 100644 --- a/src/client/spark/Modules/Trading/Ladder/View/LadderView.xaml.cs +++ b/src/client/spark/Modules/Trading/Ladder/View/LadderView.xaml.cs @@ -1,9 +1,13 @@ using bts; using bts.collections; using bts.utils; +using mbts.ui; using mbts.ui.converters; +using mbts.ui.extensions; using mbts.utils; +using System; using System.Linq; +using System.Reactive.Linq; using System.Windows; using System.Windows.Controls; using System.Windows.Data; @@ -80,6 +84,7 @@ namespace mbts.spark { _ladder.visible_columns_changed += () => _column_headers_presenter.Columns.SetItems( _ladder.visible_column_types.Select(t => ColumnFactory.create(_ladder, t))); this.vm = vm; + this.ConnectDataContext(lvm => lvm.bus.subscribe(SparkTags.SecuritiesTag).OnDispatcher().Subscribe(sec_map => Resources[ResourceKeys.Securities] = sec_map)); DataContext = vm; DataContextChanged += datacontext_changed; _ui_scaler.MouseDoubleClick += _ui_scaler_MouseDoubleClick; diff --git a/src/client/spark/Modules/Trading/ManualFills/ManualFillRowEditor.xaml b/src/client/spark/Modules/Trading/ManualFills/ManualFillRowEditor.xaml index 50a3369c2..b7ed9edf3 100644 --- a/src/client/spark/Modules/Trading/ManualFills/ManualFillRowEditor.xaml +++ b/src/client/spark/Modules/Trading/ManualFills/ManualFillRowEditor.xaml @@ -62,15 +62,15 @@ Header="{Binding security, Converter={StaticResource EditableSecurityTextConverter}}" IsEditMode="False" IsLazy="True" - SecurityMap="{DynamicResource ResourceKey={x:Static usp:ManualFillRowEditor.SecuritiesKey}}" + SecurityMap="{DynamicResource ResourceKey={x:Static u:ResourceKeys.Securities}}" SelectedSecurity="{Binding security, UpdateSourceTrigger=Explicit}" ShowDialogSkirt="True" ToolTip="{Binding SelectedCriteria, RelativeSource={RelativeSource Self}, Converter={x:Static c:ContractPickerPopup.SecurityCriteriaTextConverter}, Mode=OneWay}" OKClick="{x:Static c:ContractPickerPopup.FlushSelectedSecurityHandler}" /> - - - + + + diff --git a/src/client/spark/Modules/Trading/ManualFills/ManualFillRowEditor.xaml.cs b/src/client/spark/Modules/Trading/ManualFills/ManualFillRowEditor.xaml.cs index 0c597bf14..48a5d5ffa 100644 --- a/src/client/spark/Modules/Trading/ManualFills/ManualFillRowEditor.xaml.cs +++ b/src/client/spark/Modules/Trading/ManualFills/ManualFillRowEditor.xaml.cs @@ -6,12 +6,6 @@ namespace mbts.ui.spark { /// Interaction logic for ManualFillRowEditor.xaml /// partial class ManualFillRowEditor : UserControl { - public const string - SecuritiesKey = nameof(SecuritiesKey), - AccountsKey = nameof(AccountsKey), - UsersKey = nameof(UsersKey), - SidesKey = nameof(SidesKey); - public static readonly RoutedCommand CommitCommand = new RoutedCommand(), CancelCommand = new RoutedCommand(); diff --git a/src/client/spark/Modules/Trading/ManualFills/ManualFillsVM.cs b/src/client/spark/Modules/Trading/ManualFills/ManualFillsVM.cs index df904fd80..df4e03f43 100644 --- a/src/client/spark/Modules/Trading/ManualFills/ManualFillsVM.cs +++ b/src/client/spark/Modules/Trading/ManualFills/ManualFillsVM.cs @@ -70,7 +70,7 @@ namespace bts.shell.manual_fills { mfd => new ManualFillRow(mfd), mfr => mfr.data), _bus.subscribe(SparkTags.SecuritiesTag).OnDispatcher().Subscribe(m => { - security_map = m; + security_map = new FilteredDictionary(m, smd => smd.security.ContractCase == Security.ContractOneofCase.Future); RaisePropertyChanged(nameof(security_map)); }), ObservableAutoWrapper.connect_item_hooks(_persisted_rows, r => Disposable.Create(dirty_mon.join(r), selection_mon.join(r))), diff --git a/src/client/spark/Modules/Trading/ManualFills/ManualFillsView.xaml.cs b/src/client/spark/Modules/Trading/ManualFills/ManualFillsView.xaml.cs index 78fb5b10b..cbc3ed8bc 100644 --- a/src/client/spark/Modules/Trading/ManualFills/ManualFillsView.xaml.cs +++ b/src/client/spark/Modules/Trading/ManualFills/ManualFillsView.xaml.cs @@ -50,11 +50,11 @@ namespace mbts.ui.spark { row_editor.CommandBindings.Add(new CommandBinding(ManualFillRowEditor.CommitCommand, commit_executed)); row_editor.CommandBindings.Add(new CommandBinding(ManualFillRowEditor.CancelCommand, cancel_executed)); // dynamic resources - row_editor.Resources[ManualFillRowEditor.AccountsKey] = vm.accounts; - row_editor.Resources[ManualFillRowEditor.UsersKey] = vm.users; - row_editor.Resources[ManualFillRowEditor.SidesKey] = vm.sides; + row_editor.Resources[ResourceKeys.Accounts] = vm.accounts; + row_editor.Resources[ResourceKeys.Users] = vm.users; + row_editor.Resources[ResourceKeys.Sides] = vm.sides; FocusManager.SetIsFocusScope(row_editor, true); - var sec_hook = vm.as_demuxer().subscribe(nameof(ManualFillsVM.security_map)).OnDispatcher().Subscribe(v => row_editor.Resources[ManualFillRowEditor.SecuritiesKey] = v); + var sec_hook = vm.as_demuxer().subscribe(nameof(ManualFillsVM.security_map)).OnDispatcher().Subscribe(v => row_editor.Resources[ResourceKeys.Securities] = v); // popup adorner var popup_adorner = new PopupAdorner( data_grid, data_grid, diff --git a/src/client/spark/Modules/Trading/OrderBook/OrderBookColumns.cs b/src/client/spark/Modules/Trading/OrderBook/OrderBookColumns.cs index b9330c030..eb5a11bfb 100644 --- a/src/client/spark/Modules/Trading/OrderBook/OrderBookColumns.cs +++ b/src/client/spark/Modules/Trading/OrderBook/OrderBookColumns.cs @@ -121,7 +121,7 @@ namespace mbts.spark { r => r.user, DefaultColumnWidths.User, DiscreteValuesFilterVM.Create( - model.factory.subscribe_users().Select(d => d.Values), + model.factory.subscribe_users().Select(d => d.Values.Ordered()), ort => ort.order.order.user(), nameof(OrderBookRowVM.user)), DataGridFilterKnownTemplates.DiscreteValues, @@ -133,7 +133,7 @@ namespace mbts.spark { r => r.account, DefaultColumnWidths.Account, DiscreteValuesFilterVM.Create( - model.factory.subscribe_accounts().Select(d => d.Values), + model.factory.subscribe_accounts().Select(d => d.Values.Ordered()), ort => ort.order.order.account(), vals => vals.ToArray(), vals => Observable.Return(vals), diff --git a/src/client/spark/Modules/Trading/OrderBook/OrderBookVM.cs b/src/client/spark/Modules/Trading/OrderBook/OrderBookVM.cs index 54ffe800c..612a0adeb 100644 --- a/src/client/spark/Modules/Trading/OrderBook/OrderBookVM.cs +++ b/src/client/spark/Modules/Trading/OrderBook/OrderBookVM.cs @@ -239,7 +239,7 @@ namespace mbts.spark { grid_settings_manager, order_mod_vm.algo_vm.connect_data(Observable.Return(model)), order_mod_vm.algo_vm.connect_data(_selected_params_mon.subscribe_spreader_security()), - order_mod_vm.algo_vm, + order_mod_vm, attach_vm, _show_fills.observe_current(true) .Subscribe(is_visible => { @@ -288,7 +288,14 @@ namespace mbts.spark { double price = attach_vm.price; uint? size = attach_vm.size; uint algo_template_key = attach_vm.algo_vm.algo_root.id.is_persisted ? attach_vm.algo_vm.algo_root.id.persisted : 0u; - var order_tups = selected_rows.Select(r => r.order_tuple).ToList(); + // Don't pass any order tuples that don't have a security defined. + // The actual security is passed to the OrdersModel - instead of just the security key - + // so it can prevent GlbxIceberg orders from being sent to a non-glbx exchange (such as Ice). + /// Impl details: + var order_tups = selected_rows + .Select(r => r.order_tuple) + .Where(ot => ot.order.security.HasValue) + .ToList(); var mod_algo_vm = attach_vm.algo_vm.algo_root; await Task.WhenAll(order_tups.Select(t => { var algo = t.order.order.Def.CustomDef.actual().CloneToMutable(); @@ -302,6 +309,7 @@ namespace mbts.spark { }; } return _model.send_order( + t.order.security.Value, t.order.order.Def.with( attach_target_order_key: t.order.order.Key, custom_def: algo @@ -727,17 +735,13 @@ namespace mbts.spark { _size_permissions = AlgoParamPermissions.ReadOnly | AlgoParamPermissions.Write; SecurityMD? _security_md; - public OrderParamsVM(OrderBookVM owner, AlgoFormats init_format, AlgoEditingPermissions init_permissions) - : this(owner, - AlgoDefinitionVM.create( - owner._model, - AlgoData.create_unpersisted(OrderDataHelper.Defaults.CustomOrderDefinitionImmutable(CustomOrderDefinition.CustomOneofCase.Iceberg), CustomOrderDefinition.CustomOneofCase.Iceberg.display_name()), - init_format: init_format, - editing_permissions: init_permissions)) { } - - OrderParamsVM(OrderBookVM owner, AlgoDefinitionVM algo_vm) { + public OrderParamsVM(OrderBookVM owner, AlgoFormats init_format, AlgoEditingPermissions init_permissions) { _owner = owner; - this.algo_vm = algo_vm; + algo_vm = AlgoDefinitionVM.create( + owner._model, + AlgoData.create_unpersisted(OrderDataHelper.Defaults.CustomOrderDefinitionImmutable(CustomOrderDefinition.CustomOneofCase.Iceberg), CustomOrderDefinition.CustomOneofCase.Iceberg.display_name()), + init_format: init_format, + editing_permissions: init_permissions); _accounts = backing( nameof(selected_account), _model.factory.subscribe_account_tuples(), @@ -776,7 +780,8 @@ namespace mbts.spark { appended_param_rows.insert_front(new AlgoParamRowVM(root, _price_param, null, AlgoParamTemplateKeys.Price)); } }), - ObservableAutoWrapper.connect_item_hooks(param_rows, dirty_params_mon.join)); + ObservableAutoWrapper.connect_item_hooks(param_rows, dirty_params_mon.join), + algo_vm); // payup methods void payup_bid(int ticks) => payup(bbo.bid, -ticks); void payup_ask(int ticks) => payup(bbo.ask, ticks); diff --git a/src/client/spark/Modules/Trading/OrderBook/OrderBookWindow.cs b/src/client/spark/Modules/Trading/OrderBook/OrderBookWindow.cs index bcbc64aca..b416e6937 100644 --- a/src/client/spark/Modules/Trading/OrderBook/OrderBookWindow.cs +++ b/src/client/spark/Modules/Trading/OrderBook/OrderBookWindow.cs @@ -66,6 +66,10 @@ namespace mbts.spark { _vm = new OrderBookVM(_bus, _notification_service, _model, _settings_manager); _vm.load_from(_state); _view = new OrderBookView(_vm); + _vm.SubscribeDisposed( + _bus.subscribe(SparkTags.SecuritiesTag) + .OnDispatcher() + .Subscribe(sec_map => _view.Resources[ResourceKeys.Securities] = sec_map)); var show_fills_command = SparkCommands.ShowFills(SparkCommands.KeyGestures.ShowFills).complete(() => new FillsSettings(_vm.selected_orders.Select(r => r.Key).ToArray())); diff --git a/src/client/spark/Modules/Trading/PNL/Data/PNLData.cs b/src/client/spark/Modules/Trading/PNL/Data/PNLData.cs index 9c941fc55..b6a83ae6a 100644 --- a/src/client/spark/Modules/Trading/PNL/Data/PNLData.cs +++ b/src/client/spark/Modules/Trading/PNL/Data/PNLData.cs @@ -501,6 +501,12 @@ namespace bts.shell.pnl { account_name == other.account_name && sod == other.sod && day == other.day && + bid_price.Equals(other.bid_price) && + ask_price.Equals(other.ask_price) && + last_price.Equals(other.last_price) && + mid_price.Equals(other.mid_price) && + settlement_price.Equals(other.settlement_price) && + calc_type.Equals(other.calc_type) && raw_day == other.raw_day && working.summary == other.working.summary; diff --git a/src/client/spark/Modules/Trading/PNL/VM/PNLRowVM.cs b/src/client/spark/Modules/Trading/PNL/VM/PNLRowVM.cs index f32cee536..52cfa3ff0 100644 --- a/src/client/spark/Modules/Trading/PNL/VM/PNLRowVM.cs +++ b/src/client/spark/Modules/Trading/PNL/VM/PNLRowVM.cs @@ -97,22 +97,22 @@ namespace bts.shell.pnl { if (security.HasValue && security.Value.ContractCase != Security.ContractOneofCase.Future) { day_long = day_short = sod_long = sod_short = 0; } else { - day_long = (decimal)DoubleUtil.nan_to_zero(data.day.long_pos); - day_short = (decimal)DoubleUtil.nan_to_zero(data.day.short_pos); - sod_long = (decimal)DoubleUtil.nan_to_zero(data.sod.long_pos); - sod_short = (decimal)DoubleUtil.nan_to_zero(data.sod.short_pos); + day_long = DoubleUtil.round_to_decimal(data.day.long_pos); + day_short = DoubleUtil.round_to_decimal(data.day.short_pos); + sod_long = DoubleUtil.round_to_decimal(data.sod.long_pos); + sod_short = DoubleUtil.round_to_decimal(data.sod.short_pos); } var nd = new PNLNormalizedData( - (decimal)DoubleUtil.nan_to_zero(day_pnl.value), + DoubleUtil.round_to_decimal(day_pnl.value), day_long, day_short, - (decimal)DoubleUtil.nan_to_zero(data.raw_day.long_pos), - (decimal)DoubleUtil.nan_to_zero(data.raw_day.short_pos), - (decimal)DoubleUtil.nan_to_zero(sod_pnl.value), + DoubleUtil.round_to_decimal(data.raw_day.long_pos), + DoubleUtil.round_to_decimal(data.raw_day.short_pos), + DoubleUtil.round_to_decimal(sod_pnl.value), sod_long, sod_short, - (decimal)DoubleUtil.nan_to_zero(data.working_long_pos), - (decimal)DoubleUtil.nan_to_zero(data.working_short_pos)); + DoubleUtil.round_to_decimal(data.working_long_pos), + DoubleUtil.round_to_decimal(data.working_short_pos)); if (_normalized_data != nd) { _normalized_data = nd; updated?.Invoke(nd); diff --git a/src/client/spark/Modules/Trading/PNL/VM/PNLVM.cs b/src/client/spark/Modules/Trading/PNL/VM/PNLVM.cs index b04c03ead..19b6ff8da 100644 --- a/src/client/spark/Modules/Trading/PNL/VM/PNLVM.cs +++ b/src/client/spark/Modules/Trading/PNL/VM/PNLVM.cs @@ -103,7 +103,7 @@ namespace bts.shell.pnl { load_from(settings); _dispose = Disposable.Create( pnl_source.Operate( - _calculation_type.observe_current(), + _calculation_type.observe_current(true), (data, calc) => data.with(calc_type: calc) ).FilterWithRaw(filter_src, PNLRowData.key_fn, null) .OnDispatcher() diff --git a/src/client/spark/Modules/Trading/PNL/View/PNLRowCollectionView.cs b/src/client/spark/Modules/Trading/PNL/View/PNLRowCollectionView.cs index 083556ab7..d3dec5aa3 100644 --- a/src/client/spark/Modules/Trading/PNL/View/PNLRowCollectionView.cs +++ b/src/client/spark/Modules/Trading/PNL/View/PNLRowCollectionView.cs @@ -373,7 +373,9 @@ namespace bts.shell.pnl { var less_last = data - last_data; var plus_new = less_last + new_data; data += new_data - last_data; - Log.assert(plus_new == data); + if (plus_new != data) { + Log.Ui.warn($"Detected PNL arithmetic rounding error in the group sum for '{string.Join(" ", GroupNamePath)}' (this bug is under investigation)."); + } last_data = new_data; } void on_is_selected_changed(bool is_selected) { diff --git a/src/client/spark/Orders/AlgoParamVM.cs b/src/client/spark/Orders/AlgoParamVM.cs index e8ded6009..cc7a43a2a 100644 --- a/src/client/spark/Orders/AlgoParamVM.cs +++ b/src/client/spark/Orders/AlgoParamVM.cs @@ -1,4 +1,5 @@ -using mbts.ui.shell; +using mbts.ui; +using mbts.ui.shell; using System; using System.Collections.Generic; using System.ComponentModel; @@ -6,8 +7,9 @@ using System.Reactive.Linq; using System.Runtime.CompilerServices; namespace mbts.spark { - class AlgoParamVM : IAlgoParamVM, INotifyPropertyChanged { + class AlgoParamVM : IAlgoParamVM, INotifyPropertyChanged, ITemplateSelectorKey { readonly IEqualityComparer _comparer; + readonly string _template_key; AlgoParamPermissions _permissions; bool _is_enabled, _has_value; T _committed_value, _value; @@ -15,15 +17,16 @@ namespace mbts.spark { public AlgoParamVM(string id, string display_name) : this(id, display_name, default, EqualityComparer.Default) { } - public AlgoParamVM(string id, string display_name, T value) - : this(id, display_name, value, EqualityComparer.Default) { } + public AlgoParamVM(string id, string display_name, T value, string template_key = null) + : this(id, display_name, value, EqualityComparer.Default, template_key) { } - public AlgoParamVM(string id, string display_name, T value, IEqualityComparer comparer) { + public AlgoParamVM(string id, string display_name, T value, IEqualityComparer comparer, string template_key = null) { this.id = id; this.display_name = display_name; _value = value; _committed_value = value; _comparer = comparer; + _template_key = template_key; } public event PropertyChangedEventHandler PropertyChanged; @@ -50,7 +53,9 @@ namespace mbts.spark { public bool is_enabled { get => _is_enabled; set => _set(ref _is_enabled, value); } public bool has_value { get => _has_value; set => _set(ref _has_value, value); } public bool is_dirty { get; private set; } - object IAlgoParamVM.value { get => value; set => this.value = (T)value; } + object IAlgoParamVM.value { get => value; set => this.value = (T)value; } + + string ITemplateSelectorKey.key => _template_key; public void revert() { if (is_dirty) { diff --git a/src/client/spark/Orders/NewOrderTicketVM.cs b/src/client/spark/Orders/NewOrderTicketVM.cs index f9945b649..4e5b36122 100644 --- a/src/client/spark/Orders/NewOrderTicketVM.cs +++ b/src/client/spark/Orders/NewOrderTicketVM.cs @@ -53,7 +53,7 @@ namespace mbts.spark { send_order = new DelegateCommand(side => { bool price_valid = _validate_price(), size_valid = size_calculator.validate(); if (price_valid && size_valid && !HasErrors && !properties_vm.HasErrors) { - model.send_order(properties_vm.create(side), price, size_calculator.value.Value).ignore(); + model.send_order(security, properties_vm.create(side), price, size_calculator.value.Value).ignore(); RequestClose?.Invoke(this, new EventArgs()); } }); diff --git a/src/client/spark/Orders/OrderDataHelper.cs b/src/client/spark/Orders/OrderDataHelper.cs index 27df49a4c..6d26b2a3a 100644 --- a/src/client/spark/Orders/OrderDataHelper.cs +++ b/src/client/spark/Orders/OrderDataHelper.cs @@ -67,7 +67,6 @@ namespace mbts.spark.utilities { public IObservable> subscribe_accounts() => _admin_model.on_accounts; public IObservable> subscribe_users() => _admin_model.on_users; - public IObservable> subscribe_user_tuples() => _admin_model.on_user_tuples; public IObservable> subscribe_account_tuples() => _admin_model.on_account_tuples; #region Defaults diff --git a/src/client/spark/Orders/OrderTypeVMs/DynamicOrderHelper.cs b/src/client/spark/Orders/OrderTypeVMs/DynamicOrderHelper.cs index 81178590c..7a657e393 100644 --- a/src/client/spark/Orders/OrderTypeVMs/DynamicOrderHelper.cs +++ b/src/client/spark/Orders/OrderTypeVMs/DynamicOrderHelper.cs @@ -62,6 +62,7 @@ namespace mbts.spark { case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveInt: param.AsInt32 = 0; break; case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveOrderDef: param.AsOrderDef = new CustomOrderDefinition(); break; case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveString: param.AsString = string.Empty; break; + case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveSecurityKey: case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveUint: param.AsUint32 = 0u; break; case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveQuantity: param.AsQuantity = new bts.spark.QuantityMeasure(); break; } diff --git a/src/client/spark/Orders/OrderTypeVMs/DynamicOrderParamVM.cs b/src/client/spark/Orders/OrderTypeVMs/DynamicOrderParamVM.cs index ade3260a4..b2cf2b1d3 100644 --- a/src/client/spark/Orders/OrderTypeVMs/DynamicOrderParamVM.cs +++ b/src/client/spark/Orders/OrderTypeVMs/DynamicOrderParamVM.cs @@ -12,7 +12,7 @@ using System.Reactive.Linq; namespace mbts.spark { interface IDynamicOrderParamVM { DynamicOrderParamDescriptor descriptor { get; } - DynamicOrderParam param { get; set; } + DynamicOrderParam param { get; } } interface IDynamicOrderParamVM : IDynamicOrderParamVM { @@ -21,26 +21,16 @@ namespace mbts.spark { } abstract class DynamicOrderParamVM : DisposableViewModelBase, IDynamicOrderParamVM { - DynamicOrderParam _param; - protected DynamicOrderParamVM(DynamicOrderParamDescriptor descriptor) : this(descriptor, DynamicOrderHelper.init_param(descriptor)) { } private DynamicOrderParamVM(DynamicOrderParamDescriptor descriptor, DynamicOrderParam param) { this.descriptor = descriptor; - _param = param; + this.param = param; } public DynamicOrderParamDescriptor descriptor { get; set; } - public DynamicOrderParam param { - get => _param; - set { - _param = value; - on_param_changed(); - } - } - - protected abstract void on_param_changed(); + public DynamicOrderParam param { get; } } class DynamicOrderParamVM : DynamicOrderParamVM, IDynamicOrderParamVM { @@ -53,14 +43,12 @@ namespace mbts.spark { _field_desc = field_descriptor; _convert = convert; _convert_back = convert_back; + value = _convert(ConvertUtils.try_convert(_field_desc.Accessor.GetValue(param))); } public TView value { get => _value; - set { - _field_desc.Accessor.SetValue(param, _convert_back(_value = value)); - RaisePropertyChanged(); - } + set => Set(ref _value, value); } public IObservable subscribe_value() => @@ -70,11 +58,6 @@ namespace mbts.spark { void on_prop_changed(object sender, PropertyChangedEventArgs e) => o.OnNext(value); }); - - protected override void on_param_changed() { - _value = _convert(ConvertUtils.try_convert(_field_desc.Accessor.GetValue(param))); - RaisePropertyChanged(nameof(value)); - } } class DynamicOrderEnumeratedVM : DynamicOrderParamVM, IDynamicOrderParamVM { @@ -105,11 +88,6 @@ namespace mbts.spark { o.OnNext(selected_element); }); - protected override void on_param_changed() { - _sync_elements(); - RaisePropertyChanged(nameof(selected_element)); - } - static IEnumerable _get_elements(DynamicOrderParamDescriptor descriptor) => descriptor.ParamType?.DescriptorCase == DynamicOrderTypeDescriptor.DescriptorOneofCase.Enumerated ? descriptor.ParamType.Enumerated.Element : Enumerable.Empty(); @@ -148,10 +126,5 @@ namespace mbts.spark { o.OnNext(quantity); }); } - - protected override void on_param_changed() - { - RaisePropertyChanged(nameof(quantity)); - } } } diff --git a/src/client/spark/Orders/OrderTypeVMs/DynamicOrderTemplateSelector.cs b/src/client/spark/Orders/OrderTypeVMs/DynamicOrderTemplateSelector.cs index cdf5a4906..d27c0e4d4 100644 --- a/src/client/spark/Orders/OrderTypeVMs/DynamicOrderTemplateSelector.cs +++ b/src/client/spark/Orders/OrderTypeVMs/DynamicOrderTemplateSelector.cs @@ -20,6 +20,7 @@ namespace mbts.ui.spark public DataTemplate StringTemplate { get; set; } public DataTemplate OrderDefTemplate { get; set; } public DataTemplate QuantityMeasureTemplate { get; set; } + public DataTemplate SecurityKeyTemplate { get; set; } public DataTemplate EnumTemplate { get; set; } @@ -38,6 +39,7 @@ namespace mbts.ui.spark case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveDouble: return DoubleTemplate; case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveInt: return IntTemplate; case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveUint: return UIntTemplate; + case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveSecurityKey: return SecurityKeyTemplate; case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveString: return StringTemplate; case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveOrderDef: return OrderDefTemplate; case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveQuantity: return QuantityMeasureTemplate; diff --git a/src/client/spark/Orders/OrderTypeVMs/DynamicOrderVM.cs b/src/client/spark/Orders/OrderTypeVMs/DynamicOrderVM.cs index d69230589..d28c554a2 100644 --- a/src/client/spark/Orders/OrderTypeVMs/DynamicOrderVM.cs +++ b/src/client/spark/Orders/OrderTypeVMs/DynamicOrderVM.cs @@ -4,6 +4,7 @@ using bts.collections; using bts.spark; using mbts.collections; using mbts.spark.utilities; +using mbts.ui; using mbts.ui.utility; using mbts.utils; using System; @@ -73,9 +74,9 @@ namespace mbts.spark { case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveDouble: return mk(DynamicOrderParam.AsDoubleFieldNumber, v => v, v => v, double.NaN); case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveInt: return mk(DynamicOrderParam.AsInt32FieldNumber, v => v, v => v.GetValueOrDefault()); case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveUint: return mk(DynamicOrderParam.AsUint32FieldNumber, v => v, v => v.GetValueOrDefault()); + case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveSecurityKey: return mk(DynamicOrderParam.AsUint32FieldNumber, v => v, v => v.GetValueOrDefault(), template_key: TemplateKeys.SecurityKey); case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveString: return mk(DynamicOrderParam.AsStringFieldNumber, v => v, v => v); - case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveQuantity: - { + case DynamicOrderPrimitiveDescriptor.DynamicOrderPrimitiveQuantity: { var qm = new DynamicOrderQuantityVM(descriptor); return attached(qm, DynamicOrderHelper.get_quantity_value, DynamicOrderHelper.set_quantity_value, new QuantityMeasure { Ratio = 1.0, Absolute = 1 }); } @@ -88,16 +89,16 @@ namespace mbts.spark { default: throw ExceptionBuilder.Argument.unexpected_case(descriptor.ParamType.DescriptorCase, nameof(descriptor)); } - DynamicOrderParamVM mk(int field_number, Func convert, Func convert_back, TView default_value = default) { + DynamicOrderParamVM mk(int field_number, Func convert, Func convert_back, TView default_value = default, string template_key = null) { var fdesc = DynamicOrderParam.Descriptor.FindFieldByNumber(field_number); var pvm = new DynamicOrderParamVM(descriptor, fdesc, convert, convert_back); - return attached>(pvm, get_from_param, assign_to_param, default_value); + return attached>(pvm, get_from_param, assign_to_param, default_value, template_key); TView get_from_param(DynamicOrderParam param) => convert(ConvertUtils.try_convert(fdesc.Accessor.GetValue(param))); void assign_to_param(DynamicOrderParam param, TView value) => fdesc.Accessor.SetValue(param, convert_back(value)); } - TParam attached(TParam pvm, Func selector, Action assign_to_param, T default_value = default) where TParam : DynamicOrderParamVM, IDynamicOrderParamVM { + TParam attached(TParam pvm, Func selector, Action assign_to_param, T default_value = default, string template_key = null) where TParam : DynamicOrderParamVM, IDynamicOrderParamVM { add_property( pvm.descriptor.ParamName, mdef => mdef.OrderParams.TryGetValue(pvm.descriptor.ParamName, out DynamicOrderParam p) ? selector(p) : default_value, @@ -105,7 +106,8 @@ namespace mbts.spark { () => pvm.value, v => pvm.value = v, pvm.subscribe_value(), - default_value); + default_value, + template_key: template_key); return pvm; } } diff --git a/src/client/spark/Orders/OrderTypeVMs/OrderTypeVMFactory.cs b/src/client/spark/Orders/OrderTypeVMs/OrderTypeVMFactory.cs index b24547c8e..4243de0be 100644 --- a/src/client/spark/Orders/OrderTypeVMs/OrderTypeVMFactory.cs +++ b/src/client/spark/Orders/OrderTypeVMs/OrderTypeVMFactory.cs @@ -89,6 +89,7 @@ namespace mbts.spark { case AlgoType.Dynamic: vm = dynamic(key, name, name, editing_permissions); break; case AlgoType.Splitter: vm = splitter(key,name,editing_permissions); break; case AlgoType.Composite: vm = composite(key, name, editing_permissions); break; + case AlgoType.None: vm = new DefaultAlgoVM(); break; default: vm = generic(key, name, algo_type, editing_permissions); break; //default: // Log.Ui.debug_fatal($"Failed to create {typeof(IAlgoVM)}: unknown algo type '{algo_type}.'"); @@ -166,7 +167,8 @@ namespace mbts.spark { }; void subscribe_new_algo(IAlgoVM old_algo, IAlgoVM new_algo) { RaisePropertyChanged(sd.prop_name); - Disposable.replace(ref def_sub, new CompositeDisposable()); + Disposable.dispose(ref def_sub); + def_sub = new CompositeDisposable(); if (assert(new_algo != null, sd.prop_name, "No algo specified.")) { def_sub.Add(new_algo.subscribe_definition().Subscribe(_ => _def_change_notifier.execute())); if (new_algo is INotifyDataErrorInfo ndei) @@ -255,14 +257,14 @@ namespace mbts.spark { _security_changed += o.OnNext; return Disposable.Create(() => _security_changed -= o.OnNext); }); - Disposable.replace(ref _pending_load, Disposable.Create( + Disposable.dispose(ref _pending_load); + _pending_load = Disposable.Create( // load children ObservableAutoWrapper.connect_item_hooks( _child_algo_selector_data.AsCollection(), sd => security_src.SubscribeEach(_ => ObservableAutoWrapper.connect_item_actions(_child_algo_selector_data.AsCollection(), selector_load_impl))), // load regular props - ObservableAutoWrapper.connect_item_actions(_auto_props.AsCollection(), p => p.load(algom)))); - + ObservableAutoWrapper.connect_item_actions(_auto_props.AsCollection(), p => p.load(algom))); } void selector_load_impl(AlgoSelectorData sd) { @@ -298,11 +300,12 @@ namespace mbts.spark { _security_changed += o.OnNext; return Disposable.Create(() => _security_changed -= o.OnNext); }); - Disposable.replace(ref _pending_load, Disposable.Create( + Disposable.dispose(ref _pending_load); + _pending_load = Disposable.Create( ObservableAutoWrapper.connect_item_hooks( _child_algo_selector_data.AsCollection(), sd => security_src.Subscribe(_ => ObservableAutoWrapper.connect_item_actions(_child_algo_selector_data.AsCollection(), selector_load_impl))), - ObservableAutoWrapper.connect_item_actions(_auto_props.AsCollection(), p => p.load_many(algos)))); + ObservableAutoWrapper.connect_item_actions(_auto_props.AsCollection(), p => p.load_many(algos))); void selector_load_impl(AlgoSelectorData sd) { // **assume that algos has more than 1 element @@ -542,8 +545,19 @@ namespace mbts.spark { } } - protected void add_property(string prop_name, Func get_from_def, Action assign_to_def, Func get_local, Action set_local, IObservable subscribe_local, TValue non_uniform_value = default, AlgoParamPermissions permissions = AlgoParamPermissions.ReadWrite) { - var param = new AlgoParamVM(prop_name, DisplayNameHelper.snake_to_title_case(prop_name), get_local()) { permissions = permissions }; + protected void add_property( + string prop_name, + Func get_from_def, + Action assign_to_def, + Func get_local, + Action set_local, + IObservable subscribe_local, + TValue non_uniform_value = default, + AlgoParamPermissions permissions = AlgoParamPermissions.ReadWrite, + string template_key = null) { + var param = new AlgoParamVM(prop_name, DisplayNameHelper.snake_to_title_case(prop_name), get_local(), template_key) { + permissions = permissions + }; _algo_params.Add(param); _auto_props[prop_name] = new AutoProp(prop_name, def => { @@ -816,12 +830,11 @@ namespace mbts.spark { var default_algo = AlgoKey.from_known_type(OrderDataHelper.get_default_algo(OrderDataHelper.get_algo_format(security.Value.ContractCase))); foreach (var sd in _child_algo_selectors) sd.load_algo_data_recursive(algo_data, default_algo); - Disposable.replace( - ref _selector_data_hooks, - ObservableAutoWrapper.connect_item_hooks(child_algo_selectors, s => { - s.load_algo_data_recursive(algo_data, default_algo); - return Disposable.empty; - })); + Disposable.dispose(ref _selector_data_hooks); + _selector_data_hooks = ObservableAutoWrapper.connect_item_hooks(child_algo_selectors, s => { + s.load_algo_data_recursive(algo_data, default_algo); + return Disposable.empty; + }); } } diff --git a/src/client/spark/Orders/OrderUtils.cs b/src/client/spark/Orders/OrderUtils.cs index 6bc4cc7c6..2b4773a67 100644 --- a/src/client/spark/Orders/OrderUtils.cs +++ b/src/client/spark/Orders/OrderUtils.cs @@ -200,6 +200,13 @@ namespace mbts.spark { public static string short_description(this OrderRecordImmutable order, ContractSpecificationImmutable? spec) => $"{(order.HasDef ? order.Def.Side.ToString() : "(unknown-side)")} {(order.HasState ? ($@"{spec.format(order.State.Price)} @ {order.State.Quantity}") : "(unknown-price) @ (unknown-quantity)")}"; + public static string short_description(SecurityImmutable security, Side side, double price, uint quantity) => + new StringBuilderEx( + security.ToString(), " ", + side.ToString(), " ", + price_quantity_description(price, quantity, security.spec_or_default()) + ).ToString(); + public static string short_description(this OrderRecordImmutable order, SecurityImmutable? security) { var spec = security.spec_or_default(); return new StringBuilderEx( diff --git a/src/client/spark/Orders/OrdersModel.cs b/src/client/spark/Orders/OrdersModel.cs index 847fd5f28..31e328c1f 100644 --- a/src/client/spark/Orders/OrdersModel.cs +++ b/src/client/spark/Orders/OrdersModel.cs @@ -59,23 +59,30 @@ namespace mbts.spark { this.md_auxiliary_model = md_auxiliary_model; var scheduler = ThreadPoolScheduler.Instance; var interval = TimeSpan.FromMilliseconds(100); - on_orders = - _service - .subscribei_orders() - .Naggle(interval, scheduler) - .PermissionFilter(admin_model.on_admin_data) - .join(sstore, order => order.security_key()) + on_orders = admin_model + .permission_filter( + _service.subscribei_orders() + .Naggle(interval, scheduler), + o => o.account_id() + ).join(sstore, order => order.security_key()) .Table() .Cache(); - on_positions = _position_service.subscribei_position_data().Naggle(interval, scheduler).PermissionFilter(admin_model.on_admin_data).join(sstore, pos => pos.SecurityId).Table().Cache(); - on_pnl = _position_service.subscribei_pnl_data() - .Naggle(interval, scheduler) + on_positions = admin_model + .permission_filter( + _position_service.subscribei_position_data().Naggle(interval, scheduler), + p => p.AccountId) + .join(sstore, pos => pos.SecurityId) + .Table() + .Cache(); + on_pnl = admin_model + .permission_filter( + _position_service.subscribei_pnl_data().Naggle(interval, scheduler), + p => p.Pos.AccountId) .join(sstore, p => p.Pos.SecurityId) .TableCheckedAutoCorrect(EqualityComparer<(SecurityImmutable? sec, PNLUnitImmutable data)>.Default) .Where(t => t.has_changes()) .Cache(); - on_account_pnl = _position_service.subscribei_pnl_account_data().Naggle(interval, scheduler).Table().Cache(); var working_orders = on_orders.Filter(tup => tup.order.is_active()); this.working_orders = on_orders.Scan( @@ -174,10 +181,11 @@ namespace mbts.spark { .Where(t => t.has_changes()) .Cache(); // fill tuples - var on_fills = - _service.subscribei_fills() - .Naggle(interval, scheduler) - .PermissionFilter(admin_model.on_admin_data) + var on_fills = admin_model + .permission_filter( + _service.subscribei_fills() + .Naggle(interval, scheduler), + f => f.Account) .join(sstore, f => f.SecurityKey) .Table().Cache(); var fill_order_tuples_join = on_fills.LeftJoin( @@ -241,7 +249,6 @@ namespace mbts.spark { .Filter(os => os.order.is_active() && os.order.State.DisplayStatus == OrderState.Types.DisplayStatus.Visible, key => key.order.Key) .Subscribe(_security_working_orders_with_piq_demuxer), on_positions.Subscribe(_security_positions_demuxer), - _bus.subscribe(SparkTags.CreateOrder).Subscribe(a => send_order(a.definition, a.price, a.size).ignore()), _bus.subscribe(SparkTags.RecreateOrder).Subscribe(a => a.ForEach(x => send_order(x.definition, x.price, x.size).ignore())), _bus.subscribe(SparkTags.CancelOrders).Subscribe(x => cancel_orders(x).ignore()), _bus.subscribe(SparkTags.DetachOrders).Subscribe(x => detach_orders(x).ignore()), @@ -266,7 +273,6 @@ namespace mbts.spark { public IObservable> working_orders { get; } public IObservable> on_pnl { get; } public IObservable> security_account_pnl { get; } - public IObservable> on_account_pnl { get; } public IObservable> on_positions { get; } public IObservable> studio_algos_with_meta { get; } public IObservable> on_manual_fills_securities { get; } @@ -287,13 +293,33 @@ namespace mbts.spark { #endregion #region Input - public async Task send_order(OrderDefinitionImmutable? order, double price, uint size) { + // the security arg is needed for a temporary hack to prevent GlbxIceberg orders from going out in non-glbx exchanges (like Ice) + // eventually, we'll need an algo type model to handle this stuff. + // This hack leverages the fact that every place that calls 'send_order' already has the security handy (not just the sec id). + public Task send_order(SecurityImmutable security, OrderDefinitionImmutable? order, double price, uint size) { if (order.HasValue) { - var result = (await send_order_get_error(order.Value, price, size)); - _activity.on_send_order(order.Value, price, size, result.key, result.error); - return result.key; +#if DEBUG + if (security.Exchange != Venue.Glbx && security.Exchange != Venue.Sim) { +#else + if (security.Exchange != Venue.Glbx) { +#endif + var algo_structure = OrderDataHelper.enumerate_algo_structure(order.Value.CustomDef); + if (algo_structure.Any(t => t.type == CustomOrderDefinition.CustomOneofCase.GlbxIceberg)) { + Log.Ui.info($"OrdersModel prevented a GlbxIceberg order from going out to a non-glbx exchange ({OrderUtils.short_description(security, order.Value.Side, price, size)})."); + return Task.FromResult(null); + } + } + return send_order(order.Value, price, size); } - return 0; + return Task.FromResult(null); + } + + /// Note: this method is private to force outside callers to use this overload . + /// This allows us to control which orders can be sent to which security/exchange (e.g., GlbxIceberg orders are only permitted for securities on the Glbx exchange). + async Task send_order(OrderDefinitionImmutable order, double price, uint size) { + var result = (await send_order_get_error(order, price, size)); + _activity.on_send_order(order, price, size, result.key, result.error); + return result.key; } async Task<(uint? key, string error)> send_order_get_error(OrderDefinitionImmutable order, double price, uint size) { diff --git a/src/client/spark/Themes/generic.xaml b/src/client/spark/Themes/generic.xaml index 78caf3cbf..c9f6ac225 100644 --- a/src/client/spark/Themes/generic.xaml +++ b/src/client/spark/Themes/generic.xaml @@ -339,7 +339,7 @@ - + diff --git a/src/client/spark/core/Bus/SparkCommands.cs b/src/client/spark/core/Bus/SparkCommands.cs index 1c2e0cea5..ee4414e48 100644 --- a/src/client/spark/core/Bus/SparkCommands.cs +++ b/src/client/spark/core/Bus/SparkCommands.cs @@ -8,6 +8,7 @@ using mbts.spark; using mbts.ui; using System; using System.Collections.Generic; +using System.Collections.Immutable; using System.Windows.Input; namespace bts.spark { @@ -41,6 +42,9 @@ namespace bts.spark { SparkCommand.create("P&L (2.0)", Icons.Accounting, SparkTags.ShowPNLTag); // Ladder + public static PartialCommand> ShowLadderAutoLookup(KeyGesture key_gesture = null) => + SparkCommand.create("Ladder", Icons.Ladder, SparkTags.ShowLadderAutoLookupTag, key_gesture); + public static PartialCommand ShowLadder(KeyGesture key_gesture = null) => SparkCommand.create("Ladder", Icons.Ladder, SparkTags.ShowLadderTag, key_gesture); diff --git a/src/client/spark/core/Bus/SparkTags.cs b/src/client/spark/core/Bus/SparkTags.cs index a05477e7f..311e7fb13 100644 --- a/src/client/spark/core/Bus/SparkTags.cs +++ b/src/client/spark/core/Bus/SparkTags.cs @@ -7,6 +7,7 @@ using bts.spark; using mbts.ui; using System; using System.Collections.Generic; +using System.Collections.Immutable; using System.Threading.Tasks; namespace mbts.spark { @@ -17,7 +18,6 @@ namespace mbts.spark { #endregion #region Order - public static ITag<(OrderDefinitionImmutable definition, double price, uint size)> CreateOrder { get; } = new SparkTag<(OrderDefinitionImmutable, double, uint)>(); public static ITag> RecreateOrder { get; } = new SparkTag>(); public static ITag> CancelOrders { get; } = new SparkTag>(); @@ -56,6 +56,8 @@ namespace mbts.spark { // The owners of the various SparkWindows may remove dependencies on the former at their leasure. // For now, both tags go through the same logic in the LadderWindowFactory anyways. public static ITag ShowLadderTag { get; } = new SparkTag("Show Ladder"); + // TODO: get rid of the SecurityMD ShowLadderTag. This has created burdensome dependency issues. + public static ITag> ShowLadderAutoLookupTag { get; } = new SparkTag>("Show Ladder Auto Lookup"); public static ITag> ShowLaddersTag { get; } = new SparkTag>(nameof(ShowLaddersTag)); public static ITag> ShowLinkedLadder { get; } = new SparkTag>("Linked Ladder"); public static ITag ShowMarketViewTag { get; } = new SparkTag("Show MarketView"); diff --git a/src/client/spark/core/ShellInterface/UISettings.cs b/src/client/spark/core/ShellInterface/UISettings.cs index 2b454f848..48385d724 100644 --- a/src/client/spark/core/ShellInterface/UISettings.cs +++ b/src/client/spark/core/ShellInterface/UISettings.cs @@ -11,6 +11,7 @@ using bts.api.persistence; using System.Text.RegularExpressions; using System.Reactive.Linq; using bts.spark; +using bts; namespace mbts.persist { @@ -146,16 +147,18 @@ namespace mbts.persist } [Export(typeof(IUserPersist))] - public class UserUISettings : UISettingsBase - { + public class UserUISettings : UISettingsBase { string _user; - [ImportingConstructor] - public UserUISettings([Import("userid")]uint user, [Import("settings")]PersistenceService persist, ErrorReporter err) + public UserUISettings(uint user_id, [Import("settings")]PersistenceService persist, ErrorReporter err) : base(persist,err) { - _user = $"User {user}"; + _user = $"User {user_id}"; } + [ImportingConstructor] + UserUISettings(UserTuple user, [Import("settings")]PersistenceService persist, ErrorReporter err) + : this(user.id, persist, err) { } + protected override string user() { return _user; } } } diff --git a/src/client/spark/core/ShellInterface/UserSettings/ISettingsMigrationRule.cs b/src/client/spark/core/ShellInterface/UserSettings/ISettingsMigrationRule.cs index 9e4b314c5..ce301199a 100644 --- a/src/client/spark/core/ShellInterface/UserSettings/ISettingsMigrationRule.cs +++ b/src/client/spark/core/ShellInterface/UserSettings/ISettingsMigrationRule.cs @@ -17,6 +17,7 @@ namespace bts.settings { interface IMarketViewGridSettingsMigrationRule : ISettingsMigrationRule { } interface IPAndLGridSettingsMigrationRule : ISettingsMigrationRule { } interface IPNLSettingsMigrationRule : ISettingsMigrationRule { } + interface ISyntheticManagementMigrationRule : ISettingsMigrationRule { } static class SettingsMigrationHelper { public static bool apply(T settings, IEnumerable> migration_rules, out T new_settings) { diff --git a/src/client/spark/core/ShellInterface/UserSettings/UserSettingsHub.cs b/src/client/spark/core/ShellInterface/UserSettings/UserSettingsHub.cs index a2b70175d..50dd84359 100644 --- a/src/client/spark/core/ShellInterface/UserSettings/UserSettingsHub.cs +++ b/src/client/spark/core/ShellInterface/UserSettings/UserSettingsHub.cs @@ -25,7 +25,8 @@ namespace mbts.persist { static readonly IImmutableProperty _fills_propi = ImmutableProperty.create(us => us.fills_grid, (us, gs) => us.with(provided_fills_gridsettings: gs)), _mview_propi = ImmutableProperty.create(us => us.market_view_grid, (us, gs) => us.with(provided_market_view_gridsettings: gs)), - _pandl_propi = ImmutableProperty.create(us => us.pnl_grid, (us, gs) => us.with(provided_pnl_gridsettings: gs)); + _pandl_propi = ImmutableProperty.create(us => us.pnl_grid, (us, gs) => us.with(provided_pnl_gridsettings: gs)), + _synth_propi = ImmutableProperty.create(us => us.synthetic_management_grid, (us, gs) => us.with(provided_synthetic_management_gridsettings: gs)); static readonly IImmutableProperty _pnl_propi = ImmutableProperty.create(us => us.pnl, (us, pnl) => us.with(pnl: pnl)); @@ -36,6 +37,7 @@ namespace mbts.persist { readonly IEnumerable _mview_migration_rules; readonly IEnumerable _pandl_migration_rules; readonly IEnumerable _pnl_migration_rules; + readonly IEnumerable _synth_migration_rules; readonly IUserPersist _persist; UserSettings _default_settings; @@ -46,13 +48,15 @@ namespace mbts.persist { [ImportMany]IEnumerable fills_migration_rules, [ImportMany]IEnumerable mview_migration_rules, [ImportMany]IEnumerable pandl_migration_rules, - [ImportMany]IEnumerable pnl_migration_rules) { + [ImportMany]IEnumerable pnl_migration_rules, + [ImportMany]IEnumerable synth_migration_rules) { // _persist = persist; _fills_migration_rules = fills_migration_rules ?? Enumerable.Empty(); _mview_migration_rules = mview_migration_rules ?? Enumerable.Empty(); _pandl_migration_rules = pandl_migration_rules ?? Enumerable.Empty(); _pnl_migration_rules = pnl_migration_rules ?? Enumerable.Empty(); + _synth_migration_rules = synth_migration_rules ?? Enumerable.Empty(); init(); async void init() { UserSettings s; @@ -62,7 +66,9 @@ namespace mbts.persist { var wrapped_rules = new[] { _fills_migration_rules.merge().wrap(_fills_propi), _mview_migration_rules.merge().wrap(_mview_propi), - _pandl_migration_rules.merge().wrap(_pandl_propi) + _pandl_migration_rules.merge().wrap(_pandl_propi), + _pnl_migration_rules.merge().wrap(_pnl_propi), + _synth_migration_rules.merge().wrap(_synth_propi) }; SettingsMigrationHelper.apply_quiet(ref s, wrapped_rules); // TODO: move ladder sanitization logic into ISettingsMigrationRule @@ -136,6 +142,12 @@ namespace mbts.persist { _pnl_migration_rules, _pnl_propi); + public IWindowSettingsManager create_synth_settings_manager(GridSettings instance_settings) => + _create_settings_manager( + instance_settings, + _synth_migration_rules, + _synth_propi); + public async Task save_default(UserSettings value) { // notify observers before persisting so the changes are reflected immediately in the ui lock (_default_settings_lock) { diff --git a/src/client/spark/core/Utility/ObjectSnapshot.cs b/src/client/spark/core/Utility/ObjectSnapshot.cs index 4e8418417..e649f55ed 100644 --- a/src/client/spark/core/Utility/ObjectSnapshot.cs +++ b/src/client/spark/core/Utility/ObjectSnapshot.cs @@ -15,12 +15,8 @@ namespace bts.utils { } static class ObjectSnapshot { - public static IEnumerable> property_delegates_of_type() where TSource : class where TValue : class { - var tval = typeof(TValue); - return typeof(TSource).GetProperties(BindingFlags.Public | BindingFlags.Instance) - .Where(p => tval.IsAssignableFrom(p.PropertyType)) - .Select(_prop_getter); - } + public static IReadOnlyList> property_delegates_of_type() => + PropertyGetterStore.value; public static IObservable> property_table(T instance, params string[] prop_names) where T : INotifyPropertyChanged => _property_table(instance, Array.ConvertAll(prop_names, PropertyInfoStore.type.GetProperty)); @@ -162,6 +158,18 @@ namespace bts.utils { } } + static class PropertyGetterStore { + public static readonly IReadOnlyList> value; + + static PropertyGetterStore() { + var tval = typeof(TValue); + value = typeof(TSource).GetProperties(BindingFlags.Public | BindingFlags.Instance) + .Where(p => tval.IsAssignableFrom(p.PropertyType)) + .Select(_prop_getter) + .ToReadOnlyList(); + } + } + class PropNode { public readonly PropertyInfo prop; diff --git a/src/client/spark/core/data/SyntheticTuple.cs b/src/client/spark/core/data/SyntheticTuple.cs new file mode 100644 index 000000000..b0717f1b5 --- /dev/null +++ b/src/client/spark/core/data/SyntheticTuple.cs @@ -0,0 +1,46 @@ +using bts.api; +using bts.spark; +using bts.utils; +using System; + +namespace bts { + struct SyntheticTuple : IEquatable, IComparable, IComparable { + public static readonly Func key_fn = a => a.security.Id; + public static readonly Func nullable_key_fn = a => a.HasValue ? a.Value.security.Id : 0u; + + public readonly SecurityImmutable security; + public readonly string user_name; + public readonly ExecutionStrategyImmutable? strategy; + public readonly int? num_working_orders; + + public SyntheticTuple(SecurityImmutable security, string user_name, ExecutionStrategyImmutable? strategy, int? num_working_orders = null) { + this.security = security; + this.user_name = user_name; + this.strategy = strategy; + this.num_working_orders = num_working_orders; + } + + public int CompareTo(SyntheticTuple other) => + SecurityHelper.Comparer.Compare(security, other.security); + + int IComparable.CompareTo(object obj) => + obj is SyntheticTuple other ? CompareTo(other) : 1; + + public bool Equals(SyntheticTuple other) => + security == other.security && + user_name == other.user_name && + EqualityComparer.nullable_equals(strategy, other.strategy); + + public override bool Equals(object obj) => + EqualityComparer.equals_obj(this, obj); + + public override int GetHashCode() => + security.Id.GetHashCode(); + + public override string ToString() => + user_name == null ? security.ToString() : $"{user_name}:{security}"; + + public static bool operator ==(SyntheticTuple a, SyntheticTuple b) => a.Equals(b); + public static bool operator !=(SyntheticTuple a, SyntheticTuple b) => !a.Equals(b); + } +} diff --git a/src/client/spark/view/Controls/DIYGrid/Columns/ColumnBase.cs b/src/client/spark/view/Controls/DIYGrid/Columns/ColumnBase.cs index c2c8db794..9e8e44cda 100644 --- a/src/client/spark/view/Controls/DIYGrid/Columns/ColumnBase.cs +++ b/src/client/spark/view/Controls/DIYGrid/Columns/ColumnBase.cs @@ -114,6 +114,7 @@ namespace mbts.ui.controls { Aggressor = "Aggressor", Volume = "Volume", Last = "Last", + S = "S", Pos = "Pos", Working = "Working", Bid = "Bid", @@ -124,6 +125,7 @@ namespace mbts.ui.controls { SessionLow = "Low", SessionHigh = "High", FillKind = "Fill Kind", + FillStatus = "Fill Status", Name = "Name", Description = "Description", N_Legs = "# Legs", diff --git a/src/client/spark/view/Controls/DIYGrid/DoubleUtil.cs b/src/client/spark/view/Controls/DIYGrid/DoubleUtil.cs index 2ac6f579b..aa2019135 100644 --- a/src/client/spark/view/Controls/DIYGrid/DoubleUtil.cs +++ b/src/client/spark/view/Controls/DIYGrid/DoubleUtil.cs @@ -1,10 +1,9 @@ using System; using System.Linq; -namespace mbts.ui.controls -{ - internal class DoubleUtil - { +namespace mbts.ui.controls { + static class DoubleUtil { + public const int API_DECIMAL_PRECISION = 9; /* smallest such that 1.0+DBL_EPSILON != 1.0 */ private const double EPSILON = 2.2204460492503131e-016; private const double BIG_EPSILON = EPSILON * 10; @@ -36,6 +35,9 @@ namespace mbts.ui.controls (double.IsNaN(right) ? double.NaN : -right) : (double.IsNaN(right) ? left : left - right); + public static decimal round_to_decimal(double value, int decimals = API_DECIMAL_PRECISION) => + double.IsNaN(value) ? 0 : (decimal)Math.Round(value, decimals, MidpointRounding.AwayFromZero); + internal static bool AreClose(double v1, double v2) { if (v1 == v2) return true; double delta = Math.Abs(v1 - v2); diff --git a/src/client/spark/view/Controls/DataGridEx/Filters/DataGridFilterHelper.cs b/src/client/spark/view/Controls/DataGridEx/Filters/DataGridFilterHelper.cs index 54822a36e..1c13e5a21 100644 --- a/src/client/spark/view/Controls/DataGridEx/Filters/DataGridFilterHelper.cs +++ b/src/client/spark/view/Controls/DataGridEx/Filters/DataGridFilterHelper.cs @@ -1,11 +1,22 @@ -using System; +using bts.utils; +using System; using System.Collections.Generic; using System.ComponentModel; using System.Linq; namespace mbts.spark.filters { static class DataGridFilterHelper { - public static IDisposable hook_filters(ICollectionView rows, params IDataGridColumnFilter[] filters) { + public static IDisposable hook_all(TSource source, ICollectionView rows) => + hook_filters(rows, + ObjectSnapshot.property_delegates_of_type>() + .Select(f => f(source)) + .ExceptNull() + .ToList()); + + public static IDisposable hook_filters(ICollectionView rows, params IDataGridColumnFilter[] filters) => + hook_filters(rows, filters.AsEnumerable()); + + public static IDisposable hook_filters(ICollectionView rows, IEnumerable> filters) { var icvls = (ICollectionViewLiveShaping)rows; icvls.IsLiveFiltering = true; return _hook_filters(set_filter, filters); @@ -22,7 +33,7 @@ namespace mbts.spark.filters { } } - static IDisposable _hook_filters(Action set_filter, params IDataGridColumnFilter[] filters) { + static IDisposable _hook_filters(Action set_filter, IEnumerable> filters) { foreach (var f in filters) f.filter_changed += set_filter; set_filter(); @@ -34,7 +45,7 @@ namespace mbts.spark.filters { }); } - static List> _get_active_filters(IDataGridColumnFilter[] filters) => + static List> _get_active_filters(IEnumerable> filters) => filters.Where(f => f.active.HasValue && f.active.Value).ToList(); } diff --git a/src/client/spark/view/Controls/DataGridEx/Filters/VM/DiscreteValuesFilterVM.cs b/src/client/spark/view/Controls/DataGridEx/Filters/VM/DiscreteValuesFilterVM.cs index ce8927104..e42550ee5 100644 --- a/src/client/spark/view/Controls/DataGridEx/Filters/VM/DiscreteValuesFilterVM.cs +++ b/src/client/spark/view/Controls/DataGridEx/Filters/VM/DiscreteValuesFilterVM.cs @@ -68,10 +68,11 @@ namespace mbts.spark.filters { Func factory, Func key, out IObserver> items_observer, - out ListCollectionView items_view) + out ListCollectionView items_view, + string prop_name = null) where TKey : struct, IEquatable => // - CreateTableUpdate(factory, key, keys => keys, keys => keys.ToArray(), out items_observer, out items_view); + CreateTableUpdate(factory, key, keys => keys, keys => keys.ToArray(), out items_observer, out items_view, prop_name); public static IDataGridColumnPersistentFilter CreateTableUpdate( Func factory, @@ -79,10 +80,11 @@ namespace mbts.spark.filters { Func> load_state, Func, TState> get_state, out IObserver> items_observer, - out ListCollectionView items_view) + out ListCollectionView items_view, + string prop_name = null) where TKey : struct, IEquatable { // - var filter = new TableUpdateItemsFilter(factory, key, load_state, get_state); + var filter = new TableUpdateItemsFilter(factory, key, load_state, get_state, prop_name); items_observer = filter; items_view = filter.items_view; return filter; @@ -291,8 +293,8 @@ namespace mbts.spark.filters { readonly HashSet _selected_keys = new HashSet(), _desired_keys = new HashSet(); ListCollectionView _items_view; - public TableUpdateItemsFilter(Func factory, Func key, Func> load_state, Func, TState> get_state) - : base(factory, null) { + public TableUpdateItemsFilter(Func factory, Func key, Func> load_state, Func, TState> get_state, string prop_name) + : base(factory, prop_name) { _factory = factory; _key = key; _load_state = load_state; diff --git a/src/client/spark/view/Controls/DataGridEx/Filters/Views/TimeFilter.cs b/src/client/spark/view/Controls/DataGridEx/Filters/Views/TimeFilter.cs index 20155813d..145228b80 100644 --- a/src/client/spark/view/Controls/DataGridEx/Filters/Views/TimeFilter.cs +++ b/src/client/spark/view/Controls/DataGridEx/Filters/Views/TimeFilter.cs @@ -24,6 +24,7 @@ namespace mbts.ui { public TimeTextTuple end => _end; public bool is_active => _mode == TimeFilterMode.Today || + _mode == TimeFilterMode.CMEOpen || (_mode == TimeFilterMode.Period && (_begin.result.HasValue || _end.result.HasValue)); diff --git a/src/client/spark/view/Controls/Dialogs/SupportRequestWindow.xaml.cs b/src/client/spark/view/Controls/Dialogs/SupportRequestWindow.xaml.cs index 89e202b05..a5a139800 100644 --- a/src/client/spark/view/Controls/Dialogs/SupportRequestWindow.xaml.cs +++ b/src/client/spark/view/Controls/Dialogs/SupportRequestWindow.xaml.cs @@ -1,10 +1,4 @@ -using bts.support; -using Google.Protobuf; -using mbts; -using mbts.ui; -using mbts.ui.shell; -using mbts.ui.utility; -using System; +using System; using System.Collections.Generic; using System.Linq; using System.Text; @@ -18,6 +12,13 @@ using System.Windows.Media; using System.Windows.Media.Imaging; using System.Windows.Navigation; using System.Windows.Shapes; +using bts.support; +using Google.Protobuf; +using mbts; +using mbts.ui; +using mbts.ui.controls; +using mbts.ui.shell; +using mbts.ui.utility; namespace bts.view.controls { partial class SupportRequestWindow : Window { @@ -50,28 +51,40 @@ namespace bts.view.controls { _base = report_base; send_command = new DelegateCommand(send_report, _ => !string.IsNullOrWhiteSpace(_text.Value)); _text = backing(nameof(user_text), ""); - _text.Changed += (_,__) => send_command.TryRaiseCanExecuteChanged(); + _text.Changed += (_, __) => send_command.TryRaiseCanExecuteChanged(); _contact = backing(nameof(user_contact), ""); - _contact.Changed += (_,__) => send_command.TryRaiseCanExecuteChanged(); + _contact.Changed += (_, __) => send_command.TryRaiseCanExecuteChanged(); } async void send_report(object _) { - ByteString screenshot = null; - if (include_screenshot) { - var stream = ScreenCapture.capture_encoded("jpeg"); - screenshot = await ByteString.FromStreamAsync(stream); - } - var reply = await _service.send_report_async(_base.with( - user_description: _text.Value, - contact: user_contact, - log: include_logs ? LogUtil.read_logs() : "", - image: screenshot - )); - if (reply.success) { - BTSMessageBox.Show("Your message has been received.", "Thank You", BTSMessageBoxButtons.OK, MessageBoxImage.Information, is_modal: false, top_most:true); - } - else { - BTSMessageBox.Show($"There was an error submitting your message. '{reply.error_message}'", "Error", BTSMessageBoxButtons.OK, MessageBoxImage.Error, is_modal: false, top_most: true); + using (var d = new BusyDialog { Text = "Sending your error report...", Topmost = true }) { + d.Show(); + ByteString screenshot = null; + if (include_screenshot) { + var stream = ScreenCapture.capture_encoded("jpeg"); + screenshot = await ByteString.FromStreamAsync(stream); + } + var request = await Task.Run(() => { + var result = _base.with( + user_description: _text.Value, + contact: user_contact, + image: screenshot); + if (include_logs) { + result = spark.ErrorReporter.append_logs(result); + } + return result; + }); + + var reply = await _service.send_report_async(request); + if (reply.success) { + Log.Ui.info("Support request message has been received."); + BTSMessageBox.Show("Your message has been received.", "Thank You", BTSMessageBoxButtons.OK, MessageBoxImage.Information, is_modal: false, top_most: true); + } + else { + var errmsg = $"There was an error submitting your message. '{reply.error_message}'"; + Log.Ui.error(errmsg); + BTSMessageBox.Show(errmsg, "Error", BTSMessageBoxButtons.OK, MessageBoxImage.Error, is_modal: false, top_most: true); + } } } diff --git a/src/client/spark/view/Controls/Order/AlgoContentTemplateSelector.xaml b/src/client/spark/view/Controls/Order/AlgoContentTemplateSelector.xaml index 656513fc3..bd0aad15e 100644 --- a/src/client/spark/view/Controls/Order/AlgoContentTemplateSelector.xaml +++ b/src/client/spark/view/Controls/Order/AlgoContentTemplateSelector.xaml @@ -1145,6 +1145,30 @@ + + + + + + + + + + + + + @@ -1181,30 +1205,35 @@ - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + diff --git a/src/client/spark/view/Controls/TickUpDown/TickUpDown.xaml b/src/client/spark/view/Controls/TickUpDown/TickUpDown.xaml index 5a392c532..1915202c2 100644 --- a/src/client/spark/view/Controls/TickUpDown/TickUpDown.xaml +++ b/src/client/spark/view/Controls/TickUpDown/TickUpDown.xaml @@ -457,15 +457,15 @@ Name="PART_TextBox" u:TextBoxBehavior.EnableArrowKeyTraverse="True" BorderThickness="0"> - + - + + diff --git a/src/client/spark/view/Controls/Time/DateTimePicker.xaml b/src/client/spark/view/Controls/Time/DateTimePicker.xaml index 9a8deb2cc..f0fc67218 100644 --- a/src/client/spark/view/Controls/Time/DateTimePicker.xaml +++ b/src/client/spark/view/Controls/Time/DateTimePicker.xaml @@ -93,7 +93,6 @@ u:TextBoxBehavior.EnableArrowKeyTraverse="True" u:Behavior.DisablePaste="True" Height="20" - BorderThickness="0" > diff --git a/src/client/spark/view/Converters/AnonymousValueConverter.cs b/src/client/spark/view/Converters/AnonymousValueConverter.cs index 29ba7db43..93ce4aa32 100644 --- a/src/client/spark/view/Converters/AnonymousValueConverter.cs +++ b/src/client/spark/view/Converters/AnonymousValueConverter.cs @@ -28,9 +28,17 @@ namespace mbts.ui.converters { IntTextConverter = new IntTextConverter(), UIntTextConverter = new UIntTextConverter(), QuantityMeasureTextConverter = ValueConverter.create(fmt), + SecuritySelectorText, SimpleString = new AnonymousValueConverter(obj => obj?.ToString()), ThicknessToDouble = new ThicknessToDoubleConverter(); + static ValueConverters() { + SecuritySelectorText = new NullSubstitutionConverterWrapper { + Converter = SimpleString, + ValueForNull = "(None)" + }; + } + public static readonly IMultiValueConverter AreEqualConverter = new AreEqualConverter(); diff --git a/src/client/spark/view/Converters/DictionaryLookupConverter.cs b/src/client/spark/view/Converters/DictionaryLookupConverter.cs new file mode 100644 index 000000000..cdc635b02 --- /dev/null +++ b/src/client/spark/view/Converters/DictionaryLookupConverter.cs @@ -0,0 +1,116 @@ +using System; +using System.Collections; +using System.Collections.Generic; +using System.Globalization; +using System.Linq; +using System.Windows.Data; + +namespace mbts.ui.converters { + class DictionaryLookupConverter : IMultiValueConverter { + [ThreadStatic] + Dictionary _type_impls_backing; + Dictionary _type_impls => _type_impls_backing ?? (_type_impls_backing = new Dictionary()); + + delegate bool Lookup(object source, object key, out object result); + + public object Convert(object[] values, Type targetType, object parameter, CultureInfo culture) { + if (values.Length == 2 && values[0] != null && values[1] != null) { + var lookup_type = values[0].GetType(); + var lookup = _type_impls.GetOrAdd(lookup_type, TypeImplResolver.resolve); + if (lookup(values[0], values[1], out var result)) + return result; + } + return null; + } + + public object[] ConvertBack(object value, Type[] targetTypes, object parameter, CultureInfo culture) => + throw new NotImplementedException(); + + static class TypeImplResolver { + public static Lookup resolve(Type lookup_type) => + _prioritized_impls(lookup_type) + .OrderBy(tup => tup.priority) + .Select(tup => tup.impl) + .FirstOrDefault() ?? _lookup_fail; + + static bool _lookup_fail(object source, object key, out object result) { + result = null; + return false; + } + + static IEnumerable<(int priority, Lookup impl)> _prioritized_impls(Type lookup_type) { + foreach (var itype in lookup_type.GetInterfaces()) { + if (itype.IsGenericType) { + var gen_def = itype.GetGenericTypeDefinition(); + if (gen_def == typeof(IDictionary<,>)) + yield return (1, _get_generic_impl(nameof(_GenericImpls.idictionary), itype)); + if (gen_def == typeof(IReadOnlyDictionary<,>)) + yield return (2, _get_generic_impl(nameof(_GenericImpls.ireadonly_dictionary), itype)); + if (gen_def == typeof(ILookup<,>)) + yield return (3, _get_generic_impl(nameof(_GenericImpls.ilookup), itype)); + } + if (itype == typeof(IDictionary)) + yield return (4, _NonGenericImpls.idictionary); + } + } + + static Lookup _get_generic_impl(string impl_name, Type impl_type) => + _get_generic_impl(impl_name, impl_type.GetGenericArguments()); + + static Lookup _get_generic_impl(string impl_name, Type[] key_value_types) => + (Lookup)typeof(_GenericImpls<,>) + .MakeGenericType(key_value_types) + .GetField(nameof(_GenericImpls.idictionary)) + .GetValue(null); + + static class _GenericImpls { + public static readonly Lookup + idictionary = _idictionary, + ireadonly_dictionary = _ireadonly_dictionary, + ilookup = _ilookup; + + static bool _idictionary(object lookup, object key, out object result) { + if (key is TKey k && lookup is IDictionary d && d.TryGetValue(k, out var value)) { + result = value; + return true; + } + result = null; + return false; + } + + static bool _ireadonly_dictionary(object lookup, object key, out object result) { + if (key is TKey k && lookup is IReadOnlyDictionary rod && rod.TryGetValue(k, out var value)) { + result = value; + return true; + } + result = null; + return false; + } + + static bool _ilookup(object lookup, object key, out object result) { + if ((key is TKey k) && lookup is ILookup l && l.Contains(k)) { + result = l[k]; + return true; + } + result = null; + return false; + } + } + + static class _NonGenericImpls { + public static readonly Lookup + idictionary = _idictionary; + + static bool _idictionary(object lookup, object key, out object result) { + var dict = (IDictionary)lookup; + if (dict.Contains(key)) { + result = dict[key]; + return true; + } + result = null; + return false; + } + } + } + } +} \ No newline at end of file diff --git a/src/client/spark/view/Resources/ResourceKeys.cs b/src/client/spark/view/Resources/ResourceKeys.cs new file mode 100644 index 000000000..634de738b --- /dev/null +++ b/src/client/spark/view/Resources/ResourceKeys.cs @@ -0,0 +1,9 @@ +namespace mbts.ui { + static class ResourceKeys { + public const string + Securities = nameof(Securities), + Accounts = nameof(Accounts), + Users = nameof(Users), + Sides = nameof(Sides); + } +} \ No newline at end of file diff --git a/src/client/spark/view/Resources/TemplateKeys.cs b/src/client/spark/view/Resources/TemplateKeys.cs new file mode 100644 index 000000000..d376785ed --- /dev/null +++ b/src/client/spark/view/Resources/TemplateKeys.cs @@ -0,0 +1,6 @@ +namespace mbts.ui { + static class TemplateKeys { + public const string + SecurityKey = nameof(SecurityKey); + } +} \ No newline at end of file diff --git a/src/client/spark/view/Shell/Login/LoginModel.cs b/src/client/spark/view/Shell/Login/LoginModel.cs index 35f666503..9fb963b61 100644 --- a/src/client/spark/view/Shell/Login/LoginModel.cs +++ b/src/client/spark/view/Shell/Login/LoginModel.cs @@ -15,8 +15,7 @@ namespace mbts.ui.shell { // The LoginContainer participates in the startup Import/Export dependency system // Since the login form should not depend on *anything* it will be shown before this is created // Once this is created it access the global LoginModel and retrieves the username - internal class LoginContainer - { + class LoginContainer { [ImportingConstructor] public LoginContainer(ServiceContext context) { var model = LoginModel.instance; @@ -28,10 +27,8 @@ namespace mbts.ui.shell { model.Dispose(); } - [Export(Exports.UserName)] public string UserName { get; private set; } - [Export(Exports.UserID)] public uint UserId { get; private set; } [Export] @@ -42,13 +39,6 @@ namespace mbts.ui.shell { [Export("workspace")] public string Workspace { get; private set; } - - public static class Exports { - public const string - UserName = "username", - UserID = "userid", - UserTuple = "usertuple"; - } } public struct User { diff --git a/src/client/spark/view/Shell/ShellModel.cs b/src/client/spark/view/Shell/ShellModel.cs index 776ee66cd..7832bf1c5 100644 --- a/src/client/spark/view/Shell/ShellModel.cs +++ b/src/client/spark/view/Shell/ShellModel.cs @@ -114,10 +114,16 @@ namespace mbts.ui } public void Dispose() { - foreach (IShellModule module in _modules.OrderBy(m => m.priority).Reverse()) { + foreach (var module in _modules.OrderBy(m => m.priority).Reverse()) { module.Dispose(); } _modules.Clear(); + foreach (var daemon in _services) { + if (daemon is IDisposable d) { + d.Dispose(); + } + } + _services.Clear(); } } } \ No newline at end of file diff --git a/src/client/spark/view/Shell/ViewModels/LauncherVM.cs b/src/client/spark/view/Shell/ViewModels/LauncherVM.cs index 63b760725..de9a13c52 100644 --- a/src/client/spark/view/Shell/ViewModels/LauncherVM.cs +++ b/src/client/spark/view/Shell/ViewModels/LauncherVM.cs @@ -12,8 +12,7 @@ using System.Windows.Input; namespace mbts.ui.shell { [Export(typeof(ILauncher))] - public class LauncherVM : ViewModelBase, ILauncher - { + public class LauncherVM : ViewModelBase, ILauncher { Window _window; string _user; readonly IBus _bus; @@ -23,9 +22,9 @@ namespace mbts.ui.shell { [ImportingConstructor] public LauncherVM( ObservableCollection modules, - [Import(LoginContainer.Exports.UserName)]string username, + UserTuple user, ErrorReporter errors, IBus bus) { - _user = username; + _user = user.name; _modules = new ObservableAutoWrapper(modules, m => new ShellModuleVM(m, bus)); _error_service = errors; _bus = bus; @@ -123,7 +122,7 @@ namespace mbts.ui.shell { _shown = true; if (_window != null) { Application.Current.ShutdownMode = ShutdownMode.OnMainWindowClose; - if (shown != null) shown(); + shown?.Invoke(); _window.Show(); } } @@ -140,4 +139,4 @@ namespace mbts.ui.shell { #endregion } -} +} \ No newline at end of file diff --git a/src/client/spark/view/Shell/ViewModels/ShellThreadFactory.cs b/src/client/spark/view/Shell/ViewModels/ShellThreadFactory.cs index b28b21b41..0419641eb 100644 --- a/src/client/spark/view/Shell/ViewModels/ShellThreadFactory.cs +++ b/src/client/spark/view/Shell/ViewModels/ShellThreadFactory.cs @@ -42,8 +42,8 @@ namespace mbts.ui.shell try { Dispatcher.Run(); } - catch (Exception e) { - Log.assert(false, e.ToString()); + catch (Exception ex) { + Log.assert(false, $"Exception on dispatcher thread '{name}': {ex}."); } finally { Log.Ui.info("Disposing slow {0} and cleaning up background thread.".fmt(name)); diff --git a/src/client/spark/view/Spark/ContractPicker/ContractPickerModel.cs b/src/client/spark/view/Spark/ContractPicker/ContractPickerModel.cs index ad43e7aee..b4f3aa76b 100644 --- a/src/client/spark/view/Spark/ContractPicker/ContractPickerModel.cs +++ b/src/client/spark/view/Spark/ContractPicker/ContractPickerModel.cs @@ -20,8 +20,8 @@ namespace mbts.spark { readonly IRaiseCanExecuteChangedCommand _add_criteria_command, _remove_criteria_command, _ok_button_command; bool _allow_empty, _is_edit_mode; - SecurityImmutable? _pending_sec; - (SecurityCriteria criteria, SecurityImmutable? sec, SecurityMD? md)? _snapshot; + uint? _pending_sec_key; + (SecurityCriteria criteria, uint? sec, SecurityMD? md)? _snapshot; public ContractPickerModel(IBus bus, bool edit_mode, SecurityCriteria existing_criteria, SecurityCriteria world_view_filter = null) : this(bus.subscribe(SecuritiesTag), edit_mode, existing_criteria, world_view_filter) { } @@ -209,9 +209,9 @@ namespace mbts.spark { _update_security_types(); _update_symbols(); _update_securities(); - if (_pending_sec.HasValue && changed_securities.ContainsKey(_pending_sec.Value.Id)) { - select_security_request?.Invoke(_pending_sec.Value); - _pending_sec = null; + if (_pending_sec_key.HasValue && changed_securities.TryGetValue(_pending_sec_key.Value, out var smd)) { + select_security_request?.Invoke(smd.security); + _pending_sec_key = null; } } internal void criteria_selection_changed(IEnumerable selected_clauses) { @@ -229,6 +229,8 @@ namespace mbts.spark { security_criteria_clause_changed?.Invoke(this, e); } + public bool is_editing => _snapshot.HasValue; + public ObservableCollection all_exchanges { get; } = new ObservableCollection(); public List selected_exchanges = new List(); @@ -298,7 +300,22 @@ namespace mbts.spark { if (all_securities.ContainsKey(value.Value.security.Id)) { select_security_request?.Invoke(value.Value.security); } else { - _pending_sec = value.Value.security; + _pending_sec_key = value.Value.security.Id; + } + } + } + } + } + + public uint? selected_security_key { + get => selected_securities.Count == 1 && all_securities.TryGetValue(selected_securities[0], out var smd) ? smd.security.Id : (uint?)null; + set { + if (value != selected_security_key) { + if (value.HasValue) { + if (all_securities.TryGetValue(value.Value, out var smd)) { + select_security_request?.Invoke(smd.security); + } else { + _pending_sec_key = value.Value; } } } @@ -313,7 +330,7 @@ namespace mbts.spark { if (all_securities.ContainsKey(value.Value.Id)) { select_security_request?.Invoke(value.Value); } else { - _pending_sec = value.Value; + _pending_sec_key = value.Value.Id; } } } @@ -336,18 +353,19 @@ namespace mbts.spark { } #region IEditableObject - public void BeginEdit() => _snapshot = (result_criteria, _pending_sec ?? selected_security, selected_security_md); + public void BeginEdit() => _snapshot = (result_criteria, _pending_sec_key ?? selected_security_key, selected_security_md); public void EndEdit() => _snapshot = null; public void CancelEdit() { if (_snapshot.HasValue) { var (criteria, sec, md) = _snapshot.Value; security_criteria_clauses.SetItems(criteria.clauses); - selected_security = sec; + selected_security_key = sec; /// A may have been used to create the snapshot when was called (this happens when securities are loaded lazily). /// If this is the case, pending sec was assigned to the security; /// check if the security is now present in , if so, use that value instead of the null. - selected_security_md = sec.HasValue && !md.HasValue && all_securities.TryGetValue(sec.Value.Id, out var md_match) ? md_match : md; + selected_security_md = sec.HasValue && !md.HasValue && all_securities.TryGetValue(sec.Value, out var md_match) ? md_match : md; _snapshot = null; + RaisePropertyChanged(nameof(selected_security)); _add_criteria_command.RaiseCanExecuteChanged(); _remove_criteria_command.RaiseCanExecuteChanged(); } diff --git a/src/client/spark/view/Spark/ContractPicker/ContractPickerPopup.xaml.cs b/src/client/spark/view/Spark/ContractPicker/ContractPickerPopup.xaml.cs index 49eb0599b..28e76b6c6 100644 --- a/src/client/spark/view/Spark/ContractPicker/ContractPickerPopup.xaml.cs +++ b/src/client/spark/view/Spark/ContractPicker/ContractPickerPopup.xaml.cs @@ -2,6 +2,7 @@ using mbts.spark; using mbts.ui.converters; using mbts.ui.shell; +using mbts.ui.utility; using System; using System.Collections.Generic; using System.Collections.Specialized; @@ -16,8 +17,11 @@ namespace mbts.ui.controls { partial class ContractPickerPopup : PopupControl { public static readonly EventHandler FlushSelectedCriteriaHandler = (s, e) => (s as ContractPickerPopup)?.GetBindingExpression(SelectedCriteriaProperty)?.UpdateSource(), + FlushSelectedSecurityKeyHandler = (s, e) => (s as ContractPickerPopup)?.GetBindingExpression(SelectedSecurityKeyProperty)?.UpdateSource(), FlushSelectedSecurityHandler = (s, e) => (s as ContractPickerPopup)?.GetBindingExpression(SelectedSecurityProperty)?.UpdateSource(); + readonly PropertyUpdateGate _selection_sync_gate = new PropertyUpdateGate(); + IReadOnlyDictionary _cached_security_map = DictionaryHelper.empty(); ContractPickerPopupController _controller; bool _allow_empty, _is_lazy; @@ -97,12 +101,55 @@ namespace mbts.ui.controls { } #endregion + #region SelectedSecurityKey + public static readonly DependencyProperty SelectedSecurityKeyProperty = + WPFHelper.create_dp( + nameof(SelectedSecurityKey), + FrameworkPropertyMetadataOptions.BindsTwoWayByDefault, + (o, v) => { + o.Controller.model.selected_security_key = v; + o._selection_sync_gate.try_update(() => o._sync_selection_props(o.SelectedSecurityMD, o.SelectedSecurity, v, v)); + }); + //(o, base_value) => base_value is uint u && u != 0 ? u : (uint?)null); + + public uint? SelectedSecurityKey { + get => GetValue(SelectedSecurityKeyProperty) as uint?; + set => SetValue(SelectedSecurityKeyProperty, value); + } + #endregion + + enum SecurityProp { + Security, + SecurityMD, + ID + } + + void _sync_selection_props(SecurityMD? curr_sec_md, SecurityImmutable? curr_sec, uint? curr_sec_key, uint? target_sec_key) { + if (curr_sec_key != target_sec_key) { + SetCurrentValue(SelectedSecurityKeyProperty, target_sec_key); + } + bool sec_md_dirty = target_sec_key != (curr_sec_md?.security)?.Id; + bool sec_dirty = target_sec_key != curr_sec?.Id; + if (sec_md_dirty || sec_dirty) { + curr_sec_md = target_sec_key.HasValue && _cached_security_map.TryGetValue(target_sec_key.Value, out var smd) ? smd : (SecurityMD?)null; + if (sec_md_dirty) + SetCurrentValue(SelectedSecurityMDProperty, curr_sec_md); + if (sec_dirty) + SetCurrentValue(SelectedSecurityProperty, curr_sec_md?.security); + } + if (!IsEditMode) + SetCurrentValue(SelectedCriteriaProperty, target_sec_key.HasValue ? SecurityCriteria.from_keys(new[] { target_sec_key.Value }) : null); + } + #region SelectedSecurity public static readonly DependencyProperty SelectedSecurityProperty = WPFHelper.create_dp( nameof(SelectedSecurity), FrameworkPropertyMetadataOptions.BindsTwoWayByDefault, - (o, v) => o.Controller.model.selected_security = v); + (o, v) => { + o.Controller.model.selected_security = v; + o._selection_sync_gate.try_update(() => o._sync_selection_props(o.SelectedSecurityMD, v, o.SelectedSecurityKey, v?.Id)); + }); public SecurityImmutable? SelectedSecurity { get => GetValue(SelectedCriteriaProperty) as SecurityImmutable?; @@ -115,7 +162,10 @@ namespace mbts.ui.controls { WPFHelper.create_dp( nameof(SelectedSecurityMD), FrameworkPropertyMetadataOptions.BindsTwoWayByDefault, - (o, v) => o.Controller.model.selected_security_md = v); + (o, v) => { + o.Controller.model.selected_security_md = v; + o._selection_sync_gate.try_update(() => o._sync_selection_props(v, o.SelectedSecurity, o.SelectedSecurityKey, (v?.security)?.Id)); + }); public SecurityMD? SelectedSecurityMD { get => GetValue(SelectedCriteriaProperty) as SecurityMD?; @@ -248,12 +298,10 @@ namespace mbts.ui.controls { } void on_cancel() { model.CancelEdit(); - propagate_to_owner(); owner.CancelClick?.Invoke(owner, EventArgs.Empty); } void propagate_to_owner() { - owner.set_current_if_changed(SelectedSecurityMDProperty, owner.Controller.model.selected_security_md); - owner.set_current_if_changed(SelectedSecurityProperty, owner.Controller.model.selected_security); + owner.set_current_if_changed(SelectedSecurityKeyProperty, owner.Controller.model.selected_security_key); owner.set_current_if_changed(SelectedCriteriaProperty, owner.Controller.model.result_criteria); } } @@ -262,4 +310,4 @@ namespace mbts.ui.controls { public SecurityCriteria selected_criteria => _selected_criteria.Value; } } -} \ No newline at end of file +} diff --git a/src/client/spark/view/Utility/BindingProxy.cs b/src/client/spark/view/Utility/BindingProxy.cs new file mode 100644 index 000000000..c00646a04 --- /dev/null +++ b/src/client/spark/view/Utility/BindingProxy.cs @@ -0,0 +1,13 @@ +using System.Windows; + +namespace mbts.ui { + class BindingProxy : FrameworkElement { + public static readonly DependencyProperty ValueProperty = + DependencyProperty.Register(nameof(Value), typeof(object), typeof(BindingProxy)); + + public object Value { + get => GetValue(ValueProperty); + set => SetValue(ValueProperty, value); + } + } +} \ No newline at end of file diff --git a/src/client/spark_api/Co2/MDConsumerHandle.cs b/src/client/spark_api/Co2/MDConsumerHandle.cs index 7b03bdaee..36b560c1a 100644 --- a/src/client/spark_api/Co2/MDConsumerHandle.cs +++ b/src/client/spark_api/Co2/MDConsumerHandle.cs @@ -133,6 +133,12 @@ namespace mbts.co2 { total_volume = _cold->total_volume; return (_cold->last_trade.load_if_new(ref time, out ps)); } + public bool get_last_trade(ref ulong time, out NativePriceSize ps, out DateTime timestamp, out ulong total_volume, out bool from_spread) { + timestamp = _cold->last_trade_time; + total_volume = _cold->total_volume; + from_spread = (_cold->from_spread != 0); + return (_cold->last_trade.load_if_new(ref time, out ps)); + } public ulong misc_timestamp { get { return _cold->misc_timestamp; } } public ulong total_volume { get { return _cold->total_volume; } } @@ -259,7 +265,7 @@ namespace mbts.co2 { // Last trade public AtomicPriceSize last_trade; public ulong total_volume; - public utils.LLDateTime last_trade_time; + public utils.LLDateTime last_trade_time; //// Settlement public UInt64 misc_timestamp; @@ -280,6 +286,7 @@ namespace mbts.co2 { public double session_low_offer; public double fixing_price; + public int from_spread; public int deleted; // FixedSecurity will not be directly addressable. Instead the security is available through the callback public Int64 security_size; diff --git a/src/client/spark_api/api/Update.cs b/src/client/spark_api/api/Update.cs index 80ed9b1cf..92bf6a471 100644 --- a/src/client/spark_api/api/Update.cs +++ b/src/client/spark_api/api/Update.cs @@ -8,7 +8,6 @@ using System.Linq; using System.Reactive.Concurrency; using System.Reactive.Disposables; using System.Reactive.Linq; -using System.Reactive.Subjects; using System.Text; namespace bts.api.core { @@ -94,38 +93,38 @@ namespace bts.api.core { new Update(new T[0], new T[0], new Change[0], new T[0], key_extractor); public Update merge(Update next) { - var comp = new RowEqualityComparer(_key); - var added = this.added as HashSet ?? new HashSet(this.added, comp); + var added = this.added as KeyedValueSet ?? new KeyedValueSet(this.added, _key); var changes = this.changed as ChangeDict ?? new ChangeDict(this.changes, _key); - var removed = this.removed as HashSet ?? new HashSet(this.removed, comp); + var removed = this.removed as KeyedValueSet ?? new KeyedValueSet(this.removed, _key); // Add - added.UnionWith(next.added); + foreach (var v in next.added) { + var k = _key(v); + if (removed.Remove(k, out var oldval)) { + changes.AddOrMerge(new Change(oldval, v)); + } else { + added[k] = v; + } + } // Change foreach (var c in next.changes) { - if (added.Contains(c.Old)) { - added.Remove(c.Old); - added.Add(c.New); - } - else { + var k = _key(c.Old); + if (!added.Replace(k, c.New)) changes.AddOrMerge(c); - } } // Remove foreach (var r in next.removed) { - if (!added.Remove(r)) { - removed.Add(r); - } - changes.Remove(_key(r)); + var k = _key(r); + if (!added.Remove(k)) + removed[k] = changes.TryRemove(k, out var ch) ? ch.Old : r; } return new Update( next.all, added, changes, removed, - _key - ); + _key); } public IEnumerable all { get; } @@ -148,6 +147,39 @@ namespace bts.api.core { #endif } + class KeyedValueSet : IEnumerable { + readonly Dictionary _cache = new Dictionary(); + + public KeyedValueSet(IEnumerable value, Func key) { + foreach (var v in value) + _cache[key(v)] = v; + } + + public TValue this[TKey key] { + set => _cache[key] = value; + } + + public bool Remove(TKey key) => + _cache.Remove(key); + + public bool Remove(TKey key, out TValue value) => + _cache.TryGetValue(key, out value) && _cache.Remove(key); + + public bool Replace(TKey key, TValue value) { + if (_cache.ContainsKey(key)) { + _cache[key] = value; + return true; + } + return false; + } + + public IEnumerator GetEnumerator() => + _cache.Values.GetEnumerator(); + + IEnumerator IEnumerable.GetEnumerator() => + GetEnumerator(); + } + class ChangeDict : IEnumerable> { Dictionary> _changes; Func, TKey> _key; @@ -167,9 +199,11 @@ namespace bts.api.core { return null; } - public void Remove(TKey key) { + public bool Remove(TKey key) => _changes.Remove(key); - } + + public bool TryRemove(TKey key, out Change result) => + _changes.TryGetValue(key, out result) && _changes.Remove(key); #region IEnumerable public IEnumerator> GetEnumerator() { @@ -181,31 +215,6 @@ namespace bts.api.core { #endregion } - struct RowEqualityComparer : IEqualityComparer, IEqualityComparer> - where TKey : IEquatable { - readonly Func _key; - - public RowEqualityComparer(Func key) { - _key = key; - } - - public bool Equals(T x, T y) { - return _key(x).Equals(_key(y)); - } - - public bool Equals(Change x, Change y) { - return Equals(x.New, y.New); - } - - public int GetHashCode(T obj) { - return _key(obj).GetHashCode(); - } - - public int GetHashCode(Change c) { - return GetHashCode(c.New); - } - } - public class TableUpdate where TKey : IEquatable { // warning: this does not have a key_extractor public static readonly TableUpdate empty = new TableUpdate(ImmutableDictionary.Empty, Array.Empty(), Array.Empty>(), Array.Empty(), null); @@ -736,13 +745,19 @@ namespace bts.api.core { return new CompositeDisposable(source.Subscribe(aggregator.on_left), right.Subscribe(aggregator.on_right)); }); - public static IObservable AggregateLatest(this IObservable source, IObservable right, - Func, TRight, TAggregate> update_left, Func, TAggregate> update_right, TAggregate seed, + public static IObservable AggregateLatest( + this IObservable source, + IObservable right, + Func, TRight, TAggregate> update_left, + Func, TAggregate> update_right, + Func seed_factory, Func result_selector, - TLeft init_left = default, TRight init_right = default) => + TLeft init_left = default, + TRight init_right = default) => + // Observable.Create(o => { Log.DataFlow.diag($"Create AggregateLatest (Left: {typeof(TLeft)}, Right: {typeof(TRight)})."); - var aggregator = new ConcurrentAggregator(update_left, update_right, agg => o.OnNext(result_selector(agg)), seed, init_left, init_right); + var aggregator = new ConcurrentAggregator(update_left, update_right, agg => o.OnNext(result_selector(agg)), seed_factory(), init_left, init_right); return new CompositeDisposable(source.Subscribe(aggregator.on_left), right.Subscribe(aggregator.on_right)); }); @@ -922,4 +937,4 @@ namespace bts.api.core { $"T is '{typeof(T)}', TKey is '{typeof(TKey)}'"; } } -} \ No newline at end of file +} diff --git a/src/client/spark_api/api/Update/TableUpdateFullJoin.cs b/src/client/spark_api/api/Update/TableUpdateFullJoin.cs index c1cc9d2b3..a66381fb3 100644 --- a/src/client/spark_api/api/Update/TableUpdateFullJoin.cs +++ b/src/client/spark_api/api/Update/TableUpdateFullJoin.cs @@ -18,7 +18,7 @@ namespace bts.api.core { return left_source.AggregateLatest, TableUpdate, (TableUpdate result, IReadOnlyDictionary left_state, IReadOnlyDictionary right_state), TableUpdate>(right_source, process_left, process_right, - (TableUpdate.mk_empty(result_key), new Dictionary(0), new Dictionary(0)), + () => (TableUpdate.mk_empty(result_key), new Dictionary(0), new Dictionary(0)), t => t.result, TableUpdate.mk_empty(left_key), TableUpdate.mk_empty(right_key)); // left diff --git a/src/client/spark_api/api/Update/TableUpdateLeftJoin.cs b/src/client/spark_api/api/Update/TableUpdateLeftJoin.cs index 1b6832260..eb9f7a56f 100644 --- a/src/client/spark_api/api/Update/TableUpdateLeftJoin.cs +++ b/src/client/spark_api/api/Update/TableUpdateLeftJoin.cs @@ -20,7 +20,7 @@ namespace bts.api.core { right_source, process_left, process_right, - (TableUpdate.mk_empty(result_key), new Dictionary>()), + () => (TableUpdate.mk_empty(result_key), new Dictionary>()), state => state.result, TableUpdate.mk_empty(left_key), TableUpdate.mk_empty(right_key)); diff --git a/src/client/spark/core/data/UserTuple.cs b/src/client/spark_api/data/UserTuple.cs similarity index 52% rename from src/client/spark/core/data/UserTuple.cs rename to src/client/spark_api/data/UserTuple.cs index b0689a74f..c4200b75c 100644 --- a/src/client/spark/core/data/UserTuple.cs +++ b/src/client/spark_api/data/UserTuple.cs @@ -1,15 +1,28 @@ using System; namespace bts { - struct UserTuple : IEquatable, IComparable, IComparable { + public readonly struct UserTuple : IEquatable, IComparable, IComparable { + const string _admin_name = "bts"; + public static readonly Func key_fn = u => u.id; + public static UserTuple admin => new UserTuple(0, _admin_name, Permissions.Admin); public readonly uint id; public readonly string name; + public readonly Permissions permissions; - public UserTuple(uint id, string name) { + public UserTuple(uint id, string name) + : this(id, name, name == _admin_name ? Permissions.Admin : Permissions.Regular) { } + + public UserTuple(uint id, string name, Permissions permissions) { this.id = id; this.name = name; + this.permissions = permissions; + } + + public enum Permissions { + Regular, + Admin } public int CompareTo(UserTuple other) { @@ -28,4 +41,9 @@ namespace bts { public static bool operator ==(UserTuple a, UserTuple b) => a.Equals(b); public static bool operator !=(UserTuple a, UserTuple b) => !a.Equals(b); } -} + + public static class UserTupleExtensions { + public static bool can_access_user_data(this UserTuple user, uint user_id) => + user.id == user_id || user.permissions == UserTuple.Permissions.Admin; + } +} \ No newline at end of file diff --git a/src/client/spark_api/md/MDSource.cs b/src/client/spark_api/md/MDSource.cs index 7281aba04..7aefc929a 100644 --- a/src/client/spark_api/md/MDSource.cs +++ b/src/client/spark_api/md/MDSource.cs @@ -13,84 +13,78 @@ namespace bts.md { enum MDSourceOptions { DeferInit = 1, } - class MDSource : IObserver>, IDisposable - { + + class MDSource : IObserver>, IDisposable { const uint _no_user_id = 0; + protected readonly MDConsumer _consumer = new MDConsumer(); protected readonly Co2RelayControl _relay; - protected DateTime _version = DateTime.MinValue; protected readonly ConcurrentDictionary _exchange_sec_map = new ConcurrentDictionary(); - protected List _dispose = new List(); protected readonly MDSourceOptions _options; - readonly uint _user_id; + protected DateTime _version = DateTime.MinValue; + protected List _dispose = new List(); - public MDSource(uint naggle_ms, MDSourceOptions options = 0, uint user_id = _no_user_id) - { + readonly UserTuple _user; + readonly BehaviorSubject> _on_securities = + new BehaviorSubject>(new ConcurrentDictionary()); + + public MDSource(uint naggle_ms, MDSourceOptions options = 0) + : this(naggle_ms, options, UserTuple.admin) { } + + public MDSource(uint naggle_ms, MDSourceOptions options, UserTuple user) { _options = options; - _user_id = user_id; + _user = user; _relay = new Co2RelayControl(_consumer, naggle_ms); _relay.preload_securities((co2.MDRequest.Types.Flags)0xff).ignore(); _dispose.Add(_consumer.on_security.Subscribe(this)); - if(!options.HasFlag(MDSourceOptions.DeferInit)) + if (!options.HasFlag(MDSourceOptions.DeferInit)) initialize(); } - readonly BehaviorSubject> _on_securities = - new BehaviorSubject>( new ConcurrentDictionary() ); public IObservable> on_securities => _on_securities; - virtual public void initialize() - { + public virtual void initialize() => Task.Run(run); - } - public void set_token(AuthTokenImmutable token) - { + public void set_token(AuthTokenImmutable token) => _relay.set_token(token); - } public void OnCompleted() { } - public void OnError(Exception error) => Log.Md.error(error.ToString()); + + public void OnError(Exception error) => + Log.Md.error(error.ToString()); + public void OnNext(List values) { if (values.Count > 0) { foreach (var v in values) { - if (_user_id != _no_user_id && - v.security.ContractCase == Security.ContractOneofCase.Synthetic && - v.security.Synthetic.UserId != _user_id && - v.security.Synthetic.UserId != _no_user_id) { - continue; + if (v.security.ContractCase != Security.ContractOneofCase.Synthetic || _user.can_access_user_data(v.security.Synthetic.UserId)) { + var smd = new SecurityMD(new SecurityImmutable(v.security), v.handle); + _exchange_sec_map.AddOrUpdate(smd.security.Id, smd, (_, __) => smd); } - var smd = new SecurityMD(new SecurityImmutable(v.security), v.handle); - _exchange_sec_map.AddOrUpdate(smd.security.Id, smd, (_, __) => smd); } _version = DateTime.UtcNow; } } - public void Dispose() => Disposable.dispose(_dispose); - async protected Task run() - { + public void Dispose() => + Disposable.dispose(_dispose); + + protected async Task run() { DateTime run_version = DateTime.MinValue; - while (true) - { + while (true) { var exch = _exchange_sec_map; var current_version = _version; - if (current_version > run_version) - { + if (current_version > run_version) { run_version = current_version; - try - { + try { Log.Md.info($"MDSource publishing {exch.Count} securities"); _on_securities.OnNext(exch); - } - catch (Exception e) - { + } catch (Exception e) { Log.Md.error($"Error in publish_securities: {e}"); } } await Task.Delay(TimeSpan.FromSeconds(1)); } } - } -} +} \ No newline at end of file diff --git a/src/client/spark_api/md/SecurityMD.cs b/src/client/spark_api/md/SecurityMD.cs index d4ce00c1b..6af6e0324 100644 --- a/src/client/spark_api/md/SecurityMD.cs +++ b/src/client/spark_api/md/SecurityMD.cs @@ -1,13 +1,11 @@ using bts.api; using mbts.co2; using System; -using System.Collections.Generic; -using System.Linq; -using System.Text; -using System.Threading.Tasks; namespace mbts { - struct SecurityMD : IEquatable { + readonly struct SecurityMD : IEquatable { + public static readonly Func key_fn = smd => smd.security.Id; + public SecurityImmutable security { get; } public MDConsumerHandle md { get; } @@ -36,4 +34,4 @@ namespace mbts { return !(left == right); } } -} +} \ No newline at end of file diff --git a/src/client/spark_core/Collections/Extensions/DictionaryExtensions.cs b/src/client/spark_core/Collections/Extensions/DictionaryExtensions.cs index d0f9c95cb..4ff0a9aa7 100644 --- a/src/client/spark_core/Collections/Extensions/DictionaryExtensions.cs +++ b/src/client/spark_core/Collections/Extensions/DictionaryExtensions.cs @@ -278,6 +278,9 @@ namespace System.Collections.Generic { public static IComparer> key_comparer() => Impl.key_comparer; + public static IReadOnlyDictionary wrap_value(TKey key, TValue value) => + new Impl.ValueWrapper(key, value); + static class Impl { public static readonly IReadOnlyDictionary empty = new ReadOnlyDictionary(new Dictionary(0)); @@ -297,6 +300,57 @@ namespace System.Collections.Generic { public int Compare(object x, object y) => x is KeyValuePair xkv ? (y is KeyValuePair ykv ? Compare(xkv, ykv) : 1) : -1; } + + public class ValueWrapper : IReadOnlyDictionary { + readonly TKey _key; + readonly TValue _value; + readonly IEqualityComparer _key_comparer; + + public ValueWrapper(TKey key, TValue value) + : this(key, value, EqualityComparer.Default) { } + + public ValueWrapper(TKey key, TValue value, IEqualityComparer key_comparer) { + _key = key; + _value = value; + _key_comparer = key_comparer; + } + + public TValue this[TKey key] => + _key_comparer.Equals(key, _key) ? _value : throw new KeyNotFoundException(); + + public int Count => 1; + + public IEnumerable Keys { + get { + yield return _key; + } + } + + public IEnumerable Values { + get { + yield return _value; + } + } + + public bool ContainsKey(TKey key) => _key_comparer.Equals(key, _key); + + public IEnumerator> GetEnumerator() { + yield return new KeyValuePair(_key, _value); + } + + IEnumerator IEnumerable.GetEnumerator() { + yield return new KeyValuePair(_key, _value); + } + + public bool TryGetValue(TKey key, out TValue value) { + if (_key_comparer.Equals(key, _key)) { + value = _value; + return true; + } + value = default; + return false; + } + } } } } \ No newline at end of file diff --git a/src/client/spark_core/Collections/Extensions/EnumerableExtensions.cs b/src/client/spark_core/Collections/Extensions/EnumerableExtensions.cs index c6fcd85e2..c1384e079 100644 --- a/src/client/spark_core/Collections/Extensions/EnumerableExtensions.cs +++ b/src/client/spark_core/Collections/Extensions/EnumerableExtensions.cs @@ -343,6 +343,18 @@ namespace System.Collections.Generic { return false; } + public static List ToListSorted(this IEnumerable enumerable, Func key_selector) => + enumerable.ToListSorted(key_selector, Comparer.Default); + + public static List ToListSorted(this IEnumerable enumerable, Func key_selector, IComparer key_comparer) => + enumerable.ToListSorted(key_comparer.wrap(key_selector)); + + public static List ToListSorted(this IEnumerable enumerable, IComparer comparer) { + var list = new List(enumerable); + list.Sort(comparer); + return list; + } + /// /// Simply returns enumerable.ToList(), under the guise ofo IReadOnlyList<> /// diff --git a/src/client/spark_core/Collections/Extensions/ImmutableSetExtensions.cs b/src/client/spark_core/Collections/Extensions/ImmutableSetExtensions.cs new file mode 100644 index 000000000..d22b9f6df --- /dev/null +++ b/src/client/spark_core/Collections/Extensions/ImmutableSetExtensions.cs @@ -0,0 +1,29 @@ +using System; +using System.Collections.Generic; +using System.Collections.Immutable; + +namespace bts.collections { + public static class ImmutableSetExtensions { + public static IImmutableSet RemoveRange(this IImmutableSet source, IEnumerable values) { + /// Prefer the implementation. + /// It uses an to remove values, which is optimized for bulk changes (less memory allocations, etc.). + if (source is ImmutableHashSet ihs) { + return ihs.RemoveRange(values); + } + if (source == null) throw new ArgumentNullException(nameof(source)); + if (values == null) throw new ArgumentNullException(nameof(values)); + foreach (var v in values) + source = source.Remove(v); + return source; + } + + public static ImmutableHashSet RemoveRange(this ImmutableHashSet source, IEnumerable values) { + if (source == null) throw new ArgumentNullException(nameof(source)); + if (values == null) throw new ArgumentNullException(nameof(values)); + var bd = source.ToBuilder(); + foreach (var v in values) + bd.Remove(v); + return bd.ToImmutable(); + } + } +} \ No newline at end of file diff --git a/src/client/spark_core/TestRunner/InProcessTestRunner.cs b/src/client/spark_core/TestRunner/InProcessTestRunner.cs index 871f06b58..4b786d1f5 100644 --- a/src/client/spark_core/TestRunner/InProcessTestRunner.cs +++ b/src/client/spark_core/TestRunner/InProcessTestRunner.cs @@ -10,9 +10,9 @@ using System.Diagnostics; namespace bts.test { public static class InProcessTestRunner { - public static void run_test_synchronously(string v, TestEnumerator enumerator, Action report) { - foreach (var t in enumerator.find(v)) { - run_test_synchronously(t, report); + public static void run_test_synchronously(string v, TestType type, TestEnumerator enumerator, Action report) { + foreach (var t in enumerator.find(v,type)) { + run_test_synchronously(t, report); } } public static void run_test_synchronously(TestMethod test, Action report) { diff --git a/src/client/spark_core/TestRunner/TestEnumerator.cs b/src/client/spark_core/TestRunner/TestEnumerator.cs index e18c9b838..0fbaf3087 100644 --- a/src/client/spark_core/TestRunner/TestEnumerator.cs +++ b/src/client/spark_core/TestRunner/TestEnumerator.cs @@ -36,18 +36,19 @@ namespace bts.test { } - public IEnumerable find(string name) { + public IEnumerable find(string name, TestType type) { foreach (var g in TestGroups) { foreach (var t in g.methods) { - if (g.name == name || t.name == name || t.full_name == name) { + if (g.test_type == type && + (g.name == name || t.name == name || t.full_name == name)) { yield return t; } } } } - public IEnumerable match(IEnumerable names) { - return names.SelectMany(find).Distinct(); + public IEnumerable match(TestType type, IEnumerable names) { + return names.SelectMany(name => find(name,type)).Distinct(); } } diff --git a/src/client/spark_core/admin/AdminModelWrapper.cs b/src/client/spark_core/admin/AdminModelWrapper.cs deleted file mode 100644 index d709c7675..000000000 --- a/src/client/spark_core/admin/AdminModelWrapper.cs +++ /dev/null @@ -1,64 +0,0 @@ -using bts; -using bts.api.admin; -using System; -using System.Collections.Generic; -using System.Linq; - -namespace bts.api.core -{ - public class AdminModelWrapper - { - readonly AdminDataImmutable? _admin_data; - public AdminDataImmutable? admin_data => _admin_data; - readonly string _user_name; - readonly List _allowed_accounts; - readonly Dictionary _allowed_account_dict = new Dictionary(); - public AdminModelWrapper(AdminDataImmutable? admin_data = null, string user_name = null) - { - _admin_data = admin_data; - _user_name = user_name ?? string.Empty; - _allowed_accounts = search_admin(u => u.Name == _user_name).Where(x => - { - return (x.Item2 != null && x.Item2.HasPermissions) || (x.Item3 != null && x.Item3.HasPermissions); - }).Select(xx => xx.Item1).ToList(); - foreach (var val in _allowed_accounts) - _allowed_account_dict[val.Name] = val.Name; - } - public IEnumerable viewable_accounts() - { - return _allowed_accounts; - } - public bool can_view_account(string account) - { - return _allowed_account_dict.ContainsKey(account) || _user_name.ToLower().Equals("bts"); - } - - IEnumerable> - search_admin(Func predicate) - { - var rv = new List>(); - if (_admin_data.HasValue) - { - var users = _admin_data.Value.Users.Where(predicate); - if (users != null && users.Count() == 1) - { - var user = users.ElementAt(0); - if (user.HasUserData && user.UserData.HasAccountPermissions) - { - foreach (var kv in user.UserData.AccountPermissions.AccountPermissions) - { - var account_id = kv.Key; - var accounts = _admin_data.Value.Accounts.Where(a => a.Id == account_id); - if (accounts != null && accounts.Count() == 1) - { - var account = accounts.ElementAt(0); - rv.Add(Tuple.Create(account, kv.Value.TradingPermission, kv.Value.AdminPermission)); - } - } - } - } - } - return rv; - } - } -} \ No newline at end of file diff --git a/src/client/spark_core/admin/PermissionFilters.cs b/src/client/spark_core/admin/PermissionFilters.cs deleted file mode 100644 index 3da6a137c..000000000 --- a/src/client/spark_core/admin/PermissionFilters.cs +++ /dev/null @@ -1,64 +0,0 @@ -using bts; -using bts.api; -using bts.api.core; -using bts.api.orderman; -using bts.api.position; -using bts.spark; -using System; -using System.Collections.Generic; -using System.Collections.Immutable; -using System.Collections.Specialized; -using System.Linq; -using System.Reactive.Concurrency; -using System.Reactive.Linq; - -namespace bts.api.core -{ - public static class PermissionFilters - { - public static IObservable> - PermissionFilter(this IObservable> source, IObservable am) - { - var pred = Observable.Create>( observer => - { - return am.Subscribe(x => - { - Func filter = (z => x.can_view_account(z.Account)); - observer.OnNext(filter); - }); - }); - - return source.Filter(pred); - } - - public static IObservable> - PermissionFilter(this IObservable> source, IObservable am) - { - var pred = Observable.Create>(observer => - { - return am.Subscribe(x => - { - Func filter = (z => x.can_view_account(z.Def.Account ?? string.Empty)); - observer.OnNext(filter); - }); - }); - - return source.Filter(pred); - } - - public static IObservable> - PermissionFilter(this IObservable> source, IObservable am) - { - var pred = Observable.Create>(observer => - { - return am.Subscribe(x => - { - Func filter = (z => x.can_view_account(z.Account)); - observer.OnNext(filter); - }); - }); - - return source.Filter(pred); - } - } -} \ No newline at end of file diff --git a/src/client/spark_core/logging/LogUtil.cs b/src/client/spark_core/logging/LogUtil.cs new file mode 100644 index 000000000..7c7d569b8 --- /dev/null +++ b/src/client/spark_core/logging/LogUtil.cs @@ -0,0 +1,96 @@ +using System; +using System.IO; +using mbts.config; +using System.IO.Compression; +using System.Runtime.InteropServices; +using Google.Protobuf; +using bts; +using System.Linq; +using bts.support; + +namespace mbts +{ + public static class LogUtil { + private static string log_path = Path.Combine(Configuration.get().data_path, "logging", "log"); + + + + public static ByteString read_logs_compressed() { + return FileUtils.read_and_gzip(log_path); + } + + public static (ByteString,DateTime) read_latest_crash_dump() { + // Find the latest crash dump + // If it is < 5 days old + // gzip it in place (delete original) + // upload the gziped dump + + var crash_dump_folder = "c:\\programdata\\bts\\crash_dumps"; + FileUtils.keep_n_matching_files(crash_dump_folder, "*.gz", 4); + try { + if (Directory.Exists(crash_dump_folder)) { + var latest = + Directory.GetFiles(crash_dump_folder, "*.dmp") + .Select(f => new { name = f, modified = File.GetLastWriteTimeUtc(f) }) + .OrderBy(f => f.modified) + .Reverse() + .FirstOrDefault(); + if (latest != null) { + if (DateTime.Now - latest.modified <= TimeSpan.FromDays(5)) { + var newname = FileUtils.gzip_file_in_place(latest.name); + if (!string.IsNullOrEmpty(newname)) { + return (ByteString.FromStream(File.OpenRead(newname)), latest.modified); + } + } + } + } + } + catch (Exception e) { + Log.Gen.error($"Failed to read crash dumps. Exception: {e.Message}"); + } + return (ByteString.Empty, DateTime.MinValue); + } + + public static ByteString read_windows_event_log() { + // try to export the events to temp + var temp = Path.Combine(Path.GetTempPath(), Path.GetTempFileName() + ".evtx"); + bool success = NativeMethods.EvtExportLog( + IntPtr.Zero, + "Application", + "*", + temp, + NativeMethods.EventExportLogFlags.ChannelPath); + if (success) { + var result = FileUtils.read_and_gzip(temp); + FileUtils.try_delete(temp); + return result; + } + else { + string msg = new System.ComponentModel.Win32Exception(Marshal.GetLastWin32Error()).Message; + Log.Gen.error($"Error exporting windows event log. {msg}"); + } + return ByteString.Empty; + } + + + static class NativeMethods { + [Flags] + public enum EventExportLogFlags { + ChannelPath = 1, + LogfilePath = 2, + TolerateQueryErrors = 0x1000 + } + + [DllImport("wevtapi.dll", CallingConvention=CallingConvention.Winapi, CharSet = CharSet.Auto, SetLastError =true)] + public static extern bool EvtExportLog( + IntPtr session, + string path, + string query, + string targetPath, + [MarshalAs(UnmanagedType.I4)] + EventExportLogFlags flags); + + } + + } +} diff --git a/src/client/spark_core/logging/Logger.cs b/src/client/spark_core/logging/Logger.cs deleted file mode 100644 index a1dacd603..000000000 --- a/src/client/spark_core/logging/Logger.cs +++ /dev/null @@ -1,32 +0,0 @@ -using System; -using System.Diagnostics; -using System.IO; -using System.Net.Mail; -using System.Reflection; -using System.Runtime.CompilerServices; -using System.Runtime.InteropServices; -using mbts.config; -using System.Collections.Generic; - - -namespace mbts -{ - public static class LogUtil { - public static string read_logs() { - var log_path = Path.Combine(Configuration.get().data_path, "logging", "log"); - if (File.Exists(log_path)) { - try { - using (var f = File.Open(log_path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)) { - var reader = new StreamReader(f); - return reader.ReadToEnd(); - } - } - catch (Exception e) { - return e.ToString(); - } - } - return "Log file does not exist"; - } - - } -} diff --git a/src/client/spark_core/utils/FileUtils.cs b/src/client/spark_core/utils/FileUtils.cs new file mode 100644 index 000000000..6be341755 --- /dev/null +++ b/src/client/spark_core/utils/FileUtils.cs @@ -0,0 +1,76 @@ +using System; +using System.IO; +using System.IO.Compression; +using System.Linq; +using Google.Protobuf; + +namespace bts.support { + static class FileUtils { + public static bool try_delete(string path) { + try { + if (File.Exists(path)) { + File.Delete(path); + } + else if (Directory.Exists(path)) { + Directory.Delete(path, true); + } + } + catch { + return false; + } + return true; + } + + public static ByteString read_and_gzip(string path) { + if (File.Exists(path)) { + try { + using (var fs = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite /*Note: ReadWrite is more permissive than Read*/)) + using (var ms = new MemoryStream()) + using (var gz = new GZipStream(ms, CompressionMode.Compress, true)) { + fs.CopyTo(gz); + gz.Close(); + return ByteString.CopyFrom(ms.GetBuffer(), 0, (int)ms.Length); + } + } + catch (Exception e) { + Log.Gen.error($"Failed to read file '{path}' -- {e.Message}"); + return ByteString.Empty; + } + } + return ByteString.Empty; + } + + + // If input exists compress it and add a .gz extension + // original file is deleted + public static string gzip_file_in_place(string file) { + if (File.Exists(file)) { + string destination = file + ".gz"; + using (var input = File.OpenRead(file)) { + using (var output = File.OpenWrite(destination)) { + using (var writer = new GZipStream(output, CompressionMode.Compress)) { + input.CopyTo(writer); + } + } + } + try_delete(file); + return destination; + } + return null; + } + + // Delete the oldest files matching pattern (*.txt) keeping {count} files + public static void keep_n_matching_files(string directory, string pattern, int count) { + var to_delete = + Directory.GetFiles(directory, pattern) + .Select(f => new { name = f, modified = File.GetLastWriteTimeUtc(f) }) + .OrderBy(f => f.modified) + .Reverse() + .Skip(count); + + foreach (var f in to_delete) { + try_delete(f.name); + } + } + } +} diff --git a/src/client/spark_core/utils/ObservableExtensions.cs b/src/client/spark_core/utils/ObservableExtensions.cs index 8544f29d0..fd1b102f8 100644 --- a/src/client/spark_core/utils/ObservableExtensions.cs +++ b/src/client/spark_core/utils/ObservableExtensions.cs @@ -212,7 +212,10 @@ namespace System.Reactive.Linq { IDisposable source_sub = source.Select(v => v).Subscribe(on_source); return Disposable.Create(dispose); // local methods - void on_source(T value) => Disposable.replace(ref value_sub, subscribe_value(value)); + void on_source(T value) { + Disposable.dispose(ref value_sub); + value_sub = subscribe_value(value); + } void dispose() { Disposable.dispose(ref source_sub); Disposable.dispose(ref value_sub); diff --git a/src/client/spark_test/Co2/Co2Tests.cs b/src/client/spark_test/Co2/Co2Tests.cs index a5f1b292a..af5edc856 100644 --- a/src/client/spark_test/Co2/Co2Tests.cs +++ b/src/client/spark_test/Co2/Co2Tests.cs @@ -142,7 +142,8 @@ namespace mbts.co2.tests Assert.IsTrue(entry.session_low_offer == 5.5); Assert.IsTrue(entry.fixing_price == 6.6); - Assert.IsTrue(entry.deleted == 1); + Assert.IsTrue(entry.from_spread == 0); + Assert.IsTrue(entry.deleted == 1); } [Test] diff --git a/src/client/spark_test/Utils/MarketDataUtilities.cs b/src/client/spark_test/Utils/MarketDataUtilities.cs index 5ab58171a..c08ab7678 100644 --- a/src/client/spark_test/Utils/MarketDataUtilities.cs +++ b/src/client/spark_test/Utils/MarketDataUtilities.cs @@ -2,6 +2,7 @@ using bts.co2; using System; using System.Collections.Generic; +using System.Linq; using System.Text; using System.Threading.Tasks; @@ -39,8 +40,29 @@ namespace bts.tests { return null; } +public static async Task> get_multiple_securities(MarketDataRelay relay, IEnumerable products, SecurityType security_type, MDRequest.Types.Flags flags = MDRequest.Types.Flags.Bbo, Venue v = Venue.Sim) { + var request = new MDRequest() { Host = _MD_HOST }; + request.Channels.AddRange(products.Select(z => + new SecurityChannel() { + Time = 0, + Flags = (uint)flags, + Type = security_type, + Exchange = v, + Symbol = z.Symbol, + TradeCursor = 0, + RfqCursor = 0 + })); + + var result = await relay.requestmd_async(request); + + if (result.success) { + return result.value.Channels.Where(s => s.Updates.Count > 0).Select(x => (SecurityImmutable) x.Updates.First().Security); + } + return Enumerable.Empty(); + } + public static async Task> get_expirations(MarketDataRelay relay, string symbol, SecurityType security_type, MDRequest.Types.Flags flags = MDRequest.Types.Flags.Bbo, Venue v = Venue.Sim) { - var request = new MDRequest() { + var request = new MDRequest() { Host = _MD_HOST, Channels = { new SecurityChannel() { diff --git a/src/client/spark_test/Utils/SpreaderUtilities.cs b/src/client/spark_test/Utils/SpreaderUtilities.cs index 5de24e9a6..6a09b97ca 100644 --- a/src/client/spark_test/Utils/SpreaderUtilities.cs +++ b/src/client/spark_test/Utils/SpreaderUtilities.cs @@ -67,7 +67,12 @@ namespace bts.tests return (s1: sec1.Value, s2: sec2.Value); } - public static async Task<(IEnumerable book, (Security security, ExecutionStrategy strategy) synthetic)> common_setup_one(OrderService order_service, SyntheticSecurityService synthetic_service, (SecurityImmutable? s1, SecurityImmutable? s2) securities, (String name, uint id) account, (String name, uint id) user) { + public static async Task<(IEnumerable book, (Security security, ExecutionStrategy strategy) synthetic)> common_setup_one( + OrderService order_service, + SyntheticSecurityService synthetic_service, + (SecurityImmutable? s1, SecurityImmutable? s2) securities, + (String name, uint id) account, (String name, uint id) user) + { var order_defs = new (OrderDefinition order_def, double price, uint size)[10]; for (int i = 0; i < 4; i ++) { diff --git a/src/client/spark_test/Utils/TestUtils.cs b/src/client/spark_test/Utils/TestUtils.cs index b36693280..8e01b02b9 100644 --- a/src/client/spark_test/Utils/TestUtils.cs +++ b/src/client/spark_test/Utils/TestUtils.cs @@ -85,6 +85,8 @@ namespace bts.tests.utils { if(timeout == null) { timeout = TimeSpan.FromMinutes(2); } + Console.WriteLine("Please Wait... this is going to take some time..."); + await Task.Delay(180000); System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch(); sw.Start(); @@ -131,6 +133,13 @@ namespace bts.tests.utils { System.Diagnostics.Stopwatch watch = new System.Diagnostics.Stopwatch(); watch.Start(); + var prts = mbts.config.Configuration.get(); + Console.WriteLine("\n\n\n\n\n\n\n\n\n\n"); + Console.WriteLine("config_port: " + prts.config_port); + Console.WriteLine("qbus_port: " + prts.qbus_port); + Console.WriteLine("glbx_sim_fix_port: " + prts.glbx_sim_fix_port); + Console.WriteLine("\n\n\n\n\n\n\n\n\n\n"); + var ctx = new ServiceContext(); var mdrelay = ctx.locate(); @@ -143,20 +152,28 @@ namespace bts.tests.utils { products = tmpp.value; break; } + else { + Console.WriteLine("mdrelay.request_products_async Error: "); + Console.WriteLine(tmpp.error_message); + } await Task.Delay(10000); } + //UserSymbol.get_all() in MDConsumer.cs bts.test.Assert.IsTrue(products != null, "No products found"); - var filtered_products = products.Products.Where(z => z.Exchange == Venue.Sim && z.Type == SecurityType.Future); - bts.test.Assert.IsTrue(filtered_products.Count() > 0, "No suitable products found"); + IEnumerable filtered_products = products.Products.Where(z => z.Exchange == Venue.Sim && z.Type == SecurityType.Future); + bts.test.Assert.IsTrue(filtered_products.Count() > 0, "No suitable products found"); //products are not guaranteed to have securities, so I need to get the product from the list of securities uint? sec_id = null; + /* get securities if the list is not empty use the top one */ + for(int i = 0; i < 10; i++) { - var test_security = await MarketDataUtilities.get_security(mdrelay, filtered_products.First().Symbol, SecurityType.Future, f => true); - if(test_security.HasValue) { + var test_security = await MarketDataUtilities.get_multiple_securities(mdrelay, filtered_products, SecurityType.Future); + + if(test_security.Count() > 0) { Console.Write("," + i); - sec_id = test_security.Value.Id; + sec_id = test_security.FirstOrDefault().Id; break; } await Task.Delay(10000); @@ -237,7 +254,7 @@ namespace bts.tests.utils { Result? md_result = await await_with_timeout( md_relay.requestmd_async(md_request), timeout); - + Console.Write(md_result); return md_result != null; } } diff --git a/src/client/spark_test/api/ApiUpdateTests.cs b/src/client/spark_test/api/ApiUpdateTests.cs index e41f9dbce..1b9e9dff7 100644 --- a/src/client/spark_test/api/ApiUpdateTests.cs +++ b/src/client/spark_test/api/ApiUpdateTests.cs @@ -347,6 +347,12 @@ namespace mbts.api.Test { var a3 = new Update(new[] { "a3" }, empty, new[] { new Change("a2", "a3") }, empty, key); var merged = a2.merge(a3); Assert.SequenceEqual(new[] { new Change("a1", "a3") }, merged.changes); + // clear the state, then add a2 back + // (this validates the fix for the Update.naggle bug where interim changes were converted into added/removed). + var cleared = new Update(empty, empty, Array.Empty>(), merged.all, key); + a2 = new Update(new[] { "a2" }, new[] { "a2" }, Array.Empty>(), empty, key); + merged = merged.merge(cleared).merge(a2); + Assert.SequenceEqual(new[] { new Change("a1", "a2") }, merged.changes); } T id(T t) => t; diff --git a/src/client/spark_test/system_test_scripts/delete_dbs.sh b/src/client/spark_test/system_test_scripts/delete_dbs.sh index 423cb3f23..8b1a24f1f 100644 --- a/src/client/spark_test/system_test_scripts/delete_dbs.sh +++ b/src/client/spark_test/system_test_scripts/delete_dbs.sh @@ -11,7 +11,7 @@ if [ -d "$sparkdata_dir/data" ] then [ -d $sparkdata_dir/data/databases/orders ] && rm -rf $sparkdata_dir/data/databases/orders [ -d $sparkdata_dir/data/databases/fills ] && rm -rf $sparkdata_dir/data/databases/fills - [ -d $sparkdata_dir/data/databases/ex_sim_orders ] && rm -rf $sparkdata_dir/data/databases/ex_sim_orders + [ -d $sparkdata_dir/data/databases/ex_sim_orders_venue_sim ] && rm -rf $sparkdata_dir/data/databases/ex_sim_orders_venue_sim [ -d $sparkdata_dir/data/jetfix ] && rm -rf $sparkdata_dir/data/jetfix [ -d $sparkdata_dir/data/exchange_sim ] && rm -rf $sparkdata_dir/data/exchange_sim diff --git a/src/client/spark_test/system_test_scripts/test_position_limits.sh b/src/client/spark_test/system_test_scripts/test_position_limits.sh index 2c94a2393..93ca9761c 100644 --- a/src/client/spark_test/system_test_scripts/test_position_limits.sh +++ b/src/client/spark_test/system_test_scripts/test_position_limits.sh @@ -30,6 +30,8 @@ cat << EOF > $sparkdata_dir/conf/spark.conf } EOF +echo "Port" +./sparkd configcheck Ports nohup ./sparkd run & pid=$! diff --git a/src/client/sparkcmd/Cmd/OrderStress.cs b/src/client/sparkcmd/Cmd/OrderStress.cs index 643703155..1a0bd48a5 100644 --- a/src/client/sparkcmd/Cmd/OrderStress.cs +++ b/src/client/sparkcmd/Cmd/OrderStress.cs @@ -54,6 +54,7 @@ namespace bts { var result = relay.requestmd(request); foreach (var ch in result.Channels) { + // if (ch.Platform != Platform.Sim) continue; foreach (var update in ch.Updates) { if (update.Security != null) { var m = new Market() { @@ -129,6 +130,7 @@ namespace bts { } var orders = ctx.locate(); + var factor = TimeSpan.TicksPerMillisecond / 1000; foreach (var m in all_markets(ctx)) { ConsoleEx.WriteLine(m.security.security_desc(), ConsoleColor.White); var tick = m.security.ContractSpec.LowTick; @@ -136,7 +138,12 @@ namespace bts { for (var x = 0; x < o.depth; ++x) { var price = m.ask.Value.price() + (tick * x); ConsoleEx.WriteLine($"Sell {o.quantity} at {price}", ConsoleColor.Red); - orders.create_order_async(mkorder(m.security.Id, price, o.quantity, Side.Sell)); + var sw = new System.Diagnostics.Stopwatch(); + sw.Start(); + var r = orders.create_order(mkorder(m.security.Id, price, o.quantity, Side.Sell)); + sw.Stop(); + // Log.Api.info($"Order: id {r.OrderKey} time {sw.ElapsedTicks/factor}us"); + Log.Api.info($"Order {m.security.security_desc()}: id {r.OrderKey} {r.ErrorCode} time {sw.ElapsedTicks/factor}us"); System.Threading.Thread.Sleep(o.sleep); } } @@ -145,7 +152,11 @@ namespace bts { for (var x=0; x < o.depth; ++x) { var price = m.bid.Value.price() - (tick * x); ConsoleEx.WriteLine($"Buy {o.quantity} at {price}", ConsoleColor.Blue); - orders.create_order_async(mkorder(m.security.Id, price, o.quantity, Side.Buy)); + var sw = new System.Diagnostics.Stopwatch(); + sw.Start(); + var r = orders.create_order(mkorder(m.security.Id, price, o.quantity, Side.Buy)); + sw.Stop(); + Log.Api.info($"Order {m.security.security_desc()}: id {r.OrderKey} {r.ErrorCode} time {sw.ElapsedTicks/factor}us"); System.Threading.Thread.Sleep(o.sleep); } } diff --git a/src/client/sparkcmd/Cmd/ServiceTestCommand.cs b/src/client/sparkcmd/Cmd/ServiceTestCommand.cs index 52dcee6cd..87a7b5776 100644 --- a/src/client/sparkcmd/Cmd/ServiceTestCommand.cs +++ b/src/client/sparkcmd/Cmd/ServiceTestCommand.cs @@ -9,6 +9,7 @@ using System.Threading; namespace bts { + [Verb("servicetest", HelpText="Scratch space for testing services")] public class ServiceTestOptions { [Option(nameof(synthetic))] public bool synthetic { get; set; } @@ -16,10 +17,10 @@ namespace bts { public bool user { get; set; } [Option(nameof(admin))] public bool admin { get; set; } - + [Option(nameof(latency))] + public bool latency { get; set; } } - [Verb("servicetest", HelpText="Scratch space for testing services")] public class ServiceTest { public static int Run(ServiceTestOptions o) { if (o.synthetic) { @@ -31,6 +32,9 @@ namespace bts { if (o.admin) { admin_test(); } + if (o.latency) { + latency_test(); + } return 0; } @@ -114,5 +118,21 @@ namespace bts { Thread.Sleep(100); } } + + private static void latency_test() { + var ctx = new ServiceContext(); + var s = ctx.locate(); + + UInt64 sequence = 0; + var factor = TimeSpan.TicksPerMillisecond / 1000; + while(true) { + var sw = new System.Diagnostics.Stopwatch(); + sw.Start(); + var reply = s.ping(new metrics.BufferMessage() { Sequence = ++sequence }); + sw.Stop(); + Console.WriteLine($"{reply.Sequence} - {sw.ElapsedMilliseconds}ms -- {sw.ElapsedTicks/factor} us"); + System.Threading.Thread.Sleep(1000); + } + } } } diff --git a/src/client/sparkcmd/Cmd/TestCommand.cs b/src/client/sparkcmd/Cmd/TestCommand.cs index 9b071eb7a..aff93988d 100644 --- a/src/client/sparkcmd/Cmd/TestCommand.cs +++ b/src/client/sparkcmd/Cmd/TestCommand.cs @@ -48,7 +48,7 @@ namespace sparkcmd { } else Configuration.initialize(); - Console.WriteLine(JsonConvert.SerializeObject(o)); + ConsoleEx.WriteLine(JsonConvert.SerializeObject(o), ConsoleColor.DarkGray); if (o.list) { list_tests((o.type)); @@ -58,13 +58,19 @@ namespace sparkcmd { if (o.timeout == 0 && o.tests.Count() == 1 && !o.self_contained) { //System.Threading.Thread.Sleep(5000); using (var trr = new TestResultReceiver(true)) { - InProcessTestRunner.run_test_synchronously(o.tests.First(), Enumerator, trr.get_reporter()); + InProcessTestRunner.run_test_synchronously(o.tests.First(), o.type, Enumerator, trr.get_reporter()); trr.print_summary(); + if (trr.ResultCount == 0) { + ConsoleEx.Write("No ", ConsoleColor.Red); + ConsoleEx.Write(o.type.ToString(), ConsoleColor.Yellow); + ConsoleEx.WriteLine($" tests matched filter '{o.tests.First()}'", ConsoleColor.Red); + return -1; + } return trr.ErrorCount; } } else if (o.tests.Count() > 0) { - return run_tests(o, Enumerator.match(o.tests)); + return run_tests(o, Enumerator.match(o.type, o.tests)); } else { return run_tests(o, Enumerator.get_of_type(o.type)); diff --git a/src/client/sparkcmd/Program.cs b/src/client/sparkcmd/Program.cs index 8999b8bcc..61b1ba983 100644 --- a/src/client/sparkcmd/Program.cs +++ b/src/client/sparkcmd/Program.cs @@ -29,8 +29,13 @@ namespace sparkcmd { // initialized from c++ instead of c# BTSNativeMethods.provide_command_line(args, args.Length); + var parser = new Parser(ps => { + ps.CaseInsensitiveEnumValues = true; + ps.HelpWriter = Console.Out; + }); + // note: this can be made to infer the options type list from the list of delegates - return Parser.Default + return parser .ParseArguments< ConfigcheckOptions, ScriptClientOptions, diff --git a/src/client/sparkcmd/sparkcmd.csproj b/src/client/sparkcmd/sparkcmd.csproj index 52d9db007..b05a9d2d3 100644 --- a/src/client/sparkcmd/sparkcmd.csproj +++ b/src/client/sparkcmd/sparkcmd.csproj @@ -22,7 +22,7 @@ - + diff --git a/src/data_files/files/onixs_ice/ConnectivityConfiguration.APITest.xml b/src/data_files/files/onixs_ice/ConnectivityConfiguration.APITest.xml index 334c3c48f..e4d4d7acc 100644 --- a/src/data_files/files/onixs_ice/ConnectivityConfiguration.APITest.xml +++ b/src/data_files/files/onixs_ice/ConnectivityConfiguration.APITest.xml @@ -11,7 +11,6 @@ --> - 63.247.113.163 @@ -19,7 +18,7 @@ bluetrd-ps Starts123 N - Legacy + New @@ -28,7 +27,7 @@ Value from ICE Support Value from ICE Support N - Legacy + New @@ -37,7 +36,7 @@ bluetrdl-ps Starts123 N - Legacy + New diff --git a/src/data_files/files/onixs_ice/ConnectivityConfiguration.PerfTest.xml b/src/data_files/files/onixs_ice/ConnectivityConfiguration.PerfTest.xml index 23680779e..94edae6c4 100644 --- a/src/data_files/files/onixs_ice/ConnectivityConfiguration.PerfTest.xml +++ b/src/data_files/files/onixs_ice/ConnectivityConfiguration.PerfTest.xml @@ -17,10 +17,10 @@ 63.247.113.214 3000 - Value from ICE Support - Value from ICE Support + bt_ps + Starts123 N - Legacy + New diff --git a/src/lib/CMakeDeps_win.cmake b/src/lib/CMakeDeps_win.cmake index 2c14978ea..0f42c5041 100644 --- a/src/lib/CMakeDeps_win.cmake +++ b/src/lib/CMakeDeps_win.cmake @@ -1 +1 @@ -set(sources api/apicore.proto;api/apigateway.proto;api/apioptions.proto;api/core/ApiClientInterop.cpp;api/core/ApiServerInterop.cpp;api/public/ApiServer.cpp;api/public/ClientChannel.cpp;api/public/QuickbusClientChannel.cpp;api/public/ServerChannel.cpp;api/public/ServiceContext.cpp;api/service_helpers/AdminServiceUtils.cpp;api/service_helpers/AlgoServiceUtils.cpp;api/service_helpers/PositionUtilities.cpp;api/services/AdminService.proto;api/services/AlgoDefinitionService.proto;api/services/AlgoService.proto;api/services/ApiUnitTestService.proto;api/services/AuthService.proto;api/services/BasePositionService.proto;api/services/ErrorReportService.proto;api/services/HelloService.proto;api/services/MBOService.proto;api/services/MarginService.proto;api/services/MarketDataAuxiliaryService.proto;api/services/MarketDataRelay.proto;api/services/MoneyFileService.proto;api/services/OrderService.proto;api/services/OutOfProcessURLListener.proto;api/services/PersistenceService.proto;api/services/PositionService.proto;api/services/ShellDiagnosticService.proto;api/services/ShellLatencyService.proto;api/services/SyntheticSecurityService.proto;api/test/ApiTests.cpp;api/test/HelloServiceServer.cpp;config/ConfigCheck.cpp;config/ConfigDownloader.cpp;config/ConfigErrorCollector.cpp;config/ConfigLog.cpp;config/ConfigPropertyEvaluator.cpp;config/ConfigValueQueue.cpp;config/ConfigurationException.cpp;config/ConfigurationState.cpp;config/PreprocessInputStream.cpp;config/config.cpp;config/defaults/MarketData.conf;config/defaults/Ports.conf;config/defaults/Symbols.conf;config/defaults/cme_reference_data.conf;config/defaults/edge.conf;config/defaults/jetfix.conf;config/defaults/keycode.conf;config/defaults/logging.conf;config/defaults/orderman_cfg.conf;config/defaults/orders_fills_cleanup.conf;config/defaults/position_cfg.conf;config/defaults/span.conf;config/defaults/spreader.conf;config/defaults/testkeys.conf;config/defaults/threads.conf;config/defaults/toggles.conf;config/extensions/extensions.proto;config/invalid/invalid.proto;config/protos/ExchangeSimCfg.proto;config/protos/MarketData.proto;config/protos/Ports.proto;config/protos/Symbols.proto;config/protos/UI.proto;config/protos/cme_reference_data.proto;config/protos/jetfix.proto;config/protos/keycode.proto;config/protos/logging.proto;config/protos/mercury_cfg.proto;config/protos/orderman_cfg.proto;config/protos/orders_fills_cleanup.proto;config/protos/perflog_cfg.proto;config/protos/position_cfg.proto;config/protos/quickbus_cfg.proto;config/protos/span.proto;config/protos/spreader.proto;config/protos/system.proto;config/protos/testkeys.proto;config/protos/threads.proto;config/protos/toggles.proto;config/tests/HTTPServer.cpp;config/tests/config_ref_tests.cpp;config/tests/config_tests.cpp;config/text_format/text_format.cpp;domain_model/Security.cpp;domain_model/Security.proto;domain_model/SecurityTests.cpp;domain_model/sod.cpp;domain_model/tests/domain_tests.cpp;expression/BooleanGrammar.cpp;expression/ExpressionEvaluator.cpp;expression/ParserExceptions.cpp;expression/Value.cpp;expression/tests/expression_parser_tests.cpp;logging/LoggerHub.cpp;logging/Sink.cpp;logging/logging.cpp;logging/tests/logging_tests.cpp;md/Diag/MDPeriodicDumper.cpp;md/Diag/ShmDumper.cpp;md/MBOProcessor.cpp;md/MBOServiceServer.cpp;md/MDClient.cpp;md/MDConsumer.cpp;md/MDConsumerHandle.cpp;md/MDConsumerInterop.cpp;md/MDProducer.cpp;md/MDProducerHandle.cpp;md/MemorySegmentUtils.cpp;md/PlatformLoader.cpp;md/PriceSize.cpp;md/SecurityCache.cpp;md/SecurityDB.cpp;md/SecurityIDRecorder.cpp;md/SecurityMapper.cpp;md/SecurityOracle.cpp;md/SettlementRecorder.cpp;md/Shm/FixedSecurity.cpp;md/SyntheticSecurityService.cpp;md/cme/MDP3Utils.cpp;md/common.cpp;md/ice/ICEUtils.cpp;md/protos/co2.proto;md/relay/QueryServer.cpp;md/relay/RelayClient.cpp;md/relay/RelayRecorder.cpp;md/test/Co2ManagedTestHelper.cpp;md/test/Co2TestInstrument.cpp;md/test/mbo_tests.cpp;md/test/md2_tests.cpp;md/test/md_test_utils.cpp;md/test/synthetic_book_merge_tests.cpp;md/vendor/OnixsUtils.cpp;os/os.cpp;os/tests/os_tests.cpp;quickbus/core/LocalNode.cpp;quickbus/core/Quickbus.cpp;quickbus/core/SubscriptionID.cpp;quickbus/core/TableID.cpp;quickbus/message_model/Update.cpp;quickbus/message_model/merge_updates.cpp;quickbus/network/QBConnection.cpp;quickbus/network/QBConnectionManager.cpp;quickbus/network/endpoint_description.cpp;quickbus/network/quickbus_msgs.proto;quickbus/pubsub/TablePublisher.cpp;quickbus/pubsub/TableSubscriber.cpp;quickbus/pubsub/tests/qb_pubsub_tests.cpp;quickbus/routing/INode.cpp;quickbus/routing/NodeID.cpp;quickbus/routing/Router.cpp;quickbus/rpc/RPCClient.cpp;quickbus/rpc/RPCServer.cpp;quickbus/rpc/RPCSessionID.cpp;quickbus/rpc/rpc_utils.cpp;quickbus/rpc/tests/qb_rpc_tests.cpp;quickbus/tests/quickbus_tests.cpp;sparkproto/SparkProtos.proto;testing/catch_test_runner.cpp;utils/AuthSessionManager.cpp;utils/AuthToken.cpp;utils/Dispatcher.cpp;utils/GrowableSPSCQueue.cpp;utils/HTTPStream.cpp;utils/PBUtils.cpp;utils/Puid.cpp;utils/SecureLogon.cpp;utils/SharedMemory.cpp;utils/Sqlite_utils.cpp;utils/Uuid.cpp;utils/crypto/CipherType.cpp;utils/crypto/CryptoInit.cpp;utils/crypto/Digest.cpp;utils/crypto/KeyCodeHelper.cpp;utils/crypto/PasswordCallback.cpp;utils/crypto/PrivateKey.cpp;utils/crypto/PublicKey.cpp;utils/crypto/iostreambio.cpp;utils/datetime.cpp;utils/download.cpp;utils/file_utils.cpp;utils/init_spark_lib.cpp;utils/is_unittest_environment.cpp;utils/sftp.cpp;utils/stats.cpp;utils/tests/AuthTokenTests.cpp;utils/tests/SingleWriterConcurrentVectorTests.cpp;utils/thread_manager.cpp) \ No newline at end of file +set(sources api/apicore.proto;api/apigateway.proto;api/apioptions.proto;api/core/ApiClientInterop.cpp;api/core/ApiServerInterop.cpp;api/public/ApiServer.cpp;api/public/ClientChannel.cpp;api/public/QuickbusClientChannel.cpp;api/public/ServerChannel.cpp;api/public/ServiceContext.cpp;api/service_helpers/AdminServiceUtils.cpp;api/service_helpers/AlgoServiceUtils.cpp;api/service_helpers/PositionUtilities.cpp;api/services/AdminService.proto;api/services/AlgoDefinitionService.proto;api/services/AlgoService.proto;api/services/ApiUnitTestService.proto;api/services/AuthService.proto;api/services/BasePositionService.proto;api/services/ErrorReportService.proto;api/services/HelloService.proto;api/services/MBOService.proto;api/services/MarginService.proto;api/services/MarketDataAuxiliaryService.proto;api/services/MarketDataRelay.proto;api/services/MoneyFileService.proto;api/services/OrderService.proto;api/services/OutOfProcessURLListener.proto;api/services/PersistenceService.proto;api/services/PositionService.proto;api/services/ShellDiagnosticService.proto;api/services/ShellLatencyService.proto;api/services/SyntheticSecurityService.proto;api/test/ApiTests.cpp;api/test/HelloServiceServer.cpp;config/ConfigCheck.cpp;config/ConfigDownloader.cpp;config/ConfigErrorCollector.cpp;config/ConfigLog.cpp;config/ConfigPropertyEvaluator.cpp;config/ConfigValueQueue.cpp;config/ConfigurationException.cpp;config/ConfigurationState.cpp;config/PreprocessInputStream.cpp;config/config.cpp;config/defaults/MarketData.conf;config/defaults/Ports.conf;config/defaults/Symbols.conf;config/defaults/cme_reference_data.conf;config/defaults/edge.conf;config/defaults/jetfix.conf;config/defaults/keycode.conf;config/defaults/logging.conf;config/defaults/orderman_cfg.conf;config/defaults/orders_fills_cleanup.conf;config/defaults/position_cfg.conf;config/defaults/span.conf;config/defaults/spreader.conf;config/defaults/testkeys.conf;config/defaults/threads.conf;config/defaults/toggles.conf;config/extensions/extensions.proto;config/invalid/invalid.proto;config/protos/ExchangeSimCfg.proto;config/protos/MarketData.proto;config/protos/Ports.proto;config/protos/Symbols.proto;config/protos/UI.proto;config/protos/cme_reference_data.proto;config/protos/jetfix.proto;config/protos/keycode.proto;config/protos/logging.proto;config/protos/mercury_cfg.proto;config/protos/orderman_cfg.proto;config/protos/orders_fills_cleanup.proto;config/protos/perflog_cfg.proto;config/protos/position_cfg.proto;config/protos/quickbus_cfg.proto;config/protos/span.proto;config/protos/spreader.proto;config/protos/system.proto;config/protos/testkeys.proto;config/protos/threads.proto;config/protos/toggles.proto;config/tests/HTTPServer.cpp;config/tests/config_ref_tests.cpp;config/tests/config_tests.cpp;config/text_format/text_format.cpp;domain_model/Security.cpp;domain_model/Security.proto;domain_model/SecurityTests.cpp;domain_model/sod.cpp;domain_model/tests/domain_tests.cpp;expression/BooleanGrammar.cpp;expression/ExpressionEvaluator.cpp;expression/ParserExceptions.cpp;expression/Value.cpp;expression/tests/expression_parser_tests.cpp;logging/LoggerHub.cpp;logging/Sink.cpp;logging/logging.cpp;logging/tests/logging_tests.cpp;md/Diag/MDPeriodicDumper.cpp;md/Diag/ShmDumper.cpp;md/MBOProcessor.cpp;md/MBOServiceServer.cpp;md/MDClient.cpp;md/MDConsumer.cpp;md/MDConsumerHandle.cpp;md/MDConsumerInterop.cpp;md/MDProducer.cpp;md/MDProducerHandle.cpp;md/MemorySegmentUtils.cpp;md/PlatformLoader.cpp;md/PriceSize.cpp;md/SecurityCache.cpp;md/SecurityDB.cpp;md/SecurityIDRecorder.cpp;md/SecurityMapper.cpp;md/SecurityOracle.cpp;md/SettlementRecorder.cpp;md/Shm/FixedSecurity.cpp;md/SyntheticSecurityService.cpp;md/cme/MDP3Utils.cpp;md/common.cpp;md/ice/ICEUtils.cpp;md/protos/co2.proto;md/relay/QueryServer.cpp;md/relay/RelayClient.cpp;md/relay/RelayRecorder.cpp;md/test/Co2ManagedTestHelper.cpp;md/test/Co2TestInstrument.cpp;md/test/mbo_tests.cpp;md/test/md2_tests.cpp;md/test/md_test_utils.cpp;md/test/synthetic_book_merge_tests.cpp;md/vendor/OnixsUtils.cpp;os/os.cpp;os/tests/os_tests.cpp;quickbus/core/LocalNode.cpp;quickbus/core/Quickbus.cpp;quickbus/core/SubscriptionID.cpp;quickbus/core/TableID.cpp;quickbus/message_model/Update.cpp;quickbus/message_model/merge_updates.cpp;quickbus/network/QBConnection.cpp;quickbus/network/QBConnectionManager.cpp;quickbus/network/endpoint_description.cpp;quickbus/network/quickbus_msgs.proto;quickbus/pubsub/TablePublisher.cpp;quickbus/pubsub/TableSubscriber.cpp;quickbus/pubsub/tests/qb_pubsub_tests.cpp;quickbus/routing/INode.cpp;quickbus/routing/NodeID.cpp;quickbus/routing/Router.cpp;quickbus/rpc/RPCClient.cpp;quickbus/rpc/RPCServer.cpp;quickbus/rpc/RPCSessionID.cpp;quickbus/rpc/rpc_utils.cpp;quickbus/rpc/tests/qb_rpc_tests.cpp;quickbus/tests/quickbus_tests.cpp;sparkproto/SparkProtos.proto;testing/catch_test_runner.cpp;utils/AuthSessionManager.cpp;utils/AuthToken.cpp;utils/Dispatcher.cpp;utils/GrowableSPSCQueue.cpp;utils/HTTPStream.cpp;utils/PBUtils.cpp;utils/Puid.cpp;utils/SecureLogon.cpp;utils/SharedMemory.cpp;utils/Sqlite_utils.cpp;utils/Uuid.cpp;utils/crypto/CipherType.cpp;utils/crypto/CryptoInit.cpp;utils/crypto/Digest.cpp;utils/crypto/KeyCodeHelper.cpp;utils/crypto/PasswordCallback.cpp;utils/crypto/PrivateKey.cpp;utils/crypto/PublicKey.cpp;utils/crypto/iostreambio.cpp;utils/datetime.cpp;utils/download.cpp;utils/file_utils.cpp;utils/init_spark_lib.cpp;utils/is_unittest_environment.cpp;utils/sftp.cpp;utils/stats.cpp;utils/tests/AuthTokenTests.cpp;utils/tests/IDGeneratorTests.cpp;utils/tests/SingleWriterConcurrentVectorTests.cpp;utils/thread_manager.cpp) \ No newline at end of file diff --git a/src/lib/api/service_compiler/templates/c#/immutable.pb.cs b/src/lib/api/service_compiler/templates/c#/immutable.pb.cs index 06c8260c6..2698f8750 100644 --- a/src/lib/api/service_compiler/templates/c#/immutable.pb.cs +++ b/src/lib/api/service_compiler/templates/c#/immutable.pb.cs @@ -10,6 +10,7 @@ namespace {{package}} { {{#each message_type}} partial class {{name}} : bts.api.IImmutableMessage { MessageDescriptor bts.api.IImmutableMessage.Descriptor => {{name}}.Descriptor; + public {{name}}Immutable immutable() => new {{name}}Immutable(this); } public readonly partial struct {{name}}Immutable : bts.api.IImmutableMessage { readonly {{name}} _impl; diff --git a/src/lib/api/service_helpers/AdminServiceUtils.cpp b/src/lib/api/service_helpers/AdminServiceUtils.cpp index 022542544..261acaeae 100644 --- a/src/lib/api/service_helpers/AdminServiceUtils.cpp +++ b/src/lib/api/service_helpers/AdminServiceUtils.cpp @@ -46,12 +46,15 @@ bool AdminServiceUtils::has_exchange(const ExchangeConnection& conn, break; case api::Venue::VENUE_UNSET: case api::Venue::VENUE_SIM: + case api::Venue::VENUE_SIMICE: rv = conn.has_sim_connection(); + break; + case api::Venue::VENUE_ICE: + rv = conn.has_ice_connection(); break; case api::Venue::VENUE_BTS: case api::Venue::VENUE_CBOE: case api::Venue::VENUE_CBOECFE: - case api::Venue::VENUE_ICE: case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_: case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_: rv = false; @@ -89,6 +92,23 @@ const ExchangeConnection* AdminServiceUtils::resolve_cme_connection( return nullptr; } +const bts::api::admin::ExchangeConnection* +AdminServiceUtils::resolve_ice_connection(const bts::api::admin::AdminData& data, + const bts::api::admin::BTSAccount& account, + const bts::api::Security&) +{ + if(account.has_connections()) + { + for (const std::string& conn_name : account.connections().exchange_connection_names()) + { + const bts::api::admin::ExchangeConnection* conn = find_connection(data, conn_name); + if (conn!=nullptr && conn->has_ice_connection()) + return conn; + } + } + return nullptr; +} + const bts::api::admin::ExchangeConnection* AdminServiceUtils::resolve_sim_connection(const bts::api::admin::AdminData& data, const bts::api::admin::BTSAccount& account, @@ -117,14 +137,22 @@ AdminServiceUtils::resolve_connection(const bts::api::admin::AdminData& data, rv = resolve_sim_connection(data, account, sec); break; } + case api::Venue::VENUE_ICE: + { + rv = resolve_ice_connection(data, account, sec); + auto deployment_type = boost::to_lower_copy(config::get().type()); + if (rv == nullptr && (deployment_type == "dev" || deployment_type == "demo")) + rv = resolve_sim_connection(data, account, sec); + break; + } case api::Venue::VENUE_UNSET: case api::Venue::VENUE_BTS: case api::Venue::VENUE_SIM: + case api::Venue::VENUE_SIMICE: rv = resolve_sim_connection(data, account, sec); break; case api::Venue::VENUE_CBOE: case api::Venue::VENUE_CBOECFE: - case api::Venue::VENUE_ICE: case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_: case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_: rv = nullptr; @@ -161,9 +189,15 @@ void AdminServiceUtils::fill_connections_from_config(bts::api::admin::AdminData& case config::JetFIX::Connection::CBOE: break; case config::JetFIX::Connection::SIM: + case config::JetFIX::Connection::SIMICE: connection.mutable_sim_connection(); add_connection(admin_data,connection); break; + case config::JetFIX::Connection::ICE: + connection.mutable_ice_connection()->set_sender_compid(c.sender_compid()); + add_connection(admin_data,connection); + break; + } } for(int i=0;iclearing_account(); break; } - case api::Venue::VENUE_ICE:{ + case api::Venue::VENUE_ICE: + { auto ice_account_data = get_account_ice_data(account, data); if (ice_account_data != nullptr) - rv = ice_account_data->clearing_account_override(); - break; + rv = ice_account_data->clearing_account(); + break; } - case api::Venue::VENUE_SIM: { + case api::Venue::VENUE_SIM: + case api::Venue::VENUE_SIMICE: + { auto sim_account_data = get_account_sim_data(account, data); if (sim_account_data != nullptr) rv = sim_account_data->clearing_account(); @@ -466,6 +503,46 @@ bts::api::orderman::RoutingRecord AdminServiceUtils::get_routing_record(const bt return rv; } switch (sec.exchange()) { + case api::Venue::VENUE_ICE: + { + const bts::api::admin::ICEUserData* ice_user_data = get_user_ice_data(*user_ptr, data); + if(ice_user_data==nullptr) + { + out_result.set_is_success(false); + out_result.set_error_text("Could not find ICE data for user"); + return rv; + } + const bts::api::admin::ICEAccountData* ice_account_data = get_account_ice_data(*account_ptr, data); + if(ice_account_data==nullptr) + { + out_result.set_is_success(false); + out_result.set_error_text("Could not find ICE data for account"); + return rv; + } + // check for data corectness + if (ice_user_data->operator_id().length() == 0) + { + out_result.set_is_success(false); + out_result.set_error_text("Operator ID is not set"); + return rv; + } + + rv.set_session(conn_ptr->name()); + rv.set_sender_compid(conn_ptr->ice_connection().sender_compid()); + rv.set_account(ice_account_data->clearing_account()); + rv.set_operator_id(ice_user_data->operator_id()); + rv.set_cti_code(ice_account_data->cti()); + rv.set_customer_or_firm(ice_account_data->origin()); + rv.set_clearing_firm(ice_account_data->clearing_firm()); + rv.set_customer_account_ref_id(ice_account_data->customer_account_ref_id()); + rv.set_account_code(ice_account_data->account_code()); + rv.set_company_id(std::to_string(ice_user_data->company_id())); + rv.set_authorized_trader(ice_user_data->authorized_trader()); + rv.set_authorized_group(ice_user_data->authorized_group()); + rv.set_cust_order_handing_inst(ice_user_data->cust_order_handing_inst()); + + return rv; + } case bts::api::Venue::VENUE_GLBX: { const bts::api::admin::CMEUserData* cme_user_data = get_user_cme_data(*user_ptr, data); if(cme_user_data==nullptr){ @@ -516,7 +593,9 @@ bts::api::orderman::RoutingRecord AdminServiceUtils::get_routing_record(const bt rv.set_customer_or_firm(cme_account_data->origin()); return rv; } - case api::Venue::VENUE_SIM: { + case api::Venue::VENUE_SIM: + case api::Venue::VENUE_SIMICE: + { const bts::api::admin::SIMUserData* sim_user_data = get_user_sim_data(*user_ptr, data); if(sim_user_data==nullptr){ out_result.set_is_success(false); @@ -562,8 +641,7 @@ bts::api::orderman::RoutingRecord AdminServiceUtils::get_routing_record(const bt case api::Venue::VENUE_UNSET: case api::Venue::VENUE_BTS: case api::Venue::VENUE_CBOE: - case api::Venue::VENUE_CBOECFE: - case api::Venue::VENUE_ICE: + case api::Venue::VENUE_CBOECFE: case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_: case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_: out_result.set_is_success(false); @@ -710,6 +788,18 @@ const bts::api::admin::CMEUserData* AdminServiceUtils::get_group_cme_data(const return nullptr; } +const bts::api::admin::ICEUserData* AdminServiceUtils::get_group_ice_data(const bts::api::admin::BTSUserGroup& group, + const bts::api::admin::AdminData& data){ + if (group.has_user_data() && group.user_data().has_exchange_user_data() && group.user_data().exchange_user_data().has_ice_data()) + return &(group.user_data().exchange_user_data().ice_data()); + else if (has_id(group.parent_group_id())){ + const bts::api::admin::BTSUserGroup* parent_group = find_group(data, group.parent_group_id()); + if (parent_group != nullptr) + return get_group_ice_data(*parent_group, data); + } + return nullptr; +} + const bts::api::admin::SIMUserData* AdminServiceUtils::get_group_sim_data(const bts::api::admin::BTSUserGroup& group, const bts::api::admin::AdminData& data){ if (group.has_user_data() && group.user_data().has_exchange_user_data() && group.user_data().exchange_user_data().has_sim_data()) @@ -840,6 +930,18 @@ const bts::api::admin::CMEUserData* AdminServiceUtils::get_user_cme_data(const b return nullptr; } +const bts::api::admin::ICEUserData* AdminServiceUtils::get_user_ice_data(const bts::api::admin::BTSUser& user, + const bts::api::admin::AdminData& data){ + if (user.has_user_data() && user.user_data().has_exchange_user_data() && user.user_data().exchange_user_data().has_ice_data()) + return &(user.user_data().exchange_user_data().ice_data()); + else{ + const bts::api::admin::BTSUserGroup* parent_group = find_group(data, user.parent_group_id()); + if (parent_group != nullptr) + return get_group_ice_data(*parent_group, data); + } + return nullptr; +} + const bts::api::admin::SIMUserData* AdminServiceUtils::get_user_sim_data(const bts::api::admin::BTSUser& user, const bts::api::admin::AdminData& data){ if (user.has_user_data() && user.user_data().has_exchange_user_data() && user.user_data().exchange_user_data().has_sim_data()) @@ -867,7 +969,16 @@ std::string AdminServiceUtils::find_user_exchange_id (const bts::api::admin::Adm rv = cme_user_data->operator_id(); break; } - case api::Venue::VENUE_SIM: { + case api::Venue::VENUE_ICE: + { + auto ice_user_data = get_user_ice_data(user, data); + if (ice_user_data != nullptr) + rv = ice_user_data->operator_id(); + break; + } + case api::Venue::VENUE_SIM: + case api::Venue::VENUE_SIMICE: + { auto sim_user_data = get_user_sim_data(user, data); if (sim_user_data != nullptr) rv = sim_user_data->operator_id(); @@ -877,8 +988,7 @@ std::string AdminServiceUtils::find_user_exchange_id (const bts::api::admin::Adm case api::Venue::VENUE_UNSET: case api::Venue::VENUE_BTS: case api::Venue::VENUE_CBOE: - case api::Venue::VENUE_CBOECFE: - case api::Venue::VENUE_ICE: + case api::Venue::VENUE_CBOECFE: case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_: case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_: out_result.set_is_success(false); out_result.set_error_text("Exchange not supported"); @@ -886,30 +996,92 @@ std::string AdminServiceUtils::find_user_exchange_id (const bts::api::admin::Adm } return rv; } -std::pair -AdminServiceUtils::find_user_account_matches (const bts::api::admin::AdminData& admin_data, - const std::string& connection, const std::string& exchange_account, + +std::string AdminServiceUtils::find_user_company_id (const bts::api::admin::AdminData& data, + const bts::api::admin::BTSUser& user, + api::Venue exchange, + bts::api::admin::AdminServiceOperationResult& out_result) +{ + out_result.set_is_success(true); + std::string rv = ""; + switch(exchange) + { + case api::Venue::VENUE_ICE: + { + auto ice_user_data = get_user_ice_data(user, data); + if (ice_user_data != nullptr) + rv = std::to_string(ice_user_data->company_id()); + break; + } + case api::Venue::VENUE_SIM: + case api::Venue::VENUE_SIMICE: + case api::Venue::VENUE_GLBX: + case api::Venue::VENUE_UNSET: + case api::Venue::VENUE_BTS: + case api::Venue::VENUE_CBOE: + case api::Venue::VENUE_CBOECFE: + case api::Venue::Venue_INT_MIN_SENTINEL_DO_NOT_USE_: + case api::Venue::Venue_INT_MAX_SENTINEL_DO_NOT_USE_: + out_result.set_is_success(false); out_result.set_error_text("Exchange not supported"); + break; + } + return rv; +} + +std::pair +AdminServiceUtils::find_user_account_matches (const AdminData& admin_data, + const std::string& connection, + const std::string& exchange_account, const std::string& exchange_user, api::Venue exchange){ - bts::api::admin::AdminServiceOperationResult find_account_result, find_user_result; - auto account_ptrs = accounts(admin_data, false); - auto user_ptrs = users(admin_data, false); - for(const bts::api::admin::BTSAccount* account_ptr : account_ptrs){ - if (account_has_connection(*account_ptr, connection)){ - if (boost::algorithm::iequals(exchange_account, - find_account_exchange_account(admin_data,*account_ptr,exchange,find_account_result)) && - find_account_result.is_success()){ - for(const bts::api::admin::BTSUser* user_ptr : user_ptrs){ - if (boost::algorithm::iequals(exchange_user, - find_user_exchange_id(admin_data,*user_ptr,exchange,find_user_result)) && - find_user_result.is_success()) - return std::pair(account_ptr,user_ptr); - } - } - } + AdminServiceOperationResult find_account_result, find_user_result; + // i. uses the exchange and exchange_user to reverse lookup the user(s) + auto user_ptrs = users(admin_data, false); + std::vector matching_users; + for (BTSUser const* u : user_ptrs){ + auto exchange_u = find_user_exchange_id(admin_data, *u, + exchange, find_user_result); + if (find_user_result.is_success() && boost::algorithm::iequals ( + exchange_u, + exchange_user)) + matching_users.push_back(u); } - return std::pair(nullptr, nullptr); + // i.5 + //At the ice, they send back the login level tag50 in exec reports which doesn't match any user level tag50s in the exchange_user + //so instead we look for the CompanyID (or client_id, tag 109) field here. If this field matches we have a good bet that + //the user we find is enough to fulfill the routing record enough to cancel or modify orphaned orders, etc + for (BTSUser const* u : user_ptrs) + { + auto exchange_u = find_user_company_id(admin_data, *u, exchange, find_user_result); + if (find_user_result.is_success() && boost::algorithm::iequals ( exchange_u, exchange_user)) + matching_users.push_back(u); + } + // ii. uses the exchange, connection and exchange_account to reverse lookup the account(s) + auto account_ptrs = accounts(admin_data, false); + std::vector matching_accounts; + for (BTSAccount const* a :account_ptrs) + if (account_has_connection(*a, connection)){ + auto exchange_a = find_account_exchange_account(admin_data, *a, + exchange, + find_account_result); + if (find_account_result.is_success() && boost::algorithm::iequals ( + exchange_a, + exchange_account)) + matching_accounts.push_back(a); + } + // iii. finds the first account from ii that the user from i is authorized to trade + for (BTSUser const* u : matching_users){ + std::vector user_auth_accounts; + find_user_trading_accounts (admin_data, u->id(), user_auth_accounts); + for (BTSAccount const* i : user_auth_accounts) + for (BTSAccount const* j : matching_accounts) + if(i->id() == j->id()) + return std::pair(i,u); + } + return std::pair(nullptr, nullptr); } + + // for a given user finds the account permissions (if permissions are // derived from the group, it traverses the group hierarcy to find the // correct permissions. @@ -995,16 +1167,26 @@ void AdminServiceUtils::validate_user_exchange_data (const bts::api::admin::BTSU } void AdminServiceUtils::get_md_entitlement_report(const AdminData& admin_data, std::vector& entitlements){ - for (const auto& user : admin_data.users()){ - const CMEUserData* cme_data=AdminServiceUtils::get_user_cme_data(user, - admin_data); - if(cme_data!=nullptr){ - for (const auto& cme_mic_data : cme_data->md_permissions()){ - MDEntitlementReportUnit rv=get_cme_md_entitlement_unit(user, - cme_mic_data); + for (const auto& user : admin_data.users()) + { + const CMEUserData* cme_data=AdminServiceUtils::get_user_cme_data(user, admin_data); + if(cme_data!=nullptr) + { + for (const auto& cme_mic_data : cme_data->md_permissions()) + { + MDEntitlementReportUnit rv=get_cme_md_entitlement_unit(user, cme_mic_data); entitlements.push_back(rv); } } + const ICEUserData* ice_data=AdminServiceUtils::get_user_ice_data(user, admin_data); + if(ice_data!=nullptr) + { + for (const auto& ice_mic_data : ice_data->md_permissions()) + { + MDEntitlementReportUnit rv=get_ice_md_entitlement_unit(user, ice_mic_data); + entitlements.push_back(rv); + } + } } } MDEntitlementReportUnit AdminServiceUtils::get_cme_md_entitlement_unit( @@ -1052,6 +1234,22 @@ MDEntitlementReportUnit AdminServiceUtils::get_cme_md_entitlement_unit( rv.set_record_id(key.str()); return rv; } +MDEntitlementReportUnit AdminServiceUtils::get_ice_md_entitlement_unit( + const BTSUser& user, + const ICEMicData& ice_mic_data){ + MDEntitlementReportUnit rv; + rv.set_user(user.name()); + rv.set_user_id(user.id()); + rv.set_exchange( bts::api::Venue_Name(bts::api::Venue::VENUE_ICE) ); + rv.set_mic( bts::api::ICEMics_Name( ice_mic_data.mic()) ); + + rv.set_enabled(ice_mic_data.enabled()); + std::ostringstream key; + key << rv.user() << rv.exchange() << rv.mic(); + rv.set_record_id(key.str()); + return rv; +} + void AdminServiceUtils::get_user_symbol_permissions(const AdminData& data, const BTSUser& user, std::vector& rv){ @@ -1126,13 +1324,13 @@ void AdminServiceUtils::set_user_exchange_data(bts::api::admin::BTSUser* user, if (!user->user_data().has_exchange_user_data()) user->mutable_user_data()->mutable_exchange_user_data()->CopyFrom(exchange_user_data); } -void AdminServiceUtils::set_user_md_entitlement_data(bts::api::admin::BTSUser* user, - const bts::api::admin::ExchangeUserMarketDataEntitlementData& exchange_user_md_entitelemnt_data){ - if (!user->has_user_data()) - user->mutable_user_data(); - if (!user->user_data().has_exchange_md_entitlement_data()) - user->mutable_user_data()->mutable_exchange_md_entitlement_data()->CopyFrom(exchange_user_md_entitelemnt_data); -} +// void AdminServiceUtils::set_user_md_entitlement_data(bts::api::admin::BTSUser* user, +// const bts::api::admin::ExchangeUserMarketDataEntitlementData& exchange_user_md_entitelemnt_data){ +// if (!user->has_user_data()) +// user->mutable_user_data(); +// if (!user->user_data().has_exchange_md_entitlement_data()) +// user->mutable_user_data()->mutable_exchange_md_entitlement_data()->CopyFrom(exchange_user_md_entitelemnt_data); +// } void AdminServiceUtils::clear_user_accounts (bts::api::admin::BTSUser* user){ if(user->has_user_data() && user->user_data().has_account_permissions()) user->mutable_user_data()->clear_account_permissions(); @@ -1377,6 +1575,7 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){ auto account_1 = create_account(data, parent_account->id(), std::string("ACC A1"), std::string("description"), true, account_result); account_1->set_is_active(true); auto account_2 = create_account(data, parent_account->id(), std::string("ACC A2"), std::string("description"), true, account_result); account_2->set_is_active(true); auto account_3 = create_account(data, parent_account->id(), std::string("ACC A3"), std::string("description"), true, account_result); account_3->set_is_active(true); + auto account_4 = create_account(data, parent_account->id(), std::string("BLUETST"), std::string("description"), true, account_result); account_4->set_is_active(true); create_account(data, parent_account->id(), std::string("ACC A4"), std::string("description"), true, account_result); // add group auto def_group = create_group(data, 0, std::string("default"), result); @@ -1387,8 +1586,7 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){ auto user_2 = create_user(data, def_group->id(), std::string("bts_2"), std::string("bts_2"), empty, empty, false, license_data, license_entitlements, user_result); user_2->set_is_active(true); auto user_3 = create_user(data, def_group->id(), std::string("bts_3"), std::string("bts_3"), empty, empty, false, license_data, license_entitlements, user_result); user_3->set_is_active(true); //associate users - bts::api::admin::UserAccountPermissionUnit permission_unit; - permission_unit.mutable_trading_permission()->set_submit_market_orders(true); + bts::api::admin::UserAccountPermissionUnit permission_unit; permission_unit.mutable_trading_permission()->set_has_permissions(true); permission_unit.mutable_admin_permission()->set_has_permissions(true); bts::api::admin::UserAccountPermissions account_permissions; @@ -1396,6 +1594,7 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){ account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair(account_1->id(), permission_unit)); account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair(account_2->id(), permission_unit)); account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair(account_3->id(), permission_unit)); + account_permissions.mutable_account_permissions()->insert(google::protobuf::MapPair(account_4->id(), permission_unit)); set_user_accounts(user_1, account_permissions); set_user_accounts(user_2, account_permissions); set_user_accounts(user_3, account_permissions); @@ -1405,6 +1604,12 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){ exchange_account_data.mutable_cme_account_data()->set_cti(bts::api::orderman::CTICode::CTI_CODE_CTI_CODE1); exchange_account_data.mutable_cme_account_data()->set_origin(bts::api::orderman::CustomerOrFirm::CUSTOMER_OR_FIRM_FIRM); exchange_account_data.mutable_cme_account_data()->set_clearing_account(parent_account->name()); + + exchange_account_data.mutable_ice_account_data()->set_cti(bts::api::orderman::CTICode::CTI_CODE_UNDEFINED); + exchange_account_data.mutable_ice_account_data()->set_origin(bts::api::orderman::CustomerOrFirm::CUSTOMER_OR_FIRM_FIRM); + exchange_account_data.mutable_ice_account_data()->set_clearing_account(parent_account->name()); + exchange_account_data.mutable_ice_account_data()->set_clearing_firm("649"); + exchange_account_data.mutable_sim_account_data()->set_cti(bts::api::orderman::CTICode::CTI_CODE_CTI_CODE1); exchange_account_data.mutable_sim_account_data()->set_origin(bts::api::orderman::CustomerOrFirm::CUSTOMER_OR_FIRM_FIRM); exchange_account_data.mutable_sim_account_data()->set_clearing_account(parent_account->name()); @@ -1412,33 +1617,87 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){ exchange_account_data.mutable_cme_account_data()->set_clearing_account(account_1->name()); + exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_1->name()); exchange_account_data.mutable_sim_account_data()->set_clearing_account(account_1->name()); set_account_exchange_data(account_1, exchange_account_data); exchange_account_data.mutable_cme_account_data()->set_clearing_account(account_2->name()); + exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_2->name()); exchange_account_data.mutable_sim_account_data()->set_clearing_account(account_2->name()); set_account_exchange_data(account_2, exchange_account_data); exchange_account_data.mutable_cme_account_data()->set_clearing_account(account_3->name()); + exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_3->name()); exchange_account_data.mutable_sim_account_data()->set_clearing_account(account_3->name()); set_account_exchange_data(account_3, exchange_account_data); + exchange_account_data.mutable_ice_account_data()->set_clearing_account(account_4->name()); + set_account_exchange_data(account_4, exchange_account_data); + + // add cme and sim user data bts::api::admin::ExchangeUserData exchange_user_data; exchange_user_data.mutable_cme_data()->set_operator_id("op_id_" + user_1->name()); exchange_user_data.mutable_cme_data()->set_location("us,nc"); + + exchange_user_data.mutable_ice_data()->set_operator_id("op_id_" + user_1->name()); + exchange_user_data.mutable_ice_data()->set_company_id(51251); + exchange_user_data.mutable_ice_data()->set_authorized_trader("bluetrd-fx"); + exchange_user_data.mutable_ice_data()->set_authorized_group("Group"); + exchange_user_data.mutable_ice_data()->set_password("Starts123"); + exchange_user_data.mutable_ice_data()->set_mifid_direct_electronic_access(-1); + exchange_user_data.mutable_ice_data()->set_mifid_trading_capacity(-1); + exchange_user_data.mutable_ice_data()->set_mifid_liquidity_provision(-1); + exchange_user_data.mutable_ice_data()->set_mifid_commodity_deriv_indicator(-1); + exchange_user_data.mutable_ice_data()->set_mifid_investment_decision(-1); + exchange_user_data.mutable_ice_data()->set_mifid_execution_decision(-1); + exchange_user_data.mutable_ice_data()->set_mifid_client_id_code(-1); + exchange_user_data.mutable_ice_data()->set_mifid_id(1); + exchange_user_data.mutable_sim_data()->set_operator_id("op_id_" + user_1->name()); exchange_user_data.mutable_sim_data()->set_location("us,nc"); set_user_exchange_data(user_1, exchange_user_data); exchange_user_data.mutable_cme_data()->set_operator_id("op_id_" + user_2->name()); exchange_user_data.mutable_cme_data()->set_location("us,nc"); + + exchange_user_data.mutable_ice_data()->set_operator_id("op_id_" + user_2->name()); + exchange_user_data.mutable_ice_data()->set_company_id(51251); + exchange_user_data.mutable_ice_data()->set_authorized_trader("bluetrd-fx"); + exchange_user_data.mutable_ice_data()->set_authorized_group("Group"); + exchange_user_data.mutable_ice_data()->set_password("Starts123"); + exchange_user_data.mutable_ice_data()->set_mifid_direct_electronic_access(-1); + exchange_user_data.mutable_ice_data()->set_mifid_trading_capacity(-1); + exchange_user_data.mutable_ice_data()->set_mifid_liquidity_provision(-1); + exchange_user_data.mutable_ice_data()->set_mifid_commodity_deriv_indicator(-1); + exchange_user_data.mutable_ice_data()->set_mifid_investment_decision(-1); + exchange_user_data.mutable_ice_data()->set_mifid_execution_decision(-1); + exchange_user_data.mutable_ice_data()->set_mifid_client_id_code(-1); + exchange_user_data.mutable_ice_data()->set_mifid_id(1); + + exchange_user_data.mutable_sim_data()->set_operator_id("op_id_" + user_2->name()); exchange_user_data.mutable_sim_data()->set_location("us,nc"); set_user_exchange_data(user_2, exchange_user_data); exchange_user_data.mutable_cme_data()->set_operator_id("op_id_" + user_3->name()); exchange_user_data.mutable_cme_data()->set_location("us,nc"); + + exchange_user_data.mutable_ice_data()->set_operator_id("op_id_" + user_3->name()); + exchange_user_data.mutable_ice_data()->set_company_id(51251); + exchange_user_data.mutable_ice_data()->set_authorized_trader("bluetrd-fx"); + exchange_user_data.mutable_ice_data()->set_authorized_group("Group"); + exchange_user_data.mutable_ice_data()->set_password("Starts123"); + exchange_user_data.mutable_ice_data()->set_mifid_direct_electronic_access(-1); + exchange_user_data.mutable_ice_data()->set_mifid_trading_capacity(-1); + exchange_user_data.mutable_ice_data()->set_mifid_liquidity_provision(-1); + exchange_user_data.mutable_ice_data()->set_mifid_commodity_deriv_indicator(-1); + exchange_user_data.mutable_ice_data()->set_mifid_investment_decision(-1); + exchange_user_data.mutable_ice_data()->set_mifid_execution_decision(-1); + exchange_user_data.mutable_ice_data()->set_mifid_client_id_code(-1); + exchange_user_data.mutable_ice_data()->set_mifid_id(1); + + exchange_user_data.mutable_sim_data()->set_operator_id("op_id_" + user_3->name()); exchange_user_data.mutable_sim_data()->set_location("us,nc"); set_user_exchange_data(user_3, exchange_user_data); @@ -1480,6 +1739,13 @@ void AdminServiceUtils::bootstrap_data (bts::api::admin::AdminData& data){ add_security_clip_sizes_to_account(account_1, api::Venue::VENUE_GLBX, api::SecurityType::SECURITYTYPE_SPREAD,100); add_security_clip_sizes_to_account(account_1, api::Venue::VENUE_SIM, api::SecurityType::SECURITYTYPE_FUTURE,100); add_security_clip_sizes_to_account(account_1, api::Venue::VENUE_SIM, api::SecurityType::SECURITYTYPE_SPREAD,100); + + //ICE limits + add_product_position_limits(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_FUTURE,50000); + add_product_position_limits(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_SPREAD,50000); + add_security_clip_sizes_to_account(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_FUTURE,50); + add_security_clip_sizes_to_account(account_4, api::Venue::VENUE_ICE, api::SecurityType::SECURITYTYPE_SPREAD,50); + } bool AdminServiceUtils::has_id (uint32_t id){return id>0;} bool AdminServiceUtils::is_symbol_set (const std::string& symbol){ diff --git a/src/lib/api/service_helpers/AdminServiceUtils.h b/src/lib/api/service_helpers/AdminServiceUtils.h index ebd941cf6..dc6095e41 100644 --- a/src/lib/api/service_helpers/AdminServiceUtils.h +++ b/src/lib/api/service_helpers/AdminServiceUtils.h @@ -35,6 +35,10 @@ public: const AdminData& data, const BTSAccount& account, const api::Security& sec); + static const ExchangeConnection* resolve_ice_connection( + const AdminData& data, + const BTSAccount& account, + const api::Security& sec); static const ExchangeConnection* resolve_sim_connection( const AdminData& data, const BTSAccount& account, @@ -164,6 +168,8 @@ public: // travels up the group tree until it finds a populated cme data static const CMEUserData* get_group_cme_data(const BTSUserGroup& group, const AdminData& data); + static const ICEUserData* get_group_ice_data(const BTSUserGroup& group, + const AdminData& data); static const SIMUserData* get_group_sim_data(const BTSUserGroup& group, const AdminData& data); static const UserAccountPermissions* const get_group_user_permissions ( @@ -198,6 +204,8 @@ public: static uint32_t find_max_user_id (const AdminData& data); static const CMEUserData* get_user_cme_data(const BTSUser& user, const AdminData& data); + static const ICEUserData* get_user_ice_data(const BTSUser& user, + const AdminData& data); static const SIMUserData* get_user_sim_data(const BTSUser& user, const AdminData& data); // finds the "exchange user" (ex: "operator_id" at CME or "authorized_trader" @@ -207,6 +215,15 @@ public: const BTSUser& user, api::Venue exchange, AdminServiceOperationResult& out_result); + // i. uses the exchange and exchange_user to reverse lookup the user(s) + // ii. uses the exchange, connection and exchange_account to reverse lookup the account(s) + // iii. finds the first account from ii that a user from i is authorized to trade + static std::string find_user_company_id ( + const AdminData& data, + const BTSUser& user, + api::Venue exchange, + AdminServiceOperationResult& out_result); + static std::pair find_user_account_matches (const AdminData& admin_data, const std::string& connection, @@ -246,6 +263,9 @@ public: static MDEntitlementReportUnit get_cme_md_entitlement_unit( const BTSUser& user, const CMEMicData& cme_mic_data); + static MDEntitlementReportUnit get_ice_md_entitlement_unit( + const BTSUser& user, + const ICEMicData& ice_mic_data); static void get_user_symbol_permissions( const AdminData& data, const BTSUser& user, @@ -272,8 +292,6 @@ public: static void set_user_exchange_data( BTSUser* user, const ExchangeUserData& exchange_user_data); - static void set_user_md_entitlement_data(BTSUser* user, - const ExchangeUserMarketDataEntitlementData&); static void clear_user_accounts (BTSUser* user); static void set_user_accounts (BTSUser* user, const UserAccountPermissions& user_account_permissions); diff --git a/src/lib/api/services/AdminService.proto b/src/lib/api/services/AdminService.proto index a274c3cb6..1a1f57fff 100644 --- a/src/lib/api/services/AdminService.proto +++ b/src/lib/api/services/AdminService.proto @@ -55,11 +55,12 @@ message BTSUser{ string name = 30; string password = 40; string full_name = 42; - string company = 44; + string company = 44; bool is_active = 50; bool can_create = 55; - LicenseData license_data = 60; //unused - LicenseEntitlements license_entitlements = 70; //unused + LicenseData license_data = 60; + LicenseEntitlements license_entitlements = 70; + BTSUserDerivableData user_data = 80; } @@ -76,7 +77,7 @@ message BTSUserDerivableData{ // b. else its overridden at the user level UserAccountPermissions account_permissions = 70; ExchangeUserData exchange_user_data = 80; - ExchangeUserMarketDataEntitlementData exchange_md_entitlement_data = 90; + reserved 90; reserved "exchange_md_entitlement_data"; //ExchangeUserMarketDataEntitlementData exchange_md_entitlement_data = 90; } message LicenseData{ @@ -91,21 +92,22 @@ message LicenseEntitlements{ bool spreader = 30; } -message AlgoTradingPermissionsUnit{ - bool submit_iceberg_orders = 10; //unused - bool submit_time_box_orders = 20; //unused - bool submit_spreader_orders = 30; //unused -} +// message AlgoTradingPermissionsUnit{ +// bool submit_iceberg_orders = 10; //unused +// bool submit_time_box_orders = 20; //unused +// bool submit_spreader_orders = 30; //unused +// } message TradingPermissionUnit{ + reserved 10; reserved "submit_market_orders"; //bool submit_market_orders = 10; //unused + reserved 20; reserved "submit_gtc_orders"; //bool submit_gtc_orders = 20; //unused + reserved 30; reserved "algo_trading_permission"; //AlgoTradingPermissionsUnit algo_trading_permission = 30; //unused + bool has_permissions = 5; - bool submit_market_orders = 10; //unused - bool submit_gtc_orders = 20; //unused - AlgoTradingPermissionsUnit algo_trading_permission = 30; //unused } message AdminPermissionUnit{ - // TODO: needs to be more ellaborate (account writing, + // TODO: needs to be more ellaborate (account writing, // reading etc, routing etc) bool has_permissions = 10; bool has_md_permissions = 20; @@ -119,15 +121,15 @@ message UserAccountPermissionUnit{ message UserAccountPermissions{ //uint32 = account_id - map account_permissions = 10; + map account_permissions = 10; } -// unused now -message UserSecurityTradingLimit{ - SecurityScope security = 10; - uint32 clip_size = 20; - bool can_trade = 30; -} +// // unused now +// message UserSecurityTradingLimit{ +// SecurityScope security = 10; +// uint32 clip_size = 20; +// bool can_trade = 30; +// } /************************************************************************/ /********************************BTS Account*****************************/ @@ -152,7 +154,7 @@ message BTSAccount{ AccountLossLimit account_loss_limit = 130; } -message RiskLimitEnforcement{ +message RiskLimitEnforcement{ bool product_limit_in_net_terms = 10; bool ignore_spread_position_limits = 20; } @@ -204,23 +206,22 @@ message AccountLossLimit{ message CMEMicData{ api.CMEMics mic=10; bool enabled=20; - // there are other things here + // there are other things here } message ICEMicData{ - api.CMEMics mic=10; + api.ICEMics mic=10; bool enabled=20; } -message ExchangeUserMarketDataEntitlementData{ - CMEUserMarketDataEntitlementData cme_data = 10; - ICEUserMarketDataEntitlementData ice_data = 20; -} message ExchangeUserData{ CMEUserData cme_data = 10; ICEUserData ice_data = 20; SIMUserData sim_data = 30; + + enum AdminDisplay { All = 0; MD = 1; Routing = 2; None = 100; } + AdminDisplay admin_display_flag = 120; } message ExchangeAccountData{ @@ -229,9 +230,6 @@ message ExchangeAccountData{ SIMAccountData sim_account_data = 30; } -message MarketDataEntitlement{ - bool can_view = 10; -} message ExchangeConnection{ uint32 id = 10; @@ -274,19 +272,19 @@ message CMEMSGWConnection{ uint32 market_segment_id = 60; } -message CMEAccountData{ - // should perhaps be an enum +message CMEAccountData { + reserved 30; reserved "takeup_member_code"; //string takeup_member_code = 30; + reserved 50; reserved "direct_electronic_access"; //string direct_electronic_access = 50; + reserved 60; reserved "trading_capacity"; //string trading_capacity = 60; + reserved 70; reserved "liquidity_provision"; //string liquidity_provision = 70; + reserved 80; reserved "commodity_derivative_indicator"; //string commodity_derivative_indicator = 80; + reserved 90; reserved "investment_decision_id"; //string investment_decision_id = 90; + reserved 100; reserved "execution_decision_id"; //string execution_decision_id = 100; + reserved 110; reserved "client_id"; //string client_id = 110; + api.orderman.CTICode cti = 10; api.orderman.CustomerOrFirm origin = 20; - string takeup_member_code = 30; string clearing_account = 40; - string direct_electronic_access = 50; - string trading_capacity = 60; - string liquidity_provision = 70; - string commodity_derivative_indicator = 80; - string investment_decision_id = 90; - string execution_decision_id = 100; - string client_id = 110; map misc_tags = 120; } @@ -309,21 +307,22 @@ enum DisplayDesignation } message CMEUserData{ + reserved 20; reserved "ats_operator_id"; //string ats_operator_id = 20; + reserved 40; reserved "text_a"; //string text_a = 40; + reserved 50; reserved "text_b"; //string text_b = 50; + reserved 80; reserved "max_order_rate_per_connection"; //string max_order_rate_per_connection = 80; + //enum AdminDisplay { All = 0; MD = 1; Routing = 2; None = 100; } + reserved 120; reserved "admin_display_flag"; //AdminDisplay admin_display_flag = 120; + repeated CMEMicData md_permissions = 1000; + string operator_id = 10; - string ats_operator_id = 20; string smp_id = 30; - string text_a = 40; - string text_b = 50; map misc_tags = 60; string smp_instruction = 70; - string max_order_rate_per_connection = 80; + string location = 90; ProfessionalDesignation pro_nonpro = 100; DisplayDesignation display_nondisplay = 110; - - enum AdminDisplay { All = 0; MD = 1; Routing = 2; None = 100; } - AdminDisplay admin_display_flag = 120; - repeated CMEMicData md_permissions = 1000; } message SIMUserData { @@ -331,58 +330,67 @@ message SIMUserData { string location = 90; } -message CMEUserMarketDataEntitlementData{ - MarketDataEntitlement cbot_entitlement = 10; - MarketDataEntitlement cme_entitlement = 20; - MarketDataEntitlement nymex_entitlement = 30; - MarketDataEntitlement bmandf_entitlement = 40; - MarketDataEntitlement comex_entitlement = 50; - MarketDataEntitlement cme_europe_entitlement = 60; -} - /********************************ICE Data*********************************/ -message ICEConnection{ - string authorized_member_id = 10; - string sender_company_id = 20; - string sender_sub_id = 30; - string session_id = 40; - string session_password = 50; - string host_ip = 60; - string port = 70; - string default_routing_trader = 80; - string routing_trader_password = 90; - int32 max_order_rate = 100; +message ICEConnection { + string sender_compid = 10; } -message ICEAccountData{ - string nybit_house = 10; - int32 wce_house = 20; - string clearing_account_override = 30; - bool direct_electronic_access = 40; - string trading_capacity = 50; - bool liquidity_provision = 60; - bool commodity_derivative_indicator = 70; - string investment_decision_id = 80; - string execution_decision_id = 90; - string client_id = 100; - string mfid_id = 110; +message ICEAccountData { + reserved 10; reserved "nybit_house"; //string nybit_house = 10; + reserved 20; reserved "wce_house"; //int32 wce_house = 20; + reserved 30; reserved "clearing_account_override"; //string clearing_account_override = 30; + reserved 40; reserved "direct_electronic_access"; //bool direct_electronic_access = 40; + reserved 50; reserved "trading_capacity"; //string trading_capacity = 50; + reserved 60; reserved "liquidity_provision"; //bool liquidity_provision = 60; + reserved 70; reserved "commodity_derivative_indicator"; //bool commodity_derivative_indicator = 70; + reserved 80; reserved "investment_decision_id"; //string investment_decision_id = 80; + reserved 90; reserved "execution_decision_id"; //string execution_decision_id = 90; + reserved 100; reserved "client_id"; //string client_id = 100; + reserved 110; reserved "mfid_id"; //string mfid_id = 110; + + api.orderman.CTICode cti = 3; + api.orderman.CustomerOrFirm origin = 5; + string clearing_firm = 200; + string clearing_account = 210; + string customer_account_ref_id = 220; + string account_code = 230; + map misc_tags = 500; } -message ICEUserData{ - string authorized_trader = 10; - string memo_field = 20; - string text_b = 30; - uint32 max_order_rate = 40; -} +message ICEUserData { + reserved 20; reserved "memo_field"; //string memo_field = 20; + reserved 30; reserved "text_b"; //string text_b = 30; + reserved 40; reserved "max_order_rate"; //uint32 max_order_rate = 40; -message ICEUserMarketDataEntitlementData{ + int32 company_id = 3; //tag 109 clientid + string operator_id = 5; //tag 50 + string authorized_trader = 10; //tag 553 user_name + string authorized_group = 15; //tag 553 user_name + string password = 21; //tag 96 raw data + string cust_order_handing_inst = 25; + + //mifid + int32 mifid_direct_electronic_access = 500; + int32 mifid_trading_capacity = 510; + int32 mifid_liquidity_provision = 520; + int32 mifid_commodity_deriv_indicator = 530; + int64 mifid_investment_decision = 540; + int64 mifid_execution_decision = 550; + int64 mifid_client_id_code = 560; + int64 mifid_id = 570; + + map misc_tags = 60; + ProfessionalDesignation pro_nonpro = 100; + DisplayDesignation display_nondisplay = 110; + + repeated ICEMicData md_permissions = 1000; } /////////////////////////////////////////////////////////// // Order Auxiliary Service //////////////////////////////// -message OrdersSummaryUnit{ +message OrdersSummaryUnit{ string account = 10; string user = 20; uint32 total_orders = 30; @@ -393,11 +401,11 @@ message OrdersSummary{ } message OrderSummaryRequest{ } -// top level orders that have account +// top level orders that have account // and user that are present here, will // be cancelled. wild cards are not // supported. -message OrderCancelRequest{ +message OrderCancelRequest{ repeated string accounts = 10; repeated string users = 20; } @@ -433,7 +441,7 @@ message AccountCreateAPIArgument{ string creator_user = 40; } message AccountUpdateAPIArgument{ - BTSAccount complete_data=10; + BTSAccount complete_data=10; } message UserCreateAPIArgument{ uint32 parent_group_id = 20; @@ -477,7 +485,7 @@ service AdminService { // users rpc create_user(UserCreateAPIArgument) returns (AdminServiceCreateEntityResult); rpc update_user(UserUpdateAPIArgument) returns (AdminServiceOperationResult); - rpc delete_user(IDAPIArgument) returns (AdminServiceOperationResult); + rpc delete_user(IDAPIArgument) returns (AdminServiceOperationResult); // subscription rpc admin_data(bts.api.VoidMessage) returns (stream AdminData); rpc get_admin_data(bts.api.VoidMessage) returns (AdminData); diff --git a/src/lib/api/services/AuthService.proto b/src/lib/api/services/AuthService.proto index 2dd9997d2..f7b984497 100644 --- a/src/lib/api/services/AuthService.proto +++ b/src/lib/api/services/AuthService.proto @@ -57,4 +57,7 @@ service AuthService { rpc get_users(GetUsersRequest) returns (GetUsersResponse) {option (bts.api.method_description)="Returns list of valid users";} + + rpc clear_all_sessions(bts.api.VoidMessage) returns (bts.api.ResultMessage) + {option (bts.api.method_description)="Clears all reserved auth sessions";} } diff --git a/src/lib/api/services/ErrorReportService.proto b/src/lib/api/services/ErrorReportService.proto index d85ec4e4c..d5d382d9d 100644 --- a/src/lib/api/services/ErrorReportService.proto +++ b/src/lib/api/services/ErrorReportService.proto @@ -4,6 +4,11 @@ option csharp_namespace="bts.support"; import "lib/api/apioptions.proto"; import "google/protobuf/timestamp.proto"; +message FileData { + string name = 1; + bytes contents = 2; +} + message ErrorReport { google.protobuf.Timestamp timestamp = 1; string user = 2; @@ -13,6 +18,7 @@ message ErrorReport { string log = 6; bytes image = 7; string contact = 8; + repeated FileData files = 9; } message ErrorResponse { } diff --git a/src/lib/api/services/OrderService.proto b/src/lib/api/services/OrderService.proto index 091487e66..3448f79ea 100644 --- a/src/lib/api/services/OrderService.proto +++ b/src/lib/api/services/OrderService.proto @@ -310,6 +310,13 @@ message Fill { FILL_KIND_SOD = 2; } FillKind fill_kind = 10; + + enum FillStatus { + FILL_STATUS_ACTIVE = 0; + FILL_STATUS_CORRECTED = 1; + FILL_STATUS_BUSTED = 2; + } + FillStatus fill_status = 20; } enum OrderSubType { @@ -327,6 +334,17 @@ message RoutingRecord { string smp_id = 6 ; SelfMatchPreventionInstruction smp_instruction = 7; string location = 8; + string cust_order_handling_inst = 9; + + string sender_compid = 15; + string clearing_firm = 20; + string customer_account_ref_id = 25; + string account_code = 30; + string company_id = 45; + string authorized_trader = 50; + string authorized_group = 60; + string cust_order_handing_inst = 70; + string order_server_id = 200; } message NativeOrderState { @@ -585,6 +603,7 @@ enum DynamicOrderPrimitiveDescriptor { DYNAMIC_ORDER_PRIMITIVE_BOOL = 4; DYNAMIC_ORDER_PRIMITIVE_ORDER_DEF = 5; DYNAMIC_ORDER_PRIMITIVE_QUANTITY = 6; + DYNAMIC_ORDER_PRIMITIVE_SECURITY_KEY = 7; } message DynamicOrderTypeDescriptor { diff --git a/src/lib/api/services/ShellLatencyService.proto b/src/lib/api/services/ShellLatencyService.proto index 272310703..8d1a253ae 100644 --- a/src/lib/api/services/ShellLatencyService.proto +++ b/src/lib/api/services/ShellLatencyService.proto @@ -15,8 +15,13 @@ message Report { string user = 3; uint64 timestamp = 10; } +message BufferMessage { + uint64 sequence = 1; + string data = 10; +} service ShellLatencyService { rpc clock(bts.api.VoidMessage) returns (stream Tick); rpc report(Report) returns (bts.api.VoidMessage); + rpc ping(BufferMessage) returns (BufferMessage); } \ No newline at end of file diff --git a/src/lib/config/ConfigurationState.cpp b/src/lib/config/ConfigurationState.cpp index b7eabbb88..0678fbd61 100755 --- a/src/lib/config/ConfigurationState.cpp +++ b/src/lib/config/ConfigurationState.cpp @@ -24,6 +24,7 @@ #include #include + #define URDL_HEADER_ONLY 1 #define URDL_DISALE_SSL 1 #include @@ -908,3 +909,4 @@ std::string get_fallback(const proto::Message& message) } }}}//end namespaces + diff --git a/src/lib/config/defaults/MarketData.conf b/src/lib/config/defaults/MarketData.conf index a8dd25828..7ffb0b467 100644 --- a/src/lib/config/defaults/MarketData.conf +++ b/src/lib/config/defaults/MarketData.conf @@ -62,15 +62,44 @@ ICEOptions { value { channel { name: "oil" #these names are arbitrary and are for debugging + market_type: 3 + uds_market_type: 3 + market_type: 4 #ICE IPE Gas Oil Futures + uds_market_type: 4 market_type: 5 #ICE Futures Brent, IPE Brent Futures - market_type: 9 #ICE Futures WTI, ICE WTI Crude Futures - #udp_buffer_size: 33554432 #32mb - } - channel { - name: "juice" - market_type: 23 #ICE Futures US Agricultures, FCOJ A + uds_market_type: 5 + market_type: 9 #ICE Futures WTI, ICE WTI Crude Futures + uds_market_type: 9 + market_type: 13 #ICE Heating Oil Futures + uds_market_type: 13 + market_type: 15 #ICE NYH (RBOB) Gasoline Futures + uds_market_type: 15 + ##market_type: 21 #ICE Futures US Agricultures, Coffee C + ##uds_market_type: 21 + ##market_type: 23 #ICE Futures US Agricultures, FCOJ A + ##uds_market_type: 23 + + ##market_type: 26 #ICE Futures US Financials DX + ##uds_market_type: 26 + + uds_market_type: 11 #ICE WTI-Brent spreads + uds_market_type: 14 #ICE Heating Oil/WTI Futures Crack + uds_market_type: 16 #ICE NYH (RBOB) Gasoline/WTI Futures Crack + uds_market_type: 58 #ICE Futures UI Heating Oil / Brent + uds_market_type: 59 #ICE NYH (RBOB) Gasoline/Brent Futures Crack + uds_market_type: 66 #ICE NYH (RBOB) Gasoline/Heating Oil Spread + uds_market_type: 70 #ICE Heating Oil/LS Gasoil Futures Spread + uds_market_type: 71 #ICE LS Gasoil/Brent Futures Crack + uds_market_type: 72 #ICE NYH (RBOB) Gasoline/LS Gasoil Futures Spread + + #onixs_log_mode: Debug } + # channel { + # name: "juice" + # market_type: 26 #ICE Futures US Agricultures, FCOJ A + # } ice_config_file: "ConnectivityConfiguration.APITest.xml" + #ice_config_file: "ConnectivityConfiguration.PerfTest.xml" } } diff --git a/src/lib/config/defaults/Ports.conf b/src/lib/config/defaults/Ports.conf index 266c65c9e..a8059b4f7 100644 --- a/src/lib/config/defaults/Ports.conf +++ b/src/lib/config/defaults/Ports.conf @@ -3,7 +3,8 @@ Ports { value { config_port: 8000 qbus_port: 8001 - exchange_sim_fix_port: 8002 + glbx_sim_fix_port: 8002 + ice_sim_fix_port: 8003 replicator_port: 8080 } } \ No newline at end of file diff --git a/src/lib/config/defaults/Symbols.conf b/src/lib/config/defaults/Symbols.conf index 8494cc578..c95aaba61 100644 --- a/src/lib/config/defaults/Symbols.conf +++ b/src/lib/config/defaults/Symbols.conf @@ -19,7 +19,6 @@ SymbolList { symbol: "FOB" symbol: "GC" symbol: "GE" - symbol: "GLI" symbol: "HE" symbol: "HG" symbol: "HO" @@ -66,9 +65,16 @@ SymbolLists list { key: "ICE" - # symbol: "BRN" - # symbol: "WBS" - # symbol: "OJ" + symbol: "BRN" + symbol: "WBS" + symbol: "OJ" + symbol: "G" + symbol: "KC" + symbol: "UHO" + symbol: "UHU" + symbol: "DX" + + symbol: "*" } list { @@ -112,5 +118,9 @@ Symbols { future: @SymbolLists.list[ICE].symbol spread: @SymbolLists.list[ICE].symbol } + simice { + future: @SymbolLists.list[ICE].symbol + spread: @SymbolLists.list[ICE].symbol + } } } diff --git a/src/lib/config/defaults/cme_reference_data.conf b/src/lib/config/defaults/cme_reference_data.conf index ac365cefb..ae2a5d4d7 100644 --- a/src/lib/config/defaults/cme_reference_data.conf +++ b/src/lib/config/defaults/cme_reference_data.conf @@ -7,7 +7,6 @@ CppConfig { value { - cpp_entry { key: "098" value: 2000 } cpp_entry { key: "0A" value: 100 } cpp_entry { key: "0B" value: 100 } cpp_entry { key: "0C" value: 100 } @@ -354,6 +353,7 @@ CppConfig cpp_entry { key: "BOB" value: 1000 } cpp_entry { key: "BOO" value: 6350 } cpp_entry { key: "BOS" value: 250 } + cpp_entry { key: "BPU" value: 1000 } cpp_entry { key: "BR7" value: 1000 } cpp_entry { key: "BSF" value: 25 } cpp_entry { key: "BTC" value: 5 } @@ -442,12 +442,12 @@ CppConfig cpp_entry { key: "D3L" value: 80 } cpp_entry { key: "D4L" value: 5 } cpp_entry { key: "D7L" value: 80 } + cpp_entry { key: "DAB" value: 1000 } cpp_entry { key: "DAX" value: 5 } cpp_entry { key: "DAZ" value: 80 } cpp_entry { key: "DBB" value: 1000 } cpp_entry { key: "DBI" value: 1000 } cpp_entry { key: "DBL" value: 100 } - cpp_entry { key: "DBR" value: 1000 } cpp_entry { key: "DBT" value: 100 } cpp_entry { key: "DC" value: 2000 } cpp_entry { key: "DCB" value: 1000 } @@ -458,14 +458,13 @@ CppConfig cpp_entry { key: "DDE" value: 1000 } cpp_entry { key: "DDI" value: 1000 } cpp_entry { key: "DDO" value: 1000 } - cpp_entry { key: "DEB" value: 744 } + cpp_entry { key: "DEB" value: 745 } cpp_entry { key: "DEN" value: 250 } - cpp_entry { key: "DEP" value: 264 } + cpp_entry { key: "DEP" value: 276 } cpp_entry { key: "DFL" value: 100 } cpp_entry { key: "DFN" value: 100 } cpp_entry { key: "DG2" value: 100 } cpp_entry { key: "DG3" value: 100 } - cpp_entry { key: "DGH" value: 1000 } cpp_entry { key: "DGI" value: 100 } cpp_entry { key: "DGS" value: 100 } cpp_entry { key: "DGT" value: 100 } @@ -473,19 +472,13 @@ CppConfig cpp_entry { key: "DHI" value: 1000 } cpp_entry { key: "DJR" value: 100 } cpp_entry { key: "DKR" value: 1000 } - cpp_entry { key: "DMB" value: 100 } - cpp_entry { key: "DOB" value: 1000 } - cpp_entry { key: "DOP" value: 1000 } - cpp_entry { key: "DRB" value: 1000 } cpp_entry { key: "DRI" value: 100 } cpp_entry { key: "DRS" value: 100 } cpp_entry { key: "DRT" value: 100 } cpp_entry { key: "DSD" value: 1000 } cpp_entry { key: "DSE" value: 1000 } - cpp_entry { key: "DSG" value: 1000 } cpp_entry { key: "DTR" value: 2 } cpp_entry { key: "DTT" value: 2 } - cpp_entry { key: "DUA" value: 6350 } cpp_entry { key: "DUP" value: 1000 } cpp_entry { key: "DVE" value: 200 } cpp_entry { key: "DVT" value: 200 } @@ -530,8 +523,11 @@ CppConfig cpp_entry { key: "EP1" value: 1000 } cpp_entry { key: "EPN" value: 1000 } cpp_entry { key: "EPZ" value: 500000 } + cpp_entry { key: "EQ1" value: 50 } cpp_entry { key: "ERL" value: 5 } cpp_entry { key: "ES" value: 50 } + cpp_entry { key: "ES1" value: 50 } + cpp_entry { key: "ES2" value: 50 } cpp_entry { key: "ESB" value: 1000 } cpp_entry { key: "ESK" value: 125000 } cpp_entry { key: "ESQ" value: 50 } @@ -733,8 +729,8 @@ CppConfig cpp_entry { key: "IPO" value: 10 } cpp_entry { key: "IPT" value: 10 } cpp_entry { key: "IS" value: 50 } - cpp_entry { key: "ITB" value: 744 } - cpp_entry { key: "ITP" value: 264 } + cpp_entry { key: "ITB" value: 745 } + cpp_entry { key: "ITP" value: 276 } cpp_entry { key: "IW" value: 50 } cpp_entry { key: "J4L" value: 80 } cpp_entry { key: "J7" value: 6250000 } @@ -748,7 +744,7 @@ CppConfig cpp_entry { key: "JKB" value: 10000 } cpp_entry { key: "JKF" value: 10000 } cpp_entry { key: "JKM" value: 10000 } - cpp_entry { key: "JML" value: 480 } + cpp_entry { key: "JML" value: 240 } cpp_entry { key: "JNC" value: 1000 } cpp_entry { key: "JNG" value: 10000 } cpp_entry { key: "JNL" value: 80 } @@ -811,6 +807,7 @@ CppConfig cpp_entry { key: "LIY" value: 1000 } cpp_entry { key: "LL" value: 42000 } cpp_entry { key: "LLR" value: 1000 } + cpp_entry { key: "LNG" value: 10000 } cpp_entry { key: "LPE" value: 47000 } cpp_entry { key: "LPS" value: 1000 } cpp_entry { key: "LSF" value: 1 } @@ -1147,7 +1144,7 @@ CppConfig cpp_entry { key: "SBT" value: 50 } cpp_entry { key: "SD" value: 1000 } cpp_entry { key: "SDA" value: 250 } - cpp_entry { key: "SDD" value: 1760 } + cpp_entry { key: "SDD" value: 1840 } cpp_entry { key: "SDG" value: 250 } cpp_entry { key: "SDI" value: 1000 } cpp_entry { key: "SDM" value: 100 } @@ -1164,10 +1161,12 @@ CppConfig cpp_entry { key: "SFR" value: 250 } cpp_entry { key: "SG" value: 250 } cpp_entry { key: "SGB" value: 1000 } + cpp_entry { key: "SGC" value: 1000 } cpp_entry { key: "SGE" value: 42000 } cpp_entry { key: "SGF" value: 1000 } cpp_entry { key: "SGG" value: 1000 } cpp_entry { key: "SGT" value: 250 } + cpp_entry { key: "SGU" value: 32.15 } cpp_entry { key: "SI" value: 5000 } cpp_entry { key: "SIL" value: 1000 } cpp_entry { key: "SIR" value: 500 } @@ -1186,7 +1185,7 @@ CppConfig cpp_entry { key: "SR3" value: 2500 } cpp_entry { key: "SR5" value: 1000 } cpp_entry { key: "SRB" value: 1000 } - cpp_entry { key: "SRT" value: 1760 } + cpp_entry { key: "SRT" value: 1840 } cpp_entry { key: "SSI" value: 1000 } cpp_entry { key: "SSP" value: 5000 } cpp_entry { key: "SSW" value: 1000 } @@ -1197,31 +1196,32 @@ CppConfig cpp_entry { key: "STY" value: 1000 } cpp_entry { key: "SU" value: 250 } cpp_entry { key: "SUT" value: 250 } - cpp_entry { key: "T03" value: 800 } + cpp_entry { key: "T01" value: 800 } cpp_entry { key: "T04" value: 800 } + cpp_entry { key: "T05" value: 800 } cpp_entry { key: "T06" value: 800 } cpp_entry { key: "T07" value: 800 } cpp_entry { key: "T08" value: 800 } - cpp_entry { key: "T09" value: 800 } + cpp_entry { key: "T11" value: 800 } cpp_entry { key: "T12" value: 800 } cpp_entry { key: "T13" value: 800 } - cpp_entry { key: "T14" value: 800 } cpp_entry { key: "T15" value: 800 } cpp_entry { key: "T16" value: 800 } - cpp_entry { key: "T19" value: 800 } + cpp_entry { key: "T17" value: 800 } + cpp_entry { key: "T18" value: 800 } cpp_entry { key: "T1U" value: 1000 } - cpp_entry { key: "T20" value: 800 } cpp_entry { key: "T21" value: 800 } cpp_entry { key: "T22" value: 800 } cpp_entry { key: "T23" value: 800 } - cpp_entry { key: "T26" value: 800 } - cpp_entry { key: "T27" value: 800 } + cpp_entry { key: "T24" value: 800 } + cpp_entry { key: "T25" value: 800 } cpp_entry { key: "T28" value: 800 } cpp_entry { key: "T29" value: 800 } cpp_entry { key: "T2B" value: 1000 } cpp_entry { key: "T2D" value: 1000 } cpp_entry { key: "T2M" value: 100 } cpp_entry { key: "T30" value: 800 } + cpp_entry { key: "T31" value: 800 } cpp_entry { key: "T3B" value: 1000 } cpp_entry { key: "T3L" value: 80 } cpp_entry { key: "T4B" value: 1000 } @@ -1276,7 +1276,7 @@ CppConfig cpp_entry { key: "TT" value: 50000 } cpp_entry { key: "TTD" value: 24 } cpp_entry { key: "TTE" value: 10000 } - cpp_entry { key: "TTF" value: 744 } + cpp_entry { key: "TTF" value: 745 } cpp_entry { key: "TUB" value: 1000 } cpp_entry { key: "TUF" value: 1000 } cpp_entry { key: "TUL" value: 1000 } @@ -1295,7 +1295,6 @@ CppConfig cpp_entry { key: "UDL" value: 80 } cpp_entry { key: "UFB" value: 100 } cpp_entry { key: "UFE" value: 100 } - cpp_entry { key: "UFG" value: 100 } cpp_entry { key: "UFN" value: 100 } cpp_entry { key: "UFU" value: 100 } cpp_entry { key: "UFV" value: 100 } @@ -1308,6 +1307,9 @@ CppConfig cpp_entry { key: "UN" value: 1000 } cpp_entry { key: "UNB" value: 1000 } cpp_entry { key: "UNS" value: 1000 } + cpp_entry { key: "UP5" value: 1000 } + cpp_entry { key: "UPB" value: 1000 } + cpp_entry { key: "UPM" value: 1000 } cpp_entry { key: "UR" value: 1000 } cpp_entry { key: "USC" value: 1000 } cpp_entry { key: "USDBRL" value: 1 } @@ -1462,7 +1464,6 @@ CmeSymbolMics { value { - cme_mic { key: "098" value: CBT } cme_mic { key: "0A" value: NYMEX } cme_mic { key: "0B" value: NYMEX } cme_mic { key: "0C" value: NYMEX } @@ -1809,6 +1810,7 @@ CmeSymbolMics cme_mic { key: "BOB" value: CBT } cme_mic { key: "BOO" value: NYMEX } cme_mic { key: "BOS" value: CME } + cme_mic { key: "BPU" value: NYMEX } cme_mic { key: "BR7" value: NYMEX } cme_mic { key: "BSF" value: CBT } cme_mic { key: "BTC" value: CME } @@ -1897,12 +1899,12 @@ CmeSymbolMics cme_mic { key: "D3L" value: NYMEX } cme_mic { key: "D4L" value: NYMEX } cme_mic { key: "D7L" value: NYMEX } + cme_mic { key: "DAB" value: NYMEX } cme_mic { key: "DAX" value: NYMEX } cme_mic { key: "DAZ" value: NYMEX } cme_mic { key: "DBB" value: NYMEX } cme_mic { key: "DBI" value: DME } cme_mic { key: "DBL" value: NYMEX } - cme_mic { key: "DBR" value: DME } cme_mic { key: "DBT" value: NYMEX } cme_mic { key: "DC" value: CME } cme_mic { key: "DCB" value: NYMEX } @@ -1920,7 +1922,6 @@ CmeSymbolMics cme_mic { key: "DFN" value: CBT } cme_mic { key: "DG2" value: CBT } cme_mic { key: "DG3" value: CBT } - cme_mic { key: "DGH" value: DME } cme_mic { key: "DGI" value: CBT } cme_mic { key: "DGS" value: CBT } cme_mic { key: "DGT" value: CBT } @@ -1928,19 +1929,13 @@ CmeSymbolMics cme_mic { key: "DHI" value: DME } cme_mic { key: "DJR" value: CBT } cme_mic { key: "DKR" value: DME } - cme_mic { key: "DMB" value: DME } - cme_mic { key: "DOB" value: DME } - cme_mic { key: "DOP" value: DME } - cme_mic { key: "DRB" value: DME } cme_mic { key: "DRI" value: CBT } cme_mic { key: "DRS" value: CBT } cme_mic { key: "DRT" value: CBT } cme_mic { key: "DSD" value: DME } cme_mic { key: "DSE" value: DME } - cme_mic { key: "DSG" value: DME } cme_mic { key: "DTR" value: CBT } cme_mic { key: "DTT" value: CBT } - cme_mic { key: "DUA" value: DME } cme_mic { key: "DUP" value: DME } cme_mic { key: "DVE" value: CME } cme_mic { key: "DVT" value: CME } @@ -1985,8 +1980,11 @@ CmeSymbolMics cme_mic { key: "EP1" value: NYMEX } cme_mic { key: "EPN" value: NYMEX } cme_mic { key: "EPZ" value: CME } + cme_mic { key: "EQ1" value: CME } cme_mic { key: "ERL" value: NYMEX } cme_mic { key: "ES" value: CME } + cme_mic { key: "ES1" value: CME } + cme_mic { key: "ES2" value: CME } cme_mic { key: "ESB" value: NYMEX } cme_mic { key: "ESK" value: CME } cme_mic { key: "ESQ" value: CME } @@ -2266,6 +2264,7 @@ CmeSymbolMics cme_mic { key: "LIY" value: CBT } cme_mic { key: "LL" value: NYMEX } cme_mic { key: "LLR" value: NYMEX } + cme_mic { key: "LNG" value: NYMEX } cme_mic { key: "LPE" value: NYMEX } cme_mic { key: "LPS" value: NYMEX } cme_mic { key: "LSF" value: COMEX } @@ -2619,10 +2618,12 @@ CmeSymbolMics cme_mic { key: "SFR" value: CME } cme_mic { key: "SG" value: CME } cme_mic { key: "SGB" value: NYMEX } + cme_mic { key: "SGC" value: COMEX } cme_mic { key: "SGE" value: NYMEX } cme_mic { key: "SGF" value: NYMEX } cme_mic { key: "SGG" value: NYMEX } cme_mic { key: "SGT" value: CME } + cme_mic { key: "SGU" value: COMEX } cme_mic { key: "SI" value: COMEX } cme_mic { key: "SIL" value: COMEX } cme_mic { key: "SIR" value: CME } @@ -2652,31 +2653,32 @@ CmeSymbolMics cme_mic { key: "STY" value: NYMEX } cme_mic { key: "SU" value: CME } cme_mic { key: "SUT" value: CME } - cme_mic { key: "T03" value: NYMEX } + cme_mic { key: "T01" value: NYMEX } cme_mic { key: "T04" value: NYMEX } + cme_mic { key: "T05" value: NYMEX } cme_mic { key: "T06" value: NYMEX } cme_mic { key: "T07" value: NYMEX } cme_mic { key: "T08" value: NYMEX } - cme_mic { key: "T09" value: NYMEX } + cme_mic { key: "T11" value: NYMEX } cme_mic { key: "T12" value: NYMEX } cme_mic { key: "T13" value: NYMEX } - cme_mic { key: "T14" value: NYMEX } cme_mic { key: "T15" value: NYMEX } cme_mic { key: "T16" value: NYMEX } - cme_mic { key: "T19" value: NYMEX } + cme_mic { key: "T17" value: NYMEX } + cme_mic { key: "T18" value: NYMEX } cme_mic { key: "T1U" value: CBT } - cme_mic { key: "T20" value: NYMEX } cme_mic { key: "T21" value: NYMEX } cme_mic { key: "T22" value: NYMEX } cme_mic { key: "T23" value: NYMEX } - cme_mic { key: "T26" value: NYMEX } - cme_mic { key: "T27" value: NYMEX } + cme_mic { key: "T24" value: NYMEX } + cme_mic { key: "T25" value: NYMEX } cme_mic { key: "T28" value: NYMEX } cme_mic { key: "T29" value: NYMEX } cme_mic { key: "T2B" value: NYMEX } cme_mic { key: "T2D" value: NYMEX } cme_mic { key: "T2M" value: NYMEX } cme_mic { key: "T30" value: NYMEX } + cme_mic { key: "T31" value: NYMEX } cme_mic { key: "T3B" value: NYMEX } cme_mic { key: "T3L" value: NYMEX } cme_mic { key: "T4B" value: NYMEX } @@ -2750,7 +2752,6 @@ CmeSymbolMics cme_mic { key: "UDL" value: NYMEX } cme_mic { key: "UFB" value: CBT } cme_mic { key: "UFE" value: CBT } - cme_mic { key: "UFG" value: CBT } cme_mic { key: "UFN" value: CBT } cme_mic { key: "UFU" value: CBT } cme_mic { key: "UFV" value: CBT } @@ -2763,6 +2764,9 @@ CmeSymbolMics cme_mic { key: "UN" value: NYMEX } cme_mic { key: "UNB" value: NYMEX } cme_mic { key: "UNS" value: NYMEX } + cme_mic { key: "UP5" value: NYMEX } + cme_mic { key: "UPB" value: NYMEX } + cme_mic { key: "UPM" value: NYMEX } cme_mic { key: "UR" value: NYMEX } cme_mic { key: "USC" value: NYMEX } cme_mic { key: "USDBRL" value: CME } diff --git a/src/lib/config/defaults/jetfix.conf b/src/lib/config/defaults/jetfix.conf index b4c0b4b51..4e4a4f558 100644 --- a/src/lib/config/defaults/jetfix.conf +++ b/src/lib/config/defaults/jetfix.conf @@ -5,7 +5,7 @@ JetFIXSimConnection { name: "sim" platform: SIM socket_connect_host_ip: "127.0.0.1" - socket_connect_port: @Ports.exchange_sim_fix_port + socket_connect_port: @Ports.glbx_sim_fix_port sender_compid: "SPARK" # this is the FIX session name sender_subid: "BTS" target_compid: "BTS" @@ -18,9 +18,506 @@ JetFIXSimConnection { seconds_to_pause : 1 max_connection_attempts: 9999999 } + sim_ice { + name: "sim_ice" + platform: SIMICE + socket_connect_host_ip: "127.0.0.1" + socket_connect_port: @Ports.ice_sim_fix_port + sender_compid: "51239" # this is the FIX session name + sender_subid: "BTS" + target_compid: "BTS" + target_subid: "BTS" + cme_market_segment_id: 0 + end_time: "02:00:00" + start_time: "02:05:00" + raw_data: "1JE9X" + use_cme_secure_logon: false + seconds_to_pause : 1 + } } } +JetFIX { + rule: "CannedCertSession.name='ice_cert'" + value { + #connection: @JetFIXSimConnection.sim_ice + connection { + name: "ice_cert" + platform: ICE + use_cme_secure_logon: false + log_fix_stream: true + socket_connect_host_ip: "63.247.113.154" + socket_connect_port: 80 + sender_compid: "51251" + target_compid: "ICE" + start_time: "22:55:00" + end_time: "22:05:00" + sender_subid: "BTS" + user_name: "bluetrd-os" #this is the order_server_id + raw_data: "Starts123" + max_order_msgs_per_window: 300 + auto_fix_seq_num: true + + seconds_to_replay: 6000 + } + connection: @JetFIXSimConnection.sim + #connection: @JetFIXSimConnection.sim_ice + + connection { + name: "cme_mc5_99" + platform: GLBX + socket_connect_host_ip: "69.50.112.199" + socket_connect_port: 61922 + sender_compid: "MC5004N" + target_compid: "CME" + target_subid: "99" + cme_market_segment_id: 99 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc5_58" + platform: GLBX + socket_connect_host_ip: "69.50.112.172" + socket_connect_port: 61922 + sender_compid: "MC5004N" + target_compid: "CME" + target_subid: "58" + cme_market_segment_id: 58 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc5_54" + platform: GLBX + socket_connect_host_ip: "69.50.112.191" + socket_connect_port: 61922 + sender_compid: "MC5004N" + target_compid: "CME" + target_subid: "54" + cme_market_segment_id: 54 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc5_82" + platform: GLBX + socket_connect_host_ip: "69.50.112.179" + socket_connect_port: 61922 + sender_compid: "MC5004N" + target_compid: "CME" + target_subid: "82" + cme_market_segment_id: 82 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc5_56" + platform: GLBX + socket_connect_host_ip: "69.50.112.174" + socket_connect_port: 61922 + sender_compid: "MC5004N" + target_compid: "CME" + target_subid: "56" + cme_market_segment_id: 56 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc5_50" + platform: GLBX + socket_connect_host_ip: "69.50.112.202" + socket_connect_port: 61922 + sender_compid: "MC5004N" + target_compid: "CME" + target_subid: "50" + cme_market_segment_id: 50 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc5_52" + platform: GLBX + socket_connect_host_ip: "69.50.112.181" + socket_connect_port: 61922 + sender_compid: "MC5004N" + target_compid: "CME" + target_subid: "52" + cme_market_segment_id: 52 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + } +} + +JetFIX { + rule: "CannedCertSession.name='gt_cert'" + value { + connection { + name: "ice_cert" + platform: ICE + use_cme_secure_logon: false + log_fix_stream: true + socket_connect_host_ip: "63.247.113.154" + socket_connect_port: 80 + sender_compid: "51251" + target_compid: "ICE" + start_time: "22:55:00" + end_time: "22:05:00" + sender_subid: "BTS" + user_name: "bluetrd-os" #this is the order_server_id + raw_data: "Starts123" + max_order_msgs_per_window: 300 + auto_fix_seq_num: true + + seconds_to_replay: 6000 + } + connection: @JetFIXSimConnection.sim + connection { + name: "cme_mc7_99" + platform: GLBX + socket_connect_host_ip: "69.50.112.199" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "99" + cme_market_segment_id: 99 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_58" + platform: GLBX + socket_connect_host_ip: "69.50.112.172" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "58" + cme_market_segment_id: 58 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_54" + platform: GLBX + socket_connect_host_ip: "69.50.112.191" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "54" + cme_market_segment_id: 54 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_82" + platform: GLBX + socket_connect_host_ip: "69.50.112.179" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "82" + cme_market_segment_id: 82 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_56" + platform: GLBX + socket_connect_host_ip: "69.50.112.174" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "56" + cme_market_segment_id: 56 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_50" + platform: GLBX + socket_connect_host_ip: "69.50.112.202" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "50" + cme_market_segment_id: 50 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_52" + platform: GLBX + socket_connect_host_ip: "69.50.112.181" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "52" + cme_market_segment_id: 52 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_60" + platform: GLBX + socket_connect_host_ip: "69.50.112.195" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "60" + cme_market_segment_id: 60 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_76" + platform: GLBX + socket_connect_host_ip: "69.50.112.204" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "76" + cme_market_segment_id: 76 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_88" + platform: GLBX + socket_connect_host_ip: "69.50.112.169" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "88" + cme_market_segment_id: 88 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_74" + platform: GLBX + socket_connect_host_ip: "69.50.112.197" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "74" + cme_market_segment_id: 74 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_64" + platform: GLBX + socket_connect_host_ip: "69.50.112.177" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "64" + cme_market_segment_id: 64 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_72" + platform: GLBX + socket_connect_host_ip: "69.50.112.189" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "72" + cme_market_segment_id: 72 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_70" + platform: GLBX + socket_connect_host_ip: "69.50.112.183" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "70" + cme_market_segment_id: 70 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_68" + platform: GLBX + socket_connect_host_ip: "69.50.112.168" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "68" + cme_market_segment_id: 68 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_80" + platform: GLBX + socket_connect_host_ip: "69.50.112.187" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "80" + cme_market_segment_id: 80 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_84" + platform: GLBX + socket_connect_host_ip: "69.50.112.198" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "84" + cme_market_segment_id: 84 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + connection { + name: "cme_mc7_78" + platform: GLBX + socket_connect_host_ip: "69.50.112.167" + socket_connect_port: 61923 + sender_compid: "MC7004N" + target_compid: "CME" + target_subid: "78" + cme_market_segment_id: 78 + start_time: "00:30:00" + end_time: "23:30:00" + start_day: "sunday" + end_day: "friday" + sender_subid: "BTS" + raw_data: "S4PMA" + auto_fix_seq_num: true + } + } +} + + JetFIX { rule: "CannedCertSession.name='mc7'" value { @@ -456,193 +953,193 @@ JetFIX { raw_data: "S4PMA" auto_fix_seq_num: true } - connection { - name: "cme_mc5_60" - platform: GLBX - socket_connect_host_ip: "69.50.112.195" - socket_connect_port: 61922 - sender_compid: "MC5004N" - target_compid: "CME" - target_subid: "60" - cme_market_segment_id: 60 - start_time: "00:30:00" - end_time: "23:30:00" - start_day: "sunday" - end_day: "friday" - sender_subid: "BTS" - raw_data: "S4PMA" - auto_fix_seq_num: true - } - connection { - name: "cme_mc5_76" - platform: GLBX - socket_connect_host_ip: "69.50.112.204" - socket_connect_port: 61922 - sender_compid: "MC5004N" - target_compid: "CME" - target_subid: "76" - cme_market_segment_id: 76 - start_time: "00:30:00" - end_time: "23:30:00" - start_day: "sunday" - end_day: "friday" - sender_subid: "BTS" - raw_data: "S4PMA" - auto_fix_seq_num: true - } - connection { - name: "cme_mc5_88" - platform: GLBX - socket_connect_host_ip: "69.50.112.169" - socket_connect_port: 61922 - sender_compid: "MC5004N" - target_compid: "CME" - target_subid: "88" - cme_market_segment_id: 88 - start_time: "00:30:00" - end_time: "23:30:00" - start_day: "sunday" - end_day: "friday" - sender_subid: "BTS" - raw_data: "S4PMA" - auto_fix_seq_num: true - } - connection { - name: "cme_mc5_74" - platform: GLBX - socket_connect_host_ip: "69.50.112.197" - socket_connect_port: 61922 - sender_compid: "MC5004N" - target_compid: "CME" - target_subid: "74" - cme_market_segment_id: 74 - start_time: "00:30:00" - end_time: "23:30:00" - start_day: "sunday" - end_day: "friday" - sender_subid: "BTS" - raw_data: "S4PMA" - auto_fix_seq_num: true - } - connection { - name: "cme_mc5_64" - platform: GLBX - socket_connect_host_ip: "69.50.112.177" - socket_connect_port: 61922 - sender_compid: "MC5004N" - target_compid: "CME" - target_subid: "64" - cme_market_segment_id: 64 - start_time: "00:30:00" - end_time: "23:30:00" - start_day: "sunday" - end_day: "friday" - sender_subid: "BTS" - raw_data: "S4PMA" - auto_fix_seq_num: true - } - connection { - name: "cme_mc5_72" - platform: GLBX - socket_connect_host_ip: "69.50.112.189" - socket_connect_port: 61922 - sender_compid: "MC5004N" - target_compid: "CME" - target_subid: "72" - cme_market_segment_id: 72 - start_time: "00:30:00" - end_time: "23:30:00" - start_day: "sunday" - end_day: "friday" - sender_subid: "BTS" - raw_data: "S4PMA" - auto_fix_seq_num: true - } - connection { - name: "cme_mc5_70" - platform: GLBX - socket_connect_host_ip: "69.50.112.183" - socket_connect_port: 61922 - sender_compid: "MC5004N" - target_compid: "CME" - target_subid: "70" - cme_market_segment_id: 70 - start_time: "00:30:00" - end_time: "23:30:00" - start_day: "sunday" - end_day: "friday" - sender_subid: "BTS" - raw_data: "S4PMA" - auto_fix_seq_num: true - } - connection { - name: "cme_mc5_68" - platform: GLBX - socket_connect_host_ip: "69.50.112.168" - socket_connect_port: 61922 - sender_compid: "MC5004N" - target_compid: "CME" - target_subid: "68" - cme_market_segment_id: 68 - start_time: "00:30:00" - end_time: "23:30:00" - start_day: "sunday" - end_day: "friday" - sender_subid: "BTS" - raw_data: "S4PMA" - auto_fix_seq_num: true - } - connection { - name: "cme_mc5_80" - platform: GLBX - socket_connect_host_ip: "69.50.112.187" - socket_connect_port: 61922 - sender_compid: "MC5004N" - target_compid: "CME" - target_subid: "80" - cme_market_segment_id: 80 - start_time: "00:30:00" - end_time: "23:30:00" - start_day: "sunday" - end_day: "friday" - sender_subid: "BTS" - raw_data: "S4PMA" - auto_fix_seq_num: true - } - connection { - name: "cme_mc5_84" - platform: GLBX - socket_connect_host_ip: "69.50.112.198" - socket_connect_port: 61922 - sender_compid: "MC5004N" - target_compid: "CME" - target_subid: "84" - cme_market_segment_id: 84 - start_time: "00:30:00" - end_time: "23:30:00" - start_day: "sunday" - end_day: "friday" - sender_subid: "BTS" - raw_data: "S4PMA" - auto_fix_seq_num: true - } - connection { - name: "cme_mc5_78" - platform: GLBX - socket_connect_host_ip: "69.50.112.167" - socket_connect_port: 61922 - sender_compid: "MC5004N" - target_compid: "CME" - target_subid: "78" - cme_market_segment_id: 78 - start_time: "00:30:00" - end_time: "23:30:00" - start_day: "sunday" - end_day: "friday" - sender_subid: "BTS" - raw_data: "S4PMA" - auto_fix_seq_num: true - } + # connection { + # name: "cme_mc5_60" + # platform: GLBX + # socket_connect_host_ip: "69.50.112.195" + # socket_connect_port: 61922 + # sender_compid: "MC5004N" + # target_compid: "CME" + # target_subid: "60" + # cme_market_segment_id: 60 + # start_time: "00:30:00" + # end_time: "23:30:00" + # start_day: "sunday" + # end_day: "friday" + # sender_subid: "BTS" + # raw_data: "S4PMA" + # auto_fix_seq_num: true + # } + # connection { + # name: "cme_mc5_76" + # platform: GLBX + # socket_connect_host_ip: "69.50.112.204" + # socket_connect_port: 61922 + # sender_compid: "MC5004N" + # target_compid: "CME" + # target_subid: "76" + # cme_market_segment_id: 76 + # start_time: "00:30:00" + # end_time: "23:30:00" + # start_day: "sunday" + # end_day: "friday" + # sender_subid: "BTS" + # raw_data: "S4PMA" + # auto_fix_seq_num: true + # } + # connection { + # name: "cme_mc5_88" + # platform: GLBX + # socket_connect_host_ip: "69.50.112.169" + # socket_connect_port: 61922 + # sender_compid: "MC5004N" + # target_compid: "CME" + # target_subid: "88" + # cme_market_segment_id: 88 + # start_time: "00:30:00" + # end_time: "23:30:00" + # start_day: "sunday" + # end_day: "friday" + # sender_subid: "BTS" + # raw_data: "S4PMA" + # auto_fix_seq_num: true + # } + # connection { + # name: "cme_mc5_74" + # platform: GLBX + # socket_connect_host_ip: "69.50.112.197" + # socket_connect_port: 61922 + # sender_compid: "MC5004N" + # target_compid: "CME" + # target_subid: "74" + # cme_market_segment_id: 74 + # start_time: "00:30:00" + # end_time: "23:30:00" + # start_day: "sunday" + # end_day: "friday" + # sender_subid: "BTS" + # raw_data: "S4PMA" + # auto_fix_seq_num: true + # } + # connection { + # name: "cme_mc5_64" + # platform: GLBX + # socket_connect_host_ip: "69.50.112.177" + # socket_connect_port: 61922 + # sender_compid: "MC5004N" + # target_compid: "CME" + # target_subid: "64" + # cme_market_segment_id: 64 + # start_time: "00:30:00" + # end_time: "23:30:00" + # start_day: "sunday" + # end_day: "friday" + # sender_subid: "BTS" + # raw_data: "S4PMA" + # auto_fix_seq_num: true + # } + # connection { + # name: "cme_mc5_72" + # platform: GLBX + # socket_connect_host_ip: "69.50.112.189" + # socket_connect_port: 61922 + # sender_compid: "MC5004N" + # target_compid: "CME" + # target_subid: "72" + # cme_market_segment_id: 72 + # start_time: "00:30:00" + # end_time: "23:30:00" + # start_day: "sunday" + # end_day: "friday" + # sender_subid: "BTS" + # raw_data: "S4PMA" + # auto_fix_seq_num: true + # } + # connection { + # name: "cme_mc5_70" + # platform: GLBX + # socket_connect_host_ip: "69.50.112.183" + # socket_connect_port: 61922 + # sender_compid: "MC5004N" + # target_compid: "CME" + # target_subid: "70" + # cme_market_segment_id: 70 + # start_time: "00:30:00" + # end_time: "23:30:00" + # start_day: "sunday" + # end_day: "friday" + # sender_subid: "BTS" + # raw_data: "S4PMA" + # auto_fix_seq_num: true + # } + # connection { + # name: "cme_mc5_68" + # platform: GLBX + # socket_connect_host_ip: "69.50.112.168" + # socket_connect_port: 61922 + # sender_compid: "MC5004N" + # target_compid: "CME" + # target_subid: "68" + # cme_market_segment_id: 68 + # start_time: "00:30:00" + # end_time: "23:30:00" + # start_day: "sunday" + # end_day: "friday" + # sender_subid: "BTS" + # raw_data: "S4PMA" + # auto_fix_seq_num: true + # } + # connection { + # name: "cme_mc5_80" + # platform: GLBX + # socket_connect_host_ip: "69.50.112.187" + # socket_connect_port: 61922 + # sender_compid: "MC5004N" + # target_compid: "CME" + # target_subid: "80" + # cme_market_segment_id: 80 + # start_time: "00:30:00" + # end_time: "23:30:00" + # start_day: "sunday" + # end_day: "friday" + # sender_subid: "BTS" + # raw_data: "S4PMA" + # auto_fix_seq_num: true + # } + # connection { + # name: "cme_mc5_84" + # platform: GLBX + # socket_connect_host_ip: "69.50.112.198" + # socket_connect_port: 61922 + # sender_compid: "MC5004N" + # target_compid: "CME" + # target_subid: "84" + # cme_market_segment_id: 84 + # start_time: "00:30:00" + # end_time: "23:30:00" + # start_day: "sunday" + # end_day: "friday" + # sender_subid: "BTS" + # raw_data: "S4PMA" + # auto_fix_seq_num: true + # } + # connection { + # name: "cme_mc5_78" + # platform: GLBX + # socket_connect_host_ip: "69.50.112.167" + # socket_connect_port: 61922 + # sender_compid: "MC5004N" + # target_compid: "CME" + # target_subid: "78" + # cme_market_segment_id: 78 + # start_time: "00:30:00" + # end_time: "23:30:00" + # start_day: "sunday" + # end_day: "friday" + # sender_subid: "BTS" + # raw_data: "S4PMA" + # auto_fix_seq_num: true + # } } } JetFIX { diff --git a/src/lib/config/defaults/keycode.conf b/src/lib/config/defaults/keycode.conf index 49832fb86..5b41d0705 100644 --- a/src/lib/config/defaults/keycode.conf +++ b/src/lib/config/defaults/keycode.conf @@ -1,9 +1,13 @@ + + #nusers: 1 -#allow concurrent : false -#expires: 2019-Sep-30 00:00:00 637053984000000000 -#generation_time: 2019-Jun-18 18:25:44.333280 5248650809870720704 +#allow concurrent : true +#expires: 2019-Dec-31 00:00:00 637133472000000000 +#generation_time: 2019-Sep-30 15:52:10.434845 5248740573731736354 EncryptedKeyCode { value { - keycode: "pgtv/OmievnpmNlgxTYuN801Hbetgu7ZGkHnlGWtK+DBrKSEgKxr5lGvCW1ld/jsztkWhKHDh0iediMDjrPm4+C5qmXlt4JJlAFvKolMN8HVC59m82fECvakcqKq28WSEjf6/DuPJNP1ZwxjcmKouxx1LZ15VlNkECtNrC9pkCnifazbfQZGV3kD08zMEyw/RPmOaVLFFeav8lwI5UpcjPYBIK1rJowXdGSvm1CJViwG5Z2JbdHR39QbKjVx112d5VbVtBq9nO3i0FEKvobjN/Gzcz/5x4TgQtEShKsbmgEN+MBVL3AItQlcFzZvE/05JLmfACUFmk5Eu0a3DlLs3A==" + keycode: "fVZlub92ulkXME+uZSyO9+WmyAXYKD735Qzxxfdj4G+SqEDTuwfNxOZVPS3TCFQqWPDmgjKTbbvfqIrGe3icxkVObUcUoS7lBXRXFrSvzOfs9eTz9RqHRvquDidnOzKxmSXsxV2bMb21UqBCJFeTzb4Q6Agp8EtjXJRc57UMVVygvdXg528S3ocBIq1V4W5NYraFn5WpqrTuBhpYbnsZAHi77s/EEwUT+6la3J2OHPtbiQnB6JFb0zsExMuzE4gwp79ojuQ5pnV2DXBZFNwxYvFyBUU/ac6oU89saZ/ay1DbOjz76TNqnNFewZvzdXZl3y6+X1eblX2streH2pn7gg==" } -} \ No newline at end of file +} + + diff --git a/src/lib/config/defaults/orders_fills_cleanup.conf b/src/lib/config/defaults/orders_fills_cleanup.conf index 73597ff53..b26e96b6c 100644 --- a/src/lib/config/defaults/orders_fills_cleanup.conf +++ b/src/lib/config/defaults/orders_fills_cleanup.conf @@ -2,4 +2,14 @@ OrdersSoftCleanupCutoff { value { extra_days: 1 } +} + +SODTime +{ + rule: "DeploymentType.type == 'dev'" + value + { + sod_hh: 20 + sod_mm: 30 + } } \ No newline at end of file diff --git a/src/lib/config/protos/MarketData.proto b/src/lib/config/protos/MarketData.proto index 18c08950a..b20424632 100755 --- a/src/lib/config/protos/MarketData.proto +++ b/src/lib/config/protos/MarketData.proto @@ -155,6 +155,7 @@ message ICEOptions message ICEChannel { required string name = 1; repeated uint32 market_type = 2; + repeated uint32 uds_market_type = 7; optional uint32 udp_buffer_size = 3 [default = 4194304]; //4mb optional uint32 heartbeat_interval = 4 [default = 36000]; optional uint32 sequence_gap_margin = 5 [default = 10000]; diff --git a/src/lib/config/protos/Ports.proto b/src/lib/config/protos/Ports.proto index 11ce66eae..2176315c1 100644 --- a/src/lib/config/protos/Ports.proto +++ b/src/lib/config/protos/Ports.proto @@ -5,6 +5,7 @@ message Ports { // see ports.conf for actual defaults optional int32 config_port = 1 ; optional int32 qbus_port = 2 ; - optional int32 exchange_sim_fix_port = 3 ; + optional int32 glbx_sim_fix_port = 3 ; + optional int32 ice_sim_fix_port = 5 ; optional int32 replicator_port = 4 ; } diff --git a/src/lib/config/protos/Symbols.proto b/src/lib/config/protos/Symbols.proto index 6d859c13f..0d71ab37f 100644 --- a/src/lib/config/protos/Symbols.proto +++ b/src/lib/config/protos/Symbols.proto @@ -20,4 +20,5 @@ message Symbols { optional SecurityExchangeList glbx = 10; optional SecurityExchangeList sim = 20; optional SecurityExchangeList ice = 30; + optional SecurityExchangeList simice = 40; } \ No newline at end of file diff --git a/src/lib/config/protos/jetfix.proto b/src/lib/config/protos/jetfix.proto index c3666f5ef..6bc986d94 100644 --- a/src/lib/config/protos/jetfix.proto +++ b/src/lib/config/protos/jetfix.proto @@ -7,9 +7,11 @@ message JetFIX { required string name = 10; enum JetFIXPlatform { - GLBX = 1; - CBOE = 6; - SIM = 2; + GLBX = 1; + CBOE = 6; + SIM = 2; + ICE = 3; + SIMICE = 4; } optional JetFIXPlatform platform = 15 [default = GLBX]; @@ -47,6 +49,7 @@ message JetFIX { optional string target_subid = 130 [default = ""]; optional string sender_locationid = 135 [default = "US,NC"]; optional string raw_data = 136 [default =""]; + optional string user_name = 138 [default =""]; optional string self_match_prevention_id = 140 [default=""]; optional string cti_code = 145 [default="1"]; optional int32 customer_or_firm = 147 [default=1]; @@ -91,6 +94,7 @@ message JetFIX { message JetFIXSimConnection { required JetFIX.Connection sim = 10; + required JetFIX.Connection sim_ice = 20; } message CannedCertSession { diff --git a/src/lib/config/protos/orderman_cfg.proto b/src/lib/config/protos/orderman_cfg.proto index 5d899ea95..aee3e0802 100644 --- a/src/lib/config/protos/orderman_cfg.proto +++ b/src/lib/config/protos/orderman_cfg.proto @@ -9,4 +9,5 @@ message OrdermanCfg { } repeated Executor executor = 1; optional uint32 default_iceberg_pause_secs = 2 [default = 0]; + optional string fallback_cust_order_handling_inst = 3 [default = "Y"]; } diff --git a/src/lib/config/protos/toggles.proto b/src/lib/config/protos/toggles.proto index f1733290f..787cbf7e0 100644 --- a/src/lib/config/protos/toggles.proto +++ b/src/lib/config/protos/toggles.proto @@ -58,3 +58,13 @@ message EnsureOnScreenWithWorkAreaRect { // Should verify that the code works with every monitor arrangement before enabling by default. optional bool enabled = 1 [default = false]; } + +message IgnoreNansInDayPnL { + optional bool enabled = 1 [default = true]; +} + +message DynamicAlgos{ + optional bool enable_dorman = 1 [default = false]; + optional bool enable_pyramid = 2 [default = false]; + optional bool enable_tas = 3 [default = false]; +} \ No newline at end of file diff --git a/src/lib/domain_model/Security.cpp b/src/lib/domain_model/Security.cpp index 94df17488..40b2a2534 100644 --- a/src/lib/domain_model/Security.cpp +++ b/src/lib/domain_model/Security.cpp @@ -29,9 +29,10 @@ std::string descriptor(api::Security const &sec) { else if (sec.has_ice()) return sec.ice().security_desc(); else - return "No descriptor for non glbx exchange"; + return "No descriptor for non glbx / ice exchange"; } + boost::gregorian::date smallest_leg( ::google::protobuf::RepeatedPtrField<::bts::api::ContractLeg> const &l) { bts_assert(l.size()); diff --git a/src/lib/domain_model/Security.h b/src/lib/domain_model/Security.h index 722e6670b..80444d860 100644 --- a/src/lib/domain_model/Security.h +++ b/src/lib/domain_model/Security.h @@ -191,5 +191,6 @@ inline bool gr_than(double p1, double p2, double tick_size) { return gr_than_eq(p1, p2, tick_size) && !test_price_equality(p1, p2, tick_size); } -} // namespace securities -} // namespace bts + + +}} diff --git a/src/lib/domain_model/Security.proto b/src/lib/domain_model/Security.proto index 52ca35299..d0ee67220 100644 --- a/src/lib/domain_model/Security.proto +++ b/src/lib/domain_model/Security.proto @@ -12,6 +12,7 @@ enum Venue { VENUE_CBOECFE = 207; VENUE_GLBX = 301; VENUE_ICE = 306; + VENUE_SIMICE = 3; } enum SecurityType { @@ -54,11 +55,12 @@ enum CMEMics { } enum ICEMics{ - ICE_UNSET=0; - ICE_US = 1; - ICE_EU = 2; - ICE_LIFFE = 3; - ICE_CA = 4; + IF_UNSET = 0; + IFU = 1; + IFE = 2; + IFL = 3; + IFC = 4; + IFEN = 5; } enum Side { @@ -103,8 +105,9 @@ message CBOECFEExchangeData { message ICEExchangeData { int64 exchange_id = 10; string security_desc = 15; //contractSymbol - int32 deal_display_factor = 20; - int32 settle_display_factor = 30; + int32 price_denominator = 17; + int32 deal_denominator = 20; + int32 settle_denominator = 30; string market_description = 100; string cleared_alias = 110; diff --git a/src/lib/jetfix/codec/FIXEncodingStream.h b/src/lib/jetfix/codec/FIXEncodingStream.h index 541e426b6..502c3786c 100644 --- a/src/lib/jetfix/codec/FIXEncodingStream.h +++ b/src/lib/jetfix/codec/FIXEncodingStream.h @@ -1,11 +1,11 @@ #pragma once -#include #include "lib/jetfix/codec/FIXBuffer.h" -#include "lib/jetfix/codec/MessageTypes.h" #include "lib/jetfix/codec/FIXFragment.h" -#include "lib/jetfix/codec/Fields.h" #include "lib/jetfix/codec/FieldDefinitions.h" +#include "lib/jetfix/codec/Fields.h" +#include "lib/jetfix/codec/MessageTypes.h" +#include #include "lib/config/protos/jetfix.pb.h" @@ -14,185 +14,186 @@ namespace jetfix { // Helper to batch the encoding of multiple FIX messages all sharing // the same FIX headers. -class FIXEncodingStream -{ -public: +class FIXEncodingStream { + public: FIXEncodingStream(config::JetFIX::Connection const &cfg, - uint32_t initial_capacity = 1024*10) - : - _msg_header_template (), - _header_template_size (), - _header_template_checksum (), - _buffer (initial_capacity), - _message_map (), - _current_msg (_buffer), - _state (State::OPEN), - _body_length_bm (), - _msg_type_bm (), - _seq_num_bm (), - _last_seq_num_bm (), - _sending_time_bm () - { - _message_map.reserve (10*1024); + uint32_t initial_capacity = 1024 * 10) + : _msg_header_template(), _header_template_size(), + _header_template_checksum(), _buffer(initial_capacity), _message_map(), + _current_msg(_buffer), _state(State::OPEN), _seq_num_bm(), + _last_seq_num_bm(), _sending_time_bm() { + _message_map.reserve(10 * 1024); - FIXBuffer header_buffer (&_msg_header_template[0],_msg_header_template.size ()); - FIXFragment header (header_buffer); - header.add("FIX.4.2"); + FIXBuffer header_buffer(&_msg_header_template[0], + _msg_header_template.size()); + FIXFragment header(header_buffer); - _body_length_bm = header.add(0,6); - _msg_type_bm = header.add(" ",1); - _seq_num_bm = header.add(0,9); - _last_seq_num_bm = header.add(0,9); + // We cannot include body lenght and msg type in the header + // because message type is not fixed lenght, and therefore its + // not possible to have bookmarks to fields after it, ahead of + // knowing the message type value. The rest of the header is + // fixed lenght so we form it here. + _seq_num_bm = header.add(0, 9); + _last_seq_num_bm = header.add(0, 9); _sending_time_bm = header.add( - "20150427-19:05:18.923",sizeof("20150427-19:05:18.923")-1); + "20150427-19:05:18.923", sizeof("20150427-19:05:18.923") - 1); - header.add(cfg.sender_compid ()); - header.add(cfg.target_compid ()); - header.add(cfg.target_subid ()); - _header_template_size = header.size (); - _header_template_checksum = header.checksum (); + header.add(cfg.sender_compid()); + header.add(cfg.target_compid()); + header.add(cfg.target_subid()); + _header_template_size = header.size(); + _header_template_checksum = header.checksum(); } - inline FIXFragment &new_msg (MessageTypes type) - { - bts_assert (_state == State::OPEN); - tidy_last_msg (); - - auto msg_offset = _buffer.size (); - _message_map.emplace_back (msg_offset,0u,0u,0u,FIXFragment::Bookmark>()); - _current_msg = FIXFragment (_buffer); - _current_msg.append (&_msg_header_template[0], - _header_template_size, - _header_template_checksum); - char msg_type_str[1] = {(char) type}; - _current_msg.overwrite (_msg_type_bm.shift (msg_offset),msg_type_str,1); + inline FIXFragment &new_msg(MessageTypes type) { + bts_assert(_state == State::OPEN); + tidy_last_msg(); + + auto msg_offset = _buffer.size(); + + _current_msg = FIXFragment(_buffer); + _begin_string_bm = _current_msg.add("FIX.4.2"); + _body_length_bm = _current_msg.add(0, 6); + + char mt_encoding[4]; + auto start = encode_message_type(type, mt_encoding); + _msg_type_bm = _current_msg.add( + mt_encoding + start, sizeof(mt_encoding) - start); + + auto header_offset = _buffer.size(); + _message_map.emplace_back(msg_offset, header_offset, 0u, 0u, 0u, + FIXFragment::Bookmark>()); + _current_msg.append(&_msg_header_template[0], _header_template_size, + _header_template_checksum); return _current_msg; } - - inline void close () - { - bts_assert (_state == State::OPEN); - tidy_last_msg (); + + inline void close() { + bts_assert(_state == State::OPEN); + tidy_last_msg(); _state = State::CLOSED; } - inline void reopen_empty () - { - bts_assert (_state == State::CLOSED); - bts_assert (_message_map.empty ()); + inline void reopen_empty() { + bts_assert(_state == State::CLOSED); + bts_assert(_message_map.empty()); _state = State::OPEN; } - - inline uint32_t size () const - {return _buffer.size ();} - inline const char* data () const - {return _buffer.data ();} + inline uint32_t size() const { return _buffer.size(); } - struct MessageDesc - { - MessageDesc (uint32_t offset, - uint32_t size, - uint32_t checksum, - uint32_t seq_num, - FIXFragment::Bookmark checksum_bm) - : offset (offset), - size (size), - checksum (checksum), - seq_num (seq_num), - checksum_bm (checksum_bm) - {} - + inline const char *data() const { return _buffer.data(); } + + struct MessageDesc { + MessageDesc(uint32_t offset, uint32_t header_offset, uint32_t size, + uint32_t checksum, uint32_t seq_num, + FIXFragment::Bookmark checksum_bm) + : offset(offset), header_offset(header_offset), size(size), + checksum(checksum), seq_num(seq_num), checksum_bm(checksum_bm) {} uint32_t offset; + uint32_t header_offset; uint32_t size; uint32_t checksum; uint32_t seq_num; FIXFragment::Bookmark checksum_bm; }; - - std::string to_str () const - { + + std::string to_str() const { std::string result; - - if (size ()) { + + if (size()) { // return std::string(data(),size() is simpler but valgrind // reports a warning in that call - result.resize (size ()); - std::copy (data (), data ()+size(),result.begin ()); + result.resize(size()); + std::copy(data(), data() + size(), result.begin()); } return result; } - std::string to_human_str () const - { - auto s = to_str (); - std::replace( s.begin(), s.end(), '\001', '|' ); + std::string to_human_str() const { + auto s = to_str(); + std::replace(s.begin(), s.end(), '\001', '|'); return s; } - - inline uint32_t stamp (uint32_t seq_num,uint32_t last_processed, std::string const&time_str) - { - bts_assert (_state == State::CLOSED); + + inline uint32_t stamp(uint32_t seq_num, uint32_t last_processed, + std::string const &time_str) { + bts_assert(_state == State::CLOSED); _state = State::STAMPED; for (auto &m : _message_map) { ++seq_num; m.seq_num = seq_num; - FIXFragment frag (_buffer,m.checksum); - frag.overwrite (_seq_num_bm.shift (m.offset),m.seq_num,9); - frag.overwrite (_last_seq_num_bm.shift (m.offset),last_processed,9); - frag.overwrite (_sending_time_bm.shift (m.offset),time_str); - frag.overwrite_checksum (m.checksum_bm,frag.checksum () % 256); + FIXFragment frag(_buffer, m.checksum); + frag.overwrite(_seq_num_bm.shift(m.header_offset), m.seq_num, 9); + frag.overwrite(_last_seq_num_bm.shift(m.header_offset), last_processed, + 9); + frag.overwrite(_sending_time_bm.shift(m.header_offset), time_str); + frag.overwrite_checksum(m.checksum_bm, frag.checksum() % 256); } return seq_num; } - inline void reset () - { - _buffer.reset (); - _message_map.clear (); + inline void reset() { + _buffer.reset(); + _message_map.clear(); _state = State::OPEN; } - inline bool is_closed () const - {return _state == State::CLOSED;} + inline bool is_closed() const { return _state == State::CLOSED; } - size_t n_msgs () const {return _message_map.size ();} - - std::vector const &msg_map () const {return _message_map;} -private: - std::array _msg_header_template; + size_t n_msgs() const { return _message_map.size(); } + + std::vector const &msg_map() const { return _message_map; } + + private: + std::array _msg_header_template; uint32_t _header_template_size; uint32_t _header_template_checksum; FIXBuffer _buffer; std::vector _message_map; FIXFragment _current_msg; - enum class State { OPEN,CLOSED,STAMPED}; + enum class State { OPEN, CLOSED, STAMPED }; State _state; - + + // These bookmarks point to the current msg (not the header) + FIXFragment::Bookmark _begin_string_bm; FIXFragment::Bookmark _body_length_bm; FIXFragment::Bookmark _msg_type_bm; + + // These bookmarks point to the header and thus have to be shifted + // by the offset of the end of the message type. FIXFragment::Bookmark _seq_num_bm; FIXFragment::Bookmark _last_seq_num_bm; typedef StringField SendingTimeStr; FIXFragment::Bookmark _sending_time_bm; - inline void tidy_last_msg () - { - if (_message_map.size ()) { - auto &this_msg_map = _message_map.back (); - auto start = - this_msg_map.offset + _msg_type_bm.offset - fix::MsgType::tag_sz-1; - auto end = _buffer.size (); - _current_msg.overwrite (_body_length_bm.shift (this_msg_map.offset), - end-start,6); - this_msg_map.checksum = _current_msg.checksum (); - this_msg_map.checksum_bm = _current_msg.add_checksum (0); - this_msg_map.size = _buffer.size () - this_msg_map.offset; + inline void tidy_last_msg() { + if (_message_map.size()) { + auto &this_msg_map = _message_map.back(); + auto start = _msg_type_bm.offset - fix::MsgType::tag_sz - 1; + auto end = _buffer.size(); + _current_msg.overwrite(_body_length_bm, end - start, 6); + this_msg_map.checksum = _current_msg.checksum(); + this_msg_map.checksum_bm = _current_msg.add_checksum(0); + this_msg_map.size = _buffer.size() - this_msg_map.offset; } } + + uint32_t encode_message_type(MessageTypes m, char out[4]) { + auto mt = (uint32_t)m; + for (int i = 0; i < 4; i++) { + out[i] = mt >> (8 * (3 - i)); + mt &= ((1 << (8 * (3 - i))) - 1); + } + for (int i = 0; i < 4; i++) + if (out[i]) + return i; + return 0; + } }; -}} +} // namespace jetfix +} // namespace bts diff --git a/src/lib/jetfix/codec/FieldDefinitions.h b/src/lib/jetfix/codec/FieldDefinitions.h index f1bacd704..5ba96346a 100644 --- a/src/lib/jetfix/codec/FieldDefinitions.h +++ b/src/lib/jetfix/codec/FieldDefinitions.h @@ -9,6 +9,7 @@ namespace jetfix { namespace fix { typedef BoolField<1028> ManualOrderIndicator; +typedef StringField<1031> CustOrderHandlingInst; typedef BoolField<123> GapFillFlag; typedef BoolField<43> PossDupFlag; typedef BoolField<9768> InFlightMitigation; @@ -69,6 +70,9 @@ typedef StringField<143> TargetLocationID; typedef StringField<1603> ApplicationSystemName; typedef StringField<1604> ApplicationSystemVersion; typedef StringField<1605> ApplicationSystemVendor; +typedef StringField<9450> ICEApplicationSystemName; +typedef StringField<9451> ICEApplicationSystemVersion; +typedef StringField<9452> ICEApplicationSystemVendor; typedef StringField<167> SecurityType; typedef StringField<17> ExecID; typedef StringField<19> ExecRefID; @@ -97,6 +101,7 @@ typedef StringField<7928> SelfMatchPreventionID; typedef StringField<8> BeginString; typedef StringField<9324> OrderOptionalData; typedef StringField<96> RawData; +typedef StringField<553> UserName; typedef StringField<9717> CorrelationClOrdID; typedef StringField<9771> MMAccount; typedef TimeField<122> OrigSendingTime; @@ -104,6 +109,38 @@ typedef TimeField<52> SendingTime; typedef TimeField<5979> RequestTime; typedef TimeField<60> TransactTime; +//ICE +typedef IntField<9208> Int_CtiCode; +typedef BigIntField<11> Int_ClOrdID; +typedef BigIntField<37> Int_OrderID; +typedef BigIntField<41> Int_OrigClOrdID; +typedef IntField<109> CompanyID; +typedef IntField<55> Int_Symbol; +typedef IntField<439> Int_ClearingFirm; +typedef StringField<9121> Memo; +typedef IntField<103> OrdRejReason; +typedef IntField<285> DeleteReason; +typedef CharField<9195> AccountCode; +typedef StringField<9207> CustomerAccountRefId; +typedef StringField<115> OnBehalfOfCompID; +typedef StringField<116> OnBehalfOfSubID; +typedef StringField<144> OnBehalfOfLocationID; +typedef StringField<9139> OriginatorUserID; +typedef CharField<18> ExecInst; +typedef IntField<210> MaxShow; +typedef IntField<9006> ICEStrategyPreference; +typedef IntField<9007> ICETimeStampPreference; +typedef StringField<1031> CustOrderHandlingInst; + +typedef IntField<9700> MifidDirectElectronicAccess; +typedef IntField<9701> MifidTradingCapacity; +typedef IntField<9702> MifidLiquidityProvision; +typedef IntField<9703> MifidCommodityDerivIndicator; +typedef BigIntField<9704> MifidInvestmentDecision; +typedef BigIntField<9705> MifidExecutionDecision; +typedef BigIntField<9706> MifidClientIDCode; +typedef BigIntField<9707> MifidId; + typedef Group +class BigIntField : public BaseField +{ +public: + typedef BaseField Parent; + + void read (ReaderCursor &cursor) + { + auto loc = cursor.read_field (); + Parent::_value = atoll (&(cursor.data ()[loc.first])); + } + + static uint32_t max_value_size (int64_t,int pad = 0) + { + if (pad == 0) + return 20; + else + return pad; + } + + static std::pair write (char*buffer,int64_t i,int pad = 0) + { + auto written = i64_toa (i,buffer,pad); + return std::make_pair (written,sum_bytes (buffer,written)); + } +}; + template class StringField : public BaseField { diff --git a/src/lib/jetfix/codec/MessageDefinitions.h b/src/lib/jetfix/codec/MessageDefinitions.h index 197f5580f..75f45f3e6 100644 --- a/src/lib/jetfix/codec/MessageDefinitions.h +++ b/src/lib/jetfix/codec/MessageDefinitions.h @@ -45,7 +45,8 @@ typedef FIXMsg + TransactTimeStr, + CustOrderHandlingInst > ExecutionReportMsg; typedef FIXMsg @@ -66,5 +67,58 @@ typedef FIXMsg CancelRejectMsg; +typedef FIXMsg +ICETraderLoginResponseMsg; + +typedef FIXMsg +ICETraderLogoutResponseMsg; + +typedef FIXMsg +ICEExecutionReportMsg; + +typedef FIXMsg +ICECancelRejectMsg; + }}} diff --git a/src/lib/jetfix/codec/MessageTypes.h b/src/lib/jetfix/codec/MessageTypes.h index 39dfd7b34..a7154e8a9 100644 --- a/src/lib/jetfix/codec/MessageTypes.h +++ b/src/lib/jetfix/codec/MessageTypes.h @@ -5,7 +5,7 @@ namespace bts { namespace jetfix { -enum class MessageTypes +enum class MessageTypes : uint32_t { Advertisement = '7', Allocation = 'J', @@ -48,7 +48,12 @@ enum class MessageTypes SettlementInstructions = 'T', TestRequest = '1', TradingSessionStatus = 'h', - TradingSessionStatusRequest = 'g' + TradingSessionStatusRequest = 'g', + ICETraderLogin = ('U' << 16) + ('C' << 8) + 'G', + ICETraderLoginResponse = ('U' << 16) + ('C' << 8) + 'I', + ICETraderLogout = ('U' << 16) + ('C' << 8) + 'H', + ICETraderLogoutResponse = ('U' << 16) + ('C' << 8) + 'J', + }; }} diff --git a/src/lib/jetfix/codec/i32_toa.h b/src/lib/jetfix/codec/i32_toa.h index be546f39b..afc257ebc 100644 --- a/src/lib/jetfix/codec/i32_toa.h +++ b/src/lib/jetfix/codec/i32_toa.h @@ -259,3 +259,199 @@ inline int u32_toa(uint32_t u, char* buffer, int pad) { } +///https://github.com/miloyip/itoa-benchmark + +#define BEGIN2(n) \ + do { \ + int t = (n); \ + if(t < 10) *p++ = '0' + t; \ + else { \ + t *= 2; \ + *p++ = gDigitsLut[t]; \ + *p++ = gDigitsLut[t + 1]; \ + } \ + } while(0) +#define MIDDLE2(n) \ + do { \ + int t = (n) * 2; \ + *p++ = gDigitsLut[t]; \ + *p++ = gDigitsLut[t + 1]; \ + } while(0) +#define BEGIN4(n) \ + do { \ + int t4 = (n); \ + if(t4 < 100) BEGIN2(t4); \ + else { BEGIN2(t4 / 100); MIDDLE2(t4 % 100); } \ + } while(0) +#define MIDDLE4(n) \ + do { \ + int t4 = (n); \ + MIDDLE2(t4 / 100); MIDDLE2(t4 % 100); \ + } while(0) +#define BEGIN8(n) \ + do { \ + uint32_t t8 = (n); \ + if(t8 < 10000) BEGIN4(t8); \ + else { BEGIN4(t8 / 10000); MIDDLE4(t8 % 10000); } \ + } while(0) +#define MIDDLE8(n) \ + do { \ + uint32_t t8 = (n); \ + MIDDLE4(t8 / 10000); MIDDLE4(t8 % 10000); \ + } while(0) +#define MIDDLE16(n) \ + do { \ + uint64_t t16 = (n); \ + MIDDLE8(t16 / 100000000); MIDDLE8(t16 % 100000000); \ + } while(0) + +inline int u64_toa(uint64_t value, char* p) +{ + auto start = p; + if(value < 100000000) BEGIN8(value); + else if(value < 10000000000000000) { BEGIN8(value / 100000000); MIDDLE8(value % 100000000); } + else { BEGIN4(value / 10000000000000000); MIDDLE16(value % 10000000000000000); } + *p = 0; + return p-start; +} + +inline int count_digits (int64_t value) +{ + static auto P01 = 10ull; + static auto P02 = P01*10; + static auto P03 = P02*10; + static auto P04 = P03*10; + static auto P05 = P04*10; + static auto P06 = P05*10; + static auto P07 = P06*10; + static auto P08 = P07*10; + static auto P09 = P08*10; + static auto P10 = P09*10; + static auto P11 = P10*10; + static auto P12 = P11*10; + static auto P13 = P12*10; + static auto P14 = P13*10; + static auto P15 = P14*10; + static auto P16 = P15*10; + static auto P17 = P16*10; + static auto P18 = P17*10; + static auto P19 = P18*10; + + uint64_t u = static_cast(value); + if (value < 0) + u = ~u + 1; + + if (u < P01) return 1; + if (u < P02) return 2; + if (u < P03) return 3; + if (u < P04) return 4; + if (u < P05) return 5; + if (u < P06) return 6; + if (u < P07) return 7; + if (u < P08) return 8; + if (u < P09) return 9; + if (u < P10) return 10; + if (u < P11) return 11; + if (u < P12) return 12; + if (u < P13) return 13; + if (u < P14) return 14; + if (u < P15) return 15; + if (u < P16) return 16; + if (u < P17) return 17; + if (u < P18) return 18; + if (u < P19) return 19; + return 20; +} + +inline int count_digits (uint64_t u) +{ + static auto P01 = 10ull; + static auto P02 = P01*10; + static auto P03 = P02*10; + static auto P04 = P03*10; + static auto P05 = P04*10; + static auto P06 = P05*10; + static auto P07 = P06*10; + static auto P08 = P07*10; + static auto P09 = P08*10; + static auto P10 = P09*10; + static auto P11 = P10*10; + static auto P12 = P11*10; + static auto P13 = P12*10; + static auto P14 = P13*10; + static auto P15 = P14*10; + static auto P16 = P15*10; + static auto P17 = P16*10; + static auto P18 = P17*10; + static auto P19 = P18*10; + + if (u < P01) return 1; + if (u < P02) return 2; + if (u < P03) return 3; + if (u < P04) return 4; + if (u < P05) return 5; + if (u < P06) return 6; + if (u < P07) return 7; + if (u < P08) return 8; + if (u < P09) return 9; + if (u < P10) return 10; + if (u < P11) return 11; + if (u < P12) return 12; + if (u < P13) return 13; + if (u < P14) return 14; + if (u < P15) return 15; + if (u < P16) return 16; + if (u < P17) return 17; + if (u < P18) return 18; + if (u < P19) return 19; + return 20; +} + +inline int i64_toa(int64_t value, char* buffer, int pad) +{ + auto start = buffer; + uint64_t u = static_cast(value); + if (value < 0) { + *buffer++ = '-'; + u = ~u + 1; + } + if (pad) + { + auto digits = count_digits (value); + auto offset = pad - digits; + if (offset > 0) + { + auto start = buffer + offset; + while (buffer != start) + { + *buffer='0'; + buffer++; + } + } + else if (offset < 0) + return 0; + } + return (buffer-start) + u64_toa(u, buffer); +} + +inline int u64_toa(uint64_t u, char* buffer, int pad) +{ + auto start = buffer; + if (pad) + { + auto digits = count_digits (u); + auto offset = pad - digits; + if (offset > 0) + { + auto start = buffer + offset; + while (buffer != start) + { + *buffer='0'; + buffer++; + } + } + else if (offset < 0) + return 0; + } + return (buffer-start) + u64_toa(u, buffer); +} \ No newline at end of file diff --git a/src/lib/jetfix/gateway/Session.cpp b/src/lib/jetfix/gateway/Session.cpp index 4cc176620..5b669766c 100644 --- a/src/lib/jetfix/gateway/Session.cpp +++ b/src/lib/jetfix/gateway/Session.cpp @@ -783,42 +783,63 @@ OutFIXPtr make_logon(uint32_t seq_num, uint32_t last_seq_num_processed, const char *application = "O2"; #endif - buffer.add(application); - buffer.add("1.0"); - buffer.add("BTS"); - - if (session_cfg.has_trading_session_id()) { - buffer.add(1); - buffer.add(session_cfg.trading_session_id()); - } - if (session_cfg.has_udf_support_indicator()) - buffer.add(session_cfg.udf_support_indicator()); - - if (session_cfg.use_cme_secure_logon()) { - bts_assert(session_cfg.sender_compid().size() >= 3); - auto session_id = session_cfg.sender_compid().substr(0, 3); - std::string seq_num_str(9, ' '); - i32_toa(seq_num, &seq_num_str[0], 9); - - std::string last_seq_num_str(9, ' '); - i32_toa(last_seq_num_processed, &last_seq_num_str[0], 9); - auto ld = glbx::calculate_hmac( - session_id, seq_num_str, session_cfg.sender_compid(), - session_cfg.sender_subid(), time_str, session_cfg.target_subid(), - std::to_string(session_cfg.seconds_between_hbs()), - session_cfg.sender_locationid(), last_seq_num_str, application, "1.0", - "BTS"); -#ifndef BTS_EDGE - LOG(Jetfix, Info, "Canonical string for secure logon: {}", ld.canonical); -#endif - buffer.add(ld.access_key.size()); - buffer.add(ld.access_key); - buffer.add("CME-1-SHA-256"); - buffer.add(ld.signature.size()); - buffer.add(ld.signature); - } else { + if(session_cfg.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_ICE || + session_cfg.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_SIMICE) + { + buffer.add(application); + buffer.add("1.0"); + buffer.add("BTS"); + + buffer.add(session_cfg.user_name()); buffer.add(session_cfg.raw_data().size()); - buffer.add(session_cfg.raw_data()); + buffer.add(session_cfg.raw_data()); + buffer.add(0); + buffer.add(1); + buffer.add(1); + } + else + { + buffer.add(application); + buffer.add("1.0"); + buffer.add("BTS"); + + if (session_cfg.has_trading_session_id()) { + buffer.add(1); + buffer.add(session_cfg.trading_session_id()); + } + if (session_cfg.has_udf_support_indicator()) + buffer.add(session_cfg.udf_support_indicator()); + + if (session_cfg.use_cme_secure_logon()) { + bts_assert(session_cfg.sender_compid().size() >= 3); + auto session_id = session_cfg.sender_compid().substr(0, 3); + std::string seq_num_str(9, ' '); + i32_toa(seq_num, &seq_num_str[0], 9); + + std::string last_seq_num_str(9, ' '); + i32_toa(last_seq_num_processed, &last_seq_num_str[0], 9); + auto ld = glbx::calculate_hmac( + session_id, seq_num_str, session_cfg.sender_compid(), + session_cfg.sender_subid(), time_str, session_cfg.target_subid(), + std::to_string(session_cfg.seconds_between_hbs()), + session_cfg.sender_locationid(), last_seq_num_str, application, "1.0", + "BTS"); + #ifndef BTS_EDGE + LOG(Jetfix, Info, "Canonical string for secure logon: {}", ld.canonical); + #endif + buffer.add(ld.access_key.size()); + buffer.add(ld.access_key); + buffer.add("CME-1-SHA-256"); + buffer.add(ld.signature.size()); + buffer.add(ld.signature); + } + else + { + buffer.add(session_cfg.raw_data().size()); + buffer.add(session_cfg.raw_data()); + if(session_cfg.user_name() != "") + buffer.add(session_cfg.user_name()); + } } stream.close(); diff --git a/src/lib/md/MDConsumerHandle.cpp b/src/lib/md/MDConsumerHandle.cpp index 5b585213b..442473281 100644 --- a/src/lib/md/MDConsumerHandle.cpp +++ b/src/lib/md/MDConsumerHandle.cpp @@ -296,7 +296,7 @@ TimeStamp copy_synthetic_depth(const SyntheticStrategy& synth, size_t count, Pri // END SYNTHETIC DEPTH ///////////////////////////// -bool MDConsumerHandle::get_last_trade(TimeStamp& time, PriceSize& trade, datetime::LLDateTime& trade_timestamp, size_t* total_volume) const { +bool MDConsumerHandle::get_last_trade(TimeStamp& time, PriceSize& trade, datetime::LLDateTime& trade_timestamp, size_t* total_volume, int* from_spread) const { switch(_type) { case HANDLETYPE_VOID: return false; case HANDLETYPE_DEPTH: @@ -306,6 +306,7 @@ bool MDConsumerHandle::get_last_trade(TimeStamp& time, PriceSize& trade, datetim if (result) { trade_timestamp = pcold->last_trade_time; if (total_volume) { *total_volume = pcold->total_volume; } + if (from_spread) { *from_spread = pcold->from_spread; } } return result; } diff --git a/src/lib/md/MDConsumerHandle.h b/src/lib/md/MDConsumerHandle.h index bc094189f..c271083fd 100644 --- a/src/lib/md/MDConsumerHandle.h +++ b/src/lib/md/MDConsumerHandle.h @@ -261,7 +261,7 @@ public: return get_book(read_combined, time, copydepth, bids, asks); } - bool get_last_trade(TimeStamp& time, PriceSize& out_trade, datetime::LLDateTime& out_trade_timestamp, size_t* total_volume=nullptr) const; + bool get_last_trade(TimeStamp& time, PriceSize& out_trade, datetime::LLDateTime& out_trade_timestamp, size_t* total_volume=nullptr,int* from_spread=nullptr) const; IntPrice settlement_price() const { switch(_type) { diff --git a/src/lib/md/MDProducerHandle.cpp b/src/lib/md/MDProducerHandle.cpp index c38e53daf..8ee9c09e3 100644 --- a/src/lib/md/MDProducerHandle.cpp +++ b/src/lib/md/MDProducerHandle.cpp @@ -101,8 +101,9 @@ void MDProducerHandle::set_status(CoconutSecurityStatus status) { } } -void MDProducerHandle::add_trade(TimeStamp time, const PriceSize price, LLDateTime tradetime, co2::AggressorSide agside, uint32_t volume) { - // Add this trade to the last trade if price match +void MDProducerHandle::add_trade(TimeStamp time, const PriceSize price, LLDateTime tradetime, + co2::AggressorSide agside, uint32_t volume, bool from_spread) { + // Add this trade to the last trade if price/side match auto current = std::get(cdata().last_trade.load()); if (current.int_price() == price.int_price()) { cdata().last_trade.store(PriceSize { price.int_price(), price.size() + current.size() }, time); @@ -111,7 +112,9 @@ void MDProducerHandle::add_trade(TimeStamp time, const PriceSize price, LLDateTi cdata().last_trade.store(price,time); } cdata().last_trade_time = tradetime; - cdata().total_volume = volume; + cdata().from_spread = (from_spread ? 1 : 0); + if (volume > 0) + cdata().total_volume = volume; if (key() != invalid_security_key) { co2::TradeMessage trade; @@ -125,9 +128,15 @@ void MDProducerHandle::add_trade(TimeStamp time, const PriceSize price, LLDateTi _channel.add_trade(trade); } } -void MDProducerHandle::set_last_trade(TimeStamp time, const PriceSize ps, datetime::LLDateTime tradetime, uint32_t total_volume) { +void MDProducerHandle::set_last_trade(TimeStamp time, const PriceSize ps, datetime::LLDateTime tradetime, uint32_t total_volume, bool from_spread) { cdata().last_trade.store(ps,time); cdata().last_trade_time = tradetime; + cdata().from_spread = (from_spread ? 1 : 0); + if(total_volume > 0) + cdata().total_volume = total_volume; +} + +void MDProducerHandle::set_total_volume(uint32_t total_volume) { cdata().total_volume = total_volume; } diff --git a/src/lib/md/MDProducerHandle.h b/src/lib/md/MDProducerHandle.h index da5afcf2d..78466987a 100644 --- a/src/lib/md/MDProducerHandle.h +++ b/src/lib/md/MDProducerHandle.h @@ -92,9 +92,11 @@ public: void set_status(co2::CoconutSecurityStatus status); // hot // Updates trade log and last trade - void add_trade(TimeStamp time, const PriceSize pricesize, datetime::LLDateTime date, co2::AggressorSide agside, uint32_t total_volume=0); // cold + void add_trade(TimeStamp time, const PriceSize pricesize, datetime::LLDateTime date, + co2::AggressorSide agside, uint32_t total_volume=0, bool from_spread=false); // cold // Update last_trade without updating trade log - void set_last_trade(TimeStamp time, const PriceSize ps, datetime::LLDateTime date, uint32_t total_volume); + void set_last_trade(TimeStamp time, const PriceSize ps, datetime::LLDateTime date, uint32_t total_volume=0,bool from_spread=false); + void set_total_volume(uint32_t total_volume); void add_rfq(TimeStamp time, const co2::QuoteRequestMessage& rfq); // cold void update_exchange_status(TimeStamp time, TradingStatus status); // cold diff --git a/src/lib/md/PlatformLoader.cpp b/src/lib/md/PlatformLoader.cpp index 4188210f5..0de76bdcf 100644 --- a/src/lib/md/PlatformLoader.cpp +++ b/src/lib/md/PlatformLoader.cpp @@ -90,7 +90,7 @@ void add_symbols_to_result( size_t get_system_md_symbols(api::Venue exchange, SecurityType type, std::set &result) { - bts_assert(config::system_symbols::Symbols::descriptor()->field_count() == 3); + bts_assert(config::system_symbols::Symbols::descriptor()->field_count() == 4); { auto const &e = config::get().glbx(); if (exchange == api::VENUE_UNSET || exchange == api::VENUE_GLBX) { @@ -146,34 +146,47 @@ size_t get_system_md_symbols(api::Venue exchange, SecurityType type, { auto const &e = config::get().sim(); - if (exchange == api::VENUE_UNSET || exchange == api::VENUE_SIM) { - if (type == api::SECURITYTYPE_FUTURE || - type == api::SECURITYTYPE_UNSET) { - add_symbols_to_result(e.future(), api::VENUE_SIM, - api::SECURITYTYPE_FUTURE, &result); - } - if (type == api::SECURITYTYPE_OPTION || - type == api::SECURITYTYPE_UNSET) { - add_symbols_to_result(e.option(), api::VENUE_SIM, - api::SECURITYTYPE_OPTION, &result); - } - if (type == api::SECURITYTYPE_SPREAD || - type == api::SECURITYTYPE_UNSET) { - add_symbols_to_result(e.spread(), api::VENUE_SIM, - api::SECURITYTYPE_SPREAD, &result); - } + if (exchange == api::VENUE_UNSET || exchange == api::VENUE_SIM) + { + if (type == api::SECURITYTYPE_FUTURE || type == api::SECURITYTYPE_UNSET) + add_symbols_to_result(e.future(), api::VENUE_SIM, api::SECURITYTYPE_FUTURE, &result); + if (type == api::SECURITYTYPE_OPTION || type == api::SECURITYTYPE_UNSET) + add_symbols_to_result(e.option(), api::VENUE_SIM, api::SECURITYTYPE_OPTION, &result); + if (type == api::SECURITYTYPE_SPREAD || type == api::SECURITYTYPE_UNSET) + add_symbols_to_result(e.spread(), api::VENUE_SIM, api::SECURITYTYPE_SPREAD, &result); } - if (exchange == api::VENUE_BTS || exchange == api::VENUE_UNSET) { + if (exchange == api::VENUE_BTS || exchange == api::VENUE_UNSET) + { std::set all_symbols; all_symbols.insert(e.future().begin(), e.future().end()); all_symbols.insert(e.option().begin(), e.option().end()); all_symbols.insert(e.spread().begin(), e.spread().end()); - for (const auto &symbol : all_symbols) { - result.insert(UserSymbol(symbol, api::VENUE_BTS, - api::SECURITYTYPE_SYNTHETIC)); - } + for (const auto &symbol : all_symbols) + result.insert(UserSymbol(symbol, api::VENUE_BTS, api::SECURITYTYPE_SYNTHETIC)); } } + + { + auto const &e = config::get().simice(); + if (exchange == api::VENUE_UNSET || exchange == api::VENUE_SIMICE) + { + if (type == api::SECURITYTYPE_FUTURE || type == api::SECURITYTYPE_UNSET) + add_symbols_to_result(e.future(), api::VENUE_SIMICE, api::SECURITYTYPE_FUTURE, &result); + if (type == api::SECURITYTYPE_OPTION || type == api::SECURITYTYPE_UNSET) + add_symbols_to_result(e.option(), api::VENUE_SIMICE, api::SECURITYTYPE_OPTION, &result); + if (type == api::SECURITYTYPE_SPREAD || type == api::SECURITYTYPE_UNSET) + add_symbols_to_result(e.spread(), api::VENUE_SIMICE, api::SECURITYTYPE_SPREAD, &result); + } + if (exchange == api::VENUE_BTS || exchange == api::VENUE_UNSET) + { + std::set all_symbols; + all_symbols.insert(e.future().begin(), e.future().end()); + all_symbols.insert(e.option().begin(), e.option().end()); + all_symbols.insert(e.spread().begin(), e.spread().end()); + for (const auto &symbol : all_symbols) + result.insert(UserSymbol(symbol, api::VENUE_BTS, api::SECURITYTYPE_SYNTHETIC)); + } + } return result.size(); } diff --git a/src/lib/md/Shm/FixedSecurity.cpp b/src/lib/md/Shm/FixedSecurity.cpp index fd5f7bfec..c3b5ce25c 100644 --- a/src/lib/md/Shm/FixedSecurity.cpp +++ b/src/lib/md/Shm/FixedSecurity.cpp @@ -31,8 +31,9 @@ FixedSecurity::FixedSecurity() : cme_main_fraction(0), cme_sub_fraction(0), cme_price_display_format(0), - ice_deal_display_factor(0), - ice_settle_display_factor(0), + ice_price_denominator(0), + ice_deal_denominator(0), + ice_settle_denominator(0), _symbol(), _cfi_code(), _security_desc(), @@ -99,7 +100,10 @@ void write_leg_descriptor(std::ostringstream& result, char separator, const api: std::string security_descriptor(const api::Security& security) { const char separator = ' '; std::ostringstream result; - result << security.symbol() << separator << Venue_Name(security.exchange()); + if(security.exchange() == Venue::VENUE_ICE || security.exchange() == Venue::VENUE_SIMICE) + result << security.ice().security_desc() << separator << Venue_Name(security.exchange()); + else + result << security.symbol() << separator << Venue_Name(security.exchange()); switch(security.Contract_case()) { case Security::ContractCase::kOption: { result << separator; @@ -181,8 +185,9 @@ void convert_security(const FixedSecurity& source, Security* presult) { case Venue::VENUE_ICE: dest.mutable_ice()->set_exchange_id(source.numeric_id); dest.mutable_ice()->set_security_desc(source.security_desc()); - dest.mutable_ice()->set_deal_display_factor(source.ice_deal_display_factor); - dest.mutable_ice()->set_settle_display_factor(source.ice_settle_display_factor); + dest.mutable_ice()->set_price_denominator(source.ice_price_denominator); + dest.mutable_ice()->set_deal_denominator(source.ice_deal_denominator); + dest.mutable_ice()->set_settle_denominator(source.ice_settle_denominator); dest.mutable_ice()->set_market_description(source.ice_market_description()); dest.mutable_ice()->set_cleared_alias(source.ice_cleared_alias()); @@ -312,8 +317,9 @@ void convert_security(const Security& src, FixedSecurity& dest, bool set_securit case Security::ExchangeDataCase::kIce: dest.numeric_id = src.ice().exchange_id(); dest.security_desc(src.ice().security_desc()); - dest.ice_deal_display_factor = src.ice().deal_display_factor(); - dest.ice_settle_display_factor = src.ice().settle_display_factor(); + dest.ice_price_denominator = src.ice().price_denominator(); + dest.ice_deal_denominator = src.ice().deal_denominator(); + dest.ice_settle_denominator = src.ice().settle_denominator(); dest.ice_market_description(src.ice().market_description()); dest.ice_cleared_alias(src.ice().cleared_alias()); diff --git a/src/lib/md/Shm/FixedSecurity.h b/src/lib/md/Shm/FixedSecurity.h index 853db1308..f0828d7e9 100644 --- a/src/lib/md/Shm/FixedSecurity.h +++ b/src/lib/md/Shm/FixedSecurity.h @@ -59,8 +59,9 @@ public: uint8_t cme_main_fraction; uint8_t cme_sub_fraction; uint8_t cme_price_display_format; - int8_t ice_deal_display_factor; - int8_t ice_settle_display_factor; + int8_t ice_price_denominator; + int8_t ice_deal_denominator; + int8_t ice_settle_denominator; STRING_PROPERTY(ice_market_description); STRING_PROPERTY(ice_cleared_alias); diff --git a/src/lib/md/Shm/MDStructure.h b/src/lib/md/Shm/MDStructure.h index 3553b71d8..2bad7f6ee 100644 --- a/src/lib/md/Shm/MDStructure.h +++ b/src/lib/md/Shm/MDStructure.h @@ -26,7 +26,7 @@ namespace md { using utils::TimeStamp; -const uint32_t shm_version = 102; +const uint32_t shm_version = 107; struct ALIGNED_(CACHE_LINE) HotHeader { size_t count; @@ -213,9 +213,10 @@ struct ColdEntry { exchange_status(co2::TRADINGSTATUS_UNKNOWN), last_trade(), total_volume(0), - last_trade_time(), + last_trade_time(), misc(), - deleted(false) + from_spread(0), + deleted(0) { this->security.size = security_size; } @@ -227,12 +228,13 @@ struct ColdEntry { // Last Trade AtomicPriceSize last_trade; - size_t total_volume; + size_t total_volume; datetime::LLDateTime last_trade_time; - + MiscData misc; - bool deleted; + int from_spread; + int deleted; FixedSecurity security; }; diff --git a/src/lib/md/ice/ICEUtils.cpp b/src/lib/md/ice/ICEUtils.cpp index 1533964a0..98d7a6600 100644 --- a/src/lib/md/ice/ICEUtils.cpp +++ b/src/lib/md/ice/ICEUtils.cpp @@ -27,6 +27,8 @@ std::string ICEChannel::to_string() const std::string mt = name; for(const auto& x : market_types) mt += " " + std::to_string(x); + for(const auto& x : uds_market_types) + mt += " " + std::to_string(x); return mt; } @@ -45,6 +47,9 @@ const vector parse_ice_channels(const vector& symbols) chan.symbols.insert(symbols.begin(), symbols.end()); for(const auto& xx : x.market_type()) chan.market_types.push_back(xx); + for(const auto& xx : x.uds_market_type()) + chan.uds_market_types.push_back(xx); + result.push_back(chan); } return result; diff --git a/src/lib/md/ice/ICEUtils.h b/src/lib/md/ice/ICEUtils.h index 5e0df291e..7ce5f78b9 100644 --- a/src/lib/md/ice/ICEUtils.h +++ b/src/lib/md/ice/ICEUtils.h @@ -13,6 +13,7 @@ struct ICEChannel { std::string name; std::vector market_types; + std::vector uds_market_types; std::set symbols; std::string to_string() const; }; diff --git a/src/lib/md/protos/co2.proto b/src/lib/md/protos/co2.proto index 23a27ff2a..c44566db6 100644 --- a/src/lib/md/protos/co2.proto +++ b/src/lib/md/protos/co2.proto @@ -152,6 +152,7 @@ message TradeMessage { uint32 total_volume = 7; bool gap = 8; AggressorSide aggressor_side = 9; + bool from_spread = 10; } message QuoteRequestMessage { enum Side { diff --git a/src/lib/md/relay/QueryServer.cpp b/src/lib/md/relay/QueryServer.cpp index f0ede96a5..e404ff2b1 100644 --- a/src/lib/md/relay/QueryServer.cpp +++ b/src/lib/md/relay/QueryServer.cpp @@ -78,7 +78,8 @@ struct MDConsumerGroup { symbol(symbol), platform(platform_for_exchange(symbol.exchange())), channel(consumer.get_channel(symbol)), - handles() + handles(), + mic() { set_consumer(consumer); @@ -118,9 +119,38 @@ struct MDConsumerGroup { bts_assert_d(handle.is_initialized()); bts_assert(handle.get_usersymbol() == symbol); DatedHandle h = { TimeStamp::now(), handle }; - handles->push_back(h); + handles->push_back(h); + if(this->mic == "") + set_mic(h); } - + + void set_mic(const DatedHandle& h) + { + if(channel.exchange() == api::VENUE_ICE || channel.exchange() == api::VENUE_SIMICE) + { + this->mic = h.handle.get_normalized_security().ice().mic_code(); + } + else if(channel.exchange() == api::VENUE_GLBX || channel.exchange() == api::VENUE_SIM) + { + const auto& cme_symb_mics = config::get().cme_mic(); + for(auto mic=cme_symb_mics.begin();mic!=cme_symb_mics.end();mic++) + { + const auto& micsym = mic->first; + if(micsym == symbol.symbol()) + { + this->mic = api::CMEMics_Name(mic->second); + break; + } + } + } + else if(channel.exchange() == api::VENUE_BTS || channel.exchange() == api::VENUE_UNSET) + { + this->mic = "BTS"; + } + else + bts_assert_msg(false, "unknown venue in QueryServer"); + } + UserSymbol symbol; Platform platform; MDConsumerChannel channel; @@ -128,7 +158,7 @@ struct MDConsumerGroup { // Can add to handles without invalidating readers (no lock needed) // When we recover we swap a new vector into this unique_ptr and delayed-delete shared_ptr> handles; - + std::string mic; }; void convert(PriceSize ps, PriceSizeMessage* pps) { @@ -219,12 +249,18 @@ public: } // the hot part - void fill(const SecurityChannel& chan, MDConsumerGroup& group, ChannelReply* reply, bool entitled) const { + void fill(const SecurityChannel& chan, MDConsumerGroup& group, ChannelReply* reply, + const Entitlements* entitlements, bool bypass_entitlement) const { bts_assert(reply); + reply->mutable_channel()->CopyFrom(chan); reply->set_platform(group.platform); reply->mutable_channel()->set_time(TimeStamp::now().to_uint64()); - reply->mutable_channel()->set_entitled(entitled); + + bool entitled = bypass_entitlement || chan.exchange() == api::VENUE_BTS; + if(entitlements && !entitled) + entitled = entitlements->count(group.mic); + reply->mutable_channel()->set_entitled(entitled); auto flags = (MDRequest::Flags)chan.flags(); // Get a copy of handles. If recovery happens during this call @@ -242,8 +278,10 @@ public: ChannelUpdate* pupd = &temp_update; pupd->set_key(instrument.handle.key()); // If the security was just discovered try to ensure all data is sent - if (instrument.discovered > TimeStamp(chan.time())) { - pupd->mutable_security()->CopyFrom(handle.get_normalized_security()); + if (instrument.discovered > TimeStamp(chan.time())) + { + auto security = handle.get_normalized_security(); + pupd->mutable_security()->CopyFrom(security); snapshot = true; } //if we are not entitled AND this is the first request since we lost our entitlement, zero out books @@ -323,12 +361,13 @@ public: TimeStamp time(snapshot ? 0 : chan.time()); PriceSize price; datetime::LLDateTime trade_time; - size_t volume; - if (handle.get_last_trade(time, price, trade_time, &volume)) { + size_t volume; int from_spread; + if (handle.get_last_trade(time, price, trade_time, &volume, &from_spread)) { auto* trade = pupd->mutable_last_trade(); trade->set_key(handle.key()); trade->set_time(trade_time); trade->set_total_volume(volume); + trade->set_from_spread(from_spread); set_price(price, trade); } } @@ -373,10 +412,7 @@ public: auto groupiter = _groups.find(UserSymbol(ch.symbol(), ch.exchange(), ch.type())); if (groupiter != _groups.end()) { auto& group = groupiter->second; - bool is_entitled = bypass_entitlement || - ch.exchange() == api::VENUE_BTS || - (entitlements && entitlements->count(ch.symbol())); - fill(ch, *group, reply->add_channels(), is_entitled); + fill(ch, *group, reply->add_channels(), entitlements, bypass_entitlement); } } } @@ -538,27 +574,21 @@ QueryServerPtr run_query_server(const bts::api::ServiceContext& ctx, MDClient& c container->admin_client.subscribe_md_entitlement_data().subscribe([=](const api::admin::Md_entitlement_dataUpdate& update) { const auto& all_updates = update.all(); utils::Dispatcher::background().post([all_updates, container]() - { - const auto& cme_symb_mics = config::get().cme_mic(); + { auto entitlements = std::make_shared>(); - for (const auto& row : all_updates) { - if (row && row->enabled()) { - for(auto mic=cme_symb_mics.begin();mic!=cme_symb_mics.end();mic++){ - if(api::CMEMics_Name(mic->second) == row->mic()) { - const auto& symbol = mic->first; - auto iter = entitlements->find(row->user_id()); - if (iter == entitlements->end()) { - Entitlements ents; - ents.insert(symbol); - ents.insert("SIM:" + symbol); - entitlements->insert(iter, std::make_pair(row->user_id(), std::move(ents))); - } - else { - iter->second.insert(symbol); - iter->second.insert("SIM:" + symbol); - } - } + for (const auto& row : all_updates) + { + if (row && row->enabled()) + { + auto iter = entitlements->find(row->user_id()); + if (iter == entitlements->end()) { + Entitlements ents; + ents.insert(row->mic()); + entitlements->insert(iter, std::make_pair(row->user_id(), std::move(ents))); } + else { + iter->second.insert(row->mic()); + } } } container->relay.update_entitlements(entitlements); diff --git a/src/lib/md/relay/RelayClient.cpp b/src/lib/md/relay/RelayClient.cpp index b7d4da066..0cc92b7a4 100644 --- a/src/lib/md/relay/RelayClient.cpp +++ b/src/lib/md/relay/RelayClient.cpp @@ -86,7 +86,7 @@ class RelayProducer : public IRelayWriter { handle.update_fixing_price(updated, misc.fixing_price()); } if (update.has_last_trade()) { - handle.set_last_trade(updated, trade_pricesize(update.last_trade()), update.last_trade().time(), update.last_trade().total_volume()); + handle.set_last_trade(updated, trade_pricesize(update.last_trade()), update.last_trade().time(), update.last_trade().total_volume(), update.last_trade().from_spread()); } } diff --git a/src/lib/md/test/Co2ManagedTestHelper.cpp b/src/lib/md/test/Co2ManagedTestHelper.cpp index f38ca3325..a59e9a618 100644 --- a/src/lib/md/test/Co2ManagedTestHelper.cpp +++ b/src/lib/md/test/Co2ManagedTestHelper.cpp @@ -87,7 +87,8 @@ extern "C" BTS_EXPORT void co2_get_sample_cold(size_t& size, ColdEntry& entry) { entry.misc.session_high_bid = 4.4; entry.misc.session_low_offer = 5.5; entry.misc.fixing_price = 6.6; - entry.deleted = true; + entry.deleted = 1; + entry.from_spread = 0; } extern "C" BTS_EXPORT void co2_get_sample_cold_header(ColdHeader& header) { diff --git a/src/lib/orderman/actors/ExecutionGateway.h b/src/lib/orderman/actors/ExecutionGateway.h deleted file mode 100644 index 2e559ee47..000000000 --- a/src/lib/orderman/actors/ExecutionGateway.h +++ /dev/null @@ -1,53 +0,0 @@ -#pragma once - -#include "lib/config/protos/jetfix.pb.h" -#include "lib/jetfix/codec/MessageDefinitions.h" -#include "lib/jetfix/common/OutFIXPtr.h" -#include "lib/mercury/events/Update.h" -#include "lib/orderman/orderman_utils/orderman_common.h" - -#include - -namespace bts { -namespace md { class SecurityOracle;} -namespace trace {struct CorID;class PerfLog;} -namespace orderman { -class NewSecuritiesEvent; -class ExecutionReport; - -using OnExeReport = - std::function; -using OnGwStatusChange = std::function; -using OnCancelReject = std::function; - -// fix execution gateway for a single FIX session -class ExecutionGateway { - public: - ExecutionGateway(config::JetFIX::Connection const &, trace::PerfLog &, - md::SecurityOracle); - ~ExecutionGateway(); - - void start(); - void dispatch_incoming_fix_messages(); - void on_securities(const mercury::Update &); - void send_fix(jetfix::OutFIXPtr const &fix, trace::CorID trace); - - OnExeReport& on_exe_report(); - OnGwStatusChange& on_gw_status_change(); - OnCancelReject &on_cancel_reject(); - - private: - enum class State { WAITING_FOR_SECURITIES, READY }; - struct PImpl; - std::unique_ptr me; - - // fix gateway events - void on_exe_report(jetfix::fix::ExecutionReportMsg const &); - void on_cancel_reject(jetfix::fix::CancelRejectMsg const &); - void on_connection_status_change(bool connected); - void on_business_reject(jetfix::fix::BusinessMessageRejectMsg const &); - void on_session_reject(jetfix::fix::RejectMsg const &); -}; - -} // namespace orderman -} // namespace bts diff --git a/src/lib/orderman/actors/FillStore.cpp b/src/lib/orderman/actors/FillStore.cpp index c5a492fa7..d977ec17d 100644 --- a/src/lib/orderman/actors/FillStore.cpp +++ b/src/lib/orderman/actors/FillStore.cpp @@ -58,12 +58,10 @@ void FillStore::init() { fill.set_fill_kind(Fill::FILL_KIND_SOD); auto sec = me->oracle.get_security(SecurityKey(fill.security_key())); if (sec) { - if (config::get().enabled() && - securities::is_expired(*sec)) - continue; - else { - auto fk = kvp.first; - me->known_fills.insert(fk); + auto fk = kvp.first; + me->known_fills.insert(fk); + if (!config::get().enabled() || + !securities::is_expired(*sec)) { stored_batch.emplace_back(fill, *sec); } } else @@ -95,12 +93,12 @@ void FillStore::on_execution_report( if (u.events) { for (auto const &ev : *u.events) { auto &er = *ev.data; + auto fk = fmt::format("{}:{}", er.order_key.value(), + to_str(er.exchange_fill_id)); + auto is_known = me->known_fills.count(fk); if (er.exec_type == api::orderman::EXEC_TYPE_PARTIAL_FILL || er.exec_type == api::orderman::EXEC_TYPE_FILL) { - - auto fk = fmt::format("{}:{}", er.order_key.value(), - to_str(er.exchange_fill_id)); - auto is_known = me->known_fills.count(fk); + api::orderman::Fill fill; if (!is_known) { fill.set_order_key(er.order_key.value()); @@ -116,6 +114,9 @@ void FillStore::on_execution_report( fill.set_avg_price(er.avg_price.value()); fill.set_side(er.side); fill.set_last_shares(er.last_shares.value()); + fill.set_fill_status(api::orderman::Fill::FILL_STATUS_ACTIVE); + if(er.exec_type == api::orderman::EXEC_TYPE_TRADE_CORRECT) + fill.set_fill_status(api::orderman::Fill::FILL_STATUS_CORRECTED); fill.set_fill_key(fk); auto exchange_ts = utils::pb::to_timestamp( @@ -139,10 +140,12 @@ void FillStore::on_execution_report( "Discarding known execution report for fill {}", fk); } // if partial_fill or fill - } // for loop + } } // if u.events } + + void FillStore::on_order_definition_update( mercury::Update const &u) { if (u.events) diff --git a/src/lib/orderman/actors/ExecutionGateway.cpp b/src/lib/orderman/actors/GLBXExecutionGateway.cpp similarity index 89% rename from src/lib/orderman/actors/ExecutionGateway.cpp rename to src/lib/orderman/actors/GLBXExecutionGateway.cpp index c14fff3f0..8adc93eee 100644 --- a/src/lib/orderman/actors/ExecutionGateway.cpp +++ b/src/lib/orderman/actors/GLBXExecutionGateway.cpp @@ -1,4 +1,4 @@ -#include "ExecutionGateway.h" +#include "GLBXExecutionGateway.h" #include "lib/jetfix/codec/MessageDefinitions.h" #include "lib/jetfix/common/OutFIX.h" #include "lib/jetfix/gateway/JetFIXGateway.h" @@ -24,7 +24,7 @@ using JFGw = JetFIXGateway; -struct ExecutionGateway::PImpl { +struct GLBXExecutionGateway::PImpl { PImpl(config::JetFIX::Connection const &cfg, trace::PerfLog &l, md::SecurityOracle o) : gw(cfg, l), exe_report_pool(), state(State::WAITING_FOR_SECURITIES), @@ -46,14 +46,16 @@ struct ExecutionGateway::PImpl { md::SecurityOracle sec_oracle; }; -ExecutionGateway::ExecutionGateway(config::JetFIX::Connection const &cfg, +GLBXExecutionGateway::GLBXExecutionGateway(config::JetFIX::Connection const &cfg, trace::PerfLog &log, md::SecurityOracle o) : me(new PImpl(cfg, log, o)) { me->state = State::WAITING_FOR_SECURITIES; } -ExecutionGateway::~ExecutionGateway() {} +GLBXExecutionGateway::~GLBXExecutionGateway() {} -void ExecutionGateway::on_securities( +void GLBXExecutionGateway::on_admin_data(const mercury::Update &) {} + +void GLBXExecutionGateway::on_securities( const mercury::Update &update) { if (update.events) { @@ -96,19 +98,19 @@ void ExecutionGateway::on_securities( } } -void ExecutionGateway::start() { me->gw.start(); } +void GLBXExecutionGateway::start() { me->gw.start(); } -OnExeReport &ExecutionGateway::on_exe_report() { return me->on_exe_report; } +OnExeReport &GLBXExecutionGateway::on_exe_report() { return me->on_exe_report; } -OnGwStatusChange &ExecutionGateway::on_gw_status_change() { +OnGwStatusChange &GLBXExecutionGateway::on_gw_status_change() { return me->on_gw_status_change; } -OnCancelReject &ExecutionGateway::on_cancel_reject() { +OnCancelReject &GLBXExecutionGateway::on_cancel_reject() { return me->on_cancel_reject; } -void ExecutionGateway::on_exe_report( +void GLBXExecutionGateway::on_exe_report( const jetfix::fix::ExecutionReportMsg &msg) { OrderKey ok; SecurityKey sk; @@ -116,7 +118,7 @@ void ExecutionGateway::on_exe_report( auto scoped_log = trace::log_scope(me->perf_log, msg.trace(), - "ExecutionGateway::on_exe_report", ok.value()); + "GLBXExecutionGateway::on_exe_report", ok.value()); LOG(Latency, Info, "Started handling of execution report for order {}", ok.value()); @@ -260,13 +262,18 @@ void ExecutionGateway::on_exe_report( if (msg.has()) str_to(msg.get(), &data->exchange_fill_id); str_to(msg.get(), &data->transact_time); + + if (msg.has()) + data->cust_order_handling_inst = msg.get(); + data->cfg = &me->cfg; data->msg = text_msg; + data->exchange = api::Venue::VENUE_GLBX; me->on_exe_report(*data, msg.trace()); } } -void ExecutionGateway::on_cancel_reject( +void GLBXExecutionGateway::on_cancel_reject( const jetfix::fix::CancelRejectMsg &msg) { OrderKey ok; SecurityKey sk; @@ -281,7 +288,7 @@ void ExecutionGateway::on_cancel_reject( me->on_cancel_reject(ok, reject_msg); } -void ExecutionGateway::on_connection_status_change(bool is_connected) { +void GLBXExecutionGateway::on_connection_status_change(bool is_connected) { LOG(OrderMan, Info, "Connection state of {} changed to {}", me->gw_name.value(), is_connected ? std::string("connected") : std::string("disconnected")); @@ -289,7 +296,7 @@ void ExecutionGateway::on_connection_status_change(bool is_connected) { me->on_gw_status_change(is_connected); } -void ExecutionGateway::on_business_reject( +void GLBXExecutionGateway::on_business_reject( jetfix::fix::BusinessMessageRejectMsg const &m) { LOG(OrderMan, Error, "Unexpected business reject in {} gateway . Reason: " @@ -298,7 +305,7 @@ void ExecutionGateway::on_business_reject( m.has() ? m.get() : std::string("Unknown")); } -void ExecutionGateway::on_session_reject(jetfix::fix::RejectMsg const &m) { +void GLBXExecutionGateway::on_session_reject(jetfix::fix::RejectMsg const &m) { LOG(OrderMan, Error, "Unexpected reject message in {} gateway . Reason: {}. " "This should be investigated", @@ -306,14 +313,14 @@ void ExecutionGateway::on_session_reject(jetfix::fix::RejectMsg const &m) { m.has() ? m.get() : std::string("Unknown")); } -void ExecutionGateway::dispatch_incoming_fix_messages() { +void GLBXExecutionGateway::dispatch_incoming_fix_messages() { me->gw.dispatch_events(); } -void ExecutionGateway::send_fix(jetfix::OutFIXPtr const &fix, +void GLBXExecutionGateway::send_fix(jetfix::OutFIXPtr const &fix, trace::CorID trace) { auto scoped_log = - trace::log_scope(me->perf_log, trace, "ExecutionGateway::send_fix"); + trace::log_scope(me->perf_log, trace, "GLBXExecutionGateway::send_fix"); me->gw.send(fix, trace); me->gw.dispatch_events(); } diff --git a/src/lib/orderman/actors/GLBXExecutionGateway.h b/src/lib/orderman/actors/GLBXExecutionGateway.h new file mode 100644 index 000000000..3c0edcc83 --- /dev/null +++ b/src/lib/orderman/actors/GLBXExecutionGateway.h @@ -0,0 +1,41 @@ +#pragma once + +#include "IExecutionGateway.h" + +namespace bts { +namespace orderman { + + +// fix execution gateway for a single FIX session +class GLBXExecutionGateway : public IExecutionGateway { + public: + GLBXExecutionGateway(config::JetFIX::Connection const &, trace::PerfLog &, + md::SecurityOracle); + virtual ~GLBXExecutionGateway(); + + void start() override; + void dispatch_incoming_fix_messages() override; + void on_securities(const mercury::Update &) override; + void on_admin_data(const mercury::Update &) override; + void send_fix(jetfix::OutFIXPtr const &fix, trace::CorID trace) override; + + OnExeReport& on_exe_report() override; + OnGwStatusChange& on_gw_status_change() override; + OnCancelReject &on_cancel_reject() override; + + private: + BTS_NO_COPY(GLBXExecutionGateway); + enum class State { WAITING_FOR_SECURITIES, READY }; + struct PImpl; + std::unique_ptr me; + + // fix gateway events + void on_exe_report(jetfix::fix::ExecutionReportMsg const &); + void on_cancel_reject(jetfix::fix::CancelRejectMsg const &); + void on_connection_status_change(bool connected); + void on_business_reject(jetfix::fix::BusinessMessageRejectMsg const &); + void on_session_reject(jetfix::fix::RejectMsg const &); +}; + +} // namespace orderman +} // namespace bts diff --git a/src/lib/orderman/actors/ICEExecutionGateway.cpp b/src/lib/orderman/actors/ICEExecutionGateway.cpp new file mode 100644 index 000000000..c97d729c2 --- /dev/null +++ b/src/lib/orderman/actors/ICEExecutionGateway.cpp @@ -0,0 +1,490 @@ +#include "ICEExecutionGateway.h" +#include "lib/jetfix/codec/MessageDefinitions.h" +#include "lib/jetfix/codec/i32_toa.h" +#include "lib/jetfix/common/OutFIX.h" +#include "lib/jetfix/gateway/JetFIXGateway.h" +#include "lib/logging/logging.h" +#include "lib/orderman/events/CancelRejectEvent.h" +#include "lib/orderman/events/GatewayStatusChangeEvent.h" +#include "lib/orderman/events/NewExecutionReportEvent.h" +#include "lib/orderman/events/NewSecuritiesEvent.h" +#include "lib/orderman/orderman_utils/fix_order_codec.h" +#include "lib/api/services/AdminService.pb.h" +#include "lib/orderman/events/NewAdminDataEvent.h" +#include "lib/tracing/bts_trace.h" +#include "lib/utils/TimeStamp.h" +#include "lib/utils/helper_macros.h" +#include "lib/utils/Dispatcher.h" +#include "lib/utils/bts_chrono.h" + +#include +#include +#include + +namespace bts { +namespace orderman { + +using namespace jetfix; + +using JFGw = JetFIXGateway; + + + +struct ICEExecutionGateway::PImpl +{ + PImpl(const config::JetFIX::Connection &cfg, trace::PerfLog &l, + md::SecurityOracle o) + : gw(cfg, l), exe_report_pool(), state(State::WAITING_FOR_SECURITIES), + securities(), securities_by_exchange_id(), gw_name(cfg.name()), + cfg(cfg), perf_log(l), admin_data(), nusers(-1), connection_time(), exec_id_to_memo(), + sec_oracle(o) + {} + + JFGw gw; + boost::object_pool exe_report_pool; + State state; + std::unordered_map securities; + std::unordered_map securities_by_exchange_id; + GatewayName gw_name; + config::JetFIX::Connection cfg; + OnExeReport on_exe_report; + OnGwStatusChange on_gw_status_change; + OnCancelReject on_cancel_reject; + trace::PerfLog &perf_log; + api::admin::AdminData admin_data; + std::atomic nusers; + TimeStamp connection_time; + std::unordered_map exec_id_to_memo; //for the bullshit adjusts + md::SecurityOracle sec_oracle; +}; + +ICEExecutionGateway::ICEExecutionGateway(const config::JetFIX::Connection &cfg, trace::PerfLog &log, md::SecurityOracle o) + : IExecutionGateway(), me(new PImpl(cfg, log, o)) +{ + me->state = State::WAITING_FOR_ADMIN; +} + +ICEExecutionGateway::~ICEExecutionGateway() {} + +void ICEExecutionGateway::on_admin_data(const mercury::Update& update) +{ + if (update.events) + { + for (auto const &ev : *(update.events)) + me->admin_data.CopyFrom(*ev.admin_data); + + if (me->state == State::WAITING_FOR_ADMIN) + { + LOG(OrderMan, Warn, "Received admin data. Waiting for securities {}", me->gw_name.value()); + me->state = State::WAITING_FOR_SECURITIES; + } + } +} + +void ICEExecutionGateway::on_securities(const mercury::Update &update) +{ + if (update.events) + { + for (auto const &ev : *(update.events)) + { + auto sec_ptr = ev.security; + me->securities[sec_ptr->id()] = sec_ptr; + if(sec_ptr->has_ice()) + me->securities_by_exchange_id[sec_ptr->ice().exchange_id()] = sec_ptr; + } + + if (me->state == State::WAITING_FOR_SECURITIES) + { + LOG(OrderMan, Warn, "Received first set of {} securities. Beginning wait 2s for trader login {}", + update.events->size(), me->gw_name.value()); + // got our security synopsis, ready to start the GW + me->state = State::READY_TO_START_GW; + + // subscribe to gateway events + me->gw.on() = [this](auto const &msg) + { + this->on_exe_report(msg); + }; + me->gw.on() = [this](auto const &msg) + { + this->on_cancel_reject(msg); + }; + me->gw.on_ready() = [this]() + { + this->on_connection_status_change(true); + }; + me->gw.on_disconnect() = [this]() + { + this->on_connection_status_change(false); + }; + me->gw.on() = [this](auto const &m) + { + this->on_business_reject(m); + }; + me->gw.on() = [this](auto const &m) + { + this->on_session_reject(m); + }; + } + } +} + +void ICEExecutionGateway::start() +{ + me->gw.start(); + bts_assert_msg(me->state == State::READY_TO_START_GW, "ICEExecutionGateway statemachine failure"); +} + +void ICEExecutionGateway::on_connection_status_change(bool is_connected) +{ + LOG(OrderMan, Info, "Connection state of {} changed to {}", + me->gw_name.value(), is_connected ? std::string("connected") : std::string("disconnected")); + + if(is_connected && me->state == State::READY_TO_START_GW) + { + me->state = State::TRADER_LOGIN_TIMEOUT; + me->connection_time = Timer::now(); + } + + if(me->state == State::READY) + { + me->on_gw_status_change(is_connected); + if(!is_connected) //if we disconnect, set state so we log in again + me->state = State::READY_TO_START_GW; + } +} + + +void ICEExecutionGateway::on_trader_login_timeout_expired() +{ + if (me->state == State::TRADER_LOGIN_TIMEOUT) + { + LOG(OrderMan, Warn, "Done waiting for login timeout, logging on traders {}", me->gw_name.value()); + + // got our security synopsis, ready to start the GW + me->state = State::LOGIN_TIMEOUT_COMPLETE; + me->gw.on() = [&](const fix::ICETraderLoginResponseMsg& lmsg) + { + std::string u = lmsg.get(); + std::string m; + if(lmsg.has()) + m = lmsg.get(); + LOG(OrderMan, Error, "Succesffully logging trader {} into ICE {} {}", u, m, me->gw_name.value()); + if(--me->nusers == 0) + this->on_login(); + }; + + me->gw.on() = [&](const fix::ICETraderLogoutResponseMsg& lmsg) + { + std::string u = lmsg.get(); + std::string m; + if(lmsg.has()) + m = lmsg.get(); + + LOG(OrderMan, Error, "Error logging trader {} into ICE {} {}", u, m, me->gw_name.value()); + if(--me->nusers == 0) + this->on_login(); + }; + + std::set auth_traders; + for(const auto& u : me->admin_data.users()) + { + if(u.has_user_data() && u.user_data().has_exchange_user_data() && u.user_data().exchange_user_data().has_ice_data()) + { + const auto& ice = u.user_data().exchange_user_data().ice_data(); + if(auth_traders.find(ice.authorized_trader()) == auth_traders.end()) + { + if(me->nusers == -1) + me->nusers = 0; + me->nusers++; + + auto result = std::make_shared(me->cfg); + auto &stream = result->fix.back(); + auto &buffer = stream.new_msg(MessageTypes::ICETraderLogin); + buffer.add(ice.company_id()); + buffer.add(ice.authorized_trader()); + buffer.add(ice.password()); + + if(ice.mifid_direct_electronic_access() >= 0) + buffer.add(ice.mifid_direct_electronic_access()); + if(ice.mifid_trading_capacity() >= 0) + buffer.add(ice.mifid_trading_capacity()); + if(ice.mifid_liquidity_provision() >= 0) + buffer.add(ice.mifid_liquidity_provision()); + if(ice.mifid_commodity_deriv_indicator() >= 0) + buffer.add(ice.mifid_commodity_deriv_indicator()); + if(ice.mifid_investment_decision() > 0) + buffer.add(ice.mifid_investment_decision()); + if(ice.mifid_execution_decision() > 0) + buffer.add(ice.mifid_execution_decision()); + if(ice.mifid_client_id_code() > 0) + buffer.add(ice.mifid_client_id_code()); + if(ice.mifid_id() > 0) + buffer.add(ice.mifid_id()); + stream.close(); + + trace::CorID trace; + send_fix(result, trace); + auth_traders.insert(ice.authorized_trader()); + } + } + } + LOG(OrderMan, Warn, "{} traders identified as ICE traders {}", auth_traders.size(), me->gw_name.value()); + } +} + +void ICEExecutionGateway::on_login() +{ + if (me->state == State::LOGIN_TIMEOUT_COMPLETE) + { + LOG(OrderMan, Warn, "Successful login traders {}", me->gw_name.value()); + me->state = State::READY; + me->on_gw_status_change(true); + } +} + +OnExeReport &ICEExecutionGateway::on_exe_report() { return me->on_exe_report; } + +OnGwStatusChange &ICEExecutionGateway::on_gw_status_change() +{ + return me->on_gw_status_change; +} + +OnCancelReject &ICEExecutionGateway::on_cancel_reject() +{ + return me->on_cancel_reject; +} + +bool ICEExecutionGateway::process_memo(const jetfix::fix::ICEExecutionReportMsg &msg, OrderKey* ok, SecurityKey* sk) +{ + if(msg.has()) + { + decode_memo(msg.get(), ok, sk); + return true; + } + else + { + if (msg.get() == api::orderman::EXEC_TYPE_TRADE_CORRECT && msg.has()) + { + std::string ref_exec_id = msg.get(); + auto found = me->exec_id_to_memo.find(ref_exec_id); + if(found == me->exec_id_to_memo.end()) + { + auto new_price = Price(0); + if(msg.has()) + new_price = Price(msg.get()); + + LOG(OrderMan, Error, + "Received unsupported trade correction msg. This correction will not be forwarded. The trader should be alerted. " + "Partial original fill id: {}. New price: {}", ref_exec_id, new_price.value()); + return false; + } + else + { + decode_memo(found->second, ok, sk); + return true; + } + } + } + return false; +} + +void ICEExecutionGateway::on_exe_report(const jetfix::fix::ICEExecutionReportMsg &msg) +{ + OrderKey ok; + SecurityKey sk; + if(!process_memo(msg, &ok, &sk)) + { + std::string execid = ""; + if(msg.has()) + execid = msg.get(); + LOG(OrderMan, Error, + "Received execution report with no memo execid: {}. Will not be able to process this report. This is likely a bug that could lead " + "to lost fills. Please investigate.", execid); + return; + } + auto scoped_log = trace::log_scope(me->perf_log, msg.trace(), "ICEExecutionGateway::on_exe_report", ok.value()); + + LOG(Latency, Info, "Started handling of execution report for order {}", ok.value()); + auto &sec = me->securities[sk.value()]; + if (!sec) + { + sec = me->sec_oracle.get_security(SecurityKey(sk.value())); + } + if (!sec) + LOG(OrderMan, Error, + "Received execution report for an unknown security {}. Will not be able to process this report. This is likely a bug that could lead " + "to lost fills. Please investigate.", + sk.value()); + else + { + if(sec->has_ice()) + me->securities_by_exchange_id[sec->ice().exchange_id()] = sec; + + api::Security const *leg_security = nullptr; + if (msg.has() && msg.get() == 2) + { + auto it = me->securities_by_exchange_id.find(msg.get()); + if (it == me->securities_by_exchange_id.end()) + { + LOG(OrderMan, Error, + "Received execution report for an unknown spread leg" + "security {}. Will not be able to process this report. " + "This is likely a bug that could lead to " + "lost leg fills. Please investigate. Spread sec_key: {} leg " + "exchange id: {}", + sk.value(), msg.get()); + return; + } + leg_security = it->second; + } + std::string text_msg; + if (msg.has()) + text_msg = msg.get(); + if (msg.has()) + text_msg += " Rejected OrdRejReason: " + std::to_string(msg.get()); + + if (msg.has()) + text_msg += " Cancelled DeleteReason: " + std::to_string(msg.get()); + + // allocate from our pool + auto data = me->exe_report_pool.construct(); + + if(msg.has()) + data->memo = msg.get(); + data->exec_type = api::orderman::ExecType(msg.get()); + data->msg_seq_num = msg.get(); + + // copy data from fix msg to our structure + if (msg.has()) + str_to(msg.get(), &data->account_name); + if (msg.has()) + str_to(msg.get(), &data->operator_id); + if (msg.has()) + data->ice_company_id = msg.get(); + + data->order_key = ok; + data->security = sec; + data->leg_security = leg_security; + data->side = api::Side(msg.get()); + if (msg.has()) + data->time_in_force = api::orderman::TimeInForce(msg.get()); + else + data->time_in_force = api::orderman::TIME_IN_FORCE_UNDEFINED; + + if (msg.has()) + data->order_type = api::orderman::NativeOrderType(msg.get()); + else + data->order_type = api::orderman::NATIVE_ORDER_TYPE_UNDEFINED; + + data->order_status = api::orderman::NativeOrderStatus(msg.get()); + data->manual_order_indicator = api::orderman::MANUAL_ORDER_INDICATOR_NO; + + std::string oid_string = get_ice_order_id_string(msg); + str_to(oid_string, &data->exchange_order_id); + data->ice_cl_ord_id = 0; + if(msg.has()) + data->ice_cl_ord_id = msg.get(); + + if (msg.has()) + data->price = securities::decode_exchange_price(*sec, Price(msg.get())); + else + data->price = Price(os::nan()); + + if (msg.has()) + data->stop_price = securities::decode_exchange_price(*sec, Price(msg.get())); + else + data->stop_price = Price(os::nan()); + + if (msg.has()) + data->max_show = Quantity(msg.get()); + else + data->max_show = Quantity(0); + + if (msg.has()) + data->quantity = Quantity(msg.get()); + else + data->quantity = Quantity(0); + + data->filled_quantity = Quantity(msg.get()); + + data->last_shares = msg.has() + ? Quantity(msg.get()) + : Quantity(0); + auto last_px_sec = data->leg_security ? data->leg_security : sec; + data->last_price = msg.has() + ? securities::decode_exchange_price(*last_px_sec, Price(msg.get())) + : Price(0); + data->avg_price = msg.has() + ? securities::decode_exchange_price(*last_px_sec, Price(msg.get())) + : Price(0); + if (msg.has()) + { + auto execid = msg.get(); + str_to(execid, &data->exchange_fill_id); + if(msg.has()) + me->exec_id_to_memo[execid] = msg.get(); + } + + if (msg.has()) + { + std::string stamp = msg.get(); + stamp.resize(sizeof(mercury::TimeStampStr)); + str_to(stamp, &data->transact_time); + } + data->cfg = &me->cfg; + data->msg = text_msg; + data->exchange = api::Venue::VENUE_ICE; + me->on_exe_report(*data, msg.trace()); + } +} + +void ICEExecutionGateway::on_cancel_reject(const jetfix::fix::ICECancelRejectMsg &msg) +{ + OrderKey ok(msg.get()); + + std::string reject_msg; + if (msg.has()) + reject_msg = msg.get(); + else if (msg.has()) + reject_msg = fmt::format("Order cancel was rejected with error code {}", + msg.get()); + me->on_cancel_reject(ok, reject_msg); +} + + +void ICEExecutionGateway::on_business_reject(const jetfix::fix::BusinessMessageRejectMsg& m) +{ + LOG(OrderMan, Error, + "Unexpected business reject in {} gateway . Reason: {}. This should be investigated", + me->gw_name.value(), + m.has() ? m.get() : std::string("Unknown")); +} + +void ICEExecutionGateway::on_session_reject(const jetfix::fix::RejectMsg& m) +{ + LOG(OrderMan, Error, + "Unexpected reject message in {} gateway . Reason: {}. This should be investigated", + me->gw_name.value(), + m.has() ? m.get() : std::string("Unknown")); +} + +void ICEExecutionGateway::dispatch_incoming_fix_messages() +{ + me->gw.dispatch_events(); + if(me->state == State::TRADER_LOGIN_TIMEOUT && ms_diff(Timer::now(), me->connection_time) >= 2000) + { + on_trader_login_timeout_expired(); + } +} + +void ICEExecutionGateway::send_fix(const jetfix::OutFIXPtr& fix, trace::CorID trace) +{ + auto scoped_log = trace::log_scope(me->perf_log,trace,"ICEExecutionGateway::send_fix"); + me->gw.send(fix,trace); + me->gw.dispatch_events(); +} + +} // namespace orderman +} // namespace bts diff --git a/src/lib/orderman/actors/ICEExecutionGateway.h b/src/lib/orderman/actors/ICEExecutionGateway.h new file mode 100644 index 000000000..c885a90af --- /dev/null +++ b/src/lib/orderman/actors/ICEExecutionGateway.h @@ -0,0 +1,45 @@ +#pragma once + +#include "IExecutionGateway.h" + +namespace bts { +namespace orderman { + +// fix execution gateway for a single FIX session +class ICEExecutionGateway : public IExecutionGateway { + public: + ICEExecutionGateway(const config::JetFIX::Connection&, trace::PerfLog &, + md::SecurityOracle); + virtual ~ICEExecutionGateway(); + + void start() override; + void dispatch_incoming_fix_messages() override; + void on_securities(const mercury::Update &) override; + void on_admin_data(const mercury::Update &) override; + void send_fix(const jetfix::OutFIXPtr& fix, trace::CorID trace) override; + + OnExeReport& on_exe_report() override; + OnGwStatusChange& on_gw_status_change() override; + OnCancelReject &on_cancel_reject() override; + + private: + BTS_NO_COPY(ICEExecutionGateway); + enum class State { WAITING_FOR_ADMIN, WAITING_FOR_SECURITIES, READY_TO_START_GW, TRADER_LOGIN_TIMEOUT, LOGIN_TIMEOUT_COMPLETE, READY }; + struct PImpl; + std::unique_ptr me; + + void on_trader_login_timeout_expired(); + void on_login(); + + // fix gateway events + void on_exe_report(const jetfix::fix::ICEExecutionReportMsg&); + void on_cancel_reject(const jetfix::fix::ICECancelRejectMsg&); + void on_connection_status_change(bool connected); + void on_business_reject(const jetfix::fix::BusinessMessageRejectMsg&); + void on_session_reject(const jetfix::fix::RejectMsg&); + + bool process_memo(const jetfix::fix::ICEExecutionReportMsg &msg, OrderKey*, SecurityKey*); +}; + +} // namespace orderman +} // namespace bts diff --git a/src/lib/orderman/actors/IExecutionGateway.cpp b/src/lib/orderman/actors/IExecutionGateway.cpp new file mode 100644 index 000000000..ebe2a9e2b --- /dev/null +++ b/src/lib/orderman/actors/IExecutionGateway.cpp @@ -0,0 +1,20 @@ + +#include "IExecutionGateway.h" +#include "ICEExecutionGateway.h" +#include "GLBXExecutionGateway.h" + +namespace bts { +namespace orderman { + +std::unique_ptr +ExecutionGatewayFactory::create_gateway(const config::JetFIX::Connection& conn, trace::PerfLog& pl, md::SecurityOracle o) +{ + if(conn.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_ICE || + conn.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_SIMICE) + return std::make_unique(conn, pl, o); + else + return std::make_unique(conn, pl, o); +} + +} // namespace orderman +} // namespace bts diff --git a/src/lib/orderman/actors/IExecutionGateway.h b/src/lib/orderman/actors/IExecutionGateway.h new file mode 100644 index 000000000..14eff752d --- /dev/null +++ b/src/lib/orderman/actors/IExecutionGateway.h @@ -0,0 +1,52 @@ +#pragma once + +#include "lib/config/protos/jetfix.pb.h" +#include "lib/jetfix/codec/MessageDefinitions.h" +#include "lib/jetfix/common/OutFIXPtr.h" +#include "lib/md/SecurityOracle.h" +#include "lib/mercury/events/Update.h" +#include "lib/orderman/orderman_utils/orderman_common.h" +#include "lib/utils/bts_no_copy.h" + +#include + +namespace bts { +namespace trace {struct CorID;class PerfLog;} +namespace orderman { +class NewSecuritiesEvent; +class NewAdminDataEvent; +class ExecutionReport; + + +using OnGwStatusChange = std::function; +using OnExeReport = std::function; +using OnCancelReject = std::function; + +// fix execution gateway for a single FIX session +class IExecutionGateway { + public: + IExecutionGateway() = default; + virtual ~IExecutionGateway() {} + + virtual void start() = 0; + virtual void dispatch_incoming_fix_messages() = 0; + virtual void on_securities(const mercury::Update &) = 0; + virtual void on_admin_data(const mercury::Update &) = 0; + virtual void send_fix(const jetfix::OutFIXPtr&, trace::CorID) = 0; + + virtual OnExeReport& on_exe_report() = 0; + virtual OnGwStatusChange& on_gw_status_change() = 0; + virtual OnCancelReject &on_cancel_reject() = 0; +private: + BTS_NO_COPY(IExecutionGateway); +}; + +class ExecutionGatewayFactory +{ +public: + static std::unique_ptr create_gateway(const config::JetFIX::Connection&, trace::PerfLog&, md::SecurityOracle); + +}; + +} // namespace orderman +} // namespace bts diff --git a/src/lib/orderman/actors/order_manager/OrderManager.cpp b/src/lib/orderman/actors/order_manager/OrderManager.cpp index 2dadd8b89..a347e157c 100644 --- a/src/lib/orderman/actors/order_manager/OrderManager.cpp +++ b/src/lib/orderman/actors/order_manager/OrderManager.cpp @@ -15,6 +15,7 @@ #include "lib/orderman/orders/Order.h" #include "lib/orderman/securities/ISecurityDB.h" #include "lib/utils/inplace_function.h" +#include "lib/utils/thread_manager.h" #include #include @@ -44,12 +45,28 @@ struct OrderManager::PImpl { mercury::Actor::SubHandle filled_position_sub; OrderServicesData services_data; mercury::Stream pub_ready; + std::thread background_thread; + std::atomic shutdown = false; }; OrderManager::OrderManager(DynamicOrderRegistry const &b, trace::PerfLog &l, md::SecurityOracle o) - : me(new PImpl(*this, b, l, o)) {} -OrderManager::~OrderManager() {} + : me(new PImpl(*this, b, l, o)) { + auto id_gen = &me->services_data.id_gen; + auto done = &me->shutdown; + me->background_thread = + threading::make_thread("oman_background", [id_gen, done]() { + while (!(*done)) { + id_gen->refill(); + sleep_for(10ms); + } + }); +} + +OrderManager::~OrderManager() { + me->shutdown = true; + me->background_thread.join(); +} void OrderManager::init() { subscribe( @@ -63,11 +80,13 @@ void OrderManager::init() { void OrderManager::on_admin_data(const mercury::Update &m) { if (m.events) { - if (me->state == State::WAIT_ADMIN) { LOG(OrderMan, Info, - "Order manager received admin data. Subscribing " - "to security list now "); + "Order manager received admin data. Subscribing to security list " + "now "); + for (auto &osd : me->services_data.sessions) + osd->gw->on_admin_data(m); + me->state = State::WAIT_SEC; subscribe([this](auto const &sec_update) { this->on_securities(sec_update); @@ -100,7 +119,7 @@ void OrderManager::on_securities( // forward to execution gateways for (auto &osd : me->services_data.sessions) - osd->gw.on_securities(update); + osd->gw->on_securities(update); if (update.events) { @@ -440,19 +459,20 @@ void OrderManager::init_order_services() { // reject evets auto s_ptr = s.get(); s->exe_report_pub = make_stream(gw_name); - s->gw.on_exe_report() = [s_ptr, this](auto const &er, auto const &t) { + s->gw->on_exe_report() = [s_ptr, this](auto const &er, auto const &t) { publish(s_ptr->exe_report_pub, &er, t); }; s->gw_status_change_pub = make_stream(gw_name); - s->gw.on_gw_status_change() = [s_ptr, this, gw_name](bool b) { + s->gw->on_gw_status_change() = [s_ptr, this, gw_name](bool b) { publish(s_ptr->gw_status_change_pub, gw_name, b); }; s->cancel_reject_pub = make_stream(gw_name); - s->gw.on_cancel_reject() = [s_ptr, this](auto const &k, auto const &msg) { + s->gw->on_cancel_reject() = [s_ptr, this](auto const &k, + auto const &msg) { publish(s_ptr->cancel_reject_pub, k, msg); }; // handlers are ready so we can start processing fix events - s->gw.start(); + s->gw->start(); reload_fix_buffer(*s); } @@ -541,7 +561,7 @@ void OrderManager::after_event() { auto scoped_do_work = trace::log_scope(me->services_data.log, {}, "OrderManager::after_event"); fix_stream.close(); - osd->gw.send_fix(osd->current_buffer, osd->current_trace); + osd->gw->send_fix(osd->current_buffer, osd->current_trace); reload_fix_buffer(*osd); } } @@ -566,7 +586,7 @@ void OrderManager::on_poll(trace::PerfLog &log) { trace::log_scope(log, {}, "OrderManager::dispatch_incoming_fix_messages", utils::TimeStamp::Interval::from_microseconds(2)); for (auto &osd : me->services_data.sessions) - osd->gw.dispatch_incoming_fix_messages(); + osd->gw->dispatch_incoming_fix_messages(); } void OrderManager::publish_order_def_update( diff --git a/src/lib/orderman/actors/order_manager/OrderServices.h b/src/lib/orderman/actors/order_manager/OrderServices.h index 800320c58..2e1afdbb3 100644 --- a/src/lib/orderman/actors/order_manager/OrderServices.h +++ b/src/lib/orderman/actors/order_manager/OrderServices.h @@ -47,6 +47,9 @@ class OrderServices { return ss.is_gateway_connected(); } + uint64_t gen_id() { + return _data->id_gen.next(); + } Governor &governor(SessionServices &ss) { return ss.governor(); } void queue_fix(SessionServices &ss, jetfix::MessageTypes mt, diff --git a/src/lib/orderman/actors/order_manager/OrderServicesData.cpp b/src/lib/orderman/actors/order_manager/OrderServicesData.cpp index e175ba396..f6bbbd841 100644 --- a/src/lib/orderman/actors/order_manager/OrderServicesData.cpp +++ b/src/lib/orderman/actors/order_manager/OrderServicesData.cpp @@ -12,7 +12,9 @@ OrderServicesData::OrderServicesData(OrderManager &owner, order_factory(sec_db, order_router, b), last_timer_ev( Timer::now(), - to(datetime::FIX_timestamp(datetime::now()))) { + to(datetime::FIX_timestamp(datetime::now()))), + id_gen("orderman_ids",5000) +{ auto &all_connections = config::get().connection(); for (auto const &c : all_connections) diff --git a/src/lib/orderman/actors/order_manager/OrderServicesData.h b/src/lib/orderman/actors/order_manager/OrderServicesData.h index 8dedac1f1..bf1124406 100644 --- a/src/lib/orderman/actors/order_manager/OrderServicesData.h +++ b/src/lib/orderman/actors/order_manager/OrderServicesData.h @@ -9,7 +9,7 @@ #include "lib/mercury/events/TimerEvent.h" #include "lib/md/SecurityOracle.h" -#include "lib/orderman/actors/ExecutionGateway.h" +#include "lib/orderman/actors/IExecutionGateway.h" #include "lib/orderman/events/ActivateOrderEvent.h" #include "lib/orderman/events/BookUpdateEvent.h" #include "lib/orderman/events/TickerUpdateEvent.h" @@ -34,6 +34,7 @@ #include "lib/orderman/limiter/Limiter.h" #include "lib/orderman/order_router/OrderRouter.h" #include "lib/utils/datetime.h" +#include "lib/utils/Puid.h" #include @@ -52,7 +53,10 @@ struct SessionServicesData { trace::PerfLog &log, md::SecurityOracle o) : session(cfg.name()), gw_connected(false), cfg(cfg), governor(cfg.max_order_msgs_per_window(), cfg.window_duration_msecs()), - gw(cfg, log,o) {} + gw(ExecutionGatewayFactory::create_gateway(cfg, log, o)) + { + + } std::string session; std::vector free_buffers; @@ -69,7 +73,7 @@ struct SessionServicesData { Governor governor; jetfix::OutFIXPtr current_buffer; trace::CorID current_trace; - ExecutionGateway gw; + std::unique_ptr gw; mercury::Stream exe_report_pub; mercury::Stream gw_status_change_pub; mercury::Stream cancel_reject_pub; @@ -107,7 +111,8 @@ class OrderServicesData { mercury::Stream report_order_op_result_pub; mercury::TimerEvent last_timer_ev; std::vector> sessions; - + utils::IDGenerator id_gen ; + SessionServicesData &get_session(std::string const &session_name) { for (auto &s : sessions) if (s->session == session_name) diff --git a/src/lib/orderman/actors/order_manager/SessionServices.h b/src/lib/orderman/actors/order_manager/SessionServices.h index eab64257e..5ace9ced5 100644 --- a/src/lib/orderman/actors/order_manager/SessionServices.h +++ b/src/lib/orderman/actors/order_manager/SessionServices.h @@ -57,14 +57,14 @@ class SessionServices { _gateway_status_sub = src._gateway_status_sub; return *this; } + + config::JetFIX::Connection const &session_cfg() const { + return _session_data->cfg; + } protected: std::string const &session_name() const { return _session_data->session; } - config::JetFIX::Connection const &session_cfg() const { - return _session_data->cfg; - } - void queue_fix(jetfix::MessageTypes mt, jetfix::FIXFragment const &f, trace::CorID const &corid) { auto &stream = _session_data->current_buffer->first_stream(); diff --git a/src/lib/orderman/order_factory/OrderFactory.cpp b/src/lib/orderman/order_factory/OrderFactory.cpp index 5734e0242..8ffc67968 100644 --- a/src/lib/orderman/order_factory/OrderFactory.cpp +++ b/src/lib/orderman/order_factory/OrderFactory.cpp @@ -10,7 +10,8 @@ // order types #include "lib/orderman/order_factory/DynamicOrderRegistry.h" #include "lib/orderman/orders/DynamicOrder.h" -#include "lib/orderman/orders/NativeOrder.h" +#include "lib/orderman/orders/GLBXNativeOrder.h" +#include "lib/orderman/orders/ICENativeOrder.h" #include "lib/orderman/orders/TimeBoxOrder.h" #include "lib/orderman/orders/TriggerOrder.h" #include "lib/orderman/orders/algo/AlgoWrapperOrder.h" @@ -88,7 +89,7 @@ Order &OrderFactory::create(OrderKey parent, // field correctly. Algos will just propagate the value of this // field down so eventually native orders launched by algos will // get the right value. - if (dynamic_cast(&order) == nullptr && + if (dynamic_cast(&order) == nullptr && def.manual_order_indicator() == api::orderman::MANUAL_ORDER_INDICATOR_YES) { // An algo order incorrectly marked as a manual order @@ -113,6 +114,24 @@ Order &OrderFactory::recover(api::orderman::OrderRecord const &rec) { return order; } +NativeOrderBase *OrderFactory::make_order(const api::Venue exchange) { + if (exchange == api::Venue::VENUE_ICE || + exchange == api::Venue::VENUE_SIMICE) + return new ICENativeOrder(); + else + return new GLBXNativeOrder(); +} + +NativeOrderBase * +OrderFactory::make_native_order(const api::orderman::OrderDefinition &def) { + auto secptr = me->sec_db.resolve(SecurityKey(def.security_key())); + if (secptr == nullptr) + return new GLBXNativeOrder(); // GLBX Native can handle unresolved + // securities + else + return make_order(secptr->exchange()); +} + Order &OrderFactory::recreate_from_exe_report(ExecutionReport const &r) { using namespace api::orderman; @@ -121,7 +140,7 @@ Order &OrderFactory::recreate_from_exe_report(ExecutionReport const &r) { case NATIVE_ORDER_TYPE_LIMIT: case NATIVE_ORDER_TYPE_STOP: case NATIVE_ORDER_TYPE_STOP_LIMIT: { - auto native_order = std::make_unique(); + std::unique_ptr native_order(make_order(r.exchange)); native_order->_key = r.order_key; native_order->_parent_key = null_order_key(); auto def = native_order->on_init_from_exe_report( @@ -151,7 +170,7 @@ Order &OrderFactory::allocate(OrderKey k, OrderKey pk, case CustomOrderDefinition::kStop: case CustomOrderDefinition::kStopLimit: case CustomOrderDefinition::kGlbxIceberg: - order_ptr.reset(new NativeOrder()); + order_ptr.reset(make_native_order(def)); break; case CustomOrderDefinition::kIceberg: order_ptr.reset( diff --git a/src/lib/orderman/order_factory/OrderFactory.h b/src/lib/orderman/order_factory/OrderFactory.h index a7395413c..06410463c 100644 --- a/src/lib/orderman/order_factory/OrderFactory.h +++ b/src/lib/orderman/order_factory/OrderFactory.h @@ -11,6 +11,7 @@ class ISecurityDB; } namespace orderman { class Order; +class NativeOrderBase; class DynamicOrderRegistry; } // namespace orderman namespace orderman { @@ -45,6 +46,8 @@ class OrderFactory : public IOrderFactory { private: Order &allocate(OrderKey k, OrderKey pk, api::orderman::OrderDefinition const &); + NativeOrderBase* make_order(const api::Venue exchange); + NativeOrderBase* make_native_order(const api::orderman::OrderDefinition& def); struct PImpl; std::unique_ptr me; diff --git a/src/lib/orderman/orderman_utils/ExecutionReport.h b/src/lib/orderman/orderman_utils/ExecutionReport.h index 6081be732..5ffc627b3 100644 --- a/src/lib/orderman/orderman_utils/ExecutionReport.h +++ b/src/lib/orderman/orderman_utils/ExecutionReport.h @@ -34,7 +34,12 @@ public: ExchangeFillID exchange_fill_id; mercury::TimeStampStr transact_time; config::JetFIX::Connection const *cfg; + std::string cust_order_handling_inst; std::string msg; + api::Venue exchange; + int64_t ice_cl_ord_id; + int32_t ice_company_id; + std::string memo; }; }} diff --git a/src/lib/orderman/orderman_utils/fix_order_codec.h b/src/lib/orderman/orderman_utils/fix_order_codec.h index 6b83b8794..14cceb2ce 100644 --- a/src/lib/orderman/orderman_utils/fix_order_codec.h +++ b/src/lib/orderman/orderman_utils/fix_order_codec.h @@ -7,6 +7,7 @@ #include "lib/jetfix/codec/i32_toa.h" #include "lib/utils/bts_assert.h" #include "orderman_common.h" +#include "lib/utils/base62.h" namespace bts { namespace api { @@ -19,6 +20,8 @@ constexpr int clordid_sz() { return 20; } constexpr int price_sz() { return 17; } constexpr int quantity_sz() { return 6; } constexpr int max_show_sz() { return 6; } +constexpr int memo_sz() {return 12;} +constexpr int ice_orderid_sz() {return 12;} constexpr uint32_t corid_sz() { return 16; } // 5 for channel, 1 separator, 10 sn @@ -50,6 +53,49 @@ template OrderKey get_order_key(MSG const &msg) { return result; } +inline void encode_memo(OrderKey ok, SecurityKey sk, std::string *out) +{ + auto ok_string = utils::to_base_62(ok.value()); + auto sk_string = utils::to_base_62(sk.value()); + bts_assert_msg(ok_string.size() == 6 && sk_string.size() == 6, "base62 encoding didn't work"); + *out = ok_string + sk_string; +} + +inline void decode_memo(const std::string& in, OrderKey* ok, SecurityKey* sk) +{ + bts_assert(in.size() == memo_sz()); + char buffer[memo_sz()/2+1]; + buffer[memo_sz()/2] = 0; + std::memcpy(buffer,&in[0],memo_sz()/2); + *ok = OrderKey( utils::from_base_62(std::string(buffer)) ); + + std::memcpy(buffer,&in[memo_sz()/2],memo_sz()/2); + *sk = SecurityKey( utils::from_base_62(std::string(buffer)) ); +} + +template +std::string get_ice_order_id_string(const MSG& msg) +{ + bts_assert(msg.template has()); + std::string oid_string(ice_orderid_sz(), ' '); + + int64_t id = msg.template get(); + u64_toa(id, &(oid_string[0]), ice_orderid_sz() ); + return oid_string; +} + +template +std::string get_ice_clord_id_string(const MSG& msg) +{ + bts_assert(msg.template has()); + std::string oid_string(ice_orderid_sz(), ' '); + + int64_t id = msg.template get(); + u64_toa(id, &(oid_string[0]), ice_orderid_sz() ); + return oid_string; +} + + inline void encode_corid(const char *channel, uint32_t sn, std::array &out) { memset(&out[0], ' ', out.size()); auto ch_sz = std::min(strlen(channel), (size_t)5); diff --git a/src/lib/orderman/orderman_utils/orderman_utils.cpp b/src/lib/orderman/orderman_utils/orderman_utils.cpp index ce9d4af61..be9535229 100644 --- a/src/lib/orderman/orderman_utils/orderman_utils.cpp +++ b/src/lib/orderman/orderman_utils/orderman_utils.cpp @@ -133,6 +133,13 @@ api::orderman::DynamicOrderParamDescriptor param_uint32(const std::string& name) return doap; } +api::orderman::DynamicOrderParamDescriptor param_security_key(const std::string& name) { + api::orderman::DynamicOrderParamDescriptor doap; + doap.set_param_name(name); + doap.mutable_param_type()->set_primitive(api::orderman::DYNAMIC_ORDER_PRIMITIVE_SECURITY_KEY); + return doap; +} + api::orderman::DynamicOrderParamDescriptor param_double(const std::string& name) { api::orderman::DynamicOrderParamDescriptor doap; doap.set_param_name(name); diff --git a/src/lib/orderman/orderman_utils/orderman_utils.h b/src/lib/orderman/orderman_utils/orderman_utils.h index 4deda32e9..dac8c43df 100644 --- a/src/lib/orderman/orderman_utils/orderman_utils.h +++ b/src/lib/orderman/orderman_utils/orderman_utils.h @@ -93,6 +93,7 @@ std::optional get_if(const google::protobuf::Mapcti_code()[0])); rr.set_customer_or_firm(CustomerOrFirm(er.cfg->customer_or_firm())); rr.set_location(er.cfg->sender_locationid()); + rr.set_cust_order_handling_inst(er.cust_order_handling_inst); on_init_recovered(orecord, os); return orecord.def(); } @@ -1102,6 +1104,13 @@ void NativeOrder::report_state() { me->services->report_state(me->state); } +std::string NativeOrder::get_cust_order_handling_inst(api::orderman::RoutingRecord const&rr) { + if (rr.cust_order_handling_inst().size()) + return rr.cust_order_handling_inst(); + else + return config::get().fallback_cust_order_handling_inst(); +} + void NativeOrder::init_fix() { bts_assert(me->sec); @@ -1129,6 +1138,8 @@ void NativeOrder::init_fix() { create_encoding.add(rr.cti_code()); create_encoding.add(sec.glbx().security_desc()); create_encoding.add(clordid); + create_encoding.add (get_cust_order_handling_inst(rr)); + if (rr.smp_id().size()) { create_encoding.add(rr.smp_id()); if (rr.smp_instruction() != SELF_MATCH_PREVENTION_INSTRUCTION_UNDEFINED) @@ -1158,6 +1169,7 @@ void NativeOrder::init_fix() { update_encoding.add(clordid); update_encoding.add(clordid); update_encoding.add(true); + update_encoding.add (get_cust_order_handling_inst(rr)); // Self Match Protection if (rr.smp_id().size()) { update_encoding.add(rr.smp_id()); diff --git a/src/lib/orderman/orders/NativeOrder.h b/src/lib/orderman/orders/GLBXNativeOrder.h similarity index 88% rename from src/lib/orderman/orders/NativeOrder.h rename to src/lib/orderman/orders/GLBXNativeOrder.h index e3b7d6fa4..f8f106748 100644 --- a/src/lib/orderman/orders/NativeOrder.h +++ b/src/lib/orderman/orders/GLBXNativeOrder.h @@ -1,7 +1,7 @@ #pragma once #include "lib/api/services/OrderService.pb.h" -#include "lib/orderman/orders/Order.h" +#include "lib/orderman/orders/NativeOrderBase.h" #include "lib/tracing/PerfLog.h" #include @@ -13,12 +13,12 @@ class CancelRejectEvent; class GatewayStatusChangeEvent; class ExecutionReport; -class NativeOrder : public Order { +class NativeOrder : public NativeOrderBase { public: NativeOrder(); ~NativeOrder() override; - api::orderman::OrderDefinition const &definition() const; + api::orderman::OrderDefinition const &definition() const override; void on_init_new(api::orderman::OrderDefinition const &, const PreActivationOrderServices &) override; void on_init_recovered(api::orderman::OrderRecord const &, @@ -33,7 +33,7 @@ class NativeOrder : public Order { api::orderman::OrderDefinition on_init_from_exe_report(ExecutionReport const &, - const PreActivationOrderServices &); + const PreActivationOrderServices &) override; protected: void warm_up_activate(OrderServicesData &, OrderServices &) override; @@ -71,7 +71,10 @@ class NativeOrder : public Order { void ignore_or_cancel(std::string const &, api::orderman::OrderError, OrderOpResult *out) ; + std::string get_cust_order_handling_inst(api::orderman::RoutingRecord const&) ; }; +typedef NativeOrder GLBXNativeOrder; + } // namespace orderman } // namespace bts diff --git a/src/lib/orderman/orders/ICENativeOrder.cpp b/src/lib/orderman/orders/ICENativeOrder.cpp new file mode 100644 index 000000000..0155bf4f1 --- /dev/null +++ b/src/lib/orderman/orders/ICENativeOrder.cpp @@ -0,0 +1,1375 @@ +#include "ICENativeOrder.h" + +#include "lib/api/services/OrderService.pb.h" +#include "lib/domain_model/Security.h" +#include "lib/jetfix/codec/FIXFragment.h" +#include "lib/jetfix/codec/FieldDefinitions.h" +#include "lib/logging/logging.h" +#include "lib/orderman/order_factory/IOrderFactory.h" +#include "lib/orderman/order_router/OrderRouter.h" +#include "lib/orderman/orderman_utils/ExecutionReport.h" +#include "lib/orderman/orderman_utils/fix_order_codec.h" +#include "lib/orderman/orders/PreActivationOrderServices.h" +#include "lib/orderman/securities/ISecurityDB.h" +#include "lib/md/IntPrice.h" +#include "lib/utils/PBUtils.h" +#include "lib/utils/Puid.h" +#include "lib/utils/bts_chrono.h" + +#include "lib/orderman/actors/order_manager/OrderServices.h" + +#include +#include +#include +#include + +namespace bts { +namespace orderman { + +using namespace api::orderman; + +OrderStatus map_external_status(NativeOrderStatus); //in GLBXNativeOrder.cpp + +struct ICENativeOrder::PImpl +{ + PImpl() + : services(nullptr), session_services(), sec(nullptr), + previous_working_position(0), definition(), state(), + price_encoding_precision(7), exchange_order_id(), sender_compid(), id_scale() + { + exchange_order_id.reserve(100); + } + + OrderServices *services; + SessionServices *session_services; + + const api::Security *sec; + Quantity previous_working_position; + + OrderDefinition definition; + OrderState state; + int price_encoding_precision; + std::string exchange_order_id; + int sender_compid; + int64_t id_scale; + + jetfix::FIXFragment update_encoding; + jetfix::FIXFragment::Bookmark update_price_bm; + jetfix::FIXFragment::Bookmark update_quantity_bm; + jetfix::FIXFragment::Bookmark update_stop_px_bm; + jetfix::FIXFragment::Bookmark update_transact_time_bm; + jetfix::FIXFragment::Bookmark update_clordid_bm; + jetfix::FIXFragment::Bookmark update_orig_clordid_bm; + + jetfix::FIXFragment create_encoding; + jetfix::FIXFragment::Bookmark create_clordid_bm; + jetfix::FIXFragment::Bookmark create_price_bm; + jetfix::FIXFragment::Bookmark create_quantity_bm; + jetfix::FIXFragment::Bookmark create_transact_time_bm; + + jetfix::FIXFragment cancel_encoding; + jetfix::FIXFragment::Bookmark cancel_clordid_bm; + jetfix::FIXFragment::Bookmark cancel_orig_clordid_bm; + + OrderServices::ExeReportSubID exe_report_sid; + OrderServices::CancelRejectSubID cancel_reject_sid; + + bool buffered_cancel = false; + std::optional> buffered_update; + int buffered_tries = 0; +}; + +ICENativeOrder::ICENativeOrder() : me(new PImpl()) {} + +const OrderDefinition& ICENativeOrder::definition() const +{ + return me->definition; +} + +void ICENativeOrder::on_init_new(const OrderDefinition& def, const PreActivationOrderServices& serv) +{ + // order was just created. + me->definition = def; + me->sec = serv.sec_db.resolve(SecurityKey(def.security_key())); + if (!me->sec) + { + auto msg = fmt::format("Unknown security: {}", def.security_key()); + me->state.set_last_msg(msg); + transition(ORDER_STATUS_REJECTED_BY_SPARK, msg); + return; + } + + switch (def.custom_def().custom_case()) { + case CustomOrderDefinition::kMarket: + me->state.mutable_market(); + break; + case CustomOrderDefinition::kLimit: + me->state.mutable_limit(); + break; + case CustomOrderDefinition::kStop: + me->state.mutable_stop(); + break; + case CustomOrderDefinition::kStopLimit: + me->state.mutable_stop_limit(); + break; + default: + bts_assert(false); + break; + } + + auto rq = serv.order_router.get_routing_record( UserID(def.user_id()), AccountID(def.account_id()), *me->sec ); + if (!rq.has_route) + { + auto msg = fmt::format("Could not route order: {}", rq.error_msg); + LOG(OrderMan, Error, "{}", msg); + me->state.set_last_msg(msg); + transition(ORDER_STATUS_REJECTED_BY_SPARK, msg); + return; + } + + transition(ORDER_STATUS_INACTIVE); + auto &native_state = get_native_state(me->state); + *native_state.mutable_routing_record() = rq.routing_record; + setup_ids(rq.routing_record.sender_compid()); + init_fix(); +} + +void ICENativeOrder::init_recovered_common(const api::orderman::OrderRecord& rec, const PreActivationOrderServices& serv) +{ + // order was recovered from our orders database or from an execution report + const auto &def = rec.def(); + + me->definition = def; + me->state = rec.state(); + me->sec = serv.sec_db.resolve(SecurityKey(def.security_key())); + me->previous_working_position = Quantity(0); + + const auto &native_state = get_native_state(me->state); + setup_ids(native_state.routing_record().sender_compid()); + switch (me->state.status()) + { + case ORDER_STATUS_WORKING: + case ORDER_STATUS_PENDING_WORKING: + case ORDER_STATUS_DISCONNECTED: + { + // bail if we can't find the security for the order + if (!me->sec) + { + LOG(OrderMan, Error, + "Unknown security {} referenced by recovered order {}. This is a serious " + "problem which has to be corrected manually. ", + def.security_key(), rec.key()); + transition_to_error(NativeOrderState::SUB_ERROR_UNKNOWN_SECURITY, + fmt::format("Unknown security: {}", def.security_key())); + return; + } + + // bail if the session we used for the order is no longer valid + const auto &session = native_state.routing_record().session(); + if (!serv.order_router.is_valid_session(session)) + { + LOG(OrderMan, Error, + "Unknown JetFIX session {} referenced by recovered order {}. This is a serious " + "problem which has to be corrected manually. ", + session, rec.key()); + transition_to_error(NativeOrderState::SUB_ERROR_UNKNOWN_FIX_SESSION, + fmt::format("Unknown FIX session: {}", session)); + return; + } + + // if we made it this far we are good. + init_fix(); + if (me->state.status() != ORDER_STATUS_PENDING_WORKING) + { + me->exchange_order_id = native_state.exchange_order_id(); + add_exchange_order_id_to_fix(); + } + return; + } + + case ORDER_STATUS_REJECTED_BY_SPARK: + case ORDER_STATUS_ERROR: + case ORDER_STATUS_COMPLETED: + case ORDER_STATUS_CANCELED: + // terminal states, nothing to do + return; + + case ORDER_STATUS_INACTIVE: + case OrderStatus_INT_MIN_SENTINEL_DO_NOT_USE_: + case OrderStatus_INT_MAX_SENTINEL_DO_NOT_USE_: + bts_assert(false); + return; + } +} + +void ICENativeOrder::on_init_recovered(const OrderRecord& rec, const PreActivationOrderServices& serv) +{ + //TODO need to figure out if this is a day order from a prev session and cancel it + // const auto &def = rec.def(); + // auto &state = *orecord.mutable_state(); + // state.set_status(map_external_status(api::orderman::NativeOrderStatus::NATIVE_ORDER_STATUS_CANCELED); + init_recovered_common(rec, serv); +} + +api::orderman::OrderDefinition +ICENativeOrder::on_init_from_exe_report(const ExecutionReport& er, const PreActivationOrderServices& os) +{ + auto k = er.order_key; + auto nextid = me->services->gen_id(); + if(k.value() < nextid) + { + k = OrderKey(nextid); + LOG(OrderMan, Error, "Updating recovered order id with newly generated id {}", k.value()); + } + + OrderRecord orecord; + orecord.set_key(k.value()); + orecord.set_parent_key(null_order_key().value()); + + auto &def = *orecord.mutable_def(); + def.set_security_key(er.security->id()); + def.set_side(er.side); + + auto sec = os.sec_db.resolve(SecurityKey(def.security_key())); + auto rev_rec = os.order_router.get_reverse_routing_record(er.cfg->name(), std::to_string(er.ice_company_id), + to_trim_str(er.account_name), sec); + + if (rev_rec.ok) + { + def.set_user(rev_rec.user); + def.set_user_id(rev_rec.user_id.value()); + def.set_account(rev_rec.account); + def.set_account_id(rev_rec.account_id.value()); + } + else + { + LOG(OrderMan, Error,"Could not recover account and user information from order {}. The following information is available: {}. ", + k.value(), rev_rec.error_msg); + def.set_user(""); + def.set_user_id(0); + def.set_account(""); + def.set_account_id(0); + } + + def.set_time_in_force(er.time_in_force); + def.set_manual_order_indicator(er.manual_order_indicator); + def.set_cti_code(CTICode(er.cfg->cti_code()[0])); + def.set_customer_or_firm(CustomerOrFirm(er.cfg->customer_or_firm())); + + auto &state = *orecord.mutable_state(); + state.set_status(map_external_status(er.order_status)); + state.set_price(er.price.value()); + state.set_quantity(er.quantity.value()); + state.set_filled_quantity(er.filled_quantity.value()); + state.set_last_price(er.last_price.value()); + state.set_avg_price(er.last_price.value()); + state.set_last_msg(er.msg); + *state.mutable_last_change() = utils::pb::pb_now(); + + switch (er.order_type) + { + case NATIVE_ORDER_TYPE_MARKET: + def.mutable_custom_def()->mutable_market(); + state.mutable_market(); + break; + case NATIVE_ORDER_TYPE_LIMIT: + def.mutable_custom_def()->mutable_limit(); + state.mutable_limit(); + break; + case NATIVE_ORDER_TYPE_STOP: + { + auto &stop_def = *def.mutable_custom_def()->mutable_stop(); + stop_def.set_stop_price(er.stop_price.value()); + state.mutable_stop(); + break; + } + case NATIVE_ORDER_TYPE_STOP_LIMIT: + { + auto &stop_limit_def = *def.mutable_custom_def()->mutable_stop_limit(); + stop_limit_def.set_stop_price(er.stop_price.value()); + state.mutable_stop_limit(); + break; + } + case NATIVE_ORDER_TYPE_MARKET_LIMIT: + case NATIVE_ORDER_TYPE_UNDEFINED: + case NativeOrderType_INT_MIN_SENTINEL_DO_NOT_USE_: + case NativeOrderType_INT_MAX_SENTINEL_DO_NOT_USE_: + bts_assert(false); + break; + } + + auto &native_state = get_native_state(state); + native_state.set_exchange_order_id(to_trim_str(er.exchange_order_id)); + native_state.set_last_msg_seq_num(er.msg_seq_num); + native_state.set_native_type(er.order_type); + + + auto &rr = *native_state.mutable_routing_record(); + bool real_routing = false; + if (rev_rec.ok && sec) + { + const auto& real_rr = os.order_router.get_routing_record(rev_rec.user_id, rev_rec.account_id, *sec); + if(real_rr.has_route) + { + rr.CopyFrom(real_rr.routing_record); + real_routing = true; + } + } + if(!real_routing) + { + rr.set_sender_compid(er.cfg->sender_compid()); + rr.set_session(er.cfg->name()); + rr.set_account(to_trim_str(er.account_name)); + rr.set_operator_id(to_trim_str(er.operator_id)); + rr.set_cti_code(CTICode(er.cfg->cti_code()[0])); + rr.set_customer_or_firm(CustomerOrFirm(er.cfg->customer_or_firm())); + rr.set_location(er.cfg->sender_locationid()); + } + init_recovered_common(orecord, os); + return orecord.def(); +} + +ICENativeOrder::~ICENativeOrder() {} + +void ICENativeOrder::warm_up_activate(OrderServicesData &data, OrderServices &services) +{ + bts_assert(!me->services); + me->services = &services; + + init_order_services(data, services); + + // save these so we can re-instate them after we are done with the warmup + auto before_state = me->state.status(); + auto before_msg = me->state.last_msg(); + + _on_activate(NoLog, NoLog, Price(0), Quantity(0), services); + + // undo warm up side effects + me->state.set_status(before_state); + me->state.set_last_msg(before_msg); + + me->services->unsubscribe_from_exe_report(*me->session_services, me->exe_report_sid); + me->services->unsubscribe_from_cancel_reject(*me->session_services, me->cancel_reject_sid); + me->services->unsubscribe_from_gateway_status(*me->session_services); + me->services = nullptr; +} + +OrderOpResult +ICENativeOrder::_on_activate(log::Level reject_level, log::Level transition_level, + Price p, Quantity q, bts::orderman::OrderServices &services) +{ + me->state.set_price(p.value()); + me->state.set_quantity(q.value()); + me->state.set_filled_quantity(0); + me->state.set_last_price(0); + me->state.set_avg_price(0); + + me->session_services = &me->services->get_session_services( get_native_state(me->state).routing_record().session() ); + me->price_encoding_precision = me->services->price_encoding_precision(*me->session_services); + + // is our gateway connected? + if (!me->services->is_gateway_connected(*me->session_services)) + { + LOG(OrderMan, reject_level, + "Attempt to send order to a disconnected gateway. Dropping order {}", + key().value()); + auto msg = "Exchange gateway is down"; + transition(transition_level, ORDER_STATUS_REJECTED_BY_SPARK, msg); + return OrderOpResult{ORDER_ERROR_GATEWAY_NOT_READY, msg}; + } + + // can we trade? + auto lqr = me->services->limiter().can_trade( UserID(me->definition.user_id()), AccountID(me->definition.account_id()) ); + if (!lqr.ok()) + { + LOG(OrderMan, reject_level, + "Unauthorized order attempted. User: {} on " + "Account: {}. Msg: {} Order: {}. Dropping order", + me->definition.user(), me->definition.account(), lqr.error_msg, + key().value()); + + auto msg = fmt::format("Unauthorized order: {}", lqr.error_msg); + transition(transition_level, ORDER_STATUS_REJECTED_BY_SPARK, msg); + return OrderOpResult{ORDER_ERROR_UNAUTHORIZED, msg}; + } + + // can we trade this security, with this size? + lqr = me->services->limiter().can_send_new_order( AccountID(me->definition.account_id()), *me->sec, me->definition.side(), q ); + + if (!lqr.ok()) + { + LOG(OrderMan, reject_level, + "New order would breach limit. User: {} Account: " + "{} Msg: {} Order: {}. Dropping order", + me->definition.user(), me->definition.account(), lqr.error_msg, + key().value()); + + auto msg = fmt::format("Order would breach limit: {}", lqr.error_msg); + transition(transition_level, ORDER_STATUS_REJECTED_BY_SPARK, msg); + return OrderOpResult{ORDER_ERROR_LIMIT_BREACH, msg}; + } + + // are we under the messaging threshold? + if (!me->services->governor(*me->session_services).can_send()) + { + LOG(OrderMan, reject_level, + "New order would cause breach of messaging limits. Dropping order {}", key().value()); + auto msg = "Messaging threshold breached"; + transition(transition_level, ORDER_STATUS_REJECTED_BY_SPARK, msg); + return OrderOpResult(ORDER_ERROR_MESSAGING_LIMIT_BREACH, msg); + } + + // if we are here we are good. + // subscribe to usefull events + me->exe_report_sid = me->services->subscribe_to_exe_report(*me->session_services, key(), + [this](const auto &r) { this->on_exe_report(r); }); + me->cancel_reject_sid = me->services->subscribe_to_cancel_reject(*me->session_services, key(), + [this](const auto &cr) { this->on_cancel_reject(cr); }); + me->services->subscribe_to_gateway_status(*me->session_services, + [this](const auto &gw) { this->on_gateway_status(gw); }); + + // send out order + auto &encoding = me->create_encoding; + + auto create_clorid = get_new_clordid(); + me->create_clordid_bm = encoding.overwrite( me->create_clordid_bm, create_clorid, count_digits(create_clorid) ); + + switch (me->definition.custom_def().custom_case()) + { + case CustomOrderDefinition::kLimit: + case CustomOrderDefinition::kStopLimit: + { + me->create_price_bm = encoding.overwrite(me->create_price_bm, + securities::encode_exchange_price(*me->sec, p).value(), + me->price_encoding_precision, price_sz()); + break; + } + case CustomOrderDefinition::kStop: + case CustomOrderDefinition::kMarket: + break; + default: + bts_assert(false); + break; + } + + me->create_quantity_bm = encoding.overwrite(me->create_quantity_bm, q.value(), quantity_sz()); + const auto &ts = services.get_time_str(); + me->create_transact_time_bm = encoding.overwrite(me->create_transact_time_bm, &ts.value()[0], ts.value().size()); + return OrderOpResult{}; +} + +OrderOpResult ICENativeOrder::on_activate(Price p, Quantity q, bts::orderman::OrderServices &services) +{ + bts_assert(!me->services); + me->services = &services; + auto result = OrderOpResult{}; + switch (me->state.status()) + { + case ORDER_STATUS_INACTIVE: + { + auto scoped_log = services.log_scope("ICENativeOrder::on_activate", key().value()); + // this is a brand new order + + result = _on_activate(Warn, Info, p, q, services); + if (result.ok()) + { + me->services->queue_fix(*me->session_services, jetfix::MessageTypes::NewOrderSingle, me->create_encoding); + me->services->governor(*me->session_services).record_sent(); + me->state.set_status(ORDER_STATUS_PENDING_WORKING); + + LOG(OrderMan, Info, "Sent new order with key {} to gateway", key().value()); + // Early return so we skip the expensive report_state() + // call. It is OK to spkip it because we are bound to report + // our state once the excange aknowledges the order. + update_working_position(); + return OrderOpResult{}; + } + + // When the order fails its limits checks we fall through the + // switch to report the error state at the end of this function + break; + } + + // all cases below are for orders recovered from disk + case ORDER_STATUS_WORKING: + me->session_services = &me->services->get_session_services( get_native_state(me->state).routing_record().session() ); + me->services->subscribe_to_exe_report(*me->session_services, key(), + [this](const auto &r) { this->on_exe_report(r); }); + me->services->subscribe_to_cancel_reject(*me->session_services, key(), + [this](const auto &cr) { this->on_cancel_reject(cr); }); + me->services->subscribe_to_gateway_status(*me->session_services, + [this](const auto &gw) { this->on_gateway_status(gw); }); + + if (!me->services->is_gateway_connected(*me->session_services)) + { + result.error = ORDER_ERROR_GATEWAY_NOT_READY; + result.error_msg = "Gateway is disconnected"; + transition(ORDER_STATUS_DISCONNECTED); + } + break; + + case ORDER_STATUS_DISCONNECTED: + me->session_services = &me->services->get_session_services( get_native_state(me->state).routing_record().session() ); + me->services->subscribe_to_exe_report(*me->session_services, key(), + [this](const auto &r) { this->on_exe_report(r); }); + me->services->subscribe_to_cancel_reject(*me->session_services, key(), + [this](const auto &cr) { this->on_cancel_reject(cr); }); + me->services->subscribe_to_gateway_status(*me->session_services, + [this](const auto &gw) { this->on_gateway_status(gw); }); + + if (me->services->is_gateway_connected(*me->session_services)) + transition(ORDER_STATUS_WORKING); + break; + + case ORDER_STATUS_PENDING_WORKING: + me->session_services = &me->services->get_session_services( get_native_state(me->state).routing_record().session() ); + me->services->subscribe_to_exe_report(*me->session_services, key(), + [this](const auto &r) { this->on_exe_report(r); }); + me->services->subscribe_to_cancel_reject(*me->session_services, key(), + [this](const auto &cr) { this->on_cancel_reject(cr); }); + me->services->subscribe_to_gateway_status(*me->session_services, + [this](const auto &gw) { this->on_gateway_status(gw); }); + break; + + case ORDER_STATUS_REJECTED_BY_SPARK: + case ORDER_STATUS_COMPLETED: + case ORDER_STATUS_CANCELED: + case ORDER_STATUS_ERROR: + break; + + case OrderStatus_INT_MIN_SENTINEL_DO_NOT_USE_: + case OrderStatus_INT_MAX_SENTINEL_DO_NOT_USE_: + bts_assert(false); + break; + } + report_state(); + return result; +} + +OrderOpResult ICENativeOrder::on_update(Price p, Quantity q) +{ + api::orderman::Update update; + update.set_price(p.value()); + update.set_quantity(q.value()); + return on_update(update); +} + +OrderOpResult ICENativeOrder::on_update(api::orderman::Update const &update) +{ + auto p = Price(update.price()); + auto q = Quantity(update.quantity()); + auto scoped_log = me->services->log_scope("ICENativeOrder::on_update", key().value()); + auto result = OrderOpResult{}; + + switch (me->state.status()) + { + case ORDER_STATUS_WORKING: + { + auto previous_working_qty = 0; + if (me->state.quantity() > me->state.filled_quantity()) + previous_working_qty = me->state.quantity() - me->state.filled_quantity(); + + auto current_working_qty = 0; + if (q.value() > me->state.filled_quantity()) + current_working_qty = q.value() - me->state.filled_quantity(); + + auto lqr = me->services->limiter().can_update_order( AccountID(me->definition.account_id()), *me->sec, + me->definition.side(), Quantity(previous_working_qty), + Quantity(current_working_qty)); + + // are we under the position limits? + if (!lqr.ok()) + { + auto msg = fmt::format("Cannot update order due to limit breach: {}", lqr.error_msg); + LOG(OrderMan, Error, + "Order update failed due to limit breach. Order: {} Msg: {}", + key().value(), lqr.error_msg); + ignore_or_cancel(msg, ORDER_ERROR_LIMIT_BREACH, &result); + break; + } + + // are we under the messaging threshold? + if (!me->services->governor(*me->session_services).can_send()) + { + LOG(OrderMan, Warn, "Cannot update order {} because of messaging limits. ", key().value()); + auto msg = "Messaging threshold breached"; + ignore_or_cancel(msg, ORDER_ERROR_MESSAGING_LIMIT_BREACH, &result); + break; + } + + auto &encoding = me->update_encoding; + + if (update.has_custom()) + { + bts_assert(update.custom().custom_case() == me->definition.custom_def().custom_case()); + if (update.custom().custom_case() == CustomOrderDefinition::kStopLimit) + { + auto const &stop_limit = update.custom().stop_limit(); + auto stop_px = Price(stop_limit.stop_price()); + me->update_stop_px_bm = encoding.overwrite(me->update_stop_px_bm, + securities::encode_exchange_price(*me->sec, stop_px).value(), + me->price_encoding_precision, price_sz()); + } + } + me->update_price_bm = encoding.overwrite(me->update_price_bm, securities::encode_exchange_price(*me->sec, p).value(), + me->price_encoding_precision, price_sz()); + me->update_quantity_bm = encoding.overwrite(me->update_quantity_bm, q.value(), quantity_sz()); + + const auto &ts = me->services->get_time_str(); + me->update_transact_time_bm = encoding.overwrite( me->update_transact_time_bm, &ts.value()[0], ts.value().size() ); + + auto update_clorid = get_new_clordid(); + me->update_clordid_bm = encoding.overwrite( me->update_clordid_bm, update_clorid, count_digits(update_clorid) ); + + me->services->queue_fix(*me->session_services, jetfix::MessageTypes::CancelReplace, me->update_encoding); + me->services->governor(*me->session_services).record_sent(); + update_working_position(); + if (me->state.last_msg().size()) + me->state.set_last_msg(""); + + LOG(OrderMan, Warn, + "ICE may not honor this update if we recently issued an unacknowledged cancelreplace {}. Update: {}@{}. Buffering", + key().value(), q.value(), p.value()); + me->buffered_update.emplace(p, q); + + // early return so we skip report_state (for speed) + return OrderOpResult{}; + } + + case ORDER_STATUS_DISCONNECTED: + case ORDER_STATUS_PENDING_WORKING: + LOG(OrderMan, Warn, + "Cannot apply update to order {}. Update: {}@{}. Buffering", + key().value(), q.value(), p.value()); + me->buffered_update.emplace(p, q); + break; + + case ORDER_STATUS_REJECTED_BY_SPARK: + case ORDER_STATUS_COMPLETED: + case ORDER_STATUS_CANCELED: + case ORDER_STATUS_ERROR: + break; + + case ORDER_STATUS_INACTIVE: + case OrderStatus_INT_MIN_SENTINEL_DO_NOT_USE_: + case OrderStatus_INT_MAX_SENTINEL_DO_NOT_USE_: + bts_assert(false); + break; + } + report_state(); + return result; +} + +OrderOpResult ICENativeOrder::on_detach() { return OrderOpResult{}; } + +void ICENativeOrder::on_attached(api::orderman::OrderDefinition const &new_parent) +{ + switch (me->state.status()) + { + case ORDER_STATUS_PENDING_WORKING: + case ORDER_STATUS_DISCONNECTED: + case ORDER_STATUS_WORKING: { + LOG(OrderMan, Info, "Native order {} has been attached to by {}", + key().value(), utils::pb::to_string(new_parent)); + + // Update FIX tags to reflect new parent's information + auto rq = me->services->route_order(UserID(new_parent.user_id()), + AccountID(new_parent.account_id()), + *me->sec); + + if (!rq.has_route) + { + auto msg = fmt::format("Cannot update order routing information: {}", rq.error_msg); + transition(me->state.status(), msg); + report_state(); + return; + } + + auto &native_state = get_native_state(me->state); + *native_state.mutable_routing_record() = rq.routing_record; + + // setup_ids(rq.routing_record.sender_compid()); + // init_fix(); + // if (has_exchange_order_id()) + // add_exchange_order_id_to_fix(); + //ICE is a little different with some of these tags, there is no manual order reported and the user is really + //not the user we are updating here. So we can update the BTS data but not reinit the fix + //We should not reinit the fix because we need to correctly cache the orig_clordid for cancels and updates subsequent to + //the first update. + + // Update user, account and manual order indicator based on new parent + me->definition.set_manual_order_indicator(api::orderman::ManualOrderIndicator::MANUAL_ORDER_INDICATOR_NO); + me->definition.set_user(new_parent.user()); + me->definition.set_user_id(new_parent.user_id()); + me->definition.set_account(new_parent.account()); + me->definition.set_account_id(new_parent.account_id()); + me->services->report_definition(me->definition); + report_state(); + break; + } + + case ORDER_STATUS_CANCELED: + case ORDER_STATUS_REJECTED_BY_SPARK: + case ORDER_STATUS_COMPLETED: + case ORDER_STATUS_ERROR: + break; + + case ORDER_STATUS_INACTIVE: + case OrderStatus_INT_MIN_SENTINEL_DO_NOT_USE_: + case OrderStatus_INT_MAX_SENTINEL_DO_NOT_USE_: + bts_assert(false); + break; + } +} + +OrderOpResult ICENativeOrder::on_cancel() +{ + auto scoped_log = me->services->log_scope("ICENativeOrder::on_cancel", key().value()); + auto result = OrderOpResult{}; + + switch (me->state.status()) + { + case ORDER_STATUS_WORKING: + { + auto &encoding = me->cancel_encoding; + auto cancel_clorid = get_new_clordid(); + me->cancel_clordid_bm = encoding.overwrite( me->cancel_clordid_bm, cancel_clorid, count_digits(cancel_clorid) ); + + me->services->queue_fix(*me->session_services, jetfix::MessageTypes::Cancel,me->cancel_encoding); + me->buffered_cancel = true; + LOG(OrderMan, Warn, + "ICE May not honor this cancel if we'd previously issued an unacknowledged cancelreplace " + "Buffering cancel to be safe", + key().value()); + break; + } + + case ORDER_STATUS_CANCELED: + break; + case ORDER_STATUS_REJECTED_BY_SPARK: + case ORDER_STATUS_COMPLETED: + case ORDER_STATUS_ERROR: + result.error = ORDER_ERROR_INVALID_OP; + result.error_msg = "Order is already terminated"; + break; + + case ORDER_STATUS_PENDING_WORKING: + { + me->buffered_cancel = true; + LOG(OrderMan, Warn, + "Cannot cancel order {} while in the pending working state. " + "Buffering cancel", + key().value()); + break; + } + case ORDER_STATUS_DISCONNECTED: + { + me->buffered_cancel = true; + LOG(OrderMan, Warn, + "Cannot cancel order {} while in the disconnected state. Buffering " + "cancel", + key().value()); + auto msg = "Order will be canceled when re-connected"; + result.error = ORDER_ERROR_GATEWAY_NOT_READY; + result.error_msg = msg; + transition(me->state.status(), msg); + break; + } + case ORDER_STATUS_INACTIVE: + case OrderStatus_INT_MIN_SENTINEL_DO_NOT_USE_: + case OrderStatus_INT_MAX_SENTINEL_DO_NOT_USE_: + bts_assert(false); + break; + } + report_state(); + return result; +} + +bool ICENativeOrder::is_old_exe_report(api::orderman::NativeOrderState &native_state, const NewExecutionReportEvent &er) const +{ + if (native_state.last_transact_time().size() != sizeof(mercury::TimeStampStr)) + return false; + + if (std::lexicographical_compare(native_state.last_transact_time().begin(), + native_state.last_transact_time().end(), + er.data->transact_time.value().begin(), + er.data->transact_time.value().end())) + return false; + + if (std::equal(native_state.last_transact_time().begin(), + native_state.last_transact_time().end(), + er.data->transact_time.value().begin(), + er.data->transact_time.value().end())) + return native_state.last_msg_seq_num() >= er.data->msg_seq_num; + + return true; +} + +void ICENativeOrder::on_exe_report(const NewExecutionReportEvent &er) +{ + auto scoped_log = me->services->log_scope("ICENativeOrder::on_exe_report", key().value()); + + auto &state = me->state; + auto &native_state = get_native_state(state); + if(er.data->exec_type == api::orderman::ExecType::EXEC_TYPE_TRADE_CANCEL || + er.data->exec_type == api::orderman::ExecType::EXEC_TYPE_TRADE_CORRECT) + { + LOG(OrderMan, Info, "ICENativeOrder ignoring Cancel or Correct execution report {} in order {}", er.data->msg_seq_num, key().value()); + return; + } + + // if we have a previous state we want to ensure this execution + // report was not already considered. + if (is_old_exe_report(native_state, er)) + { + LOG(OrderMan, Info, "Ignoring already seen execution report {} in order {}", er.data->msg_seq_num, key().value()); + return; + } + + // we do not care about spread leg fills for purposes of order + // state maintenance + if (er.data->leg_security) + { + LOG(OrderMan, Diag, "Ignoring leg execution report for order {}", key().value()); + return; + } + + native_state.set_last_msg_seq_num(er.data->msg_seq_num); + native_state.set_last_transact_time(to_str(er.data->transact_time)); + + auto external_status = map_external_status(er.data->order_status); + if (external_status != ORDER_STATUS_CANCELED) + { + state.set_avg_price(er.data->avg_price.value()); + state.set_filled_quantity(er.data->filled_quantity.value()); + state.set_last_price(er.data->last_price.value()); + //cache the new clordid + if(er.data->ice_cl_ord_id > 0) + { + auto clordid = er.data->ice_cl_ord_id; + me->update_orig_clordid_bm = me->update_encoding.overwrite( me->update_orig_clordid_bm, clordid, count_digits(clordid) ); + me->cancel_orig_clordid_bm = me->cancel_encoding.overwrite( me->cancel_orig_clordid_bm, clordid, count_digits(clordid) ); + } + } + + state.set_last_msg(er.data->msg); + + native_state.set_native_status(er.data->order_status); + native_state.set_native_type(er.data->order_type); + bool got_first_exchange_order_id = !has_exchange_order_id(); + if (got_first_exchange_order_id) + { + me->exchange_order_id = to_trim_str(er.data->exchange_order_id); + native_state.set_exchange_order_id(me->exchange_order_id); + add_exchange_order_id_to_fix(); + } + + if (!terminal_state(me->state.status())) + { + if(!isnan(er.data->price.value())) + state.set_price(er.data->price.value()); + state.set_quantity(er.data->quantity.value()); + } + + switch (me->state.status()) + { + // terminal states, nothing can change them + case ORDER_STATUS_COMPLETED: + case ORDER_STATUS_CANCELED: + me->buffered_cancel = false; + me->buffered_update.reset(); + break; + + // receiving an execution report does does not mean + // we are connected + case ORDER_STATUS_DISCONNECTED: + break; + + case ORDER_STATUS_PENDING_WORKING: + if (external_status == ORDER_STATUS_WORKING && + !me->services->is_gateway_connected(*me->session_services)) + transition(ORDER_STATUS_DISCONNECTED); + else + { + transition(external_status); + if (external_status == ORDER_STATUS_WORKING) + { + handle_buffered_state("first ack"); + } + } + break; + + case ORDER_STATUS_WORKING: + { + transition(external_status); + if(me->buffered_update) + { + auto [p, q] = me->buffered_update.value(); + if(er.data->price.value() == p.value() && er.data->quantity.value() == q.value()) + me->buffered_update.reset(); + if(me->buffered_tries++ > 10) + { + me->buffered_tries = 0; + me->buffered_update.reset(); + } + } + handle_buffered_state("working"); + } + break; + + case ORDER_STATUS_INACTIVE: + case ORDER_STATUS_ERROR: + case ORDER_STATUS_REJECTED_BY_SPARK: + case OrderStatus_INT_MIN_SENTINEL_DO_NOT_USE_: + case OrderStatus_INT_MAX_SENTINEL_DO_NOT_USE_: + bts_assert(false); + break; + } + report_state(); + + // we only need the exhange order id and the routing record + // reported once, since they are both immutable, so we clear them + // after their first report to make subsquent state updates faster + if (got_first_exchange_order_id) + { + native_state.clear_exchange_order_id(); + native_state.mutable_routing_record()->Clear(); + } +} + +void ICENativeOrder::handle_buffered_state(const std::string& condition) +{ + if (me->buffered_cancel) + { + LOG(OrderMan, Warn, + "Applying buffered cancel to order {}, on ", + condition, + key().value()); + on_cancel(); + } + else if (me->buffered_update.has_value()) + { + auto [p, q] = me->buffered_update.value(); + LOG(OrderMan, Warn, + "Applying buffered update ({}@{}) to order {}, on ", + condition, + q.value(),p.value(), key().value()); + on_update(p, q); + } +} + +void ICENativeOrder::on_cancel_reject(const orderman::CancelRejectEvent &cr) +{ + auto &state = me->state; + state.set_last_msg(cr.reject_msg); + LOG(OrderMan, Warn, "Cancel reject received for order {}", key().value()); + + switch (me->state.status()) + { + case ORDER_STATUS_COMPLETED: + case ORDER_STATUS_CANCELED: + case ORDER_STATUS_DISCONNECTED: + case ORDER_STATUS_PENDING_WORKING: + case ORDER_STATUS_WORKING: + break; + + case ORDER_STATUS_INACTIVE: + case ORDER_STATUS_ERROR: + case ORDER_STATUS_REJECTED_BY_SPARK: + case OrderStatus_INT_MIN_SENTINEL_DO_NOT_USE_: + case OrderStatus_INT_MAX_SENTINEL_DO_NOT_USE_: + bts_assert(false); + break; + } + report_state(); +} + +void ICENativeOrder::on_gateway_status(const GatewayStatusChangeEvent &ev) +{ + if (ev.is_connected) + { + switch (me->state.status()) + { + case ORDER_STATUS_COMPLETED: + case ORDER_STATUS_CANCELED: + case ORDER_STATUS_ERROR: + case ORDER_STATUS_PENDING_WORKING: + break; + case ORDER_STATUS_WORKING: + break; + case ORDER_STATUS_DISCONNECTED: + { + const auto &native_state = get_native_state(me->state); + transition(map_external_status(native_state.native_status())); + if (me->state.status() == ORDER_STATUS_WORKING) + { + handle_buffered_state("re-connection"); + } + break; + } + case ORDER_STATUS_INACTIVE: + case ORDER_STATUS_REJECTED_BY_SPARK: + case OrderStatus_INT_MIN_SENTINEL_DO_NOT_USE_: + case OrderStatus_INT_MAX_SENTINEL_DO_NOT_USE_: + bts_assert(false); + break; + } + } + else + { + switch (me->state.status()) + { + case ORDER_STATUS_COMPLETED: + case ORDER_STATUS_CANCELED: + case ORDER_STATUS_ERROR: + case ORDER_STATUS_DISCONNECTED: + case ORDER_STATUS_PENDING_WORKING: + break; + case ORDER_STATUS_WORKING: + transition(ORDER_STATUS_DISCONNECTED); + break; + case ORDER_STATUS_INACTIVE: + case ORDER_STATUS_REJECTED_BY_SPARK: + case OrderStatus_INT_MIN_SENTINEL_DO_NOT_USE_: + case OrderStatus_INT_MAX_SENTINEL_DO_NOT_USE_: + bts_assert(false); + break; + } + } + report_state(); +} + +void ICENativeOrder::update_working_position() +{ + if (me->sec) + { + auto current_position = working_position(); + auto previous_position = me->previous_working_position; + me->previous_working_position = current_position; + + if (current_position.value() > previous_position.value()) + { + me->services->limiter().add_to_working_position( *me->sec, AccountID(me->definition.account_id()), + me->definition.side(), Quantity(current_position.value() - previous_position.value()) ); + } + else if (previous_position.value() > current_position.value()) + { + me->services->limiter().remove_from_working_position( *me->sec, AccountID(me->definition.account_id()), + me->definition.side(), Quantity(previous_position.value() - current_position.value()) ); + } + } + else + LOG(OrderMan, Error, + "Order {} ignored for purposes of limit enforcing " + "because it references unknown security {} ", + key().value(), me->definition.security_key()); +} + +void ICENativeOrder::report_state() +{ + update_working_position(); + me->services->report_state(me->state); +} + +void ICENativeOrder::setup_ids(const std::string& sender_compid) +{ + me->sender_compid = std::atoi(sender_compid.c_str()); + int digs = count_digits(me->sender_compid); + me->id_scale = md::pow10(ice_orderid_sz()-digs); +} + + +int64_t ICENativeOrder::get_clordid(uint64_t bts_id) +{ + int64_t id = me->id_scale * std::max(me->sender_compid, 1) + (bts_id % me->id_scale); + return id; +} + +int64_t ICENativeOrder::get_new_clordid() +{ + return get_clordid(me->services->gen_id()); +} + +void ICENativeOrder::init_fix() +{ + bts_assert(me->sec); + + using namespace jetfix; + me->create_encoding = FIXFragment{}; + me->update_encoding = FIXFragment{}; + me->cancel_encoding = FIXFragment{}; + + const auto &def = me->definition; + const auto &sec = *me->sec; + bts_assert(sec.has_ice()); + const auto &rr = get_native_state(me->state).routing_record(); + + std::string memo(memo_sz(), ' '); + encode_memo(key(), SecurityKey(sec.id()), &memo); + + // initialize create FIX + auto &create_encoding = me->create_encoding; + + create_encoding.add(memo); + me->create_clordid_bm = create_encoding.add(get_clordid(0), ice_orderid_sz()); + create_encoding.add(std::atoi(rr.company_id().c_str())); + //create_encoding.add(""); //optional, unclear if we need + std::string cust = rr.cust_order_handing_inst(); + if(cust == "") + cust = "Y"; + create_encoding.add(cust); + create_encoding.add(rr.operator_id()); + create_encoding.add(rr.operator_id() + "|" + rr.authorized_trader()); + create_encoding.add("ISV-BTS|" + rr.authorized_group()); + create_encoding.add(rr.authorized_trader()); + create_encoding.add('1'); + create_encoding.add(sec.ice().exchange_id()); + create_encoding.add(def.side()); + me->create_transact_time_bm = create_encoding.add(datetime::FIX_timestamp(datetime::now())); + me->create_quantity_bm = create_encoding.add(0, quantity_sz()); + create_encoding.add(def.time_in_force()); + if(rr.account() != "") + create_encoding.add(rr.account()); + if(rr.clearing_firm() != "") + create_encoding.add(std::atoi(rr.clearing_firm().c_str())); + if(rr.account_code() != "") + create_encoding.add(rr.account_code()[0]); + if(rr.cti_code() != api::orderman::CTICode::CTI_CODE_UNDEFINED) + create_encoding.add( rr.cti_code() - '0' ); + if(rr.customer_account_ref_id() != "") + create_encoding.add(rr.customer_account_ref_id()); + + // initialize update FIX + auto &update_encoding = me->update_encoding; + update_encoding.add(memo); + me->update_orig_clordid_bm = update_encoding.add(get_clordid(0), ice_orderid_sz()); + update_encoding.add(std::atoi(rr.company_id().c_str())); + //update_encoding.add(""); //optional, unclear if we need + update_encoding.add(rr.operator_id()); + update_encoding.add(rr.operator_id() + "|" + rr.authorized_trader()); + update_encoding.add("ISV-BTS|" + rr.authorized_group()); + update_encoding.add(rr.authorized_trader()); + update_encoding.add('1'); + update_encoding.add(sec.ice().exchange_id()); + update_encoding.add(def.side()); + me->update_transact_time_bm = update_encoding.add(datetime::FIX_timestamp(datetime::now())); + me->update_quantity_bm = update_encoding.add(0, quantity_sz()); + me->update_clordid_bm = update_encoding.add(get_clordid(0), ice_orderid_sz()); + update_encoding.add(def.time_in_force()); + if(rr.account() != "") + update_encoding.add(rr.account()); + if(rr.clearing_firm() != "") + update_encoding.add(std::atoi(rr.clearing_firm().c_str())); + if(rr.account_code() != "") + update_encoding.add(rr.account_code()[0]); + if(rr.cti_code() != api::orderman::CTICode::CTI_CODE_UNDEFINED) + update_encoding.add( rr.cti_code() - '0' ); + if(rr.customer_account_ref_id() != "") + update_encoding.add(rr.customer_account_ref_id()); + + + // initialize cancel FIX + auto &cancel_encoding = me->cancel_encoding; + cancel_encoding.add(memo); + me->cancel_orig_clordid_bm = cancel_encoding.add(get_clordid(0), ice_orderid_sz()); + cancel_encoding.add(std::atoi(rr.company_id().c_str())); + cancel_encoding.add(rr.authorized_trader()); + cancel_encoding.add(sec.ice().exchange_id()); + cancel_encoding.add(def.side()); + me->cancel_clordid_bm = cancel_encoding.add(get_clordid(0), ice_orderid_sz()); + + // special tags + switch (def.custom_def().custom_case()) + { + case CustomOrderDefinition::kMarket: + // type + create_encoding.add(NATIVE_ORDER_TYPE_MARKET); + update_encoding.add(NATIVE_ORDER_TYPE_MARKET); + cancel_encoding.add(NATIVE_ORDER_TYPE_MARKET); + break; + + case CustomOrderDefinition::kLimit: + // type + create_encoding.add(NATIVE_ORDER_TYPE_LIMIT); + update_encoding.add(NATIVE_ORDER_TYPE_LIMIT); + cancel_encoding.add(NATIVE_ORDER_TYPE_LIMIT); + + // price + me->create_price_bm = create_encoding.add(0, me->price_encoding_precision, price_sz()); + me->update_price_bm = update_encoding.add(0, me->price_encoding_precision, price_sz()); + break; + + case CustomOrderDefinition::kStop: + { + // type + create_encoding.add(NATIVE_ORDER_TYPE_STOP); + update_encoding.add(NATIVE_ORDER_TYPE_STOP); + cancel_encoding.add(NATIVE_ORDER_TYPE_STOP); + + // stop px + auto stop_px = def.custom_def().stop().stop_price(); + auto encoded_stop_px = securities::encode_exchange_price(*me->sec, Price(stop_px)).value(); + create_encoding.add(encoded_stop_px, me->price_encoding_precision, -1); + break; + } + + case CustomOrderDefinition::kStopLimit: + { + // type + create_encoding.add(NATIVE_ORDER_TYPE_STOP_LIMIT); + update_encoding.add(NATIVE_ORDER_TYPE_STOP_LIMIT); + cancel_encoding.add(NATIVE_ORDER_TYPE_STOP_LIMIT); + + // stop px + auto stop_px = def.custom_def().stop_limit().stop_price(); + auto encoded_stop_px = securities::encode_exchange_price(*me->sec, Price(stop_px)).value(); + create_encoding.add(encoded_stop_px, me->price_encoding_precision, -1); + me->update_stop_px_bm = update_encoding.add(encoded_stop_px, me->price_encoding_precision, -1); + + // price + me->create_price_bm = create_encoding.add(0, me->price_encoding_precision, price_sz()); + me->update_price_bm = update_encoding.add(0, me->price_encoding_precision, price_sz()); + break; + } + default: + bts_assert(false); + break; + } +} + +void ICENativeOrder::add_exchange_order_id_to_fix() +{ + bts_assert(has_exchange_order_id()); + me->update_encoding.add(me->exchange_order_id); + me->cancel_encoding.add(me->exchange_order_id); +} + +bool ICENativeOrder::has_exchange_order_id() +{ + return me->exchange_order_id.size(); +} + +void ICENativeOrder::transition(api::orderman::OrderStatus new_status, std::string const &msg) +{ + transition(Info, new_status, msg); +} + +void ICENativeOrder::transition(log::Level level, api::orderman::OrderStatus new_status, std::string const &msg) +{ + const auto &from = OrderStatus_Name(me->state.status()); + const auto &to = OrderStatus_Name(new_status); + if (from != to) + LOG(OrderMan, level, "Transitioning order {} from {} to {}. Msg: {}", key().value(), from, to, msg); + me->state.set_status(new_status); + me->state.set_last_msg(msg); +} + +void ICENativeOrder::transition(api::orderman::OrderStatus new_status) +{ + const auto &from = OrderStatus_Name(me->state.status()); + const auto &to = OrderStatus_Name(new_status); + if (from != to) + LOG(OrderMan, Info, "Transitioning order {} from {} to {}", key().value(), from, to); + else + LOG(OrderMan, Diag, "Transitioning order {} from {} to {}", key().value(), from, to); + + me->state.set_status(new_status); +} + +void ICENativeOrder::transition_to_error(api::orderman::NativeOrderState::SubError sub_error, std::string const &msg) +{ + me->state.mutable_limit()->mutable_native_state()->set_sub_error_code(sub_error); + transition(ORDER_STATUS_ERROR, msg); +} + +Quantity ICENativeOrder::working_position() +{ + switch (me->state.status()) + { + + case ORDER_STATUS_WORKING: + case ORDER_STATUS_PENDING_WORKING: + case ORDER_STATUS_DISCONNECTED: + if (me->state.quantity() > me->state.filled_quantity()) + return Quantity(me->state.quantity() - me->state.filled_quantity()); + else + return Quantity(0); + + case ORDER_STATUS_REJECTED_BY_SPARK: + case ORDER_STATUS_ERROR: + case ORDER_STATUS_COMPLETED: + case ORDER_STATUS_CANCELED: + return Quantity(0); + + case ORDER_STATUS_INACTIVE: + case OrderStatus_INT_MIN_SENTINEL_DO_NOT_USE_: + case OrderStatus_INT_MAX_SENTINEL_DO_NOT_USE_: + bts_assert(false); + return Quantity(0); + } + bts_assert(false); + return Quantity(0); +} + +bool ICENativeOrder::terminal_state(api::orderman::OrderStatus status) const +{ + switch (status) + { + + case ORDER_STATUS_COMPLETED: + case ORDER_STATUS_CANCELED: + case ORDER_STATUS_ERROR: + case ORDER_STATUS_REJECTED_BY_SPARK: + return true; + + case ORDER_STATUS_DISCONNECTED: + case ORDER_STATUS_PENDING_WORKING: + case ORDER_STATUS_WORKING: + return false; + + case OrderStatus_INT_MIN_SENTINEL_DO_NOT_USE_: + case OrderStatus_INT_MAX_SENTINEL_DO_NOT_USE_: + case ORDER_STATUS_INACTIVE: + bts_assert(false); + return false; + break; + } + bts_assert(false); + return false; +} + +api::orderman::NativeOrderState& ICENativeOrder::get_native_state(api::orderman::OrderState &st) +{ + switch (st.custom_case()) + { + case OrderState::kMarket: + return *st.mutable_market()->mutable_native_state(); + case OrderState::kLimit: + return *st.mutable_limit()->mutable_native_state(); + case OrderState::kStop: + return *st.mutable_stop()->mutable_native_state(); + case OrderState::kStopLimit: + return *st.mutable_stop_limit()->mutable_native_state(); + default: + break; + } + + // just to appease compiler + bts_assert(false); + return *st.mutable_stop_limit()->mutable_native_state(); +} + +void ICENativeOrder::ignore_or_cancel(std::string const &msg, api::orderman::OrderError error, OrderOpResult *out) +{ + if (parent_key() == null_order_key()) + transition(me->state.status(), msg); + else + { + me->state.set_last_msg(msg); + on_cancel(); + } + *out = {error, msg}; +} + +} // namespace orderman +} // namespace bts diff --git a/src/lib/orderman/orders/ICENativeOrder.h b/src/lib/orderman/orders/ICENativeOrder.h new file mode 100644 index 000000000..9a581a69a --- /dev/null +++ b/src/lib/orderman/orders/ICENativeOrder.h @@ -0,0 +1,83 @@ +#pragma once + +#include "lib/orderman/orders/NativeOrderBase.h" +#include "lib/tracing/PerfLog.h" +#include + +namespace bts { +namespace orderman { +struct RoutingRecord; +class NewExecutionReportEvent; +class CancelRejectEvent; +class GatewayStatusChangeEvent; +class ExecutionReport; + +class ICENativeOrder : public NativeOrderBase { + public: + ICENativeOrder(); + ~ICENativeOrder() override; + + const api::orderman::OrderDefinition& definition() const override; + void on_init_new(const api::orderman::OrderDefinition&, + const PreActivationOrderServices&) override; + void on_init_recovered(const api::orderman::OrderRecord&, + const PreActivationOrderServices &) override; + + OrderOpResult on_activate(Price, Quantity, OrderServices &) override; + OrderOpResult on_update(Price, Quantity) override; + OrderOpResult on_update(const api::orderman::Update&) override; + OrderOpResult on_detach() override; + OrderOpResult on_cancel() override; + void on_attached(const api::orderman::OrderDefinition &new_parent) override; + + api::orderman::OrderDefinition + on_init_from_exe_report(const ExecutionReport &, const PreActivationOrderServices &) override; + void ignore_or_cancel(const std::string&, api::orderman::OrderError, OrderOpResult* out); + + protected: + void warm_up_activate(OrderServicesData &, OrderServices &) override; + void init_recovered_common(const api::orderman::OrderRecord &, + const PreActivationOrderServices &); + void handle_buffered_state(const std::string& condition); + + + private: + struct PImpl; + std::unique_ptr me; + void on_exe_report(const NewExecutionReportEvent &); + void on_cancel_reject(const CancelRejectEvent &); + void on_gateway_status(const GatewayStatusChangeEvent &); + void report_state(); + OrderOpResult _on_activate(log::Level reject_level, + log::Level transition_level, Price p, Quantity q, + bts::orderman::OrderServices &services); + void init_fix(); + void add_exchange_order_id_to_fix(); + + void transition(log::Level, + api::orderman::OrderStatus new_state, + const std::string& msg); + void transition(api::orderman::OrderStatus new_state, + const std::string& msg); + void transition(api::orderman::OrderStatus new_state); + void transition_to_error(api::orderman::NativeOrderState::SubError sub_error, + const std::string& msg); + + void update_working_position(); + bool has_exchange_order_id(); + Quantity working_position() ; + bool is_old_exe_report(api::orderman::NativeOrderState &, + const NewExecutionReportEvent &) const; + bool terminal_state(api::orderman::OrderStatus status) const; + api::orderman::NativeOrderState & + get_native_state(api::orderman::OrderState &); + + int64_t get_new_clordid(); + int64_t get_clordid(uint64_t); + void setup_ids(const std::string& sender_compid); + void generate_update_ids(const std::string&); + void generate_cancel_ids(const std::string&); +}; + +} // namespace orderman +} // namespace bts diff --git a/src/lib/orderman/orders/NativeOrderBase.h b/src/lib/orderman/orders/NativeOrderBase.h new file mode 100644 index 000000000..158fb5424 --- /dev/null +++ b/src/lib/orderman/orders/NativeOrderBase.h @@ -0,0 +1,22 @@ +#pragma once + +#include "lib/api/services/OrderService.pb.h" +#include "lib/orderman/orders/Order.h" + +namespace bts { +namespace orderman { +class ExecutionReport; + +class NativeOrderBase : public Order { + public: + NativeOrderBase() = default; + ~NativeOrderBase() override {}; + + virtual api::orderman::OrderDefinition const &definition() const = 0; + api::orderman::OrderDefinition + virtual on_init_from_exe_report(ExecutionReport const &, const PreActivationOrderServices &) = 0; +}; + +} // namespace orderman +} // namespace bts + diff --git a/src/lib/orderman/orders/algo/IAlgoServices.h b/src/lib/orderman/orders/algo/IAlgoServices.h index 2222c90e1..77cf2450a 100644 --- a/src/lib/orderman/orders/algo/IAlgoServices.h +++ b/src/lib/orderman/orders/algo/IAlgoServices.h @@ -20,7 +20,7 @@ class IAlgoServices : public IBeforeLaunchAlgoServices { // Returns the filled quantity for this algo virtual Quantity total_fills() const = 0; - // Returns wheter the algo is in the process of shutting down (due + // Returns whether the algo is in the process of shutting down (due // to a cancel, an error, or just having completed its work). When // an algo is shutting down any requests to create, update or // cancel child orders will be ignored. diff --git a/src/lib/orderman/orders/algo/algo_orders/PyramidAlgo.cpp b/src/lib/orderman/orders/algo/algo_orders/PyramidAlgo.cpp new file mode 100644 index 000000000..09facf535 --- /dev/null +++ b/src/lib/orderman/orders/algo/algo_orders/PyramidAlgo.cpp @@ -0,0 +1,569 @@ +#include "lib/orderman/orders/algo/algo_orders/PyramidAlgo.h" + +#include "lib/domain_model/Security.h" +#include "lib/orderman/order_factory/IOrderFactory.h" +#include "lib/orderman/orderman_utils/Book.h" +#include "lib/orderman/orderman_utils/orderman_utils.h" +#include "lib/orderman/orders/algo/IAlgoServices.h" +#include "lib/orderman/orders/spreader_orders/spreader/SpreaderUtils.h" +#include "lib/os/os.h" +#include "lib/domain_model/Security.h" +#include "lib/utils/PBUtils.h" +#include + +namespace bts { +namespace orderman { + +using namespace api::orderman; + +enum class PyramidAlgo::Status { WAITING_FOR_MD, WAITING_FOR_MD_ATTACH, WORKING, DONE }; +enum class PyramidAlgo::ExitPath { CANCEL, ERR, COMPLETE }; + +struct LegData{ + const size_t index; + const SecurityKey sec_key; + const double tick_size; + const api::Side side; //trading side + ChildTemplate* order_template = nullptr; + OrderKey order_key; + ChildState order_state; + bool recv_book; + Book book; + Quantity top_delete; // this will be DONTCARE for covers + Quantity cover_payup;// this will be DONTCARE for top + LegData (size_t index_a, + SecurityKey sec_key_a, + double tick_size_a, + api::Side side_a, + ChildTemplate* order_template_a, + Quantity qty) + : + index (index_a), + sec_key (sec_key_a), + tick_size(tick_size_a), + side (side_a), + order_template (order_template_a), + order_key(null_order_key()), + order_state(), + recv_book(false), + book(), + top_delete(index == 0 ? qty.value() : 0), + cover_payup(index == 0 ? 0 : qty.value()) + { + } + const BookSide& opp_md() const{ + return book.side(SpreaderUtils::invert_side(side)); + } + bool is_top_spread() const{ + return index == 0; + } + void set_book (const Book& book_a){ + recv_book = true; + book = book_a; + } +}; + +struct PyramidAlgo::PImpl { + Status state = Status::WAITING_FOR_MD; + IAlgoServices* algo_services = nullptr; + IBeforeLaunchAlgoServices* bl_algo_services; + PyramidAlgo::PyramidAlgoParams params; + std::vector legs; + api::Side side; + Price out_price; + Quantity out_qty; + PImpl(){ + } + LegData* find_leg_from_sec (SecurityKey sec_key){ + for (auto& l : legs) + if(l.sec_key == sec_key) + return &l; + return nullptr; + } + LegData* find_leg_from_ord_key (OrderKey ord_key){ + for (auto& l : legs) + if(l.order_key == ord_key) + return &l; + return nullptr; + } +}; + +PyramidAlgo::PyramidAlgo() : me(new PImpl()) {} +PyramidAlgo::~PyramidAlgo() {} +IAlgoServices& PyramidAlgo::_algo() const { return *me->algo_services; } + +// initialization ///////////////////////////////////////////////////////////// +/////////////////////////////////////////////////////////////////////////////// +OrderOpResult PyramidAlgo::do_before_launch(IBeforeLaunchAlgoServices& serv) { + me->bl_algo_services = &serv; + auto& def = serv.definition(); + if (def.has_custom_def() && def.custom_def().has_dynamic()){ + auto conv_res = _get_creation_params(def.custom_def(), serv); + if (std::get<1>(conv_res)){ + // params are good + me->params = std::get<0>(conv_res); + me->side = def.side(); + // top spread + auto* t = _create_top_order_template(serv); + double tick_size = serv.lookup_security(me->params.top_spread)->contract_spec().low_tick(); + me->legs.emplace_back(0, me->params.top_spread, tick_size, def.side(), t, + me->params.top_delete); + // covers + size_t i = 0; + for (SecurityKey key : me->params.cover_spreads){ + auto side = SpreaderUtils::invert_side(def.side()); + auto* t = _create_hedge_order_template(serv,i); + double tick_size = serv.lookup_security(key)->contract_spec().low_tick(); + me->legs.emplace_back(i+1, key, tick_size, side, t, me->params.cover_payups[i]); + i++; + } + } + else + return {OrderError::ORDER_ERROR_INVALID_PARAMETER, + std::get<2>(conv_res)}; + } + else + return {OrderError::ORDER_ERROR_INVALID_PARAMETER, + std::string("Missing parameters")}; + return {}; +} +void PyramidAlgo::do_launch(IAlgoServices& algo_services) { + me->algo_services = &algo_services; + me->out_price = algo_services.price(); + me->out_qty = algo_services.quantity(); + for (const LegData& l : me->legs) + algo_services.subscribe_book(l.sec_key); +} +void PyramidAlgo::do_launch_attached(IAlgoServices& algo_services, + OrderKey key, + ChildState c) { + me->algo_services = &algo_services; + if(c.quantity != algo_services.quantity() || + !bts::securities::is_same(c.price, algo_services.price())){ + _cxl_and_end(ExitPath::ERR, + "Pyramid price or quantity did not match the working order", + true); + } + else{ + me->legs[0].order_key = key; + me->out_price = c.price; + me->out_qty = c.quantity; + for (const LegData& l : me->legs) + algo_services.subscribe_book(l.sec_key); + _switch_state(PyramidAlgo::Status::WAITING_FOR_MD_ATTACH); + on_child_change(key,c); + } +} +// updates //////////////////////////////////////////////////////////////////// +/////////////////////////////////////////////////////////////////////////////// +void PyramidAlgo::do_update(const api::orderman::Update& update) { + LOG(OrderMan, Diag, "PyramidAlgo::do_update order:{} state:{} " + "receiving update: {}", + _algo().order_key().value(), + static_cast(me->state), + utils::pb::to_string(update)); + auto conv_res = _get_update_params(update, me->params); + if (!std::get<1>(conv_res)){ + LOG(OrderMan, Warn, "PyramidAlgo::do_update order:{} bad update, ignoring", + _algo().order_key().value()); + return; + } + const PyramidAlgoParams& algo_params = std::get<0>(conv_res); + Price price = Price(update.price()); + Quantity qty = Quantity(update.quantity()); + switch(me->state){ + case PyramidAlgo::Status::WAITING_FOR_MD: + case PyramidAlgo::Status::WAITING_FOR_MD_ATTACH: + LOG(OrderMan, Info, "PyramidAlgo::do_update order:{} state:{} ignored", + _algo().order_key().value(),static_cast(me->state)); + break; + case PyramidAlgo::Status::WORKING:{ + bool pq_changed = _copy_update(algo_params, price, qty); + if(_target_reached()){ + _cxl_and_end (PyramidAlgo::ExitPath::CANCEL, "Target reached"); + return; + } + if (_working_state_met()){ + if(pq_changed) + _modify_top_spread_order(); + } + else + _cxl_and_end(PyramidAlgo::ExitPath::CANCEL, + "Order cancelled (Cover prices not met)"); + break; + } + case PyramidAlgo::Status::DONE: + break; + } +} +// callbacks ////////////////////////////////////////////////////////////////// +/////////////////////////////////////////////////////////////////////////////// +void PyramidAlgo::on_book(SecurityKey key, const Book& book) { + LegData* leg_data = me->find_leg_from_sec(key); + if(leg_data != nullptr){ + leg_data->set_book(book); + _analyze_md_and_act(); + } +} +// this only gets called on am explicit algo cancel +// from the outside +IAlgoOrder::OnCancelResponse PyramidAlgo::on_cancel(){ + // cancel top spread, let hedges work, cancel algo + switch(me->state){ + case PyramidAlgo::Status::WAITING_FOR_MD: + case PyramidAlgo::Status::WAITING_FOR_MD_ATTACH: + case PyramidAlgo::Status::WORKING: + _cxl_and_end(PyramidAlgo::ExitPath::CANCEL,"Algo cancelled"); + break; + case PyramidAlgo::Status::DONE: + break; + } + return IAlgoOrder::OnCancelResponse::IGNORE; +} +void PyramidAlgo::on_child_change(OrderKey key, const ChildState &st) { + LOG(OrderMan, Info, + "PyramidAlgo::on_child_change key:{} child_key:{} state:{} status:{} " + "total_fills:{} last_price:{}", + _algo().order_key(), + key, + static_cast(me->state), + static_cast(st.status), + st.total_fills, + st.last_price); + LegData* leg = me->find_leg_from_ord_key (key); + if (me->legs[0].order_key == key){ + switch(me->state){ + case PyramidAlgo::Status::WAITING_FOR_MD: + bts_assert_msg (false, "No child orders"); + break; + case PyramidAlgo::Status::WAITING_FOR_MD_ATTACH: + leg->order_state = st; + _algo().update_fill_state(st.total_fills, st.avg_price); + _check_for_completion(); + break; + case PyramidAlgo::Status::WORKING: + leg->order_state = st; + _algo().update_fill_state(st.total_fills, st.avg_price); + _send_hedges(st.last_fills); + _check_for_completion(); + break; + case PyramidAlgo::Status::DONE: + break; + } + } +} + +/////////////////////////////////////////////////////////////////////////////// +// stateless conversion /////////////////////////////////////////////////////// + +PyramidAlgo::PyramidAlgoParams PyramidAlgo::_convert_params ( + const CustomOrderDefinition& cod){ + const DynamicOrderDefinition& def = cod.dynamic(); + const auto& params = def.order_params(); + PyramidAlgoParams rv; + // top contract + auto r = get_if(params, "Top Spread"); + rv.top_spread = r.has_value() ? SecurityKey(r.value()) : SecurityKey(0); + // top delete + r = get_if(params, "Delete"); + rv.top_delete = Quantity(r.has_value() ? r.value() : UINT32_MAX); + for (size_t i = 0; i < 5; i++){ + auto idx = std::to_string(i+1); + auto b = get_if(params, "Spread " + idx); + if (b.has_value() && b.value() != 0){ + rv.cover_spreads.emplace_back(b.value()); + auto pu = get_if(params, "Spread " + idx + " Payup"); + rv.cover_payups.emplace_back(pu.has_value() ? pu.value() : UINT32_MAX); + } + } + return rv; +} + +std::tuple +PyramidAlgo::_get_creation_params(const CustomOrderDefinition& cod, + IBeforeLaunchAlgoServices& serv){ + PyramidAlgo::PyramidAlgoParams rv = _convert_params(cod); + // error checking + if(rv.top_spread.value() != serv.security().id()) + return std::make_tuple(rv,false, + "Algo can only be launched from the Top Spread"); + if(rv.cover_spreads.size() < 2) + return std::make_tuple(rv,false, + "Not enough cover spreads"); + if (rv.cover_spreads[0].value() == 0 || + rv.cover_spreads[1].value() == 0) + return std::make_tuple(rv,false, + "Not enough cover spreads"); + return std::make_tuple(rv,true,std::string("")); +} + +std::tuple +PyramidAlgo::_get_update_params(const Update& update, + const PyramidAlgoParams& curr_params){ + if(update.has_custom() && update.custom().has_dynamic()){ + // means there is a params change + PyramidAlgoParams new_params = _convert_params(update.custom()); + if(new_params.top_spread != curr_params.top_spread || + new_params.cover_spreads.size() != curr_params.cover_spreads.size()) + return std::make_tuple(new_params,false,"Contracts can not be updated"); + for (size_t i=0; i < curr_params.cover_spreads.size(); i++) + if(new_params.cover_spreads[i] != curr_params.cover_spreads[i]) + return std::make_tuple(new_params,false,"Contracts can not be updated"); + return std::make_tuple(new_params,true,std::string("")); + } + else + return std::make_tuple(curr_params,true,std::string("")); +} + +/////////////////////////////////////////////////////////////////////////////// +// order template creation///////////////////////////////////////////////////// + +ChildTemplate* PyramidAlgo::_create_top_order_template ( + IBeforeLaunchAlgoServices& serv) const{ + OrderDefinition order_def = serv.definition(); + order_def.set_security_key (me->params.top_spread.value()); + order_def.set_side(me->side); + order_def.mutable_custom_def()->mutable_limit()->MergeFrom ( + LimitOrderDefinition()); + return &(serv.make_child_template(order_def)); +} + +ChildTemplate* PyramidAlgo::_create_hedge_order_template ( + IBeforeLaunchAlgoServices& serv, + size_t hedge_idx) const{ + OrderDefinition order_def = serv.definition(); + order_def.set_attach_target_order_key(null_order_key().value()); + order_def.set_security_key (me->params.cover_spreads[hedge_idx].value()); + order_def.set_side(SpreaderUtils::invert_side(me->side)); + order_def.set_sub_type(ORDER_SUB_TYPE_HEDGING_LEG); + LeanOrderDefinition lean; + lean.mutable_cross_threshold()->set_absolute(me->params.cover_payups[hedge_idx].value()); + lean.set_max_chase_ticks(1); + lean.set_trade_toward_behavior(LeanOrderDefinition::NO_CHANGE); + lean.mutable_child()->mutable_limit()->MergeFrom ( + LimitOrderDefinition()); + order_def.mutable_custom_def()->mutable_lean()->MergeFrom (lean); + return &(serv.make_child_template(order_def)); +} + +void PyramidAlgo::_analyze_md_and_act(){ + switch(me->state){ + case PyramidAlgo::Status::WAITING_FOR_MD: + if(!_md_recvd()) + return; + // received data for the first time + if (_working_state_met()){ + // enter order, and switch state + LegData& top_leg = me->legs[0]; + // this is the only place top spread order is sent (once) + top_leg.order_key = _algo().launch_child( + *top_leg.order_template, + me->out_price, + me->out_qty); + LOG(OrderMan, Info,"PyramidAlgo::_analyze_md_and_act launching order key:{} " + "child_key:{} price:{} qty:{}", + _algo().order_key(), + top_leg.order_key, + me->out_price, + me->out_qty); + _switch_state(PyramidAlgo::Status::WORKING); + } + else + _cxl_and_end(PyramidAlgo::ExitPath::CANCEL, + "Order cancelled (Cover prices not met)"); + break; + case PyramidAlgo::Status::WAITING_FOR_MD_ATTACH: + if(!_md_recvd()) + return; + // received data for the first time + if (_working_state_met()) + _switch_state(PyramidAlgo::Status::WORKING); + else + _cxl_and_end(PyramidAlgo::ExitPath::CANCEL, + "Order detached (Cover prices not met)", + true); + break; + case PyramidAlgo::Status::WORKING: + if (!_working_state_met()) + _cxl_and_end(PyramidAlgo::ExitPath::CANCEL, + "Order cancelled (Cover prices not met)"); + break; + case PyramidAlgo::Status::DONE: + break; + } +} + +void PyramidAlgo::_switch_state(Status new_state){ + switch(new_state){ + case (PyramidAlgo::Status::WAITING_FOR_MD): + bts_assert(false); + break; + case (PyramidAlgo::Status::WAITING_FOR_MD_ATTACH): + bts_assert(me->state == PyramidAlgo::Status::WAITING_FOR_MD); + break; + case (PyramidAlgo::Status::WORKING): + bts_assert(me->state == PyramidAlgo::Status::WAITING_FOR_MD || + me->state == PyramidAlgo::Status::WAITING_FOR_MD_ATTACH); + break; + case (PyramidAlgo::Status::DONE): + bts_assert( + // when an order is entered and the criteria is not met + me->state == PyramidAlgo::Status::WAITING_FOR_MD || + // when an order is attached and the market conditions are not met + me->state == PyramidAlgo::Status::WAITING_FOR_MD_ATTACH || + // when the child order gets into an unexpected state + me->state == PyramidAlgo::Status::WORKING); + break; + } + LOG(OrderMan, Info,"PyramidAlgo::_switch_state key:{} {}->{} ", + _algo().order_key(), + static_cast(me->state), + static_cast(new_state)); + me->state = new_state; +} + +void PyramidAlgo::_send_hedges (Quantity filled_qty){ + if (filled_qty.value() > 0) + for (auto& l : me->legs) + if(!l.is_top_spread()){ + const auto& inside_md = l.opp_md()[0]; + Price hedge_price = Price(inside_md.price()); + uint32_t inside_md_qty = inside_md.size(); + if (l.cover_payup.value() != 0 && + inside_md_qty > l.cover_payup.value()) + hedge_price = securities::worsen_price(l.side, hedge_price, Price(l.tick_size)); + // check payup + l.order_key = _algo().launch_child( + *l.order_template, + hedge_price, + filled_qty); + _algo().detach_child(l.order_key); + } +} + +void PyramidAlgo::_modify_top_spread_order(){ + Update child_update; + child_update.set_price(me->out_price.value()); + child_update.set_quantity(me->out_qty.value()); + LOG(OrderMan, Info,"PyramidAlgo::_modify_top_spread_order modifying order key:{} " + "child_key:{} price:{} qty:{}", + _algo().order_key(), + me->legs[0].order_key, + me->out_price, + me->out_qty); + _algo().update_child(me->legs[0].order_key, child_update); +} +// copies the params, price and qty to the state +// retuns true is the price or qty has changed +bool PyramidAlgo::_copy_update(const PyramidAlgoParams& algo_params, + Price price, Quantity qty){ + // copy input params (which are stored seperately) + me->params.top_delete = algo_params.top_delete; + size_t cov_idx = 0; + for (Quantity payup : algo_params.cover_payups) + me->params.cover_payups[cov_idx++] = payup; + // copy leg data + cov_idx = 0; + me->legs[0].top_delete = algo_params.top_delete; + for (Quantity pu : algo_params.cover_payups){ + LegData& cover_leg = me->legs[cov_idx+1]; + if(cover_leg.cover_payup != pu){ + cover_leg.cover_payup = pu; + cover_leg.order_template = _create_hedge_order_template( + *me->bl_algo_services, + cov_idx); + } + cov_idx++; + } + // copy price and qty (returns true is they have changed) + bool pq_changed = me->out_qty.value() != qty.value() || + !bts::securities::is_same(me->out_price, price); + me->out_price = price; + me->out_qty = qty; + return pq_changed; +} +// if the top spread order is in a final state, the algo +// algo goes to the final state +void PyramidAlgo::_check_for_completion(){ + const ChildState& top_ord = me->legs[0].order_state; + switch (top_ord.status){ + case (ChildStatus::WORKING): + break; + case (ChildStatus::COMPLETED): + _cxl_and_end(PyramidAlgo::ExitPath::ERR,""); + break; + case (ChildStatus::CANCELED): + _cxl_and_end(PyramidAlgo::ExitPath::ERR, + "Top spread unexpectedly cancelled"); + break; + } +} + +void PyramidAlgo::_cxl_and_end(PyramidAlgo::ExitPath exit_path, + const std::string& msg, + bool detach_top){ + if(me->legs[0].order_key != null_order_key()){ + const ChildState& top_ord_state = me->legs[0].order_state; + if(top_ord_state.status == ChildStatus::WORKING){ + if(detach_top) + _algo().detach_child(me->legs[0].order_key); + else + _algo().cancel_child(me->legs[0].order_key); + } + } + switch(exit_path){ + case PyramidAlgo::ExitPath::ERR: + _algo().terminate_error(msg); + break; + case PyramidAlgo::ExitPath::CANCEL: + _algo().cancel(msg); + break; + case PyramidAlgo::ExitPath::COMPLETE: + _algo().complete(); + break; + } + _switch_state(PyramidAlgo::Status::DONE); +} +// true the out qty is changed it be below +// the filled qty +bool PyramidAlgo::_target_reached() const{ + return me->legs[0].order_state.total_fills.value() >= + me->out_qty.value(); +} + +bool PyramidAlgo::_md_recvd() const{ + for (const LegData& l : me->legs) + if(!l.recv_book) + return false; + return true; +} + +bool PyramidAlgo::_working_state_met() const{ + double cover_sum = 0.0; + size_t i = 0; + for (const auto& l : me->legs) + if (i++ > 0) + cover_sum += l.opp_md()[0].price(); + bool rv = !os::is_nan(cover_sum) && + ((me->side == api::Side::SIDE_BUY && + me->out_price.value() <= cover_sum) || + (me->side == api::Side::SIDE_SELL && + me->out_price.value() >= cover_sum)); + if (!rv && !os::is_nan(cover_sum)){ + // price condition is not met, but if the + // cum qty at the top leg price is > threashold + // then the condition is met () + const auto& top_leg = me->legs[0]; + Quantity cumm_level_qty; + SpreaderUtils::find_qty_upto_price_level(top_leg.book, + top_leg.side, + Price(me->out_price), + 0.0, + cumm_level_qty); + rv = cumm_level_qty.value() > top_leg.top_delete.value(); + } + return rv; +} + +} // namespace orderman +} // namespace bts diff --git a/src/lib/orderman/orders/algo/algo_orders/PyramidAlgo.h b/src/lib/orderman/orders/algo/algo_orders/PyramidAlgo.h new file mode 100644 index 000000000..c24ea468f --- /dev/null +++ b/src/lib/orderman/orders/algo/algo_orders/PyramidAlgo.h @@ -0,0 +1,74 @@ +#pragma once + +#include "lib/orderman/orders/algo/IAlgoOrder.h" + +namespace bts { +namespace orderman { + +class ChildTemplate; +class LegData; + +class PyramidAlgo : public IAlgoOrder { +public: + PyramidAlgo(); + ~PyramidAlgo(); + struct PyramidAlgoParams{ + // first one is the top + SecurityKey top_spread; + Quantity top_delete; + std::vector cover_spreads; + std::vector cover_payups; + }; +public: + OrderOpResult do_before_launch(IBeforeLaunchAlgoServices &) override; + void do_launch(IAlgoServices &) override; + void do_launch_attached(IAlgoServices &, OrderKey, ChildState) override; + void do_update(api::orderman::Update const &) override; + void on_book(SecurityKey, Book const &) override; + void on_child_change(OrderKey, ChildState const &) override; + IAlgoOrder::OnCancelResponse on_cancel() override; + +private: + enum class Status; + enum class ExitPath; + + struct PImpl; + std::unique_ptr me; +private: + IAlgoServices& _algo() const ; + // converts parameters from the CustomOrderDefinition::dynamic + // to PyramidAlgoParams + static PyramidAlgoParams _convert_params ( + const bts::api::orderman::CustomOrderDefinition&); + static std::tuple _get_creation_params( + const bts::api::orderman::CustomOrderDefinition&, + IBeforeLaunchAlgoServices&); + static std::tuple _get_update_params( + const api::orderman::Update&, + const PyramidAlgoParams& curr_params); + ChildTemplate* _create_top_order_template (IBeforeLaunchAlgoServices&) const; + ChildTemplate* _create_hedge_order_template (IBeforeLaunchAlgoServices&, + size_t) const; + // acts on md, cognizant of the state + void _analyze_md_and_act(); + void _switch_state (Status new_status); + void _send_hedges (Quantity filled_qty); + void _modify_top_spread_order(); + // returns true is the price, qty changes + bool _copy_update(const PyramidAlgoParams& algo_params, + Price price, Quantity qty); + void _check_for_completion(); + // cxl's the working top leg order (if any), detaches + // all hedges. if is_error then terminates algo, is + // is_cxl then cxl's algo, otherwise completes it + void _cxl_and_end(ExitPath exit_path, + const std::string& err_txt, + bool detach_top = false); + bool _target_reached() const; + // true is all legs have recvd md + bool _md_recvd() const; + bool _working_state_met() const; +}; + +} // namespace orderman +} // namespace bts diff --git a/src/lib/orderman/orders/algo/algo_orders/RegisterDorman.cpp b/src/lib/orderman/orders/algo/algo_orders/RegisterDorman.cpp index 0125d4e25..70cecbb4b 100644 --- a/src/lib/orderman/orders/algo/algo_orders/RegisterDorman.cpp +++ b/src/lib/orderman/orders/algo/algo_orders/RegisterDorman.cpp @@ -3,7 +3,11 @@ #include "lib/api/services/OrderService.pb.h" #include "lib/logging/logging.h" #include "lib/orderman/orderman_utils/orderman_utils.h" +#include "lib/config/protos/toggles.pb.h" +#include "lib/config/config.h" #include "DormanAlgo.h" +#include "PyramidAlgo.h" +#include "TASAlgo.h" using namespace bts; using namespace api::orderman; @@ -11,7 +15,7 @@ using namespace orderman::dorman; using namespace orderman; namespace bts { -namespace orderman { +namespace orderman { namespace dorman { template @@ -83,7 +87,7 @@ Result to_dorman_params(const api::orderman::DynamicOr return {res}; } -extern "C" void init_spark_plugin(bts::SparkServices &srv) { +extern "C" void init_dorman_plugin(bts::SparkServices &srv) { DynamicOrderDescriptor join_desc; join_desc.set_dynamic_order_type("Dorman"); @@ -115,4 +119,55 @@ extern "C" void init_spark_plugin(bts::SparkServices &srv) { srv.register_dynamic_algo_order(join_desc, []() { return std::make_unique(); }); } +extern "C" void init_pyramid_plugin(bts::SparkServices& srv) { + DynamicOrderDescriptor join_desc; + + join_desc.set_dynamic_order_type("Pyramid"); + + *join_desc.add_param_desc() = param_security_key("Top Spread"); + *join_desc.add_param_desc() = param_uint32("Delete"); + *join_desc.add_param_desc() = param_security_key("Spread 1"); + *join_desc.add_param_desc() = param_uint32("Spread 1 Payup"); + *join_desc.add_param_desc() = param_order("Spread 1 Order"); + *join_desc.add_param_desc() = param_security_key("Spread 2"); + *join_desc.add_param_desc() = param_uint32("Spread 2 Payup"); + *join_desc.add_param_desc() = param_order("Spread 2 Order"); + *join_desc.add_param_desc() = param_security_key("Spread 3"); + *join_desc.add_param_desc() = param_uint32("Spread 3 Payup"); + *join_desc.add_param_desc() = param_order("Spread 3 Order"); + *join_desc.add_param_desc() = param_security_key("Spread 4"); + *join_desc.add_param_desc() = param_uint32("Spread 4 Payup"); + *join_desc.add_param_desc() = param_order("Spread 4 Order"); + *join_desc.add_param_desc() = param_security_key("Spread 5"); + *join_desc.add_param_desc() = param_uint32("Spread 5 Payup"); + *join_desc.add_param_desc() = param_order("Spread 5 Order"); + + srv.register_dynamic_algo_order(join_desc, []() { return std::make_unique(); }); +} + +extern "C" void init_tas_plugin(bts::SparkServices& srv) { + DynamicOrderDescriptor tas_desc; + + tas_desc.set_dynamic_order_type("TAS"); + + size_t n_child_ords = 8; + for (size_t i=0; i< n_child_ords; i++){ + auto idx = std::to_string(i+1); + *tas_desc.add_param_desc() = param_double("Order " + idx + " Price"); + *tas_desc.add_param_desc() = param_uint32("Order " + idx + " Quantity"); + *tas_desc.add_param_desc() = param_bool("Order " + idx + " Buy?"); + } + srv.register_dynamic_algo_order(tas_desc, []() { return std::make_unique(); }); +} + +extern "C" void init_spark_plugin(bts::SparkServices &srv) { + auto dyn_algo_config = config::get(); + if(dyn_algo_config.enable_dorman()) + init_dorman_plugin(srv); + if(dyn_algo_config.enable_pyramid()) + init_pyramid_plugin(srv); + if(dyn_algo_config.enable_tas()) + init_tas_plugin(srv); +} + }}} \ No newline at end of file diff --git a/src/lib/orderman/orders/algo/algo_orders/TASAlgo.cpp b/src/lib/orderman/orders/algo/algo_orders/TASAlgo.cpp new file mode 100644 index 000000000..f167cdca0 --- /dev/null +++ b/src/lib/orderman/orders/algo/algo_orders/TASAlgo.cpp @@ -0,0 +1,222 @@ +#include "lib/orderman/orders/algo/algo_orders/TASAlgo.h" + +#include "lib/domain_model/Security.h" +#include "lib/orderman/order_factory/IOrderFactory.h" +#include "lib/orderman/orderman_utils/Book.h" +#include "lib/orderman/orderman_utils/orderman_utils.h" +#include "lib/orderman/orders/algo/IAlgoServices.h" +#include "lib/orderman/orders/spreader_orders/spreader/SpreaderUtils.h" +#include "lib/os/os.h" +#include "lib/domain_model/Security.h" +#include "lib/utils/PBUtils.h" +#include "lib/utils/datetime.h" +#include + +namespace bts { +namespace orderman { + +using namespace api::orderman; + +enum class TASAlgo::Status { WAITING_FOR_SIGNAL, WORKING, DONE }; + +struct TASLegData{ + api::Side side; //trading side + Price price; + Quantity quantity; + ChildTemplate* order_template = nullptr; + OrderKey order_key; + ChildState order_state; + TASLegData (api::Side side, + Price price, + Quantity quantity, + ChildTemplate* order_template) + : + side (side), + price(price), + quantity(quantity), + order_template (order_template), + order_key(null_order_key()), + order_state(){ + order_state.status = ChildStatus::WORKING; + } +}; + +struct TASAlgo::PImpl { + Status state = Status::WAITING_FOR_SIGNAL; + IAlgoServices* algo_services = nullptr; + SecurityKey sec_key = SecurityKey(0); + TASAlgo::TASAlgoParams algo_params; + std::vector legs; + bts::datetime::PTime start_time = bts::datetime::local_now(); + PImpl(){ + } + TASLegData* find_leg_from_ord_key (OrderKey ord_key){ + for (auto& l : legs) + if(l.order_key == ord_key) + return &l; + return nullptr; + } +}; + +TASAlgo::TASAlgo() : me(new PImpl()) {} +TASAlgo::~TASAlgo() {} +// initialization ///////////////////////////////////////////////////////////// +/////////////////////////////////////////////////////////////////////////////// +OrderOpResult TASAlgo::do_before_launch(IBeforeLaunchAlgoServices& serv) { + auto& def = serv.definition(); + if (def.has_custom_def() && def.custom_def().has_dynamic()){ + const auto& sec = serv.security(); + me->sec_key = SecurityKey(sec.id()); + me->algo_params = _convert_params(def.custom_def()); + if (me->algo_params.child_orders.size() > 0){ + for (const auto& t : me->algo_params.child_orders){ + Price price = std::get<0>(t); + Quantity quantity = std::get<1>(t); + api::Side side = std::get<2>(t); + auto* ordt = _create_order_template( + serv, + SecurityKey(sec.id()), + side); + me->legs.emplace_back(side, + price, + quantity, + ordt); + } + return {}; + } + } + return {OrderError::ORDER_ERROR_INVALID_PARAMETER, + std::string("Parameters are not well formed")}; +} +void TASAlgo::do_launch(IAlgoServices& algo_services) { + me->algo_services = &algo_services; + _algo().subscribe_status(me->sec_key); + //_algo().subscribe_timer(); +} +void TASAlgo::do_launch_attached(IAlgoServices& algo_services, + OrderKey, + ChildState) { + me->algo_services = &algo_services; + _cxl_and_end(true, "TAS Algo can not be attached to working orders"); +} +// updates //////////////////////////////////////////////////////////////////// +/////////////////////////////////////////////////////////////////////////////// +void TASAlgo::do_update(const api::orderman::Update&) { + LOG(OrderMan, Warn, "TASAlgo::do_update order:{} state:{} update not allowed ", + _algo().order_key().value(), + static_cast(me->state)); +} +// callbacks ////////////////////////////////////////////////////////////////// +/////////////////////////////////////////////////////////////////////////////// +void TASAlgo::on_book(SecurityKey, const Book&) { +} +void TASAlgo::on_child_change(OrderKey key, + ChildState const &st){ + LOG(OrderMan, Info, + "TASAlgo::on_child_change key:{} child_key:{} state:{} status:{} " + "total_fills:{} last_price:{}", + _algo().order_key(), + key, + static_cast(me->state), + static_cast(st.status), + st.total_fills, + st.last_price); + TASLegData* ld_ptr = me->find_leg_from_ord_key(key); + if (ld_ptr != nullptr) + ld_ptr->order_state = st; + for (const auto& l : me->legs) + if(l.order_state.status == ChildStatus::WORKING) + return; + _cxl_and_end(false,""); +} +void TASAlgo::on_status(SecurityKey key, co2::TradingStatus status){ + if(key == me->sec_key && + status == co2::TradingStatus::TRADINGSTATUS_PRE_OPEN){ + switch (me->state){ + case TASAlgo::Status::WAITING_FOR_SIGNAL: + for (auto& l : me->legs){ + l.order_key = _algo().launch_child( + *l.order_template, + l.price, + l.quantity); + LOG(OrderMan, Info,"TASAlgo::on_status launching order key:{} " + "child_key:{} price:{} qty:{}", + _algo().order_key(), + l.order_key, + l.price, + l.quantity); + } + _switch_state(TASAlgo::Status::WORKING); + break; + case TASAlgo::Status::WORKING: + case TASAlgo::Status::DONE: + break; + } + } +} +void TASAlgo::on_timer(){ + if (bts::datetime::msecs_since(me->start_time) > 10000) + on_status(me->sec_key,co2::TradingStatus::TRADINGSTATUS_PRE_OPEN); +} +IAlgoServices& TASAlgo::_algo() const { return *me->algo_services; } +/////////////////////////////////////////////////////////////////////////////// +// stateless conversion /////////////////////////////////////////////////////// +TASAlgo::TASAlgoParams TASAlgo::_convert_params ( + const bts::api::orderman::CustomOrderDefinition& cod){ + const DynamicOrderDefinition& def = cod.dynamic(); + const auto& params = def.order_params(); + TASAlgo::TASAlgoParams rv; + for (size_t i = 0; i < 1024; i++){ + auto idx = std::to_string(i+1); + auto price = get_if(params, "Order " + idx + " Price"); + auto qty = get_if(params, "Order " + idx + " Quantity"); + auto is_buy = get_if(params, "Order " + idx + " Buy?"); + if(price.has_value() && qty.has_value() && qty.value() != 0){ + api::Side side = is_buy.has_value() ? + (is_buy.value() ? api::Side::SIDE_BUY : api::Side::SIDE_SELL) : + api::Side::SIDE_SELL; + rv.child_orders.emplace_back(Price(price.value()), Quantity(qty.value()), side); + } + } + return rv; +} +ChildTemplate* TASAlgo::_create_order_template ( + IBeforeLaunchAlgoServices& serv, + SecurityKey key, + api::Side side){ + OrderDefinition order_def = serv.definition(); + order_def.set_security_key (key.value()); + order_def.set_side(side); + order_def.mutable_custom_def()->mutable_limit()->MergeFrom ( + api::orderman::LimitOrderDefinition()); + return &(serv.make_child_template(order_def)); +} +void TASAlgo::_switch_state(TASAlgo::Status new_state){ + switch (new_state){ + case (TASAlgo::Status::WAITING_FOR_SIGNAL): + bts_assert(false); + break; + case (TASAlgo::Status::WORKING): + bts_assert(me->state == TASAlgo::Status::WAITING_FOR_SIGNAL); + break; + case (TASAlgo::Status::DONE): + bts_assert(me->state != TASAlgo::Status::DONE); + break; + } + LOG(OrderMan, Info,"TASAlgo::_switch_state key:{} {}->{} ", + _algo().order_key(), + static_cast(me->state), + static_cast(new_state)); + me->state = new_state; +} + +void TASAlgo::_cxl_and_end (bool is_error, const std::string& err_msg){ + if(is_error) + _algo().terminate_error(err_msg); + else + _algo().complete(); + _switch_state(TASAlgo::Status::DONE); +} + +} // namespace orderman +} // namespace bts diff --git a/src/lib/orderman/orders/algo/algo_orders/TASAlgo.h b/src/lib/orderman/orders/algo/algo_orders/TASAlgo.h new file mode 100644 index 000000000..739b2e864 --- /dev/null +++ b/src/lib/orderman/orders/algo/algo_orders/TASAlgo.h @@ -0,0 +1,45 @@ +#pragma once + +#include "lib/orderman/orders/algo/IAlgoOrder.h" + +namespace bts { +namespace orderman { + +class ChildTemplate; +class LegData; + +class TASAlgo : public IAlgoOrder { + public: + TASAlgo(); + ~TASAlgo(); + struct TASAlgoParams{ + std::vector> child_orders; + }; + public: + OrderOpResult do_before_launch(IBeforeLaunchAlgoServices &) override; + void do_launch(IAlgoServices &) override; + void do_launch_attached(IAlgoServices &, OrderKey, ChildState) override; + void do_update(api::orderman::Update const &) override; + void on_book(SecurityKey, Book const &) override; + void on_child_change(OrderKey, ChildState const &) override; + void on_status(SecurityKey, co2::TradingStatus) override; + void on_timer() override; + + private: + enum class Status; + struct PImpl; + std::unique_ptr me; +private: + IAlgoServices& _algo() const ; + static TASAlgoParams _convert_params ( + const bts::api::orderman::CustomOrderDefinition&); + static ChildTemplate* _create_order_template (IBeforeLaunchAlgoServices&, + SecurityKey, + api::Side); + // acts on md, cognizant of the state + void _switch_state (Status new_status); + void _cxl_and_end (bool is_error, const std::string& err_msg); +}; + +} // namespace orderman +} // namespace bts diff --git a/src/lib/orderman/orders/spreader_orders/spreader/SpreaderUtils.h b/src/lib/orderman/orders/spreader_orders/spreader/SpreaderUtils.h index 0d34acfe2..559fa2c96 100644 --- a/src/lib/orderman/orders/spreader_orders/spreader/SpreaderUtils.h +++ b/src/lib/orderman/orders/spreader_orders/spreader/SpreaderUtils.h @@ -238,11 +238,9 @@ public: } } // CRITICAL PATH (for hedging) - // travserses the book to locate the target_price, - // and finds the qty at the target price and the - // cum qty upto that price. - // returns false if the target_price is not found. - // this is used to find the qty and cumm_qty + // travserses the book to locate the target_price, + // and finds the cumm qty up to that level (last our arg) + // Returns false if the target_price is not found. static bool find_qty_upto_price_level (const Book& book, api::Side book_side, Price target_price, diff --git a/src/lib/utils/AuthSessionManager.cpp b/src/lib/utils/AuthSessionManager.cpp index 8b1b74438..0c397d8f9 100644 --- a/src/lib/utils/AuthSessionManager.cpp +++ b/src/lib/utils/AuthSessionManager.cpp @@ -80,7 +80,7 @@ bool AuthSessionManager::validate_ip_port(const bts::config::KeyCode& kc) return portgood && ipgood; } -bool AuthSessionManager::add_or_replace_session(const api::user::TokenPayload& tok, const admin::BTSUser& user, std::string* msg) +bool AuthSessionManager::add_or_replace_session(const api::user::TokenPayload& tok, const std::string& username, std::string* msg) { if(tok.wheel()) return true; @@ -105,18 +105,18 @@ bool AuthSessionManager::add_or_replace_session(const api::user::TokenPayload& t if(it != _map.end()) { _map[tok.id()] = Session{tok}; - LOG(User, Warn, "KEYCODE: Replacing session {} user:{}:{} ", tok.id(), user.id(), user.name()); + LOG(User, Warn, "KEYCODE: Replacing session {} user:{} ", tok.id(), username); return true; } else if(_map.size() < _nusers) { _map[tok.id()] = Session{tok}; - LOG(User, Warn, "KEYCODE: Creating session {} user:{}:{}", tok.id(), user.id(), user.name()); + LOG(User, Warn, "KEYCODE: Creating session {} user:{}", tok.id(), username); return true; } if(msg) *msg = "KEYCODES: Not Enough Logins available for session"; - LOG(User, Error, "KEYCODES: Not Enough Logins available for session {} user:{}:{}", tok.id(), user.id(), user.name()); + LOG(User, Error, "KEYCODES: Not Enough Logins available for session {} user:{}", tok.id(), username); return false; } @@ -134,9 +134,18 @@ bool AuthSessionManager::check_session(const api::user::TokenPayload& tok) bool same_token = _concurrent || (tok.date() == it->second.token.date()); return not_expired && same_token; } + else //not in map, try to login in case server booted + { + return add_or_replace_session(tok); + } } return false; } + +void AuthSessionManager::clear_sessions() +{ + _map.clear(); +} }}} #endif \ No newline at end of file diff --git a/src/lib/utils/AuthSessionManager.h b/src/lib/utils/AuthSessionManager.h index fa68d097e..f9b1fa2be 100644 --- a/src/lib/utils/AuthSessionManager.h +++ b/src/lib/utils/AuthSessionManager.h @@ -19,8 +19,9 @@ class AuthSessionManager public: AuthSessionManager(); - bool add_or_replace_session(const api::user::TokenPayload& tok, const admin::BTSUser& user, std::string* msg = nullptr); + bool add_or_replace_session(const api::user::TokenPayload& tok, const std::string& username = "", std::string* msg = nullptr); bool check_session(const api::user::TokenPayload& tok); + void clear_sessions(); private: bool validate_ip_port(const bts::config::KeyCode& kc); diff --git a/src/lib/utils/AuthToken.cpp b/src/lib/utils/AuthToken.cpp index 9c062c592..cd9be38d6 100644 --- a/src/lib/utils/AuthToken.cpp +++ b/src/lib/utils/AuthToken.cpp @@ -51,6 +51,11 @@ AuthSessionManager& session_manager() return mgr; } +void clear_all_sessions() +{ + session_manager().clear_sessions(); +} + bool requires_auth_session(const admin::BTSUser& user) { //if the user can trade in any account or see market data, it needs a session @@ -76,6 +81,14 @@ bool requires_auth_session(const admin::BTSUser& user) return true; } } + if(user.user_data().exchange_user_data().has_ice_data()) + { + for(const auto& i: user.user_data().exchange_user_data().ice_data().md_permissions()) + { + if(i.enabled()) + return true; + } + } } return false; } @@ -89,7 +102,7 @@ AuthToken generate_token(const admin::BTSUser& user, std::string* msg) payload.set_wheel(true); if(requires_auth_session(user)) - session_manager().add_or_replace_session(payload, user, msg); + session_manager().add_or_replace_session(payload, user.name(), msg); auto token = encrypt(payload.SerializeAsString()); bts_assert_d(token.size() < 255-8); AuthToken result; diff --git a/src/lib/utils/AuthToken.h b/src/lib/utils/AuthToken.h index a85605a0b..e8dd1f700 100644 --- a/src/lib/utils/AuthToken.h +++ b/src/lib/utils/AuthToken.h @@ -10,6 +10,8 @@ namespace auth { AuthToken generate_token(const admin::BTSUser&, std::string* msg = nullptr); AuthToken generate_wheel_token(); bool decode_token(const api::AuthToken& token, user::TokenPayload* payload); +bool requires_auth_session(const admin::BTSUser& user); +void clear_all_sessions(); }}} diff --git a/src/lib/utils/Puid.cpp b/src/lib/utils/Puid.cpp index 2bb27e1a3..072a03065 100644 --- a/src/lib/utils/Puid.cpp +++ b/src/lib/utils/Puid.cpp @@ -1,3 +1,4 @@ +#ifndef _WIN32 #include "Puid.h" #include "../os/os.h" @@ -7,17 +8,17 @@ #include #include -#include -#include +#include #include #include -#include #include +#include +#include -#include #include -#include #include +#include +#include namespace bts { namespace utils { @@ -35,93 +36,99 @@ const std::string counter_file_name = "counter_file"; // relying on the fact that metadata changes (like renaming) are // atomic to provide transactional updates to a counter file which is // used to generate uuids. -// -// Write process (on Foo.txt): -// - Step W1: Acquire "write lock" on the existing file. -// - Step W2: Copy the old file in a new unique temporary file. (copy Foo.txt Foo.TmpNNNN.txt) -// - Step W3: Apply the writes to the new file (Foo.TmpNNNN.txt). -// - Step W4: Flush all the writes (for example those being remaining in the cache manager). -// - Step W5: Rename the old file in a new unique Alternate form (ren Foo.txt Foo.AltNNNN.txt) -// - Step W6: Rename the new file into the old file (ren Foo.TmpNNNN.txt Foo.txt) -// - Step W7: Delete the old Alternate file (del Foo.AltNNNN.txt). If this fails, simply ignore. The file will be deleted later during the next recovery. +// +// Write process (on Foo.txt): +// - Step W1: Acquire "write lock" on the existing file. +// - Step W2: Copy the old file in a new unique temporary file. (copy Foo.txt +// Foo.TmpNNNN.txt) +// - Step W3: Apply the writes to the new file (Foo.TmpNNNN.txt). +// - Step W4: Flush all the writes (for example those being remaining in the +// cache manager). +// - Step W5: Rename the old file in a new unique Alternate form (ren Foo.txt +// Foo.AltNNNN.txt) +// - Step W6: Rename the new file into the old file (ren Foo.TmpNNNN.txt +// Foo.txt) +// - Step W7: Delete the old Alternate file (del Foo.AltNNNN.txt). If this +// fails, simply ignore. The file will be deleted later during the next +// recovery. // - Step W8: Release "write lock" on the existing file. -// Recovery from a crash during write (on Foo.txt): -// - Step R1: If Foo.txt is missing but we have both Foo.AltNNNN.txt and Foo.TmpNNNN.txt present, then we crashed between Step W5 and Step W6. Retry from Step W6. -// - Step R2: If Foo.txt is present but Foo.TmpNNNN.txt is also present, then we crashed before Step W5. Try to delete all Foo.TmpNNNN.txt files and ignore failures. -// - Step R3: If Foo.txt is present but Foo.AltNNNN.txt is also present, then we crashed between Step W6 and Step W7. Try to delete all Foo.AltNNNN.txt files and ignore failures. +// Recovery from a crash during write (on Foo.txt): +// - Step R1: If Foo.txt is missing but we have both Foo.AltNNNN.txt and +// Foo.TmpNNNN.txt present, then we crashed between Step W5 and Step W6. Retry +// from Step W6. +// - Step R2: If Foo.txt is present but Foo.TmpNNNN.txt is also present, then we +// crashed before Step W5. Try to delete all Foo.TmpNNNN.txt files and ignore +// failures. +// - Step R3: If Foo.txt is present but Foo.AltNNNN.txt is also present, then we +// crashed between Step W6 and Step W7. Try to delete all Foo.AltNNNN.txt files +// and ignore failures. -void init_store (std::string const &path, - std::string const &lock_path, - std::string const &counter_path) -{ - if (!boost::filesystem::is_directory(path)) - { +void init_store(std::string const &path, std::string const &lock_path, + std::string const &counter_path) { + if (!boost::filesystem::is_directory(path)) { boost::filesystem::create_directories(path); - std::ofstream fs(lock_path.c_str ()); - std::ofstream fc(counter_path.c_str ()); + std::ofstream fs(lock_path.c_str()); + std::ofstream fc(counter_path.c_str()); fc << 0; } } -std::string find_file (std::string const &path, std::string const &ending) -{ +std::string find_file(std::string const &path, std::string const &ending) { for (boost::filesystem::directory_iterator p = - boost::filesystem::directory_iterator (path); - p != boost::filesystem::directory_iterator (); p++) - { - std::string fn (p->path ().string ()); + boost::filesystem::directory_iterator(path); + p != boost::filesystem::directory_iterator(); p++) { + std::string fn(p->path().string()); if (boost::algorithm::ends_with(fn, ending)) return fn; } - return std::string (); + return std::string(); } // boost::filesystem::copy_file is broken for C++ 11 unless one compiles boost -// with c++ 11. (See http://boost.2283326.n4.nabble.com/Filesystem-problems-with-g-std-c-0x-td2639716.html) +// with c++ 11. (See +// http://boost.2283326.n4.nabble.com/Filesystem-problems-with-g-std-c-0x-td2639716.html) // so we provide our own. -void copy_file(std::string const&in, std::string const &out) -{ - std::fstream f(in.c_str (), std::fstream::in|std::fstream::binary); +void copy_file(std::string const &in, std::string const &out) { + std::fstream f(in.c_str(), std::fstream::in | std::fstream::binary); f << std::noskipws; std::istream_iterator begin(f); std::istream_iterator end; - - std::fstream f2(out.c_str (), std::fstream::out|std::fstream::trunc|std::fstream::binary); + + std::fstream f2(out.c_str(), std::fstream::out | std::fstream::trunc | + std::fstream::binary); std::ostream_iterator begin2(f2); std::copy(begin, end, begin2); } -uint32_t get_host_ip_uint32 (std::string const &ip) -{ +uint32_t get_host_ip_uint32(std::string const &ip) { std::vector split_vec; - boost::algorithm::split( split_vec, ip,boost::algorithm::is_any_of(".") ); - bts_assert (split_vec.size () == 4); + boost::algorithm::split(split_vec, ip, boost::algorithm::is_any_of(".")); + bts_assert(split_vec.size() == 4); uint32_t result = 0; int idx = 0; for (std::string &byte : split_vec) { - uint32_t this_byte = atoi (byte.c_str ()); + uint32_t this_byte = atoi(byte.c_str()); this_byte = (this_byte << (idx++ * 8)); result += this_byte; } return result; } -std::string &get_host_ip_string (std::string const &ip) -{ +std::string &get_host_ip_string(std::string const &ip) { static std::string result; - if (result.size () == 0) { + if (result.size() == 0) { std::ostringstream result_stream; - - bts_assert (ip.size ()); - bts_assert (ip != "127.0.0.1"); - std::istringstream ip_str (ip); + + bts_assert(ip.size()); + bts_assert(ip != "127.0.0.1"); + std::istringstream ip_str(ip); for (int i = 0; i < 4; i++) { int b; ip_str >> b; - ip_str.get (); - + ip_str.get(); + char high_nibble = (b & 0xf0) >> 4; char low_nibble = b & 0x0f; if (high_nibble >= 10) @@ -134,14 +141,13 @@ std::string &get_host_ip_string (std::string const &ip) low_nibble += '0'; result_stream << high_nibble << low_nibble; } - result = result_stream.str (); + result = result_stream.str(); } return result; } -uint64_t reserve_ids (std::string const &scope, int n_ids) -{ - tbb::mutex::scoped_lock l (lock); +uint64_t reserve_ids(std::string const &scope, int n_ids) { + tbb::mutex::scoped_lock l(lock); // we risk race conditions if the directory is not there. // but that's OK, this is rare and easily manageable. @@ -149,83 +155,82 @@ uint64_t reserve_ids (std::string const &scope, int n_ids) std::string lock_path = path + "/" + lock_file_name; std::string counter_path = path + "/" + counter_file_name; - init_store (path,lock_path, counter_path); + init_store(path, lock_path, counter_path); // W1 - boost::interprocess::file_lock f_lock(lock_path.c_str ()); - boost::interprocess::scoped_lock scope_lock (f_lock); + boost::interprocess::file_lock f_lock(lock_path.c_str()); + boost::interprocess::scoped_lock scope_lock( + f_lock); - std::string tmp_file = find_file (path, ".tmp"); - std::string alt_file = find_file (path, ".alt"); - std::string counter_file = find_file (path, counter_file_name); + std::string tmp_file = find_file(path, ".tmp"); + std::string alt_file = find_file(path, ".alt"); + std::string counter_file = find_file(path, counter_file_name); // r1? - if (counter_file.size () == 0 && tmp_file.size () && alt_file.size ()) { - boost::filesystem::rename (tmp_file,counter_path); - boost::filesystem::remove (alt_file); - f_lock.unlock (); - lock.unlock (); - return reserve_ids (scope,n_ids); - + if (counter_file.size() == 0 && tmp_file.size() && alt_file.size()) { + boost::filesystem::rename(tmp_file, counter_path); + boost::filesystem::remove(alt_file); + f_lock.unlock(); + lock.unlock(); + return reserve_ids(scope, n_ids); + } // r2 - else if (counter_file.size () && tmp_file.size ()) { - boost::filesystem::remove (tmp_file); - f_lock.unlock (); - lock.unlock (); - return reserve_ids (scope,n_ids); + else if (counter_file.size() && tmp_file.size()) { + boost::filesystem::remove(tmp_file); + f_lock.unlock(); + lock.unlock(); + return reserve_ids(scope, n_ids); } // r3 - else if (counter_file.size () && alt_file.size ()) { - boost::filesystem::remove (alt_file); - f_lock.unlock (); - lock.unlock (); - return reserve_ids (scope,n_ids); + else if (counter_file.size() && alt_file.size()) { + boost::filesystem::remove(alt_file); + f_lock.unlock(); + lock.unlock(); + return reserve_ids(scope, n_ids); } else { - + // W2, copy counter file to a temp file - tmp_file = path + "/" + Uuid ().to_text_string () + ".tmp"; - copy_file (counter_path, tmp_file); + tmp_file = path + "/" + Uuid().to_text_string() + ".tmp"; + copy_file(counter_path, tmp_file); // W3, W4 update temp file, close and flush uint64_t counter_value; - std::ifstream (tmp_file.c_str ()) >> counter_value; - std::ofstream (tmp_file.c_str ()) << counter_value + n_ids; - - // W5, rename counter file to an alt file (renames are atomic) - alt_file = path + "/" + Uuid ().to_text_string () + ".alt"; - boost::filesystem::rename (counter_path, alt_file); + std::ifstream(tmp_file.c_str()) >> counter_value; + std::ofstream(tmp_file.c_str()) << counter_value + n_ids; - // W6, rename temp file to counter file - boost::filesystem::rename (tmp_file,counter_path); + // W5, rename counter file to an alt file (renames are atomic) + alt_file = path + "/" + Uuid().to_text_string() + ".alt"; + boost::filesystem::rename(counter_path, alt_file); + + // W6, rename temp file to counter file + boost::filesystem::rename(tmp_file, counter_path); // W7 delete alt file - boost::filesystem::remove (alt_file); + boost::filesystem::remove(alt_file); return counter_value; } } -std::vector gen_ids (std::string const &scope, int n_ids) -{ - auto counter_value = reserve_ids (scope,n_ids); +std::vector gen_ids(std::string const &scope, int n_ids) { + auto counter_value = reserve_ids(scope, n_ids); std::vector result; for (int i = 0; i < n_ids; i++) - result.push_back (counter_value + i); + result.push_back(counter_value + i); return result; } -static char base_62_alphabet [] = { - '0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f', - 'g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v', - 'w','x','y','z','A','B','C','D','E','F','G','H','I','J','K','L', - 'M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z' -}; +static char base_62_alphabet[] = { + '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', + 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', + 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', 'A', 'B', 'C', + 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', + 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z'}; -std::string to_base_62 (uint32_t src) -{ - std::string out (6, ' '); - for (int i=5; i >= 0;i--) { +std::string to_base_62(uint32_t src) { + std::string out(6, ' '); + for (int i = 5; i >= 0; i--) { int r = src % 62; out[i] = base_62_alphabet[r]; src = src / 62; @@ -233,47 +238,46 @@ std::string to_base_62 (uint32_t src) return out; } -void to_base_62 (std::vector const &id_vec, std::vector *out) -{ - for (auto src : id_vec) - for (int i=5; i >= 0;i--) { +void to_base_62(std::vector const &id_vec, std::vector *out) { + for (auto src : id_vec) + for (int i = 5; i >= 0; i--) { int r = src % 62; - out->push_back (base_62_alphabet[r]); + out->push_back(base_62_alphabet[r]); src = src / 62; } } -uint64_t gen_id (std::string const &scope) -{ - std::vector result = gen_ids (scope, 1); +uint64_t gen_id(std::string const &scope) { + std::vector result = gen_ids(scope, 1); return result[0]; } -std::vector gen_puids (std::string const &scope, int n_puids) -{ +std::vector gen_puids(std::string const &scope, int n_puids) { // first generate a series of unique uint32 numbers (For this machine) - std::vector ids = gen_ids (scope, n_puids); + std::vector ids = gen_ids(scope, n_puids); // now add the host ip and a timer string to the generated id to // guarantee uniqueness across machines - uint32_t ip = get_host_ip_uint32 (os::default_host_ip ()); - std::string encoded_ip (to_base_62 (ip)); - - uint32_t seconds = time (NULL); - std::string encoded_time (to_base_62 (seconds)); - + uint32_t ip = get_host_ip_uint32(os::default_host_ip()); + std::string encoded_ip(to_base_62(ip)); + + uint32_t seconds = time(NULL); + std::string encoded_time(to_base_62(seconds)); + std::vector result; for (uint32_t id : ids) { std::ostringstream s; - s << encoded_ip << "-" << encoded_time << "-" << to_base_62 (id); - result.push_back (s.str ()); + s << encoded_ip << "-" << encoded_time << "-" << to_base_62(id); + result.push_back(s.str()); } return result; } -Puid gen_puid (std::string const &scope) -{ - return *(gen_puids (scope,1).begin ()); +Puid gen_puid(std::string const &scope) { + return *(gen_puids(scope, 1).begin()); } -}} + +} // namespace utils +} // namespace bts +#endif \ No newline at end of file diff --git a/src/lib/utils/Puid.h b/src/lib/utils/Puid.h index 2340320a9..4a7d4e2b5 100644 --- a/src/lib/utils/Puid.h +++ b/src/lib/utils/Puid.h @@ -1,10 +1,12 @@ #pragma once #include "bts_export.h" +#include "lib/utils/bts_assert.h" +#include +#include #include #include -#include namespace bts { namespace utils { @@ -19,21 +21,77 @@ typedef std::string Puid; // machine's ip, the unix time (seconds since 1970) and a machine // unique counter. id's are 20 bytes long and can contain [a-z], [A-Z] // [0-9] and - -BTS_EXPORT std::vector gen_puids (std::string const &scope, int n_puids); -BTS_EXPORT Puid gen_puid (std::string const &scope); +BTS_EXPORT std::vector gen_puids(std::string const &scope, int n_puids); +BTS_EXPORT Puid gen_puid(std::string const &scope); // generate machine unique ids (no guarantees of uniqueness across // machines) -BTS_EXPORT std::vector gen_ids (std::string const &scope, int n_ids); +BTS_EXPORT std::vector gen_ids(std::string const &scope, int n_ids); // Similar to the above but instead of copying the ids to a vector // this simply returns the first free id (x) and skips ahead by n_ids // so that the next gen_id call will will be x+n_ids -BTS_EXPORT uint64_t reserve_ids (std::string const &scope, int n_ids); +BTS_EXPORT uint64_t reserve_ids(std::string const &scope, int n_ids); // generate a single id -BTS_EXPORT uint64_t gen_id (std::string const &scope); +BTS_EXPORT uint64_t gen_id(std::string const &scope); // convert a uint32 to a 6 byte human readeable string containing only // alpha numerical characters. -BTS_EXPORT std::string to_base_62 (uint32_t id); -BTS_EXPORT void to_base_62 (std::vector const &ids, std::vector *out); -}} +BTS_EXPORT std::string to_base_62(uint32_t id); +BTS_EXPORT void to_base_62(std::vector const &ids, + std::vector *out); + +#ifndef _WIN32 +class BTS_EXPORT IDGenerator { + public: + IDGenerator(std::string const &scope, uint32_t batch_size) + : _batch_size(batch_size), _next(0), _end(0), _scope(scope), _lock() { + bts_assert((int)batch_size > 0); + refill(); + } + + // next should only be called from a single thread + uint64_t next() { + if (_remaning() == 0) { + std::lock_guard _(_lock); + if (!_next_block.has_value()) + _get_next_block(); + + bts_assert(_next_block.has_value()); + _next = _next_block.value().first; + _end = _next_block.value().second; + _next_block.reset(); + } + bts_assert(_remaning() > 0); + return _next++; + } + + // refill can be called from any thread + void refill() { + std::lock_guard _(_lock); + if (!_next_block.has_value()) + _get_next_block(); + } + + private: + uint64_t _remaning() { + bts_assert(_end >= _next); + return _end - _next; + } + + void _get_next_block() { + auto block_start = reserve_ids(_scope, _batch_size); + auto block_end = block_start + _batch_size; + _next_block = std::make_pair(block_start, block_end); + } + + const uint32_t _batch_size; + uint64_t _next; + uint64_t _end; + const std::string _scope; + std::mutex _lock; + std::optional> _next_block; +}; +#endif + +} // namespace utils +} // namespace bts diff --git a/src/lib/utils/base62.h b/src/lib/utils/base62.h new file mode 100644 index 000000000..22b01252c --- /dev/null +++ b/src/lib/utils/base62.h @@ -0,0 +1,46 @@ +#pragma once + +#include +#include +#include +#include + +#include "lib/utils/bts_assert.h" + +namespace bts { +namespace utils { + +static char base_62_alphabet [] = +{ + '0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f', + 'g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v', + 'w','x','y','z','A','B','C','D','E','F','G','H','I','J','K','L', + 'M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z' +}; + +inline std::string to_base_62 (uint32_t src) +{ + std::string out (6, ' '); + for (int i=5; i >= 0; i--) + { + int r = src % 62; + out[i] = base_62_alphabet[r]; + src = src / 62; + } + return out; +} + +inline uint32_t from_base_62 (const std::string& src) +{ + bts_assert_msg(src.size() == 6, "bad base62 hash"); + uint32_t out = 0; + for (int i=5; i >= 0; i--) + { + uint32_t idx = std::distance(&base_62_alphabet[0], std::find(&base_62_alphabet[0], &base_62_alphabet[61], src[i])); + out += idx * pow(62, 5-i); + } + return out; +} + + +}} diff --git a/src/lib/utils/tests/AuthTokenTests.cpp b/src/lib/utils/tests/AuthTokenTests.cpp index 6bcbe52d8..6c550426d 100644 --- a/src/lib/utils/tests/AuthTokenTests.cpp +++ b/src/lib/utils/tests/AuthTokenTests.cpp @@ -19,7 +19,9 @@ TEST_CASE("AuthToken", "[security]") { // We use 1 byte for api header size REQUIRE(token.ByteSize() <= 240); api::user::TokenPayload payload; - REQUIRE(!api::auth::decode_token(token, &payload)); + api::auth::clear_all_sessions(); + REQUIRE(!api::auth::requires_auth_session(admin_user)); + REQUIRE(api::auth::decode_token(token, &payload)); REQUIRE(payload.id() == admin_user.id()); REQUIRE(payload.wheel() == false); } @@ -28,7 +30,8 @@ TEST_CASE("AuthToken", "[security]") { auto token = api::auth::generate_token(trader_user); // We use 1 byte for api header size REQUIRE(token.ByteSize() <= 240); - api::user::TokenPayload payload; + api::user::TokenPayload payload; + api::auth::clear_all_sessions(); REQUIRE(api::auth::decode_token(token, &payload)); REQUIRE(payload.id() == trader_user.id()); REQUIRE(payload.wheel() == false); diff --git a/src/lib/utils/tests/IDGeneratorTests.cpp b/src/lib/utils/tests/IDGeneratorTests.cpp new file mode 100644 index 000000000..832a17fe4 --- /dev/null +++ b/src/lib/utils/tests/IDGeneratorTests.cpp @@ -0,0 +1,55 @@ +#ifndef _WIN32 +#include "../Puid.h" +#include "catch/catch.hpp" + +namespace bts { + +TEST_CASE("IDGenerator", "[utils]") { + SECTION("1") { + utils::IDGenerator gen("test",1); + auto start = gen.next(); + for (int i = 0; i < 10;i++) { + auto previous_start = start; + start = gen.next(); + REQUIRE( start == previous_start + 1); + } + } + SECTION("10") { + utils::IDGenerator gen("test",4); + auto start = gen.next(); + for (int i = 0; i < 10;i++) { + auto previous_start = start; + start = gen.next(); + REQUIRE( start == previous_start + 1); + } + } + SECTION("10_interference") { + utils::IDGenerator gen("test",4); + auto start = gen.next(); + auto previous_start = start; + auto interferance = utils::reserve_ids("test", 3); + + previous_start = start; + start = gen.next(); + REQUIRE( start == previous_start + 1); + + previous_start = start; + start = gen.next(); + REQUIRE( start == previous_start + 1); + + previous_start = start; + start = gen.next(); + REQUIRE( start == previous_start + 1); + + previous_start = start; + start = gen.next(); + REQUIRE( interferance + 3 == start); + + previous_start = start; + start = gen.next(); + REQUIRE( start == previous_start + 1); + } +} + +}//ns +#endif diff --git a/src/server/sparkd/cmds/FIXSim.cpp b/src/server/sparkd/cmds/FIXSim.cpp index 413719a14..2ce8e9bfe 100644 --- a/src/server/sparkd/cmds/FIXSim.cpp +++ b/src/server/sparkd/cmds/FIXSim.cpp @@ -5,7 +5,7 @@ namespace bts { namespace cmd { int FIXSim::execute(std::vector const &) { - bts::sim::start_exchange_sim(); + bts::sim::start_exchange_sim(bts::api::Venue::VENUE_GLBX); return 0; } diff --git a/src/server/sparkd/cmds/RunSim.cpp b/src/server/sparkd/cmds/RunSim.cpp index 92f9f9cc5..abaf2cd53 100644 --- a/src/server/sparkd/cmds/RunSim.cpp +++ b/src/server/sparkd/cmds/RunSim.cpp @@ -6,7 +6,7 @@ namespace cmd { int RunSim::execute(std::vector const &) { - sim::start_exchange_sim(nullptr); + sim::start_exchange_sim(api::Venue::VENUE_SIM, nullptr); return 0; } diff --git a/src/server/sparkd/exchange_sim/ExchangeSim.cpp b/src/server/sparkd/exchange_sim/ExchangeSim.cpp index 11001dfa9..6333786ef 100644 --- a/src/server/sparkd/exchange_sim/ExchangeSim.cpp +++ b/src/server/sparkd/exchange_sim/ExchangeSim.cpp @@ -4,12 +4,14 @@ #include #include #include +#include #include #include #include #include "lib/os/os.h" +#include "lib/domain_model/Security.pb.h" #include "lib/config/protos/Ports.pb.h" #include "lib/config/protos/ExchangeSimCfg.pb.h" #include "lib/config/config.h" @@ -33,8 +35,8 @@ auto cfg = "SocketReuseAddress=Y\n" "StartTime=06:02:00\n" "EndTime=06:01:00\n" - "FileLogPath=$BTS_DATA_PATH/exchange_sim/log\n" - "FileStorePath=$BTS_DATA_PATH/exchange_sim/store\n" + "FileLogPath=$BTS_DATA_PATH/exchange_sim/$SIM_VENUE/log\n" + "FileStorePath=$BTS_DATA_PATH/exchange_sim/$SIM_VENUE/store\n" "HeartBtInt=30\n" "[SESSION]\n" "BeginString=FIX.4.2\n" @@ -125,20 +127,32 @@ struct MDPlayer : md::IRelayWriter }; -int start_exchange_sim(md::MBOWriter* mbo) +int start_exchange_sim(bts::api::Venue venue, md::MBOWriter* mbo) { try { auto str_settings = std::string(cfg); + auto venue_string = boost::to_lower_copy(api::Venue_Name(venue)); boost::replace_all(str_settings,"$BTS_DATA_PATH", os::bts_data_path()); - boost::replace_all(str_settings,"$EXCHANGE_SIM_FIX_PORT", - std::to_string( - config::get().exchange_sim_fix_port())); + boost::replace_all(str_settings,"$SIM_VENUE", venue_string); + std::string port_string; + switch(venue) + { + case api::VENUE_SIM: + port_string = std::to_string(config::get().glbx_sim_fix_port()); + break; + case api::VENUE_SIMICE: + port_string = std::to_string(config::get().ice_sim_fix_port()); + break; + default: + bts_assert_msg(false, "Not a valid sim venue"); + } + boost::replace_all(str_settings,"$EXCHANGE_SIM_FIX_PORT", port_string); std::istringstream str{str_settings}; FIX::SessionSettings settings( str ); - ex_sim::ExchangeSimApplication application(mbo); + ex_sim::ExchangeSimApplication application( venue, mbo ); FIX::FileStoreFactory storeFactory( settings ); FIX::FileLogFactory logFactory( settings ); FIX::SocketAcceptor acceptor( application, storeFactory, settings, logFactory ); diff --git a/src/server/sparkd/exchange_sim/ExchangeSim.h b/src/server/sparkd/exchange_sim/ExchangeSim.h index 819fb5bdb..7ae259e84 100644 --- a/src/server/sparkd/exchange_sim/ExchangeSim.h +++ b/src/server/sparkd/exchange_sim/ExchangeSim.h @@ -1,11 +1,12 @@ #pragma once #include +#include "lib/domain_model/Security.pb.h" namespace bts { namespace md { class MBOWriter; } namespace sim { -int start_exchange_sim(md::MBOWriter* = nullptr); +int start_exchange_sim(bts::api::Venue, md::MBOWriter* = nullptr); }} diff --git a/src/server/sparkd/exchange_sim/ExchangeSimApplication.cpp b/src/server/sparkd/exchange_sim/ExchangeSimApplication.cpp index 8a5708964..0cfb2998d 100644 --- a/src/server/sparkd/exchange_sim/ExchangeSimApplication.cpp +++ b/src/server/sparkd/exchange_sim/ExchangeSimApplication.cpp @@ -20,6 +20,7 @@ #include "server/sparkd/exchange_sim/book/book_transform.h" #include +#include #include #include #include @@ -106,15 +107,19 @@ inline FIX::OrderID exchange_key(const std::string &clOrdID) { } struct ExchangeSimApplication::PImpl { - PImpl(md::MBOWriter *mbo) - : producer(co2::PLATFORM_SIM), md_sn(1), + PImpl(bts::api::Venue venue, md::MBOWriter* mbo) + : venue(venue), + producer(co2::PLATFORM_SIM), + md_sn(1), timer(utils::Dispatcher::background(), 200, [=]() { producer.heartbeat(); }), do_merge_with_md(config::get().merge_with_md()), - store(new persistence::SqliteStore("ex_sim_orders")), lock(), + store( new persistence::SqliteStore("ex_sim_orders_" + boost::to_lower_copy(api::Venue_Name(venue)) )), + lock(), mbo(mbo) {} IDGenerator m_generator; + bts::api::Venue venue; md::MDProducer producer; std::map md_consumer_recs; std::map md_producer_handles; @@ -127,8 +132,7 @@ struct ExchangeSimApplication::PImpl { md::MBOWriter *mbo; }; -ExchangeSimApplication::ExchangeSimApplication(md::MBOWriter *mbo) - : me(new PImpl(mbo)) { +ExchangeSimApplication::ExchangeSimApplication(bts::api::Venue venue, md::MBOWriter* mbo) : me(new PImpl(venue, mbo)) { auto previous_book = me->store->retrieve_all(); auto all = me->store->retrieve_all()["books"]; @@ -552,7 +556,7 @@ void ExchangeSimApplication::update_md(std::string const &sec_desc) { } std::string get_sim_sec_desc(api::Security const &src_sec) { - return std::string("SIM:") + src_sec.glbx().security_desc(); + return std::string("SIM:") + securities::descriptor(src_sec); } void negate_id(api::Security *sec) { @@ -576,18 +580,28 @@ void ExchangeSimApplication::persist_book(OrderBook const &b) { void ExchangeSimApplication::on_new_security(api::Security const &s) { std::lock_guard l(me->lock); if (!s.has_synthetic()) { - md::with_security_like(s, [this, s](auto &fs) { - auto s2 = s; - *s2.mutable_glbx()->mutable_security_desc() = get_sim_sec_desc(s); - s2.set_exchange(api::VENUE_SIM); + md::with_security_like(s, [this, s](auto &fs) + { + auto s2 = s; + if(s.exchange() == api::VENUE_GLBX && me->venue == api::VENUE_SIM) + { + *s2.mutable_glbx()->mutable_security_desc() = get_sim_sec_desc(s); + s2.set_exchange(api::VENUE_SIM); + } + else if(me->venue == api::VENUE_ICE && me->venue == api::VENUE_SIMICE) + { + *s2.mutable_ice()->mutable_security_desc() = get_sim_sec_desc(s); + s2.set_exchange(api::VENUE_SIMICE); + } + else + return; negate_id(&s2); md::convert_security(s2, fs, true); md::MDProducerHandle handle = me->producer.add_security(fs); auto sec_desc = securities::descriptor(s2); me->md_producer_handles.insert(make_pair(sec_desc, handle)); - me->md_consumer_recs.insert( - make_pair(s.glbx().security_desc(), - MDConsumerRec{s2, utils::TimeStamp::now()})); + me->md_consumer_recs.insert(make_pair(securities::descriptor(s), + MDConsumerRec{s2, utils::TimeStamp::now()})); }); } } @@ -654,8 +668,9 @@ void ExchangeSimApplication::on_depth( if (!me->do_merge_with_md) return; - auto it = me->md_consumer_recs.find(sec.glbx().security_desc()); - bts_assert(it != me->md_consumer_recs.end()); + auto it = me->md_consumer_recs.find(securities::descriptor(sec)); + if(it == me->md_consumer_recs.end()) + return; auto &matcher = me->matchers[get_sim_sec_desc(sec)]; Book previous_md_book = matcher.get_book(); @@ -675,7 +690,7 @@ void ExchangeSimApplication::on_depth( // from the differences between current_md_book and previous_md_book // we generate a set of orders that we apply to our order book - merge_with_order_book(it->second.security.glbx().security_desc(), + merge_with_order_book(securities::descriptor(it->second.security), previous_md_book, current_md_book); // pass on implied market unchanged diff --git a/src/server/sparkd/exchange_sim/ExchangeSimApplication.h b/src/server/sparkd/exchange_sim/ExchangeSimApplication.h index 64077befe..121ff6838 100644 --- a/src/server/sparkd/exchange_sim/ExchangeSimApplication.h +++ b/src/server/sparkd/exchange_sim/ExchangeSimApplication.h @@ -40,6 +40,7 @@ GCC_DIAG_OFF("-Wignored-qualifiers") #include #include #include "server/sparkd/exchange_sim/IMDPlayerObserver.h" +#include "lib/domain_model/Security.pb.h" namespace bts { namespace api {class Security;} @@ -55,7 +56,7 @@ class ExchangeSimApplication public FIX::MessageCracker { public: - ExchangeSimApplication(md::MBOWriter*); + ExchangeSimApplication(bts::api::Venue venue, md::MBOWriter*); ~ExchangeSimApplication(); void merge_with_md_book(); diff --git a/src/server/sparkd/exchange_sim/book/OrderMatcher.h b/src/server/sparkd/exchange_sim/book/OrderMatcher.h index 4e4301762..f5d58ac43 100644 --- a/src/server/sparkd/exchange_sim/book/OrderMatcher.h +++ b/src/server/sparkd/exchange_sim/book/OrderMatcher.h @@ -14,27 +14,56 @@ namespace bts { namespace ex_sim { +// An order matcher maintains a book of same security orders +// and generates matches as the sides cross class OrderMatcher { public: OrderMatcher(); ~OrderMatcher(); + // Add a new order, returning any updates to the new order or + // existing ones resulting from matching void add_new(Order const &, std::vector *out); + + // Add a new order or, if a threshold of existing orders at same + // level from the same owner is met, merge its quantity with the + // last similar order. The merge is usefull to keep memory growth + // in check. Return any changes resulting from matching on the new + // order. void add_or_merge(Order const &, std::vector *out); + // Change an existing order, returning any changed orders that + // resulted matches made possible by the update. void update(std::string const &clorid, Price, Quantity, std::vector *out); + + // Cancel an existing order, removing it from the book void cancel(std::string const &clorid); + // Try to reduce the quantity available at the specified level, + // side by the specified owner, by the specified amount. The + // updated orders are not returned. void try_reduce_quantity_at_level(api::Side, Price level, Quantity by_quantity, std::string const& order_owner); + + // Take the all the orders and consolidated their quantities, per + // level, in a book structure Book get_book() const; - OrderBook get_order_book(const char*) const; - + + // Copy all the orders *not* owned by owner to a PB type that can + // be stored on disk + OrderBook get_order_book(const char* owner) const; + + // Get a single order Order const *get_order(std::string const &clorid); - + + // Orders are kept in two multisets (one for bids, one for + // offers). Each multiset uses a comparison operator that uses + // first the price and then the time_changed of each order to sort + // the orders. FIFO matching is a matter of peeling off matching + // entries from bid and offer. using OrdersSide = std::multiset; struct Trade { diff --git a/src/server/sparkd/http/WebServer.cpp b/src/server/sparkd/http/WebServer.cpp index 11a6cea27..c5d8e8160 100644 --- a/src/server/sparkd/http/WebServer.cpp +++ b/src/server/sparkd/http/WebServer.cpp @@ -84,6 +84,7 @@ namespace http { // Turn off logging _server.clear_access_channels(websocketpp::log::alevel::all); _server.clear_error_channels(websocketpp::log::alevel::all); + _server.set_open_handshake_timeout(0); run(); } diff --git a/src/server/sparkd/md/platforms/ice_onixs/ICEListener.cpp b/src/server/sparkd/md/platforms/ice_onixs/ICEListener.cpp index 7c85642ad..d9bc8a316 100644 --- a/src/server/sparkd/md/platforms/ice_onixs/ICEListener.cpp +++ b/src/server/sparkd/md/platforms/ice_onixs/ICEListener.cpp @@ -14,13 +14,21 @@ namespace md { namespace ice { template <> -double double_of_price(const onixs_ice::Rational& price, int8_t display_factor) +double double_of_price(const onixs_ice::Rational& price, int8_t denominator) { - UNUSED(display_factor); + UNUSED(denominator); double dbl = price.convertTo(); return dbl; } +template<> +void ICEListener::set_security_specific(const onixs_ice::FuturesProductDefinition& def, + FixedSecurity& fsec) +{ + //fsec.cash_per_point = double_of_price(def.tickValue, fsec.ice_price_denominator); + //fsec.cash_per_point = def.tickValue; + fsec.cash_per_point = def.unitQuantity; +} ICEListener::ICEListener(ICESecurityIndex& index, MDProducer& producer, const std::vector channels) : _index(index), @@ -35,20 +43,21 @@ ICEListener::~ICEListener() {} std::string ICEListener::parse_contract_symbol(const std::string& sym) { - const auto&& res = sym.substr(0, sym.find(' ')); - return res; + auto end = sym.find(' '); + if(end == std::string::npos) + return sym; + else + return sym.substr(0, end); } // //State change -void ICEListener::onHandlerStateChange(const onixs_ice::HandlerStateChange& ) +void ICEListener::onHandlerStateChange(const onixs_ice::HandlerStateChange&) { - //TODO } -void ICEListener::onMarketStateChange(const onixs_ice::MarketStateChange& , const onixs_ice::MessageInfo& ) +void ICEListener::onMarketStateChange(const onixs_ice::MarketStateChange& , const onixs_ice::MessageInfo&) { - //TODO } // @@ -84,18 +93,12 @@ void ICEListener::onNewExpiry(const onixs_ice::NewExpiry& msg, bool& is_interest void ICEListener::onFuturesStrategyDefinition(const onixs_ice::FuturesStrategyDefinition& def, bool& is_interested) { - //TODO need to find out how to produce data for testing - //on_uds_future_spread(def, is_interested); - UNUSED(def); - UNUSED(is_interested); + on_uds_future_spread(def, is_interested); } //This is only for UDS void ICEListener::onNewFuturesStrategyDefinition(const onixs_ice::NewFuturesStrategyDefinition& msg, bool& is_interested, const onixs_ice::MessageInfo& ) { - //TODO need to find out how to produce data for testing - //on_uds_future_spread(msg, is_interested); - UNUSED(msg); - UNUSED(is_interested); + on_uds_future_spread(msg, is_interested); } void ICEListener::onOptionsProductDefinition(const onixs_ice::OptionsProductDefinition& msg, bool& is_interested) @@ -122,19 +125,51 @@ void ICEListener::onNewOptionsStrategyDefinition(const onixs_ice::NewOptionsStra } // -// Gap Recovery -- This is not necessary if we subscribe to the onOrderBookUpdated callback, they handle the breaks transparently + void ICEListener::onMarketSnapshot(const onixs_ice::MarketSnapshot& msg, const onixs_ice::MessageInfo& msg_info) { - UNUSED(msg); UNUSED(msg_info); + auto* data = _index.get_security(msg.marketId); + if (data != nullptr) + { + bts_assert(data->id == msg.marketId); + auto ts = utils::TimeStamp::now(); + auto settle_denominator = data->handle.security().ice_settle_denominator; + if (msg.settlePriceDateTime > 0) { + data->handle.update_settlement( ts, convert_price(msg.settlementPrice, settle_denominator), date_of_timestamp(msg.settlePriceDateTime) ); + } + + LOG(md,Diag,"OnixSICE Got Market Snap : {} - {}", std::to_string(data->id), std::to_string(msg.settlementPrice)); + + auto deal_denominator = data->handle.security().ice_deal_denominator; + data->handle.update_opening( ts, double_of_price(msg.openingPrice, deal_denominator), datetime::today() ); + data->handle.update_session_high( ts, double_of_price(msg.high, deal_denominator) ); + data->handle.update_session_low( ts, double_of_price(msg.low, deal_denominator) ); + + data->handle.set_last_trade( ts, pricesize_of_price_size(msg.lastTradePrice, deal_denominator, msg.lastTradeQuantity), + ll_of_timestamp(msg.lastTradeDateTime), msg.volume ); + } } +void ICEListener::onMarketStatistics(const onixs_ice::MarketStatistics& msg, const onixs_ice::MessageInfo& msg_info) +{ + UNUSED(msg_info); + auto* data = _index.get_security(msg.marketId); + if (data != nullptr) + { + bts_assert(data->id == msg.marketId); + data->handle.set_total_volume(msg.volume); + } +} + +// Gap Recovery -- This is not necessary if we subscribe to the onOrderBookUpdated callback, they handle the breaks transparently void ICEListener::onMarketSnapshotOrder(const onixs_ice::MarketSnapshotOrder& msg, const onixs_ice::MessageInfo& msg_info) { UNUSED(msg); UNUSED(msg_info); } +// Gap Recovery -- This is not necessary if we subscribe to the onOrderBookUpdated callback, they handle the breaks transparently void ICEListener::onMarketSnapshotPriceLevel(const onixs_ice::MarketSnapshotPriceLevel& msg, const onixs_ice::MessageInfo& msg_info) { UNUSED(msg); @@ -151,7 +186,7 @@ void ICEListener::onOrderBookUpdated(const onixs_ice::OrderBook& onixs_book, con if (data == nullptr) return; bts_assert(data->id == onixs_book.marketId()); - auto display_factor = data->handle.security().display_factor; + auto denominator = data->handle.security().ice_price_denominator; // Overwrite all entries every time size_t bids_size = onixs_book.bids().size(); @@ -169,20 +204,21 @@ void ICEListener::onOrderBookUpdated(const onixs_ice::OrderBook& onixs_book, con { auto& bid = onixs_book.bids()[x]; if(x < COCONUT_MAX_DEPTH) - dbids[x] = pricesize_of_price_size(bid.price, display_factor, bid.quantity - bid.impliedQuantity); + dbids[x] = pricesize_of_price_size(bid.price, denominator, bid.quantity - bid.impliedQuantity); if(x < COCONUT_MAX_IMPLIED_DEPTH) - ibids[x] = pricesize_of_price_size(bid.price, display_factor, bid.impliedQuantity); + ibids[x] = pricesize_of_price_size(bid.price, denominator, bid.impliedQuantity); } for (size_t x = 0; x < asks_size; ++x) { auto& ask = onixs_book.offers()[x]; if(x < COCONUT_MAX_DEPTH) - dasks[x] = pricesize_of_price_size(ask.price, display_factor, ask.quantity - ask.impliedQuantity); + dasks[x] = pricesize_of_price_size(ask.price, denominator, ask.quantity - ask.impliedQuantity); if(x < COCONUT_MAX_IMPLIED_DEPTH) - iasks[x] = pricesize_of_price_size(ask.price, display_factor, ask.impliedQuantity); + iasks[x] = pricesize_of_price_size(ask.price, denominator, ask.impliedQuantity); } data->handle.update_books(TimeStamp::now(), dbids, dasks, ibids, iasks, (uint32_t) msg_info.sequenceNumber); -} +} + // // Trades @@ -199,12 +235,13 @@ AggressorSide map_aggressor(onixs_ice::AggressorSide::Enum agside) } } -void on_trade(ICESecurityData* data, const onixs_ice::Price& price, int size, onixs_ice::AggressorSide::Enum agside, const onixs_ice::DateTime& onixs_date) +void on_trade(ICESecurityData* data, const onixs_ice::Price& price, int size, + onixs_ice::AggressorSide::Enum agside, const onixs_ice::DateTime& onixs_date, bool from_spread) { auto ts = utils::TimeStamp::now(); - PriceSize ps = pricesize_of_price_size(price, data->handle.security().display_factor, size); + PriceSize ps = pricesize_of_price_size(price, data->handle.security().ice_deal_denominator, size); auto date = ll_of_timestamp(onixs_date); - data->handle.add_trade(ts, ps, date, map_aggressor(agside), size); + data->handle.add_trade(ts, ps, date, map_aggressor(agside), 0u, from_spread); } void ICEListener::onTrade(const onixs_ice::Trade& msg, const onixs_ice::MessageInfo& ) @@ -216,7 +253,7 @@ void ICEListener::onTrade(const onixs_ice::Trade& msg, const onixs_ice::MessageI auto size = msg.quantity; auto price = msg.price; onixs_ice::AggressorSide::Enum agside = msg.aggressorSide; - on_trade(data, price, size, agside, msg.transactDateTime); + on_trade(data, price, size, agside, msg.transactDateTime, msg.isSystemPricedLeg); } } @@ -264,10 +301,12 @@ void ICEListener::onSettlementPrice(const onixs_ice::SettlementPrice& msg, const if (data != nullptr) { bts_assert(data->id == msg.marketId); - onixs_ice::Price price = msg.settlementPrice; - onixs_ice::DateTime time = msg.dateTime; - auto display_factor = data->handle.security().display_factor; - data->handle.update_settlement(utils::TimeStamp::now(), convert_price(price, display_factor), date_of_timestamp(time)); + auto ts = utils::TimeStamp::now(); + auto price = msg.settlementPrice; + auto time = msg.dateTime; + auto denominator = data->handle.security().ice_settle_denominator; + LOG(md,Diag,"OnixSICE Got Settlement: {} - {}", std::to_string(data->id), std::to_string(price)); + data->handle.update_settlement(ts, convert_price(price, denominator), date_of_timestamp(time)); } } @@ -285,8 +324,8 @@ void ICEListener::onOpenPrice(const onixs_ice::OpenPrice& msg, const onixs_ice:: bts_assert(data->id == msg.marketId); onixs_ice::Price price = msg.openPrice; onixs_ice::DateTime time = msg.dateTime; - auto display_factor = data->handle.security().display_factor; - data->handle.update_opening( utils::TimeStamp::now(), double_of_price(price, display_factor), date_of_timestamp(time)); + auto denominator = data->handle.security().ice_deal_denominator; + data->handle.update_opening( utils::TimeStamp::now(), double_of_price(price, denominator), date_of_timestamp(time)); } } @@ -298,8 +337,8 @@ void ICEListener::onClosePrice(const onixs_ice::ClosePrice& msg, const onixs_ice bts_assert(data->id == msg.marketId); onixs_ice::Price price = msg.closePrice; onixs_ice::DateTime time = msg.dateTime; - auto display_factor = data->handle.security().display_factor; - data->handle.update_closing( utils::TimeStamp::now(), double_of_price(price, display_factor), date_of_timestamp(time)); + auto denominator = data->handle.security().ice_deal_denominator; + data->handle.update_closing( utils::TimeStamp::now(), double_of_price(price, denominator), date_of_timestamp(time)); } } @@ -310,11 +349,16 @@ void ICEListener::onEndOfDayMarketSummary(const onixs_ice::EndOfDayMarketSummary { bts_assert(data->id == msg.marketId); onixs_ice::DateTime time = msg.dateTime; - auto display_factor = data->handle.security().display_factor; - data->handle.update_opening( utils::TimeStamp::now(), double_of_price(msg.openingPrice, display_factor), date_of_timestamp(time) ); - data->handle.update_session_high( utils::TimeStamp::now(), double_of_price(msg.high, display_factor) ); - data->handle.update_session_low( utils::TimeStamp::now(), double_of_price(msg.low, display_factor) ); - data->handle.update_settlement( utils::TimeStamp::now(), convert_price(msg.settlementPrice, display_factor), date_of_timestamp(time) ); + auto ts = utils::TimeStamp::now(); + auto settle_denominator = data->handle.security().ice_settle_denominator; + data->handle.update_settlement( ts, convert_price(msg.settlementPrice, settle_denominator), date_of_timestamp(time) ); + + LOG(md,Warn,"OnixSICE Got EOD Market : {} - {}", std::to_string(data->id), std::to_string(msg.settlementPrice)); + + auto deal_denominator = data->handle.security().ice_deal_denominator; + data->handle.update_opening( ts, double_of_price(msg.openingPrice, deal_denominator), date_of_timestamp(time) ); + data->handle.update_session_high( ts, double_of_price(msg.high, deal_denominator) ); + data->handle.update_session_low( ts, double_of_price(msg.low, deal_denominator) ); } } diff --git a/src/server/sparkd/md/platforms/ice_onixs/ICEListener.h b/src/server/sparkd/md/platforms/ice_onixs/ICEListener.h index 25eacf3aa..b73cf1c49 100644 --- a/src/server/sparkd/md/platforms/ice_onixs/ICEListener.h +++ b/src/server/sparkd/md/platforms/ice_onixs/ICEListener.h @@ -21,7 +21,7 @@ namespace ice { class ICEListener : public onixs_ice::ExchangeListener, //One Thread public onixs_ice::HandlerStateChangeListener, //One thread - public onixs_ice::OrderBookUpdateListener, //One thread //TODO? + public onixs_ice::OrderBookUpdateListener, //One thread public onixs_ice::FeedListener, //Multi thread public onixs_ice::ErrorListener, //Multi thread public onixs_ice::WarningListener //Multi thread @@ -85,6 +85,7 @@ public: void onOpenPrice(const onixs_ice::OpenPrice& msg, const onixs_ice::MessageInfo& msgInfo) override; void onClosePrice(const onixs_ice::ClosePrice& msg, const onixs_ice::MessageInfo& msgInfo) override; void onEndOfDayMarketSummary(const onixs_ice::EndOfDayMarketSummary& msg, const onixs_ice::MessageInfo& msgInfo) override; + void onMarketStatistics(const onixs_ice::MarketStatistics& msg, const onixs_ice::MessageInfo& msgInfo) override; // Feed void onMulticastMessageBlockBegin(onixs_ice::FeedId feedId, const onixs_ice::MessageInfo& msgInfo) override; @@ -104,6 +105,8 @@ private: template void set_security_common(const Definition& def, FixedSecurity& fsec); template + void set_security_specific(const Definition& def, FixedSecurity& fsec); + template void on_future(const Definition& def, bool& is_interested); template void on_uds_future_spread(const Definition& def, bool& is_interested); @@ -121,9 +124,12 @@ private: SecurityFilterAction filter_security(int exchange_id, const Definition& def) { if (_index.has_security(exchange_id)) + { + LOG(md,Warn, "Duplicate exchange ID {} - {}", std::to_string(exchange_id), def.contractSymbol); return SKIP; - // if(_symbol_filter.count("*") > 0) - // return ADD; + } + if(_symbol_filter.count("*") > 0) + return ADD; return _symbol_filter.count( parse_contract_symbol(def.contractSymbol) ) > 0 ? ADD : SKIP; } void flush_books(); @@ -141,16 +147,25 @@ void ICEListener::set_security_common(const Definition& def, FixedSecurity& fsec { fsec.numeric_id = def.marketId; std::string sym = parse_contract_symbol(def.contractSymbol); - fsec.symbol( sym ); - //fsec.security_desc(def.marketDesc); - fsec.security_desc(def.contractSymbol); + if(!sym.empty()) + { + fsec.symbol( sym ); + fsec.security_desc(def.contractSymbol); + } + else + { + auto id = std::to_string(def.marketId); + fsec.symbol(id); + fsec.security_desc(id); + } fsec.exchange = api::Venue::VENUE_ICE; - fsec.display_factor = (int8_t)(def.orderPriceDenominator - '0'); - fsec.ice_deal_display_factor = (int8_t)(def.dealPriceDenominator - '0'); - fsec.ice_settle_display_factor = (int8_t)(def.settlePriceDenominator - '0'); - double min_tick = double_of_price(def.incrementPrice, fsec.display_factor); - fsec.low_tick = fsec.high_tick = min_tick; - fsec.price_break = 0.; + fsec.display_factor = 0.0; + fsec.ice_price_denominator = (int8_t)(def.orderPriceDenominator - '0'); + fsec.ice_deal_denominator = (int8_t)(def.dealPriceDenominator - '0'); + fsec.ice_settle_denominator = (int8_t)(def.settlePriceDenominator - '0'); + double min_tick = double_of_price(def.incrementPrice, fsec.ice_price_denominator); + fsec.low_tick = fsec.high_tick = min_tick; + fsec.price_break = 0.0; fsec.ice_market_description(def.marketDesc); fsec.ice_cleared_alias(def.clearedAlias); @@ -160,6 +175,10 @@ void ICEListener::set_security_common(const Definition& def, FixedSecurity& fsec fsec.mic_code(def.micCode); } +template +void ICEListener::set_security_specific(const Definition&, FixedSecurity& ) {} + + template void ICEListener::on_future(const Definition& def, bool& is_interested) { @@ -169,7 +188,7 @@ void ICEListener::on_future(const Definition& def, bool& is_interested) FixedFutureSecurity fsec; set_security_common(def, fsec); - //fsec.cash_per_point = double_of_price(def.tickValue, fsec.display_factor); + set_security_specific(def, fsec); fsec.expiration_year = def.maturityYear; fsec.expiration_month = def.maturityMonth; @@ -197,11 +216,18 @@ void ICEListener::on_uds_future_spread(const Definition& def, bool& is_intereste for(size_t x = 0; x < fsec.leg_count(); ++x) { const auto& leg = legs[x]; - if(leg.ratioPriceNumerator) - fsec.legs[x].leg_price = double_of_price(leg.ratioPriceNumerator.value(), leg.ratioPriceDenominator.value()); - + if(leg.ratioPriceNumerator && leg.ratioPriceDenominator) + { + auto num = *leg.ratioPriceNumerator; + auto denom = *leg.ratioPriceDenominator; + fsec.legs[x].leg_price = num / (double) denom; + } if(leg.ratioQtyNumerator && leg.ratioQtyDenominator) - fsec.legs[x].leg_delta = double_of_price(leg.ratioQtyNumerator.value(), leg.ratioQtyDenominator.value()); + { + auto num = *leg.ratioQtyNumerator; + auto denom = *leg.ratioQtyDenominator; + fsec.legs[x].leg_delta = num / (double) denom; + } fsec.legs[x].leg_quantity = leg.ratio; fsec.legs[x].leg_side = leg.side == onixs_ice::Side::Bid ? api::SIDE_BUY : api::SIDE_SELL; diff --git a/src/server/sparkd/md/platforms/ice_onixs/ICEListenerHelpers.h b/src/server/sparkd/md/platforms/ice_onixs/ICEListenerHelpers.h index 7efa7b049..34d6f09f4 100644 --- a/src/server/sparkd/md/platforms/ice_onixs/ICEListenerHelpers.h +++ b/src/server/sparkd/md/platforms/ice_onixs/ICEListenerHelpers.h @@ -16,25 +16,25 @@ namespace md { namespace ice { template -double double_of_price(const OnixPrice& price, int8_t display_factor) +double double_of_price(const OnixPrice& price, int8_t denominator) { - double dbl = price / pow10(display_factor); + double dbl = price / pow10(denominator); return dbl; } template -IntPrice convert_price(const OnixsPrice& price, int8_t display_factor) +IntPrice convert_price(const OnixsPrice& price, int8_t denominator) { - double dbl = double_of_price(price, display_factor); + double dbl = double_of_price(price, denominator); const int PRECISION = 7; //TODO config::get().connection() case ICE auto intpx = md::IntPrice_from_double(dbl); return intpx; } template -inline PriceSize pricesize_of_price_size(const OnixsPrice& price, int8_t display_factor, Size size) +inline PriceSize pricesize_of_price_size(const OnixsPrice& price, int8_t denominator, Size size) { - auto intpx = convert_price(price, display_factor); + auto intpx = convert_price(price, denominator); PriceSize result({intpx.mantissa(), (int8_t)(intpx.exponent())}, size); return result; } diff --git a/src/server/sparkd/md/platforms/ice_onixs/ICEMain.cpp b/src/server/sparkd/md/platforms/ice_onixs/ICEMain.cpp index ea2b3ef9b..c9cb6a79b 100644 --- a/src/server/sparkd/md/platforms/ice_onixs/ICEMain.cpp +++ b/src/server/sparkd/md/platforms/ice_onixs/ICEMain.cpp @@ -74,11 +74,30 @@ onixs_ice::MarketSubscriptions market_subcription_for_channel(const ICEChannel& for(const auto& x : chan.market_types) { //onixs_ice::MarketSubType::NonImplied - onixs_ice::MarketSubscription futures(x, onixs_ice::MarketSubType::FullImplied, onixs_ice::SecurityType::Futures, onixs_ice::BookDepth::FOD); - onixs_ice::MarketSubscription spreads(x, onixs_ice::MarketSubType::FullImplied, onixs_ice::SecurityType::UdsFuturesMarkets, onixs_ice::BookDepth::FOD); - subs.push_back(futures); - subs.push_back(spreads); + if(x == 26) + { + onixs_ice::MarketSubscription futures(x, onixs_ice::MarketSubType::NonImplied, onixs_ice::SecurityType::Futures, onixs_ice::BookDepth::FOD); + subs.push_back(futures); + } + else + { + onixs_ice::MarketSubscription futures(x, onixs_ice::MarketSubType::FullImplied, onixs_ice::SecurityType::Futures, onixs_ice::BookDepth::FOD); + subs.push_back(futures); + } } + for(const auto& x : chan.uds_market_types) + { + if(x == 26) + { + onixs_ice::MarketSubscription spreads(x, onixs_ice::MarketSubType::NonImplied, onixs_ice::SecurityType::UdsFuturesMarkets, onixs_ice::BookDepth::FOD); + subs.push_back(spreads); + } + else + { + onixs_ice::MarketSubscription spreads(x, onixs_ice::MarketSubType::FullImplied, onixs_ice::SecurityType::UdsFuturesMarkets, onixs_ice::BookDepth::FOD); + subs.push_back(spreads); + } + } return subs; } @@ -113,7 +132,16 @@ void ICEMain::start(std::vector& symbols) LOG(md,Info, "ICE: Init Handler {}", pair.first.to_string()); sleep_for(16s); onixs_ice::MarketSubscriptions subs = market_subcription_for_channel(pair.first); - pair.second->start(subs); + try + { + pair.second->start(subs); + } + catch (std::exception& e) + { + LOG(md,Error,"handler->start ONIXSERROR {}", std::string(e.what())); + throw; + } + } onixs_destroy_temporary_license(license); LOG(md,Info, "** ICE Publisher started **"); @@ -191,8 +219,9 @@ unique_ptr ICEMain::prepare_handler(const ICEChannel& channe auto handler = unique_ptr(new onixs_ice::Handler(settings)); return handler; } - catch (std::exception& e) { - LOG(md,Error,"ONIXSERROR{}", std::string(e.what())); + catch (std::exception& e) + { + LOG(md,Error,"handler create ONIXSERROR {}", std::string(e.what())); throw; } } @@ -212,13 +241,20 @@ unique_ptr ICEMain::prepare_listener(onixs_ice::Handler& handler, c for (auto& symbol : channel.symbols) listener->add_filter_symbol(symbol.symbol()); - handler.registerExchangeListener(listener.get()); - handler.registerFeedListener(listener.get()); - handler.registerOrderBookUpdateListener(listener.get()); - handler.registerErrorListener(listener.get()); - handler.registerWarningListener(listener.get()); - handler.registerHandlerStateChangeListener(listener.get()); - + try + { + handler.registerExchangeListener(listener.get()); + handler.registerFeedListener(listener.get()); + handler.registerOrderBookUpdateListener(listener.get()); + handler.registerErrorListener(listener.get()); + handler.registerWarningListener(listener.get()); + handler.registerHandlerStateChangeListener(listener.get()); + } + catch (std::exception& e) + { + LOG(md,Error,"handler register ONIXSERROR {}", std::string(e.what())); + throw; + } return listener; } diff --git a/src/server/sparkd/md/run_md.cpp b/src/server/sparkd/md/run_md.cpp index 35ded097d..a63c5f7c4 100644 --- a/src/server/sparkd/md/run_md.cpp +++ b/src/server/sparkd/md/run_md.cpp @@ -153,8 +153,9 @@ void run_market_datapublishers(MBOWriter* mbo) { })); if (config::get().use_sim()) { - threading::make_thread("ExchangeSim", [mbo]() { sim::start_exchange_sim(mbo); }) - .detach(); + threading::make_thread("ExchangeSimGLBX", [mbo]() { sim::start_exchange_sim(api::Venue::VENUE_SIM, mbo); }).detach(); + //threading::make_thread("ExchangeSimICE", [mbo]() { sim::start_exchange_sim(api::Venue::VENUE_SIMICE, mbo); }).detach(); + //TODO I'll need Eric's help to disambiguate replay / relay here } boot_sleep(); } diff --git a/src/server/sparkd/position/PositionServicePNLHelper.cpp b/src/server/sparkd/position/PositionServicePNLHelper.cpp index 403c9b52b..a547f6643 100644 --- a/src/server/sparkd/position/PositionServicePNLHelper.cpp +++ b/src/server/sparkd/position/PositionServicePNLHelper.cpp @@ -4,6 +4,7 @@ #include "lib/os/os.h" #include "PositionServicePNLHelper.h" #include "lib/config/protos/cme_reference_data.pb.h" +#include "lib/config/protos/toggles.pb.h" #include "lib/config/config.h" #include "lib/utils/PBUtils.h" @@ -217,6 +218,15 @@ double PositionServicePNLHelper::_compute_day_pnl ( (md - u.pos().day_long_avg_price()), short_pnl = std::abs(u.pos().day_short_pos()) * u.cash_value() * (u.pos().day_short_avg_price() - md); + + if(config::get().enabled()) + { + if(bts::os::is_nan(long_pnl)) + long_pnl = 0.0; + if(bts::os::is_nan(short_pnl)) + short_pnl = 0.0; + } + pnl = long_pnl + short_pnl; break; } diff --git a/src/server/sparkd/support/ErrorReportService.cpp b/src/server/sparkd/support/ErrorReportService.cpp index 1a90be267..5b972267a 100644 --- a/src/server/sparkd/support/ErrorReportService.cpp +++ b/src/server/sparkd/support/ErrorReportService.cpp @@ -29,6 +29,17 @@ ErrorReportServer::ErrorReportServer(bts::api::ServiceContext& context) : ErrorReportServer::~ErrorReportServer() {} +void write_file(const std::string& destination, const std::string& data) { + if (data.size() > 0) { + std::ofstream file; + file.open(destination, std::ios::binary); + if (file.is_open()) { + file << data; + file.close(); + } + } +} + void ErrorReportServer::send_report( const ErrorReport& request, const api::Response& reply) { @@ -37,7 +48,7 @@ void ErrorReportServer::send_report( auto dt = datetime::local_now(); auto dir_name = request.user() + "-" + datetime::FIX_timestamp(dt); dir_name = os::sanitize_file_name(dir_name); - dir_name = get_report_dir() + dir_name; + dir_name = get_report_dir() + dir_name + "/"; // create directory with that name boost::filesystem::create_directories(dir_name); @@ -46,7 +57,7 @@ void ErrorReportServer::send_report( // user_text.txt std::ofstream file; - file.open(dir_name + "/user_text.txt"); + file.open(dir_name + "user_text.txt"); if (file.is_open()) { file << request.user() << " - " << request.contact() << std::endl; file << "================" << std::endl; @@ -56,20 +67,13 @@ void ErrorReportServer::send_report( file.close(); } // logs.txt - if (request.log().size() > 0) { - file.open(dir_name + "/logs.txt"); - if (file.is_open()) { - file << request.log(); - file.close(); - } - } + write_file(dir_name + "logs.txt", request.log()); // screenshot.jpeg - if (request.image().size() > 0) { - file.open(dir_name + "/screenshot.jpeg", std::ios::binary); - if (file.is_open()) { - file << request.image(); - file.close(); - } + write_file(dir_name + "screenshot.jpeg", request.image()); + for (const auto& fd : request.files()) { + // boost::filesystem::path p = fd.name(); + auto name = os::sanitize_file_name(fd.name()); + write_file(dir_name + name, fd.contents()); } // log error for visibility diff --git a/src/server/sparkd/support/ShellLatencyServer.cpp b/src/server/sparkd/support/ShellLatencyServer.cpp index eb5657b87..ba0cb791e 100644 --- a/src/server/sparkd/support/ShellLatencyServer.cpp +++ b/src/server/sparkd/support/ShellLatencyServer.cpp @@ -55,6 +55,11 @@ void ShellLatencyServer::report( } } +void ShellLatencyServer::ping( + const ::bts::metrics::BufferMessage& request, + const api::Response<::bts::metrics::BufferMessage>& reply) { + reply(request); +} }} \ No newline at end of file diff --git a/src/server/sparkd/support/ShellLatencyServer.h b/src/server/sparkd/support/ShellLatencyServer.h index 3c68833ea..d31c302b0 100644 --- a/src/server/sparkd/support/ShellLatencyServer.h +++ b/src/server/sparkd/support/ShellLatencyServer.h @@ -28,6 +28,10 @@ class ShellLatencyServer : public ShellLatencyServiceBase { void report( const ::bts::metrics::Report& request, const api::Response<::bts::api::VoidMessage>& reply) override; + + void ping( + const ::bts::metrics::BufferMessage& request, + const api::Response<::bts::metrics::BufferMessage>& reply) override; }; diff --git a/src/server/sparkd/user/AdminServiceServer.cpp b/src/server/sparkd/user/AdminServiceServer.cpp index b795c7f39..db3a0c7ad 100644 --- a/src/server/sparkd/user/AdminServiceServer.cpp +++ b/src/server/sparkd/user/AdminServiceServer.cpp @@ -194,10 +194,10 @@ void AdminServiceServer::get_filtered_admin_data( newuser->CopyFrom(*acctuser); if(!unit.admin_permission().has_md_permissions()) newuser->mutable_user_data()->mutable_exchange_user_data()-> - mutable_cme_data()->set_admin_display_flag(bts::api::admin::CMEUserData_AdminDisplay::CMEUserData_AdminDisplay_Routing); + set_admin_display_flag(bts::api::admin::ExchangeUserData_AdminDisplay::ExchangeUserData_AdminDisplay_Routing); else newuser->mutable_user_data()->mutable_exchange_user_data()-> - mutable_cme_data()->set_admin_display_flag(bts::api::admin::CMEUserData_AdminDisplay::CMEUserData_AdminDisplay_All); + set_admin_display_flag(bts::api::admin::ExchangeUserData_AdminDisplay::ExchangeUserData_AdminDisplay_All); } } } @@ -212,7 +212,7 @@ void AdminServiceServer::get_filtered_admin_data( const auto newuser = filtered.add_users(); newuser->CopyFrom(*acctuser); newuser->mutable_user_data()->mutable_exchange_user_data()-> - mutable_cme_data()->set_admin_display_flag(bts::api::admin::CMEUserData_AdminDisplay::CMEUserData_AdminDisplay_MD); + set_admin_display_flag(bts::api::admin::ExchangeUserData_AdminDisplay::ExchangeUserData_AdminDisplay_MD); } } } diff --git a/src/server/sparkd/user/AuthServiceServer.cpp b/src/server/sparkd/user/AuthServiceServer.cpp index 357a683c9..9853fb0d4 100644 --- a/src/server/sparkd/user/AuthServiceServer.cpp +++ b/src/server/sparkd/user/AuthServiceServer.cpp @@ -33,6 +33,12 @@ AuthServiceServer::AuthServiceServer (bts::api::ServiceContext& context) : AuthS AuthServiceServer::~AuthServiceServer () {} +void AuthServiceServer::clear_all_sessions(const api::VoidMessage& , + const std::function& ) +{ + api::auth::clear_all_sessions(); +} + void AuthServiceServer::get_users(const api::user::GetUsersRequest& , const std::function& respond) { diff --git a/src/server/sparkd/user/AuthServiceServer.h b/src/server/sparkd/user/AuthServiceServer.h index 91df48a8e..0e00cc334 100644 --- a/src/server/sparkd/user/AuthServiceServer.h +++ b/src/server/sparkd/user/AuthServiceServer.h @@ -20,6 +20,9 @@ public: void authenticate_login(const api::user::AuthenticateLoginRequest& , const std::function& ) override; + void clear_all_sessions(const api::VoidMessage& , + const std::function& ) override; + private: struct PImpl; std::unique_ptr me; diff --git a/src/web/ui/src/Access.fs b/src/web/ui/src/Access.fs index 45d1fb9b4..883319c04 100644 --- a/src/web/ui/src/Access.fs +++ b/src/web/ui/src/Access.fs @@ -81,29 +81,19 @@ module Accounts = { cmeAccountData = { cti = CTICode.CtiCodeUndefined origin = CustomerOrFirm.CustomerOrFirmCustomer - takeupMemberCode = "" clearingAccount = "" - directElectronicAccess = "" - tradingCapacity = "" - liquidityProvision = "" - commodityDerivativeIndicator = "" - investmentDecisionId = "" - executionDecisionId = "" - clientId = "" miscTags = JSMap.empty } iceAccountData = - { nybitHouse="" - wceHouse=0 - clearingAccountOverride="" - directElectronicAccess=false - tradingCapacity="" - liquidityProvision=false - commodityDerivativeIndicator=false - investmentDecisionId="" - executionDecisionId="" - clientId="" - mfidId="" } + { + cti = CTICode.CtiCodeUndefined + origin = CustomerOrFirm.CustomerOrFirmCustomer + clearingAccount = "" + clearingFirm = "" + customerAccountRefId = "" + accountCode = "" + miscTags = JSMap.empty + } simAccountData = { cti = CTICode.CtiCodeUndefined origin = CustomerOrFirm.CustomerOrFirmCustomer diff --git a/src/web/ui/src/Accounts/State.fs b/src/web/ui/src/Accounts/State.fs index 4647425ee..48a628c19 100644 --- a/src/web/ui/src/Accounts/State.fs +++ b/src/web/ui/src/Accounts/State.fs @@ -207,14 +207,18 @@ let applyUpdate (account:BTSAccount) = function let applyCMEUpdate (data:CMEAccountData) = function | CMEExchangeField.CTI v -> { data with cti = v } | CMEExchangeField.Origin v -> { data with origin = v } - | CMEExchangeField.TakeupMember v -> { data with takeupMemberCode = v } | CMEExchangeField.ClearingAccount v -> { data with clearingAccount = v } - | CMEExchangeField.DirectElectronicAccess v -> { data with directElectronicAccess = v } - | CMEExchangeField.TradingCapacity v -> { data with tradingCapacity = v } - | CMEExchangeField.LiquidityProvision v -> { data with liquidityProvision = v } - | CMEExchangeField.CommodityDerivativeIndicator v -> { data with commodityDerivativeIndicator = v } | CMEExchangeField.MiscTagMapping mp -> { data with miscTags = mp } +let applyICEUpdate (data:ICEAccountData) = function + | ICEExchangeField.CTI v -> { data with cti = v } + | ICEExchangeField.Origin v -> { data with origin = v } + | ICEExchangeField.ClearingFirm v -> { data with clearingFirm = v } + | ICEExchangeField.ClearingAccount v -> { data with clearingAccount = v } + | ICEExchangeField.MiscTagMapping mp -> { data with miscTags = mp } + | ICEExchangeField.CustomerAccountRefId v -> { data with customerAccountRefId = v } + | ICEExchangeField.AccountCode v -> { data with accountCode = v } + let applySIMUpdate (data:SIMAccountData) = function | CTI v -> { data with cti = v } //| SIMExchangeField.Origin v -> { data with origin = v } @@ -576,6 +580,10 @@ let update msg (model:Model) : Model * Cmd = let newCme = applyCMEUpdate model.scratch.exchangeAccountData.cmeAccountData field let scratch = { model.scratch with exchangeAccountData = { cmeAccountData = newCme; iceAccountData = model.scratch.exchangeAccountData.iceAccountData; simAccountData = model.scratch.exchangeAccountData.simAccountData } } { model with scratch = scratch }, Cmd.Empty + | UpdateICE (field:ICEExchangeField) -> + let newICE = applyICEUpdate model.scratch.exchangeAccountData.iceAccountData field + let scratch = { model.scratch with exchangeAccountData = { cmeAccountData = model.scratch.exchangeAccountData.cmeAccountData; iceAccountData = newICE; simAccountData = model.scratch.exchangeAccountData.simAccountData } } + { model with scratch = scratch }, Cmd.Empty | UpdateSIM field -> let newSim = applySIMUpdate model.scratch.exchangeAccountData.simAccountData field let scratch = { model.scratch with exchangeAccountData = { cmeAccountData = model.scratch.exchangeAccountData.cmeAccountData ; iceAccountData = model.scratch.exchangeAccountData.iceAccountData; simAccountData = newSim } } diff --git a/src/web/ui/src/Accounts/Types.fs b/src/web/ui/src/Accounts/Types.fs index a0bf2442f..fa7daafcc 100644 --- a/src/web/ui/src/Accounts/Types.fs +++ b/src/web/ui/src/Accounts/Types.fs @@ -17,7 +17,7 @@ type Tab = | PositionManagementTab | MarginTab -let venues = [| Venue.GLBX; Venue.SIM |] +let venues = [| Venue.GLBX; Venue.ICE; Venue.SIM |] let securityTypes = [| SecurityType.FUTURE; SecurityType.SPREAD |] @@ -32,12 +32,16 @@ type AccountField = type CMEExchangeField = | CTI of CTICode | Origin of CustomerOrFirm - | TakeupMember of string | ClearingAccount of string - | DirectElectronicAccess of string - | TradingCapacity of string - | LiquidityProvision of string - | CommodityDerivativeIndicator of string + | MiscTagMapping of JSMap + +type ICEExchangeField = + | CTI of CTICode + | Origin of CustomerOrFirm + | ClearingAccount of string + | ClearingFirm of string + | CustomerAccountRefId of string + | AccountCode of string | MiscTagMapping of JSMap type SIMExchangeField = @@ -113,6 +117,7 @@ type Msg = | DeleteSecurityLimit of int // ExchangeData | UpdateCME of CMEExchangeField + | UpdateICE of ICEExchangeField | UpdateSIM of SIMExchangeField | RequestProducts | UpdateProducts of Product[] diff --git a/src/web/ui/src/Accounts/View.fs b/src/web/ui/src/Accounts/View.fs index 1c624e10e..5a7f68640 100644 --- a/src/web/ui/src/Accounts/View.fs +++ b/src/web/ui/src/Accounts/View.fs @@ -1108,19 +1108,30 @@ let creditLimitsView model dispatch = div [] [str "lorem ipsum"] let exchangeSettings (model:Model) dispatch = let cme = model.scratch.exchangeAccountData.cmeAccountData + let ice = model.scratch.exchangeAccountData.iceAccountData let sim = model.scratch.exchangeAccountData.simAccountData - let send = UpdateCME>>dispatch + let sendCME = UpdateCME>>dispatch + let sendICE = UpdateICE>>dispatch let sendSim = UpdateSIM>>dispatch div [] [ h2 [Class "is-size-3"] [ str "CME" ] - Fields.enumSelect "CTI" (toHuman false CTICode.values cme.cti) (allToHuman false CTICode.values) [] (fromHuman CTICode.values>>CMEExchangeField.CTI>>send) - Fields.enumSelect "Origin" (toHuman false CustomerOrFirm.values cme.origin) (allToHuman false CustomerOrFirm.values) [] (fromHuman CustomerOrFirm.values>>CMEExchangeField.Origin>>send) + Fields.enumSelect "CTI" (toHuman false CTICode.values cme.cti) (allToHuman false CTICode.values) [] (fromHuman CTICode.values>>CMEExchangeField.CTI>>sendCME) + Fields.enumSelect "Origin" (toHuman false CustomerOrFirm.values cme.origin) (allToHuman false CustomerOrFirm.values) [] (fromHuman CustomerOrFirm.values>>CMEExchangeField.Origin>>sendCME) //Fields.text "Takeup Member Code" cme.takeupMemberCode [] (TakeupMember>>send) - Fields.text "Clearing Account" cme.clearingAccount [MaxLength 12.0] (CMEExchangeField.ClearingAccount>>send) + Fields.text "Clearing Account" cme.clearingAccount [MaxLength 12.0] (CMEExchangeField.ClearingAccount>>sendCME) //Fields.text "Direct Electronic Access" cme.directElectronicAccess [] (DirectElectronicAccess>>send) //Fields.text "Trading Capacity" cme.tradingCapacity [] (TradingCapacity>>send) //Fields.text "Liquidity Provision" cme.liquidityProvision [] (LiquidityProvision>>send) //Fields.map "Misc Tag Mapping" cme.miscTags [] (MiscTagMapping>>send) + + h2 [Class "is-size-3"] [ str "ICE" ] + Fields.enumSelect "CTI" (toHuman false CTICode.values ice.cti) (allToHuman false CTICode.values) [] (fromHuman CTICode.values>>ICEExchangeField.CTI>>sendICE) + Fields.enumSelect "Origin" (toHuman false CustomerOrFirm.values ice.origin) (allToHuman false CustomerOrFirm.values) [] (fromHuman CustomerOrFirm.values>>ICEExchangeField.Origin>>sendICE) + Fields.text "Clearing Firm" ice.clearingFirm [] (ICEExchangeField.ClearingFirm>>sendICE) + Fields.text "Clearing Account" ice.clearingAccount [MaxLength 12.0] (ICEExchangeField.ClearingAccount>>sendICE) + Fields.text "Customer Account Ref ID" ice.customerAccountRefId [] (ICEExchangeField.CustomerAccountRefId>>sendICE) + Fields.text "Account Code" ice.accountCode [] (ICEExchangeField.AccountCode>>sendICE) + h2 [Class "is-size-3"] [ str "SIM" ] Fields.enumSelect "CTI" (toHuman false CTICode.values sim.cti) (allToHuman false CTICode.values) [] (fromHuman CTICode.values>>SIMExchangeField.CTI>>sendSim) Fields.enumSelect "Origin" (toHuman false CustomerOrFirm.values sim.origin) (allToHuman false CustomerOrFirm.values) [] (fromHuman CustomerOrFirm.values>>SIMExchangeField.Origin>>sendSim) diff --git a/src/web/ui/src/Controls/Fields.fs b/src/web/ui/src/Controls/Fields.fs index 6e8908f76..2a80caaa1 100644 --- a/src/web/ui/src/Controls/Fields.fs +++ b/src/web/ui/src/Controls/Fields.fs @@ -150,6 +150,7 @@ let password (lbl:string) (dv:string) (attrs:IHTMLProp list) onchange = ] @ attrs) ] ] + let number (lbl:string) (dv:string) (attrs:IHTMLProp list) (validate: string -> bool) onchange = let errorClass = match dv |> validate with @@ -183,6 +184,10 @@ let positiveIntNumber (lbl:string) (dv:string) (attrs:IHTMLProp list) onchange = number lbl dv attrs validate onchange + +let number_novalidate (lbl:string) (dv) (attrs:IHTMLProp list) onchange = + text lbl (string dv) attrs (unbox>>onchange) + let double (lbl:string) (dv:double) (attrs:IHTMLProp list) onchange = text lbl (string dv) attrs (double>>onchange) diff --git a/src/web/ui/src/Users/State.fs b/src/web/ui/src/Users/State.fs index d59dd43a1..a413d9d3f 100644 --- a/src/web/ui/src/Users/State.fs +++ b/src/web/ui/src/Users/State.fs @@ -53,16 +53,12 @@ let defaultScratchUser = { accountPermissions = { accountPermissions = JSMap.empty } exchangeUserData = { + adminDisplayFlag = bts.api.admin.ExchangeUserData.AdminDisplay.None cmeData = { - adminDisplayFlag = bts.api.admin.CMEUserData.AdminDisplay.None operatorId = "" - atsOperatorId = "" smpId = "" - textA = "" - textB = "" miscTags = JSMap.empty smpInstruction = "" - maxOrderRatePerConnection = "" location = "" proNonpro = ProfessionalDesignation.PRO displayNondisplay = DisplayDesignation.DISPLAY @@ -73,25 +69,26 @@ let defaultScratchUser = { location = "" } iceData = { + companyId = 0 + operatorId = "" authorizedTrader = "" - memoField = "" - textB = "" - maxOrderRate = 0ul + authorizedGroup = "" + password = "" + custOrderHandingInst = "" + mifidDirectElectronicAccess = -1 + mifidTradingCapacity = -1 + mifidLiquidityProvision = -1 + mifidCommodityDerivIndicator = -1 + mifidInvestmentDecision = -1L + mifidExecutionDecision = -1L + mifidClientIdCode = -1L + mifidId = -1L + miscTags = JSMap.empty + proNonpro = ProfessionalDesignation.PRO + displayNondisplay = DisplayDesignation.DISPLAY + mdPermissions = [||] } } - exchangeMdEntitlementData = - { - cmeData = - { - cbotEntitlement = { canView = false } - cmeEntitlement = { canView = false } - nymexEntitlement = { canView = false } - bmandfEntitlement = { canView = false } - comexEntitlement = { canView = false } - cmeEuropeEntitlement = { canView = false } - } - iceData = () - } } } @@ -182,21 +179,16 @@ let createUser (admin:IAdminService) (user:BTSUser) (model:Model)= promise { return ErrorOccured (sprintf "Error creating user: %s" result.errorText) } -let updateExchangeMdEntitlementDataCMEData user dta :BTSUser = { user with userData = { user.userData with exchangeMdEntitlementData = { user.userData.exchangeMdEntitlementData with cmeData = dta}}} +//let updateExchangeMdEntitlementDataCMEData user dta :BTSUser = { user with userData = { user.userData with exchangeMdEntitlementData = { user.userData.exchangeMdEntitlementData with cmeData = dta}}} let updateCMEExchangeUserData (user:BTSUser) dta = { user with userData = { user.userData with exchangeUserData = { user.userData.exchangeUserData with cmeData = dta }}} let updateSIMExchangeUserData (user:BTSUser) dta = { user with userData = { user.userData with exchangeUserData = { user.userData.exchangeUserData with simData = dta }}} +let updateICEExchangeUserData (user:BTSUser) dta = { user with userData = { user.userData with exchangeUserData = { user.userData.exchangeUserData with iceData = dta }}} let default_user_account = { - tradingPermission = + tradingPermission = { hasPermissions = false - submitMarketOrders = false - submitGtcOrders = false - algoTradingPermission = - { submitIcebergOrders = false - submitTimeBoxOrders = false - submitSpreaderOrders = false } } - adminPermission = + adminPermission = { hasPermissions = false hasMdPermissions = false } } @@ -211,15 +203,32 @@ let applyUpdate (field:UserField) (user:BTSUser) = | IsActive ia -> { user with isActive = ia } | CanCreate cr -> { user with canCreate = cr } | CMEOperatorID oid -> updateCMEExchangeUserData user { user.userData.exchangeUserData.cmeData with operatorId = oid } - | CMEATSOperatorID atsoid -> updateCMEExchangeUserData user { user.userData.exchangeUserData.cmeData with atsOperatorId = atsoid } + + | ICECompanyID oid -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with companyId = oid } + | ICEOperatorID oid -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with operatorId = oid } + | ICEAuthorizedTrader oid -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with authorizedTrader = oid } + | ICEAuthorizedGroup oid -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with authorizedGroup = oid } + | ICEPassword oid -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with password = oid } + | ICECustOrderHandingInst oid -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with custOrderHandingInst = oid } + + | MifidClientIdCode value -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with mifidClientIdCode = value } + | MifidCommodityDerivIndicator value -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with mifidCommodityDerivIndicator = value } + | MifidDirectElectronicAccess value -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with mifidDirectElectronicAccess = value } + | MifidExecutionDecision value -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with mifidExecutionDecision = value } + | MifidId value -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with mifidId = value } + | MifidInvestmentDecision value -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with mifidInvestmentDecision = value } + | MifidLiquidityProvision value -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with mifidLiquidityProvision = value } + | MifidTradingCapacity value -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with mifidTradingCapacity = value } + | SimOperatorID oid -> updateSIMExchangeUserData user { user.userData.exchangeUserData.simData with operatorId = oid } | SMPID smpid -> updateCMEExchangeUserData user { user.userData.exchangeUserData.cmeData with smpId = smpid } | CMELocation loc -> updateCMEExchangeUserData user { user.userData.exchangeUserData.cmeData with location = loc } | CMEProNonPro loc -> updateCMEExchangeUserData user { user.userData.exchangeUserData.cmeData with proNonpro = loc } | CMEDisplayNonDisplay loc -> updateCMEExchangeUserData user { user.userData.exchangeUserData.cmeData with displayNondisplay = loc } + | ICEProNonPro loc -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with proNonpro = loc } + | ICEDisplayNonDisplay loc -> updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with displayNondisplay = loc } | SimLocation loc -> updateSIMExchangeUserData user { user.userData.exchangeUserData.simData with location = loc } | SMPInstruction smpi -> updateCMEExchangeUserData user { user.userData.exchangeUserData.cmeData with smpInstruction = smpi } - | MaxOrderRatePerConnection morpc -> updateCMEExchangeUserData user { user.userData.exchangeUserData.cmeData with maxOrderRatePerConnection = morpc } | MiscTags map -> updateCMEExchangeUserData user { user.userData.exchangeUserData.cmeData with miscTags = map } | CMEMic mic -> let micArr = @@ -232,6 +241,17 @@ let applyUpdate (field:UserField) (user:BTSUser) = mdperm | None -> Array.concat [[|mic|]; mdperm] updateCMEExchangeUserData user { user.userData.exchangeUserData.cmeData with mdPermissions = micArr } + | ICEMic (mic:ICEMicData) -> + let micArr = + let mdperm = user.userData.exchangeUserData.iceData.mdPermissions + let found = Array.tryFind (fun (elm:ICEMicData) -> elm.mic = mic.mic) mdperm + match found with + | Some elm -> + let ndx = Array.findIndex (fun (e:ICEMicData) -> e.mic = elm.mic) mdperm + Array.set mdperm ndx mic + mdperm + | None -> Array.concat [[|mic|]; mdperm] + updateICEExchangeUserData user { user.userData.exchangeUserData.iceData with mdPermissions = micArr } | TradingPerm (id:string, can_trade:bool) -> let find = JSMap.tryFind id user.userData.accountPermissions.accountPermissions let current = if find.IsNone then default_user_account else find.Value diff --git a/src/web/ui/src/Users/Types.fs b/src/web/ui/src/Users/Types.fs index 51b83e88a..130711a57 100644 --- a/src/web/ui/src/Users/Types.fs +++ b/src/web/ui/src/Users/Types.fs @@ -18,6 +18,23 @@ type UserField = | CanCreate of bool | CMEOperatorID of string | CMEATSOperatorID of string +| ICEOperatorID of string +| ICECompanyID of int +| ICEAuthorizedTrader of string +| ICEAuthorizedGroup of string +| ICEPassword of string +| ICECustOrderHandingInst of string + +| MifidClientIdCode of int64 +| MifidCommodityDerivIndicator of int +| MifidDirectElectronicAccess of int +| MifidExecutionDecision of int64 +| MifidId of int64 +| MifidInvestmentDecision of int64 +| MifidLiquidityProvision of int +| MifidTradingCapacity of int + + | SimOperatorID of string | SMPID of string | CMELocation of string @@ -32,6 +49,9 @@ type UserField = | CMEMic of CMEMicData | CMEProNonPro of ProfessionalDesignation | CMEDisplayNonDisplay of DisplayDesignation +| ICEMic of ICEMicData +| ICEProNonPro of ProfessionalDesignation +| ICEDisplayNonDisplay of DisplayDesignation type Msg = @@ -71,4 +91,12 @@ let CMEMicMap:Map = Map.empty .Add("XNYM", bts.api.CMEMics.NYMEX) .Add("XCBT", bts.api.CMEMics.CBT) .Add("XCEC", bts.api.CMEMics.COMEX) - .Add("DME", bts.api.CMEMics.DME) \ No newline at end of file + .Add("DME", bts.api.CMEMics.DME) + +let ICEMicMap:Map = [ + ("IFCA", bts.api.ICEMics.IFC); + ("IFEN", bts.api.ICEMics.IFEN); + ("IFEU", bts.api.ICEMics.IFE); + ("IFLL", bts.api.ICEMics.IFL); + ("IFUS", bts.api.ICEMics.IFU) + ] |> Map.ofList diff --git a/src/web/ui/src/Users/View.fs b/src/web/ui/src/Users/View.fs index f1159f84e..4b0a7b2fe 100644 --- a/src/web/ui/src/Users/View.fs +++ b/src/web/ui/src/Users/View.fs @@ -198,22 +198,41 @@ let accountsTabView (model:Model) (dispatch:Msg->Unit) (readonly:bool) = let MarketDataTabView (model:Model) (dispatch:Msg->Unit) (readonly:bool) = let cmeData = model.scratchUser.userData.exchangeUserData.cmeData + let iceData = model.scratchUser.userData.exchangeUserData.iceData - let findCheckedValue (value:bts.api.CMEMics) = + let findCMECheckedValue (value:bts.api.CMEMics) = let checkedValue = cmeData.mdPermissions |> Array.tryFind (fun mic -> mic.mic = value && mic.enabled) match checkedValue with | Some t -> true | None -> false - let micList = [ - for micPair in CMEMicMap do - yield ( - Fields.checkbox + let findICECheckedValue (value:bts.api.ICEMics) = + let checkedValue = iceData.mdPermissions |> Array.tryFind (fun mic -> mic.mic = value && mic.enabled) + match checkedValue with + | Some t -> true + | None -> false + + let cmeMicList = [ + for micPair in CMEMicMap do + yield ( + Fields.checkbox (sprintf "%O" micPair.Key) - "" - (findCheckedValue micPair.Value) - [Disabled readonly] - (fun chk -> (({enabled = chk; mic = micPair.Value}:CMEMicData) |> (CMEMic >> UpdateField >> dispatch))) + "" + (findCMECheckedValue micPair.Value) + [Disabled readonly] + (fun chk -> (({enabled = chk; mic = micPair.Value}:CMEMicData) |> (CMEMic >> UpdateField >> dispatch))) + ) + ] + + let iceMicList = [ + for micPair in ICEMicMap do + yield ( + Fields.checkbox + (sprintf "%O" micPair.Key) + "" + (findICECheckedValue micPair.Value) + [Disabled readonly] + (fun chk -> (({enabled = chk; mic = micPair.Value}:ICEMicData) |> (ICEMic >> UpdateField >> dispatch))) ) ] @@ -223,40 +242,69 @@ let MarketDataTabView (model:Model) (dispatch:Msg->Unit) (readonly:bool) = [ h2 [Class "is-size-3"] [ str "CME" ] h3 [Class "is-size-4"] [ str "Entitlements" ] - div [Style [Padding " .5em 1em"]] micList + div [Style [Padding " .5em 1em"]] cmeMicList + + h3 [Class "is-size-4"] [ str "Designations" ] + //Fields.enumSelect "Pro / Non-Pro" (toHuman false ProfessionalDesignation.values cmeData.proNonpro) (allToHuman false ProfessionalDesignation.values) [Disabled readonly] (fromHuman ProfessionalDesignation.values>>UserField.CMEProNonPro>>send) + Fields.enumSelect "Display / Non-Display" (toHuman false DisplayDesignation.values cmeData.displayNondisplay) (allToHuman false DisplayDesignation.values) [Disabled readonly] (fromHuman DisplayDesignation.values>>UserField.CMEDisplayNonDisplay>>send) + + br [] + h2 [Class "is-size-3"] [ str "ICE" ] + h3 [Class "is-size-4"] [ str "Entitlements" ] + div [Style [Padding " .5em 1em"]] iceMicList h3 [Class "is-size-4"] [ str "Designations" ] //Pico recommends we hide this for the ICE //Fields.enumSelect "Pro / Non-Pro" (toHuman false ProfessionalDesignation.values cmeData.proNonpro) (allToHuman false ProfessionalDesignation.values) [Disabled readonly] (fromHuman ProfessionalDesignation.values>>UserField.CMEProNonPro>>send) - Fields.enumSelect "Display / Non-Display" (toHuman false DisplayDesignation.values cmeData.displayNondisplay) (allToHuman false DisplayDesignation.values) [Disabled readonly] (fromHuman DisplayDesignation.values>>UserField.CMEDisplayNonDisplay>>send) + Fields.enumSelect "Display / Non-Display" (toHuman false DisplayDesignation.values iceData.displayNondisplay) (allToHuman false DisplayDesignation.values) [Disabled readonly] (fromHuman DisplayDesignation.values>>UserField.ICEDisplayNonDisplay>>send) ] let orderRoutingTabView (model:Model) (dispatch:Msg->Unit) (readonly:bool) = let cmeData = model.scratchUser.userData.exchangeUserData.cmeData let simData = model.scratchUser.userData.exchangeUserData.simData + let iceData = model.scratchUser.userData.exchangeUserData.iceData div [] [ - h2 [Class "is-size-3"] [ str "CME" ] - div [] - [ - Fields.text "Operator ID" cmeData.operatorId [ MaxLength 18.0 ; Disabled readonly ] (CMEOperatorID >> UpdateField >> dispatch) - //Fields.text "ATS Operator ID" cmeData.atsOperatorId [] (ATSOperatorID >> UpdateField >> dispatch) - Fields.text "SMP ID" cmeData.smpId [Disabled readonly] (SMPID >> UpdateField >> dispatch) - Fields.mapSelectEx "SMP Instruction" State.SMPInstructionMap None (Some cmeData.smpInstruction) (SMPInstruction >> UpdateField >> dispatch) (Some readonly) - Fields.text "Location" cmeData.location [Disabled readonly] (CMELocation >> UpdateField >> dispatch) - //Fields.text "Max Order Rate Per Connection" cmeData.maxOrderRatePerConnection [] (MaxOrderRatePerConnection >> UpdateField >> dispatch) - //Fields.map "Misc Tag Mapping" cmeData.miscTags [] (MiscTags >> UpdateField >> dispatch) - ] - br [] - h2 [Class "is-size-3"] [ str "Sim" ] - div [] - [ - Fields.text "Operator ID" simData.operatorId [ MaxLength 18.0 ; Disabled readonly] (SimOperatorID >> UpdateField >> dispatch) - Fields.text "Location" simData.location [Disabled readonly] (SimLocation >> UpdateField >> dispatch) + h2 [Class "is-size-3"] [ str "CME" ] + div [] [ + Fields.text "Operator ID" cmeData.operatorId [ MaxLength 18.0 ; Disabled readonly ] (CMEOperatorID >> UpdateField >> dispatch) + //Fields.text "ATS Operator ID" cmeData.atsOperatorId [] (ATSOperatorID >> UpdateField >> dispatch) + Fields.text "SMP ID" cmeData.smpId [Disabled readonly] (SMPID >> UpdateField >> dispatch) + Fields.mapSelectEx "SMP Instruction" State.SMPInstructionMap None (Some cmeData.smpInstruction) (SMPInstruction >> UpdateField >> dispatch) (Some readonly) + Fields.text "Location" cmeData.location [Disabled readonly] (CMELocation >> UpdateField >> dispatch) + //Fields.text "Max Order Rate Per Connection" cmeData.maxOrderRatePerConnection [] (MaxOrderRatePerConnection >> UpdateField >> dispatch) + //Fields.map "Misc Tag Mapping" cmeData.miscTags [] (MiscTags >> UpdateField >> dispatch) + ] + br [] + h2 [Class "is-size-3"] [ str "ICE" ] + div [] [ + Fields.number_novalidate "Company ID" iceData.companyId [ ] (ICECompanyID >> UpdateField >> dispatch) + Fields.text "Operator ID" iceData.operatorId [ ] (ICEOperatorID >> UpdateField >> dispatch) + Fields.text "Authorized Trader" iceData.authorizedTrader [ ] (ICEAuthorizedTrader >> UpdateField >> dispatch) + Fields.text "Authorized Group" iceData.authorizedGroup [ ] (ICEAuthorizedGroup >> UpdateField >> dispatch) + Fields.text "Password" iceData.password [ ] (ICEPassword >> UpdateField >> dispatch) + Fields.text "Custom Order Instructions" iceData.custOrderHandingInst [ ] (ICECustOrderHandingInst >> UpdateField >> dispatch) - ] + h2 [Class "is-size-4"] [ str "MIFID" ] + div [Style [MarginLeft "2em"]] [ + Fields.number_novalidate "Client Id Code" iceData.mifidClientIdCode [ ] (MifidClientIdCode >> UpdateField >> dispatch) + Fields.number_novalidate "Commodity Deriv Indicator" iceData.mifidCommodityDerivIndicator [ ] (MifidCommodityDerivIndicator >> UpdateField >> dispatch) + Fields.number_novalidate "Direct Electronic Access" iceData.mifidDirectElectronicAccess [ ] (MifidDirectElectronicAccess >> UpdateField >> dispatch) + Fields.number_novalidate "Execution Decision" iceData.mifidExecutionDecision [ ] (MifidExecutionDecision >> UpdateField >> dispatch) + Fields.number_novalidate "Id" iceData.mifidId [ ] (MifidId >> UpdateField >> dispatch) + Fields.number_novalidate "Investment Decision" iceData.mifidInvestmentDecision [ ] (MifidInvestmentDecision >> UpdateField >> dispatch) + Fields.number_novalidate "Liquidity Provision" iceData.mifidLiquidityProvision [ ] (MifidLiquidityProvision >> UpdateField >> dispatch) + Fields.number_novalidate "Trading Capacity" iceData.mifidTradingCapacity [ ] (MifidTradingCapacity >> UpdateField >> dispatch) + ] + ] + br [] + h2 [Class "is-size-3"] [ str "Sim" ] + div [] [ + Fields.text "Operator ID" simData.operatorId [ MaxLength 18.0 ; Disabled readonly] (SimOperatorID >> UpdateField >> dispatch) + Fields.text "Location" simData.location [Disabled readonly] (SimLocation >> UpdateField >> dispatch) + ] ] @@ -439,10 +487,10 @@ let root (model:Model) (dispatch:Msg->Unit) = | _, _ -> () ) *) - let adminDisplay = model.scratchUser.userData.exchangeUserData.cmeData.adminDisplayFlag + let adminDisplay = model.scratchUser.userData.exchangeUserData.adminDisplayFlag let is_bts = model.this_username = Access.superuser - let display_md = if is_bts then true else (adminDisplay = bts.api.admin.CMEUserData.AdminDisplay.All || adminDisplay = bts.api.admin.CMEUserData.AdminDisplay.MD) - let display_risk = if is_bts then true else (adminDisplay = bts.api.admin.CMEUserData.AdminDisplay.All || adminDisplay = bts.api.admin.CMEUserData.AdminDisplay.Routing) + let display_md = if is_bts then true else (adminDisplay = bts.api.admin.ExchangeUserData.AdminDisplay.All || adminDisplay = bts.api.admin.ExchangeUserData.AdminDisplay.MD) + let display_risk = if is_bts then true else (adminDisplay = bts.api.admin.ExchangeUserData.AdminDisplay.All || adminDisplay = bts.api.admin.ExchangeUserData.AdminDisplay.Routing) let modal = match (fst model.ConfirmModal) with | "" -> span [] [] diff --git a/testing/manual/admin/order_router.txt b/testing/manual/admin/order_router.txt index c66cb1789..5030ae767 100644 --- a/testing/manual/admin/order_router.txt +++ b/testing/manual/admin/order_router.txt @@ -17,7 +17,7 @@ Prerequisites User settings ------------- 3. Navigate to http://sparkvm:8000 (or wherever your sparkd resides), Users tab -4. Between bts and bts_2 users, verify that have distinct operator ids (Order Routing tab) +4. Between bts and bts_2 users, verify they have distinct operator ids (Order Routing tab) 5. In another ssh session on your server, let's tail the sim fix log for orders tail -f ~/sparkdata/data/jetfix/sim/current/fix | grep --line-buffered 35=D | tr '\001' '|' 6. Open up a client, login as bts, pick your favorite symbol and submit an order diff --git a/testing/manual/keycodes/keycode.txt b/testing/manual/keycodes/keycode.txt index ed9535602..43106e45c 100644 --- a/testing/manual/keycodes/keycode.txt +++ b/testing/manual/keycodes/keycode.txt @@ -23,7 +23,8 @@ You should see keycode details printed to screen -You should see the data zero out on the first instance logged in as bts_2 and data ticking on the bts instance as well as the second bts_2 instance - d)Kill your sparkd + d) Kill your sparkd but leave your shells open + e) bring sparkd back up, you should see market data in the shell logged in as bts AND one of the other shells 4.5) For extra credit generate a keycode with -allow_concurrent and make sure we don't zero the market data for concurrent logins diff --git a/testing/manual/ladder/badge_spec.PNG b/testing/manual/ladder/badge_spec.PNG deleted file mode 100644 index 38a870d7f..000000000 Binary files a/testing/manual/ladder/badge_spec.PNG and /dev/null differ diff --git a/testing/manual/orders/order_book.txt b/testing/manual/orders/order_book.txt index 0b724e132..c1e7ec741 100644 --- a/testing/manual/orders/order_book.txt +++ b/testing/manual/orders/order_book.txt @@ -36,7 +36,7 @@ Left click on a cancelled parent order of a parent-child order type Order tree node collapsing testing ---------------------------------- -- Create an algo using the algo studio consisting of IB -> IB, call it IB2. +- Create an algo using the algo studio consisting of IB -> Limit, call it IB2. - Launch an instance of IB2 on a normal security away from the market so it does not execute immediately - In an order book form ensure that the setting "Show Full Algo Tree" is checked - Ensure further that there are no filters on the OB @@ -61,7 +61,7 @@ Filter/Collapse interaction --------------------------- - Now uncheck the "Show Full Algo Tree" setting and clear all filters - You should only see the top algo for CIB and the the hedges and currently working limit order -- Now filter for order type = limit, you should only the hedges and the currently working limit +- Now filter for order type = limit, you should only see the hedges and the currently working limit - Now filter for order type = IB and you should see nothing diff --git a/testing/manual/positions_and_fills/SOD.txt b/testing/manual/positions_and_fills/SOD.txt index cd5c62572..3e5f9dfbd 100644 --- a/testing/manual/positions_and_fills/SOD.txt +++ b/testing/manual/positions_and_fills/SOD.txt @@ -3,6 +3,8 @@ rm ~/sparkdata/data/databases/fills/* rm ~/sparkdata/data/databases/orders/* and bounce your server. +BE SURE NOT TO PICK AN EXPIRED SECURITY, THOSE FILLS ARE FILTERED + 0. We will force fills across for 3 securities, across 2 accounts. a. Pick 2 futures and one exchange traded spread, all on different symbols b. Choose these securities such that there is market data (sim or cert) that you can get fills against (maybe bring up the market view and select securities from there). diff --git a/testing/manual/positions_and_fills/SOD_timezone.txt b/testing/manual/positions_and_fills/SOD_timezone.txt index 78b53ffef..fb2336615 100644 --- a/testing/manual/positions_and_fills/SOD_timezone.txt +++ b/testing/manual/positions_and_fills/SOD_timezone.txt @@ -12,6 +12,8 @@ ExchangeSim { 2. Clean up your fills database. Cross the market to buy a 10 lot future (any future). Lets say the fill time is 6:03:XX EST. Wait until 6:05 and do a 20 lot sell. +BE SURE NOT TO PICK AN EXPIRED SECURITY, THOSE FILLS ARE FILTERED + 3. Now we will make 6:05 as the SOD Time. Enter this in your spark.conf SODTime{ diff --git a/testing/manual/server_performance/spreader_requote.txt b/testing/manual/server_performance/spreader_requote.txt index 3724f0fb1..19d8a7d5c 100644 --- a/testing/manual/server_performance/spreader_requote.txt +++ b/testing/manual/server_performance/spreader_requote.txt @@ -1,4 +1,4 @@ -Test Author: Pedro +Test Author: Pedro If it breaks: Pedro Spreader Requote Performance test @@ -30,13 +30,13 @@ OrdermanCfg { value { executor { thread_name: "oman_orders" - poll: true + poll: true hosted_actor: "MDSource" hosted_actor: "OrderManager" } executor { thread_name: "oman_slow" - poll: false + poll: false hosted_actor: "*" } } @@ -54,7 +54,7 @@ Threads { 5) Now you are ready to start testing. Start a shell and create a new synthetic say with 2 CL futures. Use the near month (fast ticking) future as the non quoting leg, so that the other leg gets quoted -often. Make sure to disable throtling in the synthetic definition +often. Make sure to disable throttling in the synthetic definition dialog. 6) Place an order for this synthetic, away from the market. diff --git a/testing/manual/settings/UserSettingsTest.txt b/testing/manual/settings/UserSettingsTest.txt index 573a14106..5201d95b6 100644 --- a/testing/manual/settings/UserSettingsTest.txt +++ b/testing/manual/settings/UserSettingsTest.txt @@ -1,50 +1,33 @@ -User Settings Test (15 min) +User Settings Test + (there is a REFERENCE section at the bottom, please consult it before getting frustrated, if things are still unclear, ask John.) NOTES: - - in addition to the notes below, there is a "Reference" section at the bottom of the test, - feel free to add things to it if you run into problems; impart your wisdom unto the next tester. - the settings button is located in the bottom-left corner of the Ladder - - the "Default" tab will be referred to as [DTab], "Custom" as [CTab] - - the "Use Defaults" checkbox is located at the top of the dialog - - when "Use Defaults" is checked, [CTab] is disabled ([DTab], however, is always enabled) - to Import/Export settings to files, click the hamburger icon in the upper-right corner of the settings dialog. - you cannot open multiple settings dialog for the same Ladder i) if a settings dialog is already open, clicking the settings button in the Ladder again closes said dialog, then opens a new one - - the settings dialogs do not live update - i) this only pertains to [DTab], as Custom settings are not shared. - - commit changes by clicking "Apply" or "OK" - i) the only difference between the 2 buttons is that "OK" also closes the dialog - - when you commit changes: - i) even if "Use Defaults" is NOT checked, the settings in [DTab] will be compared with the current defaults, - if they are different, the default changes will be committed as well. - ii) be advised of the following scenario: - You have 2 settings dialogs open (call them [A] and [B]), and commit changes to the defaults in [A]. - As the dialogs do NOT live update, [B] will have stale defaults. - When you click commit in [B] (even if you have not actually changed anything in [DTab]) - the stale defaults in [B] will be committed, effectively overwritting the changes to the defaults you just committed from [A]. + - the "sample" areas should always reflect the properties in your editor, if this is unclear, see the reference section at the bottom for a more detailed explanation. - - when the test directs you to "Visually Confirm blah...," just give it a quick glance, if something isn't matching it should be obvious. - - when the test directs you to repeat a step for each property, here is the list (in case it isn't clear in the dialog): - Note: the Sample column is purely output, don't bother clicking it - - Order Badge Style - - Alternate Order Quantity - - Size Presets (aka Number Pad buttons) - - Fonts and Colors section: - for each column (Orders, Bid, Price, Ask, Last) + + - when the test directs you to repeat a step for each property, here is the list: + - Ladder Columns tab -> + - Order Badge Style + - for each column (Orders, Bid, Price, Ask, Last) i) background ii) foreground iii) font (family, size, and weight) - - Market Break Lines section: - i) for each "price-line" (session high, ask, mid, bid, and session low) + - Order Quantities tab -> + - 'Single Click' Quantity + - Number Pad Quantities + - Market Break Lines tab: + i) for each "price-line" (session high, ask, mid, bid, and session low) a) toggle show/hide b) color ii) for session range (this is not actually a line, but the color between the session low/high a) toggle show/hide b) color - - when the test advises you to name files (screenshots, or ladder settings files), it is because they will be referenced later by that name. - - if you uncheck all 3 market break lines, the ladder resolves to show the bid line by design. - - just to reiterate: there is a REFERENCE section at the bottom, please consult it before getting frustrated, if things are still unclear, ask John. + - if you uncheck all market break lines, the ladder resolves to show the bid line by design. ------------TEST STEPS------------------ @@ -52,61 +35,61 @@ NOTES: - the security needs a few working orders on both sides, and a gap of 5-8 ticks between the top bid/ask. The gap is needed to separate the market break lines (Bid, Ask, and Mid). - from hence forth, the 2 ladders will be referred to as "[A]" and "[B]" and their settings dialogs as "dialog[A]" and "dialog[B]" - - at the risk of sounding redundant, the "Default" tab will be referred to as [DTab], the "Custom" as [CTab] - + 2) Open the settings dialogs for [A] and [B] 3) Make sure that both [A] and [B] are using default settings, then close [B]dialog -4) In dialog[A] (with [DTab] selected, [CTab] should be disabled anyways), change each property, then click "Apply" +4) In dialog[A] change each property, then click "Apply" 5) Visually confirm that all the changes have been applied to both ladders - the row height should adjust to fit the largest font size, - the scroll position should be maintained, assuming the AutoRecenterTimer didn't move it. - + 6) Take a screenshot, name it "A-1-confirmation.png" -7) In dialog[A], export the default settings to a file, name it "A-1.ladder" (instructions on how to import/export are in the NOTES section) +7) In dialog[A], export the settings to a file, name it "A-1.ladder" (from the irregular hamburger menu) -8) In dialog[A], click "Restore to Factory," then click "Apply" +8) In dialog[A], click "Load Factory Defaults" in the irregular hamburger menu then click "Apply" 9) Visually confirm that dialog[A], and both ladders, each changed appearance, and match one another. -10) In dialog[A], uncheck "Use Defaults", then select [CTab] - -11) Import "A-1.ladder" to Custom, then click Apply +10) In dialog[A] Import "A-1.ladder", then click Apply 12) Visually confirm that: - dialog[A] and [A] changed appearance, and look just like they did in "A-1-confirmation.png" - [B] did not change appearance 13) Click the settings button in [A] and [B] to get 2 fresh dialogs (dialog[A] should close and reopen) - - in dialog[A], "Use Default" should be false, and [CTab] selected - - in dialog[B], "Use Default" should be true, and [DTab] selected ([CTab] should be disabled) 14) Take a screenshot, then save the workspace, then restore it - visually confirm that [A] and [B] look as they did in the screenshot - repeat step 13 -15) In dialog[A], with [CTab] selected, click "Copy from Default" - - [CTab] should now have the same settings as [DTab] +15) In dialog[A] hit the plus stack icon to add a new Profile, give it a unique and memorable name -16) Click the settings button in [A] to close and reopen dialog[A] +16) In Dialog [B] select the profile you just created, if you remember the name -17) In dialog[A] select [DTab], then click "Copy from Custom" - - [DTab] should now have the same settings as [CTab] +17) Click Apply and verify [B] changes to match [A] + +18) Click the smaller than average pencil button in dialog[B] change the name of your profile and hit enter. Verify the name changed in dialog[A] + +19) Change any one setting of your choice in either dialog. Click the 3.5 inch floppy disk icon and wait for the light to go off before ejecting the disk. Verify both dialogs have the setting you changed. + +2001) Change any one setting of your choice in either dialog; click apply in the same dialog and verify both ladders and both dialogs reflect the changed setting. + +2002) Revert to factory defaults and create a new profile named "Unsafety Checks" + +2003) On the Safety Checks tab uncheck Warn on Market Cross + +2004) Click the security selection icon (with the hand), double click every entry in the Exchange column, and click OK. Save or Apply. + +2005) Open a new ladder for any symbol and verify you can cross your own orders without being pestered by a confirmation dialog DONE!!! Reference----------------------------------------------------------- - Alternate Order Quantity: - - you may send an order with this size via a RIGHT CLICK on a price cell - - Size Presets: - - aka the number-pad buttons - - click the expander-arrow button next to the "Size Presets" label to modify them - - you control the quantities in the buttons, and the order in which they appear Market Break Lines (Bid, Ask, Mid, and Session Low/High/Range): - each row corresponds to one item, and represents a "horizontal price-line" that will be displayed in the ladder (if show is checked, of course) diff --git a/tools/cpp_helper/ics_spreads.csv b/tools/cpp_helper/ics_spreads.csv index 0f58741d9..26265d4fb 100644 --- a/tools/cpp_helper/ics_spreads.csv +++ b/tools/cpp_helper/ics_spreads.csv @@ -28,4 +28,4 @@ Sym,Mult,Exch "BOB",1000,CBT "BUB",1000,CBT "FNU",1000,CBT -"SOM",500,CBT \ No newline at end of file +"SOM",500,CBT diff --git a/tools/spark_vm_shell.sh b/tools/spark_vm_shell.sh index b2bd64634..dbc0c8ab1 100755 --- a/tools/spark_vm_shell.sh +++ b/tools/spark_vm_shell.sh @@ -8,7 +8,7 @@ docker run \ -p 8001:8001 \ -p 8080:8080 \ --net host \ - --shm-size=1024m \ + --shm-size=2048m \ -v /etc/passwd:/etc/passwd \ -v /etc/group:/etc/group \ -v /etc/shadow:/etc/shadow \