Compare commits

..
215 Commits
Author SHA1 Message Date
Pete Fisher e974edfc6e Merge branch 'master' of bitbucket.org:bluetradesys/spark 2021-03-18 16:22:27 -04:00
Pete Fisher 3b7740cee0 updating submodule to latest 2021-03-18 16:22:12 -04:00
John Colvin 258d4a3fc1 import compute_synthetic_book method from spartan 2020-12-08 23:00:18 -06:00
John Colvin c53f6425cd import IntPrice arithmetic operators from spartan
The compute_synthetic_book method (also to be imported
from spartan) depends on a couple of them.
2020-12-08 22:52:28 -06:00
Pedro Pinto e5d77593d1 Changed logging 2020-10-15 14:24:02 -04:00
Pedro Pinto 6580f3883b Added logging of cancel/replace 2020-10-15 14:18:57 -04:00
Pete Fisher a66eae66bb fixing for python3 2020-10-14 21:56:49 -04:00
Pedro Pinto 090a9dfb17 Added iLink3 documentation. 2020-09-14 12:02:37 -04:00
Pete Fisher 27aa43f44d fixed time on ilink2 audit 2020-09-09 12:16:57 -04:00
Pedro Pinto 19e087aa69 Fixed time reporting in iLink3. 2020-09-09 10:58:34 -04:00
Pedro Pinto 167c8c23a0 Fixed batching of orders under iLink3. 2020-08-27 08:19:47 -04:00
Pete Fisher d235f4be41 fixed jetfix.conf ports on relay 2020-08-25 10:03:09 -04:00
Pedro Pinto ec43dbd82d Maybe fixed the self match prevention problem. 2020-08-24 17:02:58 -04:00
Pedro Pinto f07998d6a6 Fixed order rounding error in ilink3 2020-08-24 16:17:31 -04:00
Pedro Pinto b3dc2cdcd4 Changed logging subsystem from ilink3 to iLink3 2020-08-24 11:06:44 -04:00
Pete Fisher 0963aee137 added audit log scripts to be run by the docker container 2020-08-19 17:49:47 -04:00
Pedro Pinto c2f8e92b10 Hopefully fixed the ilink3 filled qty reporting problem.
This was crashing the spreader.
2020-08-13 08:42:43 -04:00
Pedro Pinto 7ae0142dff Fixed reset of sequence numbers problem 2020-08-12 11:56:08 -04:00
Pedro Pinto db0ffba8ed Fixed windows build 2020-07-10 13:00:20 -04:00
Pedro Pinto a274a214b4 Merged with master 2020-07-10 11:31:47 -04:00
Pedro Pinto bbcfa3c75a merged with prod 2020-07-10 10:45:10 -04:00
Pedro Pinto 89229d65f1 Merge branch 'master' of bitbucket.org:bluetradesys/spark 2020-07-10 10:43:34 -04:00
Pedro Pinto 0e75cde409 Fixed a bad asser in recovery, a few typos, haderned against bad securities. 2020-07-10 10:36:03 -04:00
Pedro Pinto 42fa6dfaff Fixed life cycle of exe reports and config during replay. 2020-07-09 16:06:39 -04:00
Pedro Pinto 400143c84d Protected against ticker subs for expired securities. 2020-07-09 14:45:51 -04:00
Pedro Pinto e915945880 Trying to replay fix logs. 2020-07-09 14:27:22 -04:00
Taha Afzal ca6673e98f Merge branch 'prod' 2020-07-09 13:09:57 -04:00
Taha Afzal 479f3dccff Adding OnSignal (this is a new algo which is hidden by default) 2020-07-09 13:07:28 -04:00
Pedro Pinto 9601ebab20 Adding replay of stored messages. 2020-07-09 10:16:31 -04:00
Taha Afzal c272d8a8c2 Pyramid Algo: Takes care of the rounding error when Top == sum (child orders) 2020-07-08 13:35:21 -04:00
Pedro Pinto b344ab3377 Fixed an issue with order recovery from Exec reports. 2020-07-08 12:07:48 -04:00
Pedro Pinto ae3080b031 Refactoring. 2020-07-08 10:10:14 -04:00
Pedro Pinto f0464331cb Added better handling of cancel rejects. 2020-07-08 09:52:18 -04:00
Pedro Pinto 0648b618bf Refactored. 2020-07-08 08:42:53 -04:00
Pedro Pinto 59282b8c70 Added one more test to manual order entry list. 2020-07-08 08:42:18 -04:00
Pedro Pinto 34a86009ce Doc and tool minor update. 2020-07-08 07:44:01 -04:00
Pedro Pinto 0ad9ae9001 Small tweak to logging tool. 2020-07-07 20:19:59 -04:00
Pedro Pinto a296e1e772 Fixed parsing of stop px in link3. 2020-07-07 20:19:41 -04:00
Pedro Pinto 7f53d8b161 Improved logging, fix iceberg order, removed duplicate logging. 2020-07-07 17:01:24 -04:00
John Colvin fbd8bbd4a8 algo abbreviation code uses string.Join instead of list_string (the latter method is redundant, should be removed) 2020-07-07 13:16:18 -05:00
John Colvin 3da52b350f fix pure crash (algo abbreviation code choked on names that end with a space) 2020-07-07 13:15:55 -05:00
Pedro Pinto 243d6827d4 Merge remote-tracking branch 'refs/remotes/origin/ilink3_final' into ilink3_final 2020-07-07 12:12:09 -04:00
Pedro Pinto 00221f9bd9 Fixed spread order entry. 2020-07-07 12:11:11 -04:00
Pedro Pinto a6171ae405 Fixed spread order entry. 2020-07-07 11:55:05 -04:00
Pedro Pinto be5d01a797 Fixed fills. 2020-07-07 06:47:58 -04:00
Pedro Pinto e5d9d324bf Order order entry manual test. 2020-07-06 18:22:39 -04:00
Pedro Pinto c34dee9d96 Taked order status from execution report, parse error message in il3 2020-07-06 18:21:44 -04:00
Pedro Pinto e5a98aab41 Merged with master 2020-07-06 12:49:24 -04:00
Pedro Pinto 10cd35372b Can enter, modify, cancel orders. 2020-07-06 11:38:17 -04:00
Pedro Pinto ad4515be39 Renamed NativeOrder to GLBXNativeOrder 2020-07-06 09:47:34 -04:00
Pedro Pinto ff4d87dd6b Fixed IL3 execution report parser to convert exchange prices. 2020-07-06 09:34:26 -04:00
Pedro Pinto 8ddde3f9da Part 2 of MBO service fix. 2020-07-06 09:34:09 -04:00
Pedro Pinto 51093dc3a5 Added simple tool to watch spark logs. 2020-07-06 09:33:35 -04:00
Pedro Pinto 5d5db9ad9f Fixed MBO parser to extract the right exchange key. 2020-07-06 09:33:06 -04:00
Pedro Pinto fca808fe0d Made ilink3 gw wait for start() before its starts. 2020-07-03 14:11:22 -04:00
Pedro Pinto 07d14f877b More nonsense 2020-07-02 20:28:49 -04:00
Pedro Pinto 5ea6dccdd4 Something here breaks 2020-07-02 19:04:49 -04:00
Pedro Pinto ac956a5c02 Renamed ExchangeOrderIDu64 to ExchangeOrderID64 2020-07-02 19:00:32 -04:00
Pedro Pinto 1fba09cf94 Refactor of ids. 2020-07-02 11:05:04 -04:00
Pedro Pinto 0baa6b875e Fixed merge error. 2020-07-02 09:09:20 -04:00
Pedro Pinto cdacc2d4d4 Merged 2020-07-02 07:26:43 -04:00
Pedro Pinto 1433541947 Automatically advanced our own seq number on connection failure.
It is fairly easy for our sequence number to be behind CME if we
crash. Since we can detect this issue (CME tells us about it) we can
try to fix it automatically by incrementing our own sequence
number. This should allow us to recover automatically in some
scenarios.
2020-07-02 06:57:01 -04:00
Pedro Pinto 1070ca7ace Ran clang-format. 2020-07-02 06:28:32 -04:00
Pedro Pinto 56fa27dd7e Handled non text timestamp in native order. 2020-07-02 06:23:07 -04:00
Pedro Pinto d87b522bc6 Added better logging when terminated by exchange. 2020-07-02 06:22:38 -04:00
Eric Harding 166cb03782 Provide account when parsing execution reports. 2020-07-01 14:40:10 -04:00
John Colvin 40bad84c48 algo studio includes algos with undefined children in the dropdown 2020-07-01 12:09:12 -05:00
Eric Harding c3786d9d64 ilink3: Provide SecurityOracle to execution report parser 2020-07-01 11:50:03 -04:00
Pedro Pinto 9080e3124b Filled in cust_order_handing_inst 2020-07-01 11:29:47 -04:00
Pedro Pinto d9c536124a Fixed logging of session seq numbers. 2020-07-01 11:29:30 -04:00
Pedro Pinto 7de6b2907e Filled in sendercompid in routing record.
iLink3 requires this.
2020-07-01 10:48:01 -04:00
Eric Harding 75f017e9f8 ilink3: Get PartyDetails from sender_cmp_id 2020-07-01 10:29:20 -04:00
Pedro Pinto 7757ab3ea0 Fixed a few bugs. 2020-06-30 19:35:53 -04:00
Pedro Pinto 3186dd383c We can connect now. 2020-06-30 19:03:35 -04:00
Pedro Pinto 45f751c9c6 Bad merge. 2020-06-30 19:03:35 -04:00
Pedro Pinto e4d3774980 Better logging. 2020-06-30 19:03:35 -04:00
Eric Harding eedb3ce05d Clear hmac for json logging 2020-06-30 17:11:01 -04:00
Eric Harding 6a7d38c0fd flip session bool 2020-06-30 16:10:26 -04:00
Eric Harding 7f9967834d add readme note 2020-06-30 15:34:03 -04:00
Eric Harding 31ccbfcd19 Switch to ilink3 logging group 2020-06-30 14:43:38 -04:00
Eric Harding c2abf90cc4 IL3MessageFramer owns outgoing messages so it needs to outlive ILinkGW 2020-06-30 14:36:02 -04:00
Eric Harding 8dc4735248 ilink3: set_state(Error) only after disconnect 2020-06-30 14:25:59 -04:00
Pedro Pinto d940d9f11d For now, caught serialization error in some ilink3 msgs. 2020-06-30 14:10:07 -04:00
Pedro Pinto 1e0c75c6c7 Added iLink3 subsystem for logging. 2020-06-30 14:09:55 -04:00
Eric Harding fbaa17bc40 Add some comments to marketdata.proto 2020-06-30 13:16:30 -04:00
Eric Harding 38e3cd6197 Update market data readme 2020-06-30 13:13:48 -04:00
Pedro Pinto 3bb9961af4 Refactor. 2020-06-30 11:35:40 -04:00
Pedro Pinto da8088fd8d Rebased 2020-06-30 11:27:56 -04:00
Eric Harding 748a65f671 Merge branch 'ilink3_gateway' of bitbucket.org:bluetradesys/spark into ilink3_gateway 2020-06-30 10:05:33 -04:00
Eric Harding 66d80136bd ilink3: Keep outgoing messages alive forever. 2020-06-30 10:04:42 -04:00
Eric Harding 10386311d4 Update market data readme 2020-06-29 12:25:30 -04:00
Eric Harding 629a7f8130 Remove unused class in MD 2020-06-29 12:25:14 -04:00
Pedro Pinto 81189bb2cc Forgot to add message logger. 2020-06-29 12:08:22 -04:00
Pedro Pinto e5681a50f2 Made session name come from date. 2020-06-29 10:59:15 -04:00
Pedro Pinto 69fe4569e3 Made FIX session name available as uint64_t as well as strings. 2020-06-29 10:57:40 -04:00
Pedro Pinto c7caf66f9c Changed the ILink3 log format to json. 2020-06-26 19:11:05 -04:00
Pedro Pinto 14504f4a94 Cleaned up comment. 2020-06-26 19:10:49 -04:00
Pedro Pinto 5b601a4016 Fixed encofing of clorid. 2020-06-26 19:10:23 -04:00
John Colvin e3207b921a truncate list of securities in transitive compare test (was timing out due to too many permutations) 2020-06-26 13:32:27 -05:00
Taha Afzal 98db8e61a5 Removing hardcoded ref's to Katallage 2020-06-25 17:05:10 -04:00
Pedro Pinto 45aa75e8bb Brought back the ilink3/ilink2 encoding abstraction. 2020-06-25 16:49:46 -04:00
Pedro Pinto b2471d45b2 Added tool 2020-06-25 16:49:10 -04:00
Pedro Pinto 11e2df8c76 Tweaked build tools. 2020-06-25 16:49:10 -04:00
Pedro Pinto cba6852655 Updated docker 2020-06-25 16:49:10 -04:00
Pedro Pinto 5eeaedde26 Added a few build tools. 2020-06-25 16:49:10 -04:00
Pedro Pinto b63052dbaa Updated ICE Onixs 2020-06-25 16:49:10 -04:00
Pedro Pinto 29f5bc798e Added +x 2020-06-25 16:49:10 -04:00
Pedro Pinto 04603acbb9 Added publishing of new routing info when orders are recorevered from disk 2020-06-25 16:49:10 -04:00
Pedro Pinto bc6890055e temp commit, fixing glbxnative order 2020-06-25 16:49:10 -04:00
Pedro Pinto ea94c09c78 Eric intermediate ilink3 fixes 2020-06-25 16:49:10 -04:00
Eric Harding e3c3bc78fc Note what this address list is for 2020-06-24 20:55:20 -04:00
Eric Harding a11d390dcd ilink3 message logging 2020-06-24 20:52:21 -04:00
Eric Harding 7c74a961d4 Add canned cert sessions for ilink3 nrcert 2020-06-24 20:43:27 -04:00
Eric Harding 5db21ad2ca fix build, session and clordid changes 2020-06-24 19:32:12 -04:00
Eric Harding f703dabdfe Raw msg should be 2 strings 2020-06-24 17:53:30 -04:00
Eric Harding 237d3eea4b encode_clordid needs security key 2020-06-24 17:29:06 -04:00
Taha Afzal e009316bfb Merge branch 'DJ_ICE' of bitbucket.org:bluetradesys/spark into DJ_ICE 2020-06-24 17:13:38 -04:00
Taha Afzal 5528efd2d2 Cleaning up 2020-06-24 17:13:31 -04:00
Eric Harding 818bdec130 Add NrCert keys 2020-06-24 15:07:02 -04:00
John Colvin 09f91735fe Security sort tests run with ice securities 2020-06-24 12:58:29 -05:00
John Colvin d9fc18547b SecurityHelper.Compare fallsback on 'ToString' then 'Id' comparisons to guarentee deterministic sort 2020-06-24 12:56:29 -05:00
Eric Harding d257cf2c77 Disable broken ilink3 tests for now 2020-06-24 13:28:33 -04:00
Eric Harding 079f46404f Add some testing notes 2020-06-24 10:57:15 -04:00
Eric Harding edcc73e934 Remove old ILink3ExecutionGateway. Instead use GlbxIlink3ExecutionGateway 2020-06-23 22:41:33 -04:00
Pedro Pinto bc7ac8523b Added tool 2020-06-23 13:06:15 -04:00
Eric Harding 0c4c2b7231 cme autocert msgw_failover test script 2020-06-23 13:05:32 -04:00
Eric Harding 420351c093 Replay refactor #4 2020-06-23 12:01:52 -04:00
Eric Harding f69e6b89c5 Sequence number fixes 2020-06-22 22:23:13 -04:00
Eric Harding c3e22e4b23 Recovery revamp #3 2020-06-22 20:16:21 -04:00
Pedro Pinto 880c95d92b Tweaked build tools. 2020-06-20 17:17:22 -04:00
Eric Harding 3231ffb343 Meaningless autocert code 2020-06-15 22:48:28 -04:00
Eric Harding 8d7f1738cf update cert secure logon 2020-06-15 22:47:30 -04:00
Eric Harding 9101214e66 Add message logger and revamp replay sequence handling 2020-06-15 14:51:41 -04:00
Eric Harding c6b2e2f21e OMF setting to true, always 2020-06-12 17:49:37 -04:00
Eric Harding 2eac97c1a9 meaningless cert change 2020-06-12 15:18:20 -04:00
Eric Harding b4b0e10f29 Fix IOC=FAK order type (not FOK) 2020-06-12 15:17:38 -04:00
Eric Harding 38d9db4a90 Session and sequence number fixes 2020-06-11 11:36:37 -04:00
Eric Harding 6a679ff78c Fix for cancel/replace 2020-06-10 14:13:28 -04:00
Eric Harding aba3f21020 stop order does not carry price 2020-06-09 19:18:36 -04:00
Eric Harding fede19fd11 stop-limit success 2020-06-09 16:30:00 -04:00
Eric Harding 3fa8dbf112 Cannot send price with market order 2020-06-08 23:09:10 -04:00
Eric Harding 77ae4703b3 Fix clOrdID and custOrdCapacity 2020-06-08 22:58:53 -04:00
Eric Harding a28bfa1175 execution report handling tweaks 2020-06-04 10:05:44 -04:00
Eric Harding 4e4a48283d bool parse_report(string&, SecurityOracle&, ExecutionReport*); 2020-06-03 13:09:56 -04:00
Pedro Pinto 7a761d00ce Added security oracle to SessionStore so we can resolve exe reports. 2020-06-03 08:51:31 -04:00
Pedro Pinto 968be6a99a Added storage for sessions. Still missing parsing of exec reports. 2020-06-02 17:13:53 -04:00
Pedro Pinto 03404eb3f1 Made some progress in GLBXiLink3ExecutionGateway 2020-06-02 09:16:02 -04:00
Pedro Pinto ff1a970982 Fixed build some more 2020-05-28 14:29:45 -04:00
Pedro Pinto ed890b4d84 Fixed broken tests for subscribe_exe_report change 2020-05-28 13:50:04 -04:00
Pedro Pinto e87003cc41 Merged with master 2020-05-28 12:21:51 -04:00
Eric Harding bc06037843 Fix the build 2020-05-28 12:20:30 -04:00
Pedro Pinto c4c7cebfc5 Updated externals 2020-05-28 12:12:46 -04:00
Pedro Pinto f18bcd2995 Build tweaks 2020-05-28 12:12:46 -04:00
Pedro Pinto a6a1b6ac53 Fixed clang build. 2020-05-28 12:12:46 -04:00
Eric Harding 8c3da4e43c Re-arrange some methods in ILinkGW.h 2020-05-27 16:51:36 -04:00
Eric Harding 06b955dceb Exchange seq number fix 2020-05-27 11:49:08 -04:00
Eric Harding e75d15146c cme cert test update 2020-05-26 23:55:27 -04:00
Eric Harding cb07384c9f Fix problem with party details 2020-05-26 23:55:13 -04:00
Eric Harding fc9e502539 cert refactor 2020-05-26 12:40:16 -04:00
Eric Harding fcbf3388c0 update ilinkbinaryxml templates 2020-05-26 12:40:04 -04:00
Eric Harding 899bea940f improve ilinkgw logging 2020-05-26 12:39:55 -04:00
Eric Harding 14b7509607 ensure local launch timezone 2020-05-26 12:39:41 -04:00
Eric Harding 25668e8df0 dont' build web client on wsl2 2020-05-26 11:54:19 -04:00
Eric Harding 5345ba8ff8 securities for sending orders ilink3 test 2020-05-19 15:58:56 -05:00
Eric Harding 9bf05973bd Add in/out to logs 2020-05-14 20:51:15 -05:00
Eric Harding cea4d05bc4 log every msg 2020-05-13 20:19:48 -05:00
Eric Harding 0b7208784b Can't take the double negatives anymore 2020-05-13 14:01:16 -05:00
Eric Harding 059ccbdab6 Merge branch 'master' of bitbucket.org:bluetradesys/spark into ilink3_gateway 2020-05-13 11:05:31 -05:00
Eric Harding 75eaf8c3d4 IL3ExecutionReportParser 2020-05-12 13:54:28 -05:00
Eric Harding 1e8fcd794f Update ExecutionReport::exchange_order_id 2020-05-06 12:22:33 -05:00
Eric Harding 726329c4bf Add ilink3 execution report parser and order test 2020-05-06 11:56:52 -05:00
Eric Harding 2d01f3f0f4 CME Certification requires very specific UUID generation 2020-05-06 11:56:01 -05:00
Eric Harding 2a595c0fd7 cme cert hard coding 2020-05-06 11:54:32 -05:00
Eric Harding 5f9866c9e8 SecurityCache now hands out raw pointers. SecurityOracle now indexes exchange security key 2020-05-05 11:17:16 -05:00
Eric Harding 78f1cf88e0 cmake needs love too 2020-05-04 11:16:21 -05:00
Eric Harding 4877436877 Ilink3 order unit test 2020-05-04 11:32:30 -04:00
Eric Harding 1b43124756 Add ReentrancyGuard.h 2020-05-02 13:10:49 -04:00
Eric Harding a1feb3fddf PrimaryGatewayConnectionEnforcement 2020-05-01 13:22:35 -04:00
Eric Harding 024328bf8b Refactor empty session 2020-05-01 11:44:03 -04:00
Eric Harding 80d7e34bf1 Fixes for updated sbe codegen 2020-04-28 18:16:45 -04:00
Eric Harding 8ab69c919b Improve cme provided error message 2020-04-27 16:56:32 -04:00
Eric Harding 99531bcfa0 Update ilink3 generated templates 2020-04-27 16:50:54 -04:00
Eric Harding 6bd516d4f2 cleanup 2020-04-27 11:26:42 -04:00
Eric Harding 1a1d771e1b addition ilink3 tests 2020-04-25 13:18:58 -04:00
Eric Harding 050fcf7c6e Passing simple log in test 2020-04-24 12:31:30 -04:00
Eric Harding aadeae3201 update cmake, avoid web build on wsl 2020-04-17 22:38:53 -04:00
Eric Harding b72e8aed02 ilink3 test code 2020-04-17 20:30:11 -05:00
Eric Harding 0820cc3e9a Fix a very subtle bug in serializer 2020-04-17 13:15:12 -05:00
Eric Harding b1ba9a1315 update ilink code gen readme 2020-04-16 14:03:53 -05:00
Eric Harding 39ce4323a1 Add CME cert command line tool 2020-04-13 16:18:59 -05:00
Eric Harding d9fcd290d9 source files don't need to be executable 2020-04-13 09:53:28 -05:00
Eric Harding 05939b02c5 save 2020-04-12 15:11:46 -05:00
Eric Harding aa91638fe4 ilink3 receive buffer handler + tests 2020-04-10 10:55:53 -05:00
Eric Harding 065fc54f73 IL3Reactor 2020-04-07 14:05:31 -05:00
Eric Harding 95fb13f58b Fix some build issues 2020-04-01 15:21:04 -05:00
Eric Harding 4eb2931cb8 Additional ilink3 refactoring 2020-03-28 21:08:47 -05:00
Eric Harding 5326015a48 Testcase for framer 2020-03-27 13:46:29 -05:00
Eric Harding fcbe319470 Split out functional pieces for easier testing 2020-03-27 09:14:30 -05:00
Eric Harding 5ad21c88b2 Add tests and refactor includes for faster build 2020-03-25 21:15:19 -05:00
Eric Harding 3a9c1d557b Split Retransmit of old vs new session logic 2020-03-09 16:29:15 -05:00
Eric Harding 6fb5b83249 Only send Negotiate if the session is new 2020-02-28 15:33:27 -06:00
Eric Harding 95830fafdf send_cancel 2020-02-27 14:48:56 -06:00
Eric Harding d68b03bb91 send_cancel_replace 2020-02-26 17:56:50 -06:00
Eric Harding e7062ce01e send_order call for link3 2020-02-26 17:36:18 -06:00
Pedro Pinto 932738f98e Merge branch 'master' into ilink3_gateway 2020-02-20 08:39:47 -05:00
Pedro Pinto 1426b00e83 Added fake iLink3 gateway 2020-02-19 16:21:06 -05:00
Pedro Pinto e724409b58 Added jetfix support for string_view 2020-02-19 10:55:56 -05:00
Pedro Pinto 9cf828e9a4 Implemented iLink3 order 2020-02-18 15:37:06 -05:00
Eric Harding 41408fb609 ILink3ExecutionGateway placeholder 2020-02-14 16:07:44 -06:00
Eric Harding 4c21e32838 ILink3GW core gateway 2020-02-14 16:06:58 -06:00
Eric Harding 385f3ea402 add unix_nano_now() to datetime.h 2020-02-14 16:05:00 -06:00
Eric Harding 8d6d8cab18 Update SecureLogon for ilink3 2020-02-14 16:05:00 -06:00
Eric Harding 01615f824a Generated ilink3 messages 2020-02-14 16:05:00 -06:00
Pedro Pinto c6779ddc90 Reactore GLBX native order for iLink3. 2020-02-13 15:26:33 -05:00
Pedro Pinto 46eff65435 Changed IExecutionGateway to support non FIX gateways. 2020-02-12 10:11:53 -05:00
284 changed files with 152727 additions and 2239 deletions

No files matched your search

+1
View File
@@ -37,3 +37,4 @@ trace_data/
*.rej
/build
.clangd/
*CMakeDeps.cmake
+9 -5
View File
@@ -8,13 +8,15 @@
"program": "${workspaceRoot}/build/ninja_debug/server/sparkd",
"args": [
"run",
// "test",
// "-a",
// "[ilink3]"
"test",
"-a",
"OutrightVanillaOrder"
],
"stopAtEntry": false,
"cwd": "${workspaceRoot}/build/ninja_debug/server/",
"environment": [],
"environment": [
{ "name": "TZ", "value": "America/Chicago" }
],
"externalConsole": false,
"MIMode": "gdb",
"setupCommands": [
@@ -31,7 +33,9 @@
"request": "launch",
"program": "/home/bts/spark/build/ninja_debug/server/sparkd",
"args": [
"run"
"test",
"-a",
"[ilink3]"
],
"stopAtEntry": false,
"cwd": "/home/bts/spark/build/ninja_debug/server/",
+1
View File
@@ -41,6 +41,7 @@ else ()
-Wno-missing-braces -Wno-ignored-qualifiers \
--std=c++17 -fdiagnostics-color")
set (CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_DEBUG} -fno-omit-frame-pointer -DNDEBUG -O3")
set (CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -DDEBUG")
set (CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wno-undef")
add_subdirectory(src/server/sparkd)
add_subdirectory(src/lib)
+1 -1
View File
@@ -1 +1 @@
set(sources /home/ppinto/Dropbox/bts/spark/doc/user/build.sh;content/_index.md;content/bts_spark/Glossary/_index.md;content/bts_spark/Help/_index.md;content/bts_spark/Help/about_spark.png;content/bts_spark/Help/image39.png;content/bts_spark/Help/image58.png;content/bts_spark/Launcher/_index.md;content/bts_spark/Launcher/image38.png;content/bts_spark/Launcher/image68.png;content/bts_spark/Logging-In-to-the-System/_index.md;content/bts_spark/Logging-In-to-the-System/image3.png;content/bts_spark/Logging-In-to-the-System/image64.png;content/bts_spark/Market/_index.md;content/bts_spark/Market/change_securities.png;content/bts_spark/Market/image11.png;content/bts_spark/Market/image14.png;content/bts_spark/Market/image20.png;content/bts_spark/Market/image21.png;content/bts_spark/Market/image23.png;content/bts_spark/Market/image24.png;content/bts_spark/Market/image34.png;content/bts_spark/Market/image49.png;content/bts_spark/Market/image53.png;content/bts_spark/Market/image66.png;content/bts_spark/Market/image69.png;content/bts_spark/Market/market_view.png;content/bts_spark/Market/settings_icon.png;content/bts_spark/Market/time_and_sales.png;content/bts_spark/My-Windows/_index.md;content/bts_spark/My-Windows/image15.png;content/bts_spark/My-Windows/image16.png;content/bts_spark/My-Windows/image37.png;content/bts_spark/Settings/_index.md;content/bts_spark/Settings/image51.png;content/bts_spark/Settings/image9.png;content/bts_spark/Spreader/_index.md;content/bts_spark/Spreader/image10.png;content/bts_spark/Spreader/image13.png;content/bts_spark/Spreader/image18.png;content/bts_spark/Spreader/image19.png;content/bts_spark/Spreader/image28.png;content/bts_spark/Spreader/image29.png;content/bts_spark/Spreader/image31.png;content/bts_spark/Spreader/image35.png;content/bts_spark/Spreader/image40.png;content/bts_spark/Spreader/image43.png;content/bts_spark/Spreader/image54.png;content/bts_spark/Spreader/image55.png;content/bts_spark/Spreader/image56.png;content/bts_spark/Spreader/image6.png;content/bts_spark/Spreader/image7.png;content/bts_spark/Spreader/image70.png;content/bts_spark/Spreader/image72.png;content/bts_spark/Spreader/image8.png;content/bts_spark/Trading/OrderTypes/index.md;content/bts_spark/Trading/_index.md;content/bts_spark/Trading/algo_studio.png;content/bts_spark/Trading/collapse_all_icon.png;content/bts_spark/Trading/cover_algo_control.png;content/bts_spark/Trading/expand_all_icon.png;content/bts_spark/Trading/image1.png;content/bts_spark/Trading/image17.png;content/bts_spark/Trading/image2.png;content/bts_spark/Trading/image25.png;content/bts_spark/Trading/image27.png;content/bts_spark/Trading/image33.png;content/bts_spark/Trading/image42.png;content/bts_spark/Trading/image44.png;content/bts_spark/Trading/image47.png;content/bts_spark/Trading/image48.png;content/bts_spark/Trading/image5.png;content/bts_spark/Trading/image50.png;content/bts_spark/Trading/image52.png;content/bts_spark/Trading/image57.png;content/bts_spark/Trading/image59.png;content/bts_spark/Trading/image60.png;content/bts_spark/Trading/image61.png;content/bts_spark/Trading/image62.png;content/bts_spark/Trading/image73.png;content/bts_spark/Trading/image74.png;content/bts_spark/Trading/join_algo_control.png;content/bts_spark/Trading/ladder.png;content/bts_spark/Trading/ladder_badges.png;content/bts_spark/Trading/ladder_main_part.png;content/bts_spark/Trading/lean_algo_control.png;content/bts_spark/Trading/mini_orderbook.png;content/bts_spark/Trading/oco_algo_control.png;content/bts_spark/Trading/offset_algo_control.png;content/bts_spark/Trading/pnl_calculation.png;content/bts_spark/Trading/pnl_window.png;content/bts_spark/Trading/recreate_icon.png;content/bts_spark/Trading/settings_icon.png;content/bts_spark/Trading/spark_menu_order_book.png;content/bts_spark/Trading/top_algo_control.png;content/bts_spark/Trading/trigger_algo_control.png;content/bts_spark/_index.md;content/release_notes/_index.md;content/system_requirements/_index.md;static/book.js;static/favicon.ico;static/image.js;templates/index.html;templates/page.html;templates/section.html;templates/shortcodes/back_button.html)
set(sources /home/pfisher/bts/spark/doc/user/build.sh;content/_index.md;content/bts_spark/Glossary/_index.md;content/bts_spark/Help/_index.md;content/bts_spark/Help/about_spark.png;content/bts_spark/Help/image39.png;content/bts_spark/Help/image58.png;content/bts_spark/Launcher/_index.md;content/bts_spark/Launcher/image38.png;content/bts_spark/Launcher/image68.png;content/bts_spark/Logging-In-to-the-System/_index.md;content/bts_spark/Logging-In-to-the-System/image3.png;content/bts_spark/Logging-In-to-the-System/image64.png;content/bts_spark/Market/_index.md;content/bts_spark/Market/change_securities.png;content/bts_spark/Market/image11.png;content/bts_spark/Market/image14.png;content/bts_spark/Market/image20.png;content/bts_spark/Market/image21.png;content/bts_spark/Market/image23.png;content/bts_spark/Market/image24.png;content/bts_spark/Market/image34.png;content/bts_spark/Market/image49.png;content/bts_spark/Market/image53.png;content/bts_spark/Market/image66.png;content/bts_spark/Market/image69.png;content/bts_spark/Market/market_view.png;content/bts_spark/Market/settings_icon.png;content/bts_spark/Market/time_and_sales.png;content/bts_spark/My-Windows/_index.md;content/bts_spark/My-Windows/image15.png;content/bts_spark/My-Windows/image16.png;content/bts_spark/My-Windows/image37.png;content/bts_spark/Settings/_index.md;content/bts_spark/Settings/image51.png;content/bts_spark/Settings/image9.png;content/bts_spark/Spreader/_index.md;content/bts_spark/Spreader/image10.png;content/bts_spark/Spreader/image13.png;content/bts_spark/Spreader/image18.png;content/bts_spark/Spreader/image19.png;content/bts_spark/Spreader/image28.png;content/bts_spark/Spreader/image29.png;content/bts_spark/Spreader/image31.png;content/bts_spark/Spreader/image35.png;content/bts_spark/Spreader/image40.png;content/bts_spark/Spreader/image43.png;content/bts_spark/Spreader/image54.png;content/bts_spark/Spreader/image55.png;content/bts_spark/Spreader/image56.png;content/bts_spark/Spreader/image6.png;content/bts_spark/Spreader/image7.png;content/bts_spark/Spreader/image70.png;content/bts_spark/Spreader/image72.png;content/bts_spark/Spreader/image8.png;content/bts_spark/Trading/OrderTypes/index.md;content/bts_spark/Trading/_index.md;content/bts_spark/Trading/algo_studio.png;content/bts_spark/Trading/collapse_all_icon.png;content/bts_spark/Trading/cover_algo_control.png;content/bts_spark/Trading/expand_all_icon.png;content/bts_spark/Trading/image1.png;content/bts_spark/Trading/image17.png;content/bts_spark/Trading/image2.png;content/bts_spark/Trading/image25.png;content/bts_spark/Trading/image27.png;content/bts_spark/Trading/image33.png;content/bts_spark/Trading/image42.png;content/bts_spark/Trading/image44.png;content/bts_spark/Trading/image47.png;content/bts_spark/Trading/image48.png;content/bts_spark/Trading/image5.png;content/bts_spark/Trading/image50.png;content/bts_spark/Trading/image52.png;content/bts_spark/Trading/image57.png;content/bts_spark/Trading/image59.png;content/bts_spark/Trading/image60.png;content/bts_spark/Trading/image61.png;content/bts_spark/Trading/image62.png;content/bts_spark/Trading/image73.png;content/bts_spark/Trading/image74.png;content/bts_spark/Trading/join_algo_control.png;content/bts_spark/Trading/ladder.png;content/bts_spark/Trading/ladder_badges.png;content/bts_spark/Trading/ladder_main_part.png;content/bts_spark/Trading/lean_algo_control.png;content/bts_spark/Trading/mini_orderbook.png;content/bts_spark/Trading/oco_algo_control.png;content/bts_spark/Trading/offset_algo_control.png;content/bts_spark/Trading/pnl_calculation.png;content/bts_spark/Trading/pnl_window.png;content/bts_spark/Trading/recreate_icon.png;content/bts_spark/Trading/settings_icon.png;content/bts_spark/Trading/spark_menu_order_book.png;content/bts_spark/Trading/top_algo_control.png;content/bts_spark/Trading/trigger_algo_control.png;content/bts_spark/_index.md;content/release_notes/_index.md;content/system_requirements/_index.md;static/book.js;static/favicon.ico;static/image.js;templates/index.html;templates/page.html;templates/section.html;templates/shortcodes/back_button.html)
+209 -203
View File
@@ -1,40 +1,40 @@
using bts.api;
using bts.api.core;
using bts.api.orderman;
using bts.utils;
using mbts.spark.utilities;
using bts.api;
using bts.api.core;
using bts.api.orderman;
using bts.utils;
using mbts.spark.utilities;
using mbts.ui.utility;
using mbts.utils;
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;
using System.Reactive.Linq;
namespace mbts.spark {
interface IAlgoProvider {
AlgoTypeSetImmutable get_permitted_algos(SecurityImmutable security);
AlgoTypeSetImmutable get_permitted_algos(SecurityCriteriaClause clause);
IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_algos();
IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_algos(SecurityImmutable security);
IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_algos(SecurityCriteriaClause clause);
}
// This interface is used to distinguish the attachable algo provider
// from the standard one wrt MEF export/import.
interface IAttachableAlgoProvider : IAlgoProvider { }
class AlgoProvider : IAlgoProvider {
readonly IEnumerable<SecurityAlgoTypeSetImmutable> _security_algo_types;
readonly IAlgoStudioModel _studio_model;
readonly ISecurityAuxiliaryDataModel _security_auxiliary_model;
readonly IObservable<TableUpdate<AlgoData, AlgoKey>> _studio_algos_source;
readonly DynamicOrderHelper _dynamic_helper;
readonly AlgoTypeSetImmutable _full_set;
public AlgoProvider(IEnumerable<SecurityAlgoTypeSetImmutable> algo_sets, IAlgoStudioModel studio_model, DynamicOrderHelper dynamic_helper, ISecurityAuxiliaryDataModel security_auxiliary_model) {
_security_algo_types = algo_sets;
_studio_model = studio_model;
using mbts.utils;
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.Linq;
using System.Reactive.Linq;
namespace mbts.spark {
interface IAlgoProvider {
AlgoTypeSetImmutable get_permitted_algos(SecurityImmutable security);
AlgoTypeSetImmutable get_permitted_algos(SecurityCriteriaClause clause);
IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_algos();
IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_algos(SecurityImmutable security);
IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_algos(SecurityCriteriaClause clause);
}
// This interface is used to distinguish the attachable algo provider
// from the standard one wrt MEF export/import.
interface IAttachableAlgoProvider : IAlgoProvider { }
class AlgoProvider : IAlgoProvider {
readonly IEnumerable<SecurityAlgoTypeSetImmutable> _security_algo_types;
readonly IAlgoStudioModel _studio_model;
readonly ISecurityAuxiliaryDataModel _security_auxiliary_model;
readonly IObservable<TableUpdate<AlgoData, AlgoKey>> _studio_algos_source;
readonly DynamicOrderHelper _dynamic_helper;
readonly AlgoTypeSetImmutable _full_set;
public AlgoProvider(IEnumerable<SecurityAlgoTypeSetImmutable> algo_sets, IAlgoStudioModel studio_model, DynamicOrderHelper dynamic_helper, ISecurityAuxiliaryDataModel security_auxiliary_model) {
_security_algo_types = algo_sets;
_studio_model = studio_model;
_dynamic_helper = dynamic_helper;
_full_set = new AlgoTypeSetImmutable(
ImmutableHashSet.CreateRange(
@@ -51,42 +51,42 @@ namespace mbts.spark {
_studio_algos_source = studio_model.definitions
.StartWith(TableUpdate<AlgoData, AlgoKey>.mk_empty(AlgoData.key_fn))
.Cache();
}
public AlgoTypeSetImmutable get_permitted_algos(SecurityImmutable security) =>
_security_algo_types.Where(p => p.security_criteria.includes(security))
.Select(sec_algos => sec_algos.algo_type_set)
.Aggregate(_full_set, (result, algos) => result.intersect_with(algos));
public AlgoTypeSetImmutable get_permitted_algos(SecurityCriteriaClause clause) =>
_security_algo_types.Where(p => p.security_criteria.is_subset_of(clause))
.Select(sec_algos => sec_algos.algo_type_set)
.Aggregate(_full_set, (result, algos) => result.intersect_with(algos));
public IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_algos() =>
_studio_algos_source
.AppendTable(_security_algo_types
.Select(s => s.algo_type_set)
.Aggregate(AlgoTypeSetImmutable.empty, (a, b) => a.union_with(b))
.define_algos(_dynamic_helper));
public IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_algos(SecurityCriteriaClause security_clause) {
var system_algos = _security_algo_types.match_algos(
security_clause,
_dynamic_helper,
def => _fit_to_type(def, security_clause, _security_algo_types));
return _studio_algos_source.Select(
a => (algo: a, fitted_def: _fit_to_type(a.value, security_clause, _security_algo_types)),
tup => tup.algo.key)
.Select(tup => AlgoData.create(tup.fitted_def, tup.algo.name, tup.algo.key), a => a.key)
.AppendTable(system_algos.Values);
}
public IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_algos(SecurityImmutable security) {
var clause = new SecurityCriteriaClause(
exchanges: new[] { security.Exchange },
security_types: new[] { security.security_type() });
}
public AlgoTypeSetImmutable get_permitted_algos(SecurityImmutable security) =>
_security_algo_types.Where(p => p.security_criteria.includes(security))
.Select(sec_algos => sec_algos.algo_type_set)
.Aggregate(_full_set, (result, algos) => result.intersect_with(algos));
public AlgoTypeSetImmutable get_permitted_algos(SecurityCriteriaClause clause) =>
_security_algo_types.Where(p => p.security_criteria.is_subset_of(clause))
.Select(sec_algos => sec_algos.algo_type_set)
.Aggregate(_full_set, (result, algos) => result.intersect_with(algos));
public IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_algos() =>
_studio_algos_source
.AppendTable(_security_algo_types
.Select(s => s.algo_type_set)
.Aggregate(AlgoTypeSetImmutable.empty, (a, b) => a.union_with(b))
.define_algos(_dynamic_helper));
public IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_algos(SecurityCriteriaClause security_clause) {
var system_algos = _security_algo_types.match_algos(
security_clause,
_dynamic_helper,
def => _fit_to_type(def, security_clause, _security_algo_types));
return _studio_algos_source.Select(
a => (algo: a, fitted_def: _fit_to_type(a.value, security_clause, _security_algo_types)),
tup => tup.algo.key)
.Select(tup => AlgoData.create(tup.fitted_def, tup.algo.name, tup.algo.key), a => a.key)
.AppendTable(system_algos.Values);
}
public IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_algos(SecurityImmutable security) {
var clause = new SecurityCriteriaClause(
exchanges: new[] { security.Exchange },
security_types: new[] { security.security_type() });
// i.e., non-studio algos
var system_algos = _security_algo_types.match_algos(
clause,
@@ -94,135 +94,141 @@ namespace mbts.spark {
def => _fit_to_type(def, security, _security_algo_types),
_security_auxiliary_model.get_display_quantity_divisor(security));
return _studio_algos_source.Select(
a => (algo: a, fitted_def: _fit_to_type(a.value, security, _security_algo_types)),
tup => tup.algo.key)
// filter out algos that use an unsupported root, because this would have the effect of duplicating the default algo
// (e.g., GlbxIceberg orders would be converted to Limit/Spreader orders for non-Glbx securities).
.Filter(tup => tup.fitted_def.actual().CustomCase == tup.algo.value.actual().CustomCase, tup => tup.algo.key)
.Select(tup => AlgoData.create(tup.fitted_def, tup.algo.name, tup.algo.key), AlgoData.key_fn)
return _studio_algos_source.Select(
a => (algo: a, fitted_def: _fit_to_type(a.value, security, _security_algo_types)),
tup => tup.algo.key)
// filter out algos that use an unsupported root, because this would have the effect of duplicating the default algo
// (e.g., GlbxIceberg orders would be converted to Limit/Spreader orders for non-Glbx securities).
.Filter(tup => tup.fitted_def.actual().CustomCase == tup.algo.value.actual().CustomCase, tup => tup.algo.key)
.Select(tup => AlgoData.create(tup.fitted_def, tup.algo.name, tup.algo.key), AlgoData.key_fn)
.AppendTable(system_algos.Values);
}
protected virtual void assign_fallback_algo(CustomOrderDefinition mdef, SecurityCriteriaClause security_clause) {
if (security_clause.security_types.empty_if_null().Contains(Security.ContractOneofCase.Synthetic.security_type())) {
mdef.Spreader = new SpreaderOrderDefinition();
} else {
mdef.Limit = new LimitOrderDefinition();
}
}
protected virtual void assign_fallback_algo(CustomOrderDefinition mdef, SecurityImmutable security) {
if (security.ContractCase == Security.ContractOneofCase.Synthetic) {
mdef.Spreader = new SpreaderOrderDefinition {
Security = security.CloneToMutable()
};
} else {
mdef.Limit = new LimitOrderDefinition();
}
}
CustomOrderDefinitionImmutable _fit_to_type(CustomOrderDefinitionImmutable defi, SecurityCriteriaClause security_clause, IEnumerable<SecurityAlgoTypeSetImmutable> algo_sets) {
var mroot = defi.CloneToMutable();
if (_fit_to_type(mroot, security_clause, algo_sets)) {
defi = new CustomOrderDefinitionImmutable(mroot);
}
return defi;
}
CustomOrderDefinitionImmutable _fit_to_type(CustomOrderDefinitionImmutable defi, SecurityImmutable security, IEnumerable<SecurityAlgoTypeSetImmutable> algo_sets) {
var mroot = defi.CloneToMutable();
if (_fit_to_type(mroot, security, algo_sets)) {
defi = new CustomOrderDefinitionImmutable(mroot);
}
return defi;
}
bool _fit_to_type(CustomOrderDefinition mdef, SecurityCriteriaClause security_clause, IEnumerable<SecurityAlgoTypeSetImmutable> algo_sets) {
while (mdef.CustomCase == CustomOrderDefinition.CustomOneofCase.Composite)
mdef = mdef.Composite.Definition;
var sec_sets = algo_sets.Where(s => security_clause.is_subset_of(s.security_criteria));
if (mdef.CustomCase == CustomOrderDefinition.CustomOneofCase.Dynamic) {
if (!algo_sets.All(s => s.algo_type_set.dynamic_types.Contains(mdef.Dynamic.DynamicOrderType))) {
assign_fallback_algo(mdef, security_clause);
return true;
}
} else if (!sec_sets.All(s => s.algo_type_set.factory_types.Contains(mdef.CustomCase))) {
assign_fallback_algo(mdef, security_clause);
return true;
}
return OrderDataHelper.get_child_algo_data(mdef).Count(cad => _fit_to_type(cad.get_child(mdef), security_clause, algo_sets)) > 0;
}
bool _fit_to_type(CustomOrderDefinition mdef, SecurityImmutable security, IEnumerable<SecurityAlgoTypeSetImmutable> algo_sets) {
while (mdef.CustomCase == CustomOrderDefinition.CustomOneofCase.Composite)
mdef = mdef.Composite.Definition;
var sec_sets = algo_sets.Where(s => s.security_criteria.includes(security));
if (mdef.CustomCase == CustomOrderDefinition.CustomOneofCase.Dynamic) {
if (!sec_sets.All(s => s.algo_type_set.dynamic_types.Contains(mdef.Dynamic.DynamicOrderType))) {
assign_fallback_algo(mdef, security);
return true;
}
} else if (!sec_sets.All(s => s.algo_type_set.factory_types.Contains(mdef.CustomCase))) {
assign_fallback_algo(mdef, security);
return true;
}
return OrderDataHelper.get_child_algo_data(mdef).Count(cad => _fit_to_type(cad.get_child(mdef), security, algo_sets)) > 0;
}
}
class AttachableAlgoProvider : AlgoProvider, IAttachableAlgoProvider {
public AttachableAlgoProvider(IEnumerable<SecurityAlgoTypeSetImmutable> algo_sets, IAlgoStudioModel studio_model, DynamicOrderHelper dynamic_helper, ISecurityAuxiliaryDataModel security_auxiliary_model)
: base(algo_sets, studio_model, dynamic_helper, security_auxiliary_model) { }
// arbitrarily assign LimitOrderDefinition (this is just a placeholder; the real child is the attach target algo).
// Note: although 'None' might seem approrpriate, the server requires a valid definition (perhaps this will change in the future).
protected override void assign_fallback_algo(CustomOrderDefinition mdef, SecurityCriteriaClause security_clause) =>
mdef.Limit = new LimitOrderDefinition();
// arbitrarily assign LimitOrderDefinition (this is just a placeholder; the real child is the attach target algo).
// Note: although 'None' might seem approrpriate, the server requires a valid definition (perhaps this will change in the future).
protected override void assign_fallback_algo(CustomOrderDefinition mdef, SecurityImmutable security) =>
mdef.Limit = new LimitOrderDefinition();
}
static class AlgoProviderExtensions {
// This stream filters out composite algos that are not subsets of the non-composite algo types in the table
// (normally, such composites are 'auto-fit,' meaning the algo nodes that are not part of the set are replaced with ones that are).
// For example, consider a composite algo the following structure: Offset -> GlbxIceberg.
// If an update arrives with these noncomposite types: "Iceberg, Limit, Market," that composite algo is not a subset of the non-composite algo types,
// because GlbxIceberg is not in the table, so it would only be added to the table if/when GlbxIceberg arrives.
public static IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_complete_algos(this IAlgoProvider algo_provider) =>
subscribe_complete_algos(algo_provider.subscribe_algos());
public static IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_complete_algos(IObservable<TableUpdate<AlgoData, AlgoKey>> source) =>
Observable.Create<TableUpdate<AlgoData, AlgoKey>>(o => {
var last_out = TableUpdate<AlgoData, AlgoKey>.mk_empty(AlgoData.key_fn);
var incomplete_composites = ImmutableHashSet<AlgoData>.Empty;
return source.Subscribe(table => {
var bd = last_out.to_builder();
foreach (var algo in incomplete_composites.Where(a => OrderDataHelper.enumerate_algo_structure(a.value).All(tup => _is_part_contained(tup, table.state)))) {
incomplete_composites = incomplete_composites.Remove(algo);
bd.add_or_update(algo);
}
foreach (var a in table.removed.Concat(table.changes.Old()))
bd.remove(a.key);
foreach (var a in table.added.Concat(table.changed)) {
var structure = OrderDataHelper.enumerate_algo_structure(a.value);
if (structure.All(t => _is_part_contained(t, table.state))) {
bd.add_or_update(a);
} else {
incomplete_composites = incomplete_composites.Add(a);
}
}
o.OnNext(last_out = bd.to_table());
});
});
static bool _is_part_contained((CustomOrderDefinition.CustomOneofCase type, string dyn, uint studio_id, uint spreader_sec) structure, IReadOnlyDictionary<AlgoKey, AlgoData> state) =>
structure.type == CustomOrderDefinition.CustomOneofCase.Composite ||
structure.type == CustomOrderDefinition.CustomOneofCase.Dynamic ?
state.ContainsKey(AlgoKey.from_dynamic_type(structure.dyn)) :
state.ContainsKey(AlgoKey.from_known_type(structure.type));
}
}
}
protected virtual void assign_fallback_algo(CustomOrderDefinition mdef, SecurityCriteriaClause security_clause) {
if (security_clause.security_types.empty_if_null().Contains(Security.ContractOneofCase.Synthetic.security_type())) {
mdef.Spreader = new SpreaderOrderDefinition();
} else {
mdef.Limit = new LimitOrderDefinition();
}
}
protected virtual void assign_fallback_algo(CustomOrderDefinition mdef, SecurityImmutable security) {
if (security.ContractCase == Security.ContractOneofCase.Synthetic) {
mdef.Spreader = new SpreaderOrderDefinition {
Security = security.CloneToMutable()
};
} else {
mdef.Limit = new LimitOrderDefinition();
}
}
CustomOrderDefinitionImmutable _fit_to_type(CustomOrderDefinitionImmutable defi, SecurityCriteriaClause security_clause, IEnumerable<SecurityAlgoTypeSetImmutable> algo_sets) {
var mroot = defi.CloneToMutable();
if (_fit_to_type(mroot, security_clause, algo_sets)) {
defi = new CustomOrderDefinitionImmutable(mroot);
}
return defi;
}
CustomOrderDefinitionImmutable _fit_to_type(CustomOrderDefinitionImmutable defi, SecurityImmutable security, IEnumerable<SecurityAlgoTypeSetImmutable> algo_sets) {
var mroot = defi.CloneToMutable();
if (_fit_to_type(mroot, security, algo_sets)) {
defi = new CustomOrderDefinitionImmutable(mroot);
}
return defi;
}
bool _fit_to_type(CustomOrderDefinition mdef, SecurityCriteriaClause security_clause, IEnumerable<SecurityAlgoTypeSetImmutable> algo_sets) {
while (mdef.CustomCase == CustomOrderDefinition.CustomOneofCase.Composite)
mdef = mdef.Composite.Definition;
var sec_sets = algo_sets.Where(s => security_clause.is_subset_of(s.security_criteria));
if (mdef.CustomCase == CustomOrderDefinition.CustomOneofCase.Dynamic) {
if (!algo_sets.All(s => s.algo_type_set.dynamic_types.Contains(mdef.Dynamic.DynamicOrderType))) {
assign_fallback_algo(mdef, security_clause);
return true;
}
} else if (!sec_sets.All(s => s.algo_type_set.factory_types.Contains(mdef.CustomCase))) {
assign_fallback_algo(mdef, security_clause);
return true;
}
return OrderDataHelper.get_child_algo_data(mdef).Count(cad => _fit_to_type(cad.get_child(mdef), security_clause, algo_sets)) > 0;
}
bool _fit_to_type(CustomOrderDefinition mdef, SecurityImmutable security, IEnumerable<SecurityAlgoTypeSetImmutable> algo_sets) {
while (mdef.CustomCase == CustomOrderDefinition.CustomOneofCase.Composite)
mdef = mdef.Composite.Definition;
var sec_sets = algo_sets.Where(s => s.security_criteria.includes(security));
if (mdef.CustomCase == CustomOrderDefinition.CustomOneofCase.Dynamic) {
if (!sec_sets.All(s => s.algo_type_set.dynamic_types.Contains(mdef.Dynamic.DynamicOrderType))) {
assign_fallback_algo(mdef, security);
return true;
}
} else if (!sec_sets.All(s => s.algo_type_set.factory_types.Contains(mdef.CustomCase))) {
assign_fallback_algo(mdef, security);
return true;
}
return OrderDataHelper.get_child_algo_data(mdef).Count(cad => _fit_to_type(cad.get_child(mdef), security, algo_sets)) > 0;
}
}
class AttachableAlgoProvider : AlgoProvider, IAttachableAlgoProvider {
public AttachableAlgoProvider(IEnumerable<SecurityAlgoTypeSetImmutable> algo_sets, IAlgoStudioModel studio_model, DynamicOrderHelper dynamic_helper, ISecurityAuxiliaryDataModel security_auxiliary_model)
: base(algo_sets, studio_model, dynamic_helper, security_auxiliary_model) { }
// arbitrarily assign LimitOrderDefinition (this is just a placeholder; the real child is the attach target algo).
// Note: although 'None' might seem approrpriate, the server requires a valid definition (perhaps this will change in the future).
protected override void assign_fallback_algo(CustomOrderDefinition mdef, SecurityCriteriaClause security_clause) =>
mdef.Limit = new LimitOrderDefinition();
// arbitrarily assign LimitOrderDefinition (this is just a placeholder; the real child is the attach target algo).
// Note: although 'None' might seem approrpriate, the server requires a valid definition (perhaps this will change in the future).
protected override void assign_fallback_algo(CustomOrderDefinition mdef, SecurityImmutable security) =>
mdef.Limit = new LimitOrderDefinition();
}
static class AlgoProviderExtensions {
// This stream filters out composite algos that are not subsets of the non-composite algo types in the table
// (normally, such composites are 'auto-fit,' meaning the algo nodes that are not part of the set are replaced with ones that are).
// For example, consider a composite algo the following structure: Offset -> GlbxIceberg.
// If an update arrives with these noncomposite types: "Iceberg, Limit, Market," that composite algo is not a subset of the non-composite algo types,
// because GlbxIceberg is not in the table, so it would only be added to the table if/when GlbxIceberg arrives.
public static IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_complete_algos(this IAlgoProvider algo_provider, bool allow_undefined_child = false) =>
subscribe_complete_algos(algo_provider.subscribe_algos(), allow_undefined_child);
public static IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_complete_algos(IObservable<TableUpdate<AlgoData, AlgoKey>> source, bool allow_undefined_child = false) =>
Observable.Create<TableUpdate<AlgoData, AlgoKey>>(o => {
var last_out = TableUpdate<AlgoData, AlgoKey>.mk_empty(AlgoData.key_fn);
var incomplete_composites = ImmutableHashSet<AlgoData>.Empty;
return source.Subscribe(table => {
var bd = last_out.to_builder();
foreach (var algo in incomplete_composites.Where(a => _validate_algo_structure(a.value, table.state, allow_undefined_child))) {
incomplete_composites = incomplete_composites.Remove(algo);
bd.add_or_update(algo);
}
foreach (var a in table.removed.Concat(table.changes.Old()))
bd.remove(a.key);
foreach (var a in table.added.Concat(table.changed)) {
if (_validate_algo_structure(a.value, table.state, allow_undefined_child)) {
bd.add_or_update(a);
} else {
incomplete_composites = incomplete_composites.Add(a);
}
}
o.OnNext(last_out = bd.to_table());
});
});
static bool _validate_algo_structure(CustomOrderDefinitionImmutable custom_def, IReadOnlyDictionary<AlgoKey, AlgoData> allowed_algo_parts, bool allow_undefined_child) =>
OrderDataHelper.enumerate_algo_structure(custom_def)
.Select((algo_node, index) =>
// None is an acceptable algo type as long as it's not the root algo (i.e., index > 0)
(index > 0 && allow_undefined_child && algo_node.type == CustomOrderDefinition.CustomOneofCase.None) || _is_part_contained(algo_node, allowed_algo_parts))
.All();
static bool _is_part_contained((CustomOrderDefinition.CustomOneofCase type, string dyn, uint studio_id, uint spreader_sec) structure, IReadOnlyDictionary<AlgoKey, AlgoData> state) =>
structure.type == CustomOrderDefinition.CustomOneofCase.Composite ||
structure.type == CustomOrderDefinition.CustomOneofCase.Dynamic ?
state.ContainsKey(AlgoKey.from_dynamic_type(structure.dyn)) :
state.ContainsKey(AlgoKey.from_known_type(structure.type));
}
}
@@ -138,7 +138,6 @@ namespace mbts.spark {
public event Action invalidated;
#region Bound Properties
public string symbol => security.display_symbol();
public bool inside_bids => _inside_bids.value;
public bool inside_asks => _inside_asks.value;
public int position => _position.value;
@@ -424,4 +423,4 @@ namespace mbts.spark {
}
}
}
}
}
@@ -41,7 +41,6 @@ namespace mbts.spark {
const double working_orders_col_width = 105;
IMultiValueConverter price_converter = new ui.spark.converters.TickDisplayConverter();
add_col(Columns.Security, "Security", r => r.security.ToString(), nameof(Row.security), DefaultColumnWidths.Security, security_filter, DataGridFilterKnownTemplates.Regex);
add_col(Columns.Symbol, DefaultColumnHeaders.Symbol, r => r.symbol, nameof(Row.symbol), DefaultColumnWidths.Symbol, symbol_filter, DataGridFilterKnownTemplates.DiscreteValues);
add_col(Columns.Volume, "Volume", r => r.volume_text, nameof(Row.volume), DefaultColumnWidths.Volume, volume_filter, DataGridFilterKnownTemplates.ULongMinMax);
add_col(Columns.LastSpd, "Spd", r => r.last_price_from_spread ? "S" : "", nameof(Row.last_price_from_spread), DefaultColumnWidths.Side, last_price_from_spread_filter, DataGridFilterKnownTemplates.Regex);
add_col(Columns.Last, "Last", r => r.format_price(r.last_price), nameof(Row.last_price), DefaultColumnWidths.Price, last_price_filter, DataGridFilterKnownTemplates.DoubleMinMax);
@@ -135,7 +134,6 @@ namespace mbts.spark {
}
if (s.filters is FilterStates filters) {
security_filter.load_state(filters.security);
symbol_filter.load_state(filters.symbols);
bid_price_filter.load_state(filters.bid_price);
bid_merged_size_filter.load_state(filters.bid_merged_size);
bid_direct_size_filter.load_state(filters.bid_direct_size);
@@ -186,7 +184,6 @@ namespace mbts.spark {
public static IComparer<Row> get_column_comparer(Columns column) {
switch (column) {
case Columns.Security: return wrap(r => r.security);
case Columns.Symbol: return wrap(r => r.symbol);
case Columns.SecurityKey: return wrap(r => r.security.Id);
case Columns.Volume: return wrap(r => r.volume);
case Columns.Last: return wrap(r => r.last_price);
@@ -220,7 +217,6 @@ namespace mbts.spark {
#region Filters
public IDataGridColumnPersistentFilter<Row, string> security_filter { get; }
public IDataGridColumnPersistentFilter<Row, string[]> symbol_filter { get; } = DiscreteValuesFilterVM.Common.symbol<Row>(r => r.symbol, nameof(Row.symbol));
public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> bid_price_filter { get; } = new DoubleMinMax<Row>(r => r.bid_price, nameof(Row.bid_price));
public IDataGridColumnPersistentFilter<Row, UIntMinMaxState> bid_merged_size_filter { get; } = new UIntMinMax<Row>(r => r.bid_merged_size, nameof(Row.bid_merged_size));
public IDataGridColumnPersistentFilter<Row, UIntMinMaxState> bid_direct_size_filter { get; } = new UIntMinMax<Row>(r => r.bid_direct_size, nameof(Row.bid_direct_size));
@@ -256,7 +252,6 @@ namespace mbts.spark {
public FilterStates get_filter_states() {
return new FilterStates {
security = security_filter.get_state(),
symbols = symbol_filter.get_state(),
bid_price = bid_price_filter.get_state(),
bid_merged_size = bid_merged_size_filter.get_state(),
bid_direct_size = bid_direct_size_filter.get_state(),
@@ -10,7 +10,6 @@ namespace mbts.ui.spark {
partial class MarketViewView : UserControl {
public enum Columns {
Security,
Symbol,
Volume,
Last,
LastSpd,
@@ -50,4 +49,4 @@ namespace mbts.ui.spark {
}
class MarketViewRowView : DIYGrid.HighPerformanceRow<MarketViewVM.Row> { }
}
}
@@ -1,6 +1,7 @@
using bts.api;
using bts.proto;
using bts.settings;
using bts.view;
using mbts.persist;
using mbts.spark;
using mbts.spark.filters;
@@ -100,7 +101,6 @@ namespace bts.shell {
static class DefaultColumns {
public static GridColumnSettings security => new GridColumnSettings(col_id(Columns.Security), DefaultColumnHeaders.Security, width: DefaultColumnWidths.Security);
public static GridColumnSettings symbol => new GridColumnSettings(col_id(Columns.Symbol), DefaultColumnHeaders.Symbol, width: DefaultColumnWidths.Symbol, is_visible: false);
public static GridColumnSettings volume => new GridColumnSettings(col_id(Columns.Volume), DefaultColumnHeaders.Volume, width: DefaultColumnWidths.Volume);
public static GridColumnSettings last => new GridColumnSettings(col_id(Columns.Last), DefaultColumnHeaders.Last, width: DefaultColumnWidths.Price);
public static GridColumnSettings last_from_spread => new GridColumnSettings(col_id(Columns.LastSpd), DefaultColumnHeaders.S, width: DefaultColumnWidths.Price, is_visible: false);
@@ -140,7 +140,6 @@ namespace bts.shell {
public static IEnumerable<GridColumnSettings> all {
get {
yield return security;
yield return symbol;
yield return volume;
yield return last;
yield return last_from_spread;
@@ -233,8 +232,6 @@ namespace bts.shell {
public string last_price_from_spread;
[ProtoMember(28)]
public string mic_code;
[ProtoMember(29)]
public string[] symbols;
}
// migration rules
@@ -1,13 +1,11 @@
using bts.utils;
using Google.Protobuf;
using mbts.spark.filters;
using mbts.spark.filters;
using mbts.ui;
using ProtoBuf;
using System;
namespace bts.shell.time_and_sales {
[ProtoContract]
class FilterStates : IDeepCloneable<FilterStates> {
class FilterStates {
[ProtoMember(1)]
public TimeFilterTuple time;
[ProtoMember(2)]
@@ -22,24 +20,12 @@ namespace bts.shell.time_and_sales {
public string security_key;
[ProtoMember(7)]
public co2.AggressorSide[] aggressor_sides;
[ProtoMember(8)]
public string[] symbols;
public FilterStates Clone() =>
new FilterStates {
time = time.Clone(),
security = security?.Copy(),
price = price?.Clone(),
size = size?.Clone(),
symbols = ArrayHelper.safe_convert(symbols, sym => sym?.Copy())
};
}
static class TimeAndSalesFilterStatesExtensions {
public static Func<TimeAndSalesRowVM, bool> build_predicate(this FilterStates fs) {
var pb = new ColumnPredicateBuilder<TimeAndSalesRowVM>();
pb.add_regex(fs.security, r => r.security.ToString());
pb.add_discrete_values(fs.symbols, r => r.symbol);
pb.add_time_filter(fs.time, r => r.time);
pb.add_price(fs.price, r => r.price, r => r.security);
pb.add_numeric(fs.size, r => r.size);
@@ -1,90 +1,74 @@
using bts.utils;
using mbts.spark;
using mbts.ui;
using mbts.ui.settings;
using ProtoBuf;
using System;
using System.Collections.Generic;
using System.Linq;
using static mbts.ui.controls.DIYGrid;
using Columns = bts.shell.time_and_sales.TimeAndSalesView.Columns;
namespace bts.shell.time_and_sales {
[ProtoContract]
class TimeAndSalesSettings {
[ProtoMember(1)]
public readonly SecurityCriteria security_criteria;
[ProtoMember(2)]
List<SortingCriteria> _sorting_criterion;
[ProtoMember(3)]
public FilterStates filters;
[ProtoMember(4)]
public GridSettings grid;
[ProtoMember(5)]
public SparkMenuDisplaySettings menu_display;
[ProtoMember(6)]
public bool use_defaults;
[ProtoMember(7)]
string[] _symbols;
public TimeAndSalesSettings() { }
public TimeAndSalesSettings(SecurityCriteria security_criteria) :
this(security_criteria, new FilterStates(), null, null) { }
public TimeAndSalesSettings(SecurityCriteria security_criteria, FilterStates filters, GridSettings grid = default, List<SortingCriteria> sorting_criterion = null) {
this.security_criteria = security_criteria;
this.filters = filters;
this.grid = grid;
_sorting_criterion = sorting_criterion;
}
public static GridSettings DefaultGridSettings => GridSettings.Default.with(DefaultColumns.all, DefaultColumns.default_sorting_criterion);
public List<SortingCriteria> sorting_criterion => _sorting_criterion ?? (_sorting_criterion = DefaultColumns.default_sorting_criterion.ToList());
public IReadOnlyList<string> symbols => _symbols ?? Array.Empty<string>();
public static TimeAndSalesSettings Default(SecurityCriteria security_criteria) =>
new TimeAndSalesSettings(security_criteria, new FilterStates(), null, DefaultColumns.default_sorting_criterion.ToList());
// todo: should prob clone new instances of each member;
// prob won't make a difference in practice, but it's weird that 2 instances of TimeAndSalesSettings
// share the same instances of internal members.
public TimeAndSalesSettings with(SecurityCriteria security_criteria) =>
new TimeAndSalesSettings(security_criteria, filters, grid.clone()) {
filters = filters?.Clone(),
grid = grid?.clone(),
menu_display = menu_display.Clone(),
use_defaults = use_defaults,
_symbols = ArrayHelper.safe_convert(_symbols, sym => sym?.Copy())
};
public static class DefaultColumns {
public static GridColumnSettings security => new GridColumnSettings(Columns.Security.col_id(), DefaultColumnHeaders.Security, width: DefaultColumnWidths.Security);
public static GridColumnSettings symbol => new GridColumnSettings(Columns.Symbol.col_id(), DefaultColumnHeaders.Symbol, width: DefaultColumnWidths.Symbol, is_visible: false);
public static GridColumnSettings time => new GridColumnSettings(Columns.Time.col_id(), DefaultColumnHeaders.Time, width: DefaultColumnWidths.TimeOfDay);
public static GridColumnSettings price => new GridColumnSettings(Columns.Price.col_id(), DefaultColumnHeaders.Price, width: DefaultColumnWidths.Price, horizontal_alignment: System.Windows.HorizontalAlignment.Right);
public static GridColumnSettings size => new GridColumnSettings(Columns.Size.col_id(), DefaultColumnHeaders.Size, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right);
public static GridColumnSettings aggressor => new GridColumnSettings(Columns.Aggressor.col_id(), DefaultColumnHeaders.Aggressor, width: DefaultColumnWidths.Aggressor);
public static GridColumnSettings security_key => new GridColumnSettings(Columns.SecurityKey.col_id(), DefaultColumnHeaders.SecurityKey, is_visible: false, width: DefaultColumnWidths.SecurityKey);
public static IEnumerable<SortingCriteria> default_sorting_criterion {
get {
yield return new SortingCriteria(time.id, false);
}
}
public static IEnumerable<GridColumnSettings> all {
get {
yield return security;
yield return symbol;
yield return time;
yield return price;
yield return size;
yield return aggressor;
yield return security_key;
}
}
}
}
}
using mbts.spark;
using mbts.ui;
using mbts.ui.settings;
using ProtoBuf;
using System.Collections.Generic;
using System.Linq;
using static mbts.ui.controls.DIYGrid;
using Columns = bts.shell.time_and_sales.TimeAndSalesView.Columns;
namespace bts.shell.time_and_sales {
[ProtoContract]
class TimeAndSalesSettings {
[ProtoMember(1)]
public readonly SecurityCriteria security_criteria;
[ProtoMember(2)]
List<SortingCriteria> _sorting_criterion;
[ProtoMember(3)]
public FilterStates filters;
[ProtoMember(4)]
public GridSettings grid;
[ProtoMember(5)]
public SparkMenuDisplaySettings menu_display;
[ProtoMember(6)]
public bool use_defaults;
public TimeAndSalesSettings() { }
public TimeAndSalesSettings(SecurityCriteria security_criteria) :
this(security_criteria, new FilterStates(), null, null) { }
public TimeAndSalesSettings(SecurityCriteria security_criteria, FilterStates filters, GridSettings grid = default, List<SortingCriteria> sorting_criterion = null) {
this.security_criteria = security_criteria;
this.filters = filters;
this.grid = grid;
_sorting_criterion = sorting_criterion;
}
public static GridSettings DefaultGridSettings => GridSettings.Default.with(DefaultColumns.all, DefaultColumns.default_sorting_criterion);
public List<SortingCriteria> sorting_criterion => _sorting_criterion ?? (_sorting_criterion = DefaultColumns.default_sorting_criterion.ToList());
public static TimeAndSalesSettings Default(SecurityCriteria security_criteria) =>
new TimeAndSalesSettings(security_criteria, new FilterStates(), null, DefaultColumns.default_sorting_criterion.ToList());
public TimeAndSalesSettings with(SecurityCriteria security_criteria) =>
new TimeAndSalesSettings(security_criteria, filters, grid);
public static class DefaultColumns {
public static GridColumnSettings security => new GridColumnSettings(Columns.Security.col_id(), DefaultColumnHeaders.Security, width: DefaultColumnWidths.Security);
public static GridColumnSettings time => new GridColumnSettings(Columns.Time.col_id(), DefaultColumnHeaders.Time, width: DefaultColumnWidths.TimeOfDay);
public static GridColumnSettings price => new GridColumnSettings(Columns.Price.col_id(), DefaultColumnHeaders.Price, width: DefaultColumnWidths.Price, horizontal_alignment: System.Windows.HorizontalAlignment.Right);
public static GridColumnSettings size => new GridColumnSettings(Columns.Size.col_id(), DefaultColumnHeaders.Size, width: DefaultColumnWidths.Size, horizontal_alignment: System.Windows.HorizontalAlignment.Right);
public static GridColumnSettings aggressor => new GridColumnSettings(Columns.Aggressor.col_id(), DefaultColumnHeaders.Aggressor, width: DefaultColumnWidths.Aggressor);
public static GridColumnSettings security_key => new GridColumnSettings(Columns.SecurityKey.col_id(), DefaultColumnHeaders.SecurityKey, is_visible: false, width: DefaultColumnWidths.SecurityKey);
public static IEnumerable<SortingCriteria> default_sorting_criterion {
get {
yield return new SortingCriteria(time.id, false);
}
}
public static IEnumerable<GridColumnSettings> all {
get {
yield return security;
yield return time;
yield return price;
yield return size;
yield return aggressor;
yield return security_key;
}
}
}
}
}
@@ -39,7 +39,6 @@ namespace bts.shell.time_and_sales {
public uint trade_key { get; }
public uint security_key => security.Id;
public SecurityImmutable security { get; }
public string symbol => security.display_symbol();
public DateTime time => ((ulong)_trade.Time).lldatetime().ToLocalTime();
public double price => _trade.price();
public uint size => _trade.Size;
@@ -80,13 +80,6 @@ namespace bts.shell.time_and_sales {
DefaultColumnWidths.Security,
security_filter,
DataGridFilterKnownTemplates.Regex);
add_column(
TimeAndSalesView.Columns.Symbol,
r => r.symbol,
DefaultColumnHeaders.Symbol,
DefaultColumnWidths.Symbol,
symbol_filter,
DataGridFilterKnownTemplates.DiscreteValues);
add_column(
TimeAndSalesView.Columns.Time,
r => r.time.ToString("h:mm:ss tt"),
@@ -138,10 +131,7 @@ namespace bts.shell.time_and_sales {
return Disposable.Create(() => f.filter_changed -= _refresh_filter.execute);
}));
_sort_manager.active_sorting_changed += _refresh_sort.execute;
resources.Add(Disposable.Create(() => {
_sort_manager.active_sorting_changed -= _refresh_sort.execute;
_filters.OfType<IDisposable>().DisposeAll();
}));
resources.Add(Disposable.Create(() => _sort_manager.active_sorting_changed -= _refresh_sort.execute));
_dispose = Disposable.Create(resources.ToArray());
// local methods
void add_column(
@@ -192,7 +182,6 @@ namespace bts.shell.time_and_sales {
// filters
public IDataGridColumnPersistentFilter<TimeAndSalesRowVM, TimeFilterTuple> time_filter { get; } = new TimeFilterVM<TimeAndSalesRowVM>(r => r.time);
public IDataGridColumnPersistentFilter<TimeAndSalesRowVM, string> security_filter { get; } = new RegexFilterVM<TimeAndSalesRowVM>(r => r.security.ToString(), nameof(TimeAndSalesRowVM.security));
public IDataGridColumnPersistentFilter<TimeAndSalesRowVM, string[]> symbol_filter { get; } = DiscreteValuesFilterVM.Common.symbol<TimeAndSalesRowVM>(r => r.symbol, nameof(TimeAndSalesRowVM.symbol));
public IDataGridColumnPersistentFilter<TimeAndSalesRowVM, DoubleMinMaxState> price_filter { get; } = new DoubleMinMax<TimeAndSalesRowVM>(r => r.price);
public IDataGridColumnPersistentFilter<TimeAndSalesRowVM, UIntMinMaxState> size_filter { get; } = new UIntMinMax<TimeAndSalesRowVM>(r => r.size, nameof(TimeAndSalesRowVM.size));
public IDataGridColumnPersistentFilter<TimeAndSalesRowVM, AggressorSide[]> aggressor_filter { get; } = DiscreteValuesFilterVM.Create<TimeAndSalesRowVM, AggressorSide>(EnumHelper.get_values<AggressorSide>().ToArray(), r => r.aggressor_side);
@@ -270,7 +259,6 @@ namespace bts.shell.time_and_sales {
// load filters
time_filter.load_state(filters.time);
security_filter.load_state(filters.security);
symbol_filter.load_state(filters.symbols);
price_filter.load_state(filters.price);
size_filter.load_state(filters.size);
aggressor_filter.load_state(filters.aggressor_sides);
@@ -361,7 +349,6 @@ namespace bts.shell.time_and_sales {
case TimeAndSalesView.Columns.Aggressor: return wrap(r => r.aggressor_side);
case TimeAndSalesView.Columns.Time: return wrap(r => r.time);
case TimeAndSalesView.Columns.Security: return wrap(r => r.security);
case TimeAndSalesView.Columns.Symbol: return wrap(r => r.symbol);
case TimeAndSalesView.Columns.Price: return wrap(r => r.price);
case TimeAndSalesView.Columns.Size: return wrap(r => r.size);
case TimeAndSalesView.Columns.SecurityKey: return wrap(r => r.security_key);
@@ -386,7 +373,6 @@ namespace bts.shell.time_and_sales {
new FilterStates {
time = time_filter.get_state(),
security = security_filter.get_state(),
symbols = symbol_filter.get_state(),
price = price_filter.get_state(),
size = size_filter.get_state(),
security_key = security_key_filter.get_state(),
@@ -6,7 +6,7 @@ namespace bts.shell.time_and_sales {
/// Interaction logic for TimeAndSalesView.xaml
/// </summary>
partial class TimeAndSalesView : UserControl {
public enum Columns { Time, Security, Symbol, Price, Size, SecurityKey, Aggressor, RowKey }
public enum Columns { Time, Security, Price, Size, SecurityKey, Aggressor, RowKey }
public TimeAndSalesView() =>
InitializeComponent();
@@ -1,20 +1,18 @@
using bts.api;
using bts.settings;
using bts.shell.time_and_sales;
using bts.spark;
using bts.shell.time_and_sales;
using mbts;
using mbts.co2;
using mbts.config;
using mbts.persist;
using mbts.spark;
using mbts.ui;
using mbts.ui.settings;
using System;
using System.Collections.Generic;
using System.ComponentModel.Composition;
using System.Reactive.Linq;
using System.Threading.Tasks;
using Columns = bts.shell.time_and_sales.TimeAndSalesView.Columns;
using mbts.config;
namespace bts.shell {
class TimeAndSalesWindowFactory : ShellWindowFactory<TimeAndSalesSettings> {
@@ -42,7 +40,7 @@ namespace bts.shell {
public override Task<IShellWindow> load(TimeAndSalesSettings state) =>
Task.FromResult<IShellWindow>(new TimeAndSalesWindow(this, _bus, _consumer, state, SparkCommands.PickTimeAndSales.icon,
_settings_hub.create_time_and_sales_settings_manager(state.grid),
new WindowSettingsManager<GridSettings>(_settings_hub, state.grid, s => s.time_and_sales_grid, (s, g) => s.with(time_and_sales:g)),
_time_and_sales_conf.can_pause_updates,
_security_auxiliary_model.get_display_quantity_divisor));
@@ -57,12 +55,5 @@ namespace bts.shell {
public TimeAndSalesConfig(bts.config.TimeAndSalesOptions options) =>
can_pause_updates = options.can_pause_updates;
}
// migration rules
[Export(typeof(ITimeAndSalesGridSettingsMigrationRule))]
class AddMissingColumnsMigrationRule : ITimeAndSalesGridSettingsMigrationRule {
public bool apply(GridSettings settings, out GridSettings new_settings) =>
settings.add_missing_columns(TimeAndSalesSettings.DefaultGridSettings.columns, out new_settings);
}
}
}
@@ -73,7 +73,7 @@ namespace mbts.spark {
_revert_cmd_tooltip = backing(nameof(revert_cmd_tooltip), get_revert_tooltip);
//
_dispose = new CompositeDisposable(
_model.subscribe_complete_algos().OnDispatcher().Subscribe(_definitions_update_buffer),
_model.subscribe_complete_algos(allow_undefined_child: true).OnDispatcher().Subscribe(_definitions_update_buffer),
_definitions_update_buffer.OnDispatcher()
.SubscribeEach(u => {
u.FilterIncremental(a => a.key.is_composite, out var added, out var changes, out var removed);
@@ -144,7 +144,7 @@ namespace mbts.spark {
string candidate = string.Empty;
if (a.HasValue) {
var algo_name = a.Value.name;
var words = algo_name.Split(delim);
var words = algo_name.Split(delim, StringSplitOptions.RemoveEmptyEntries);
if (words.Length == 1) {
var single_word = words[0];
if (single_word.Length > 1) {
@@ -153,7 +153,7 @@ namespace mbts.spark {
candidate = single_word;
}
} else {
candidate = words.Take(2).Select(w => w.Substring(0, 1)).list_string(string.Empty);
candidate = string.Join(string.Empty, words.Take(2).Select(w => w.Substring(0, 1)));
}
}
return candidate.ToUpper();
@@ -622,9 +622,7 @@ namespace bts.shell.pnl {
static class PNLDataExtensions {
public static string pnl_symbol(this SecurityImmutable security) {
if(security.HasIce)
return PNLICEHelper.get_symbol(security);
return security.Symbol;
return bts.api.ExchangeSpecificUitlities.get_pnl_symbol(security);
}
// while override_cpp could use double.NaN to indicate the lack of an override,
@@ -636,7 +634,7 @@ namespace bts.shell.pnl {
public static (double default_cpp, double? override_cpp) pnl_cpp_tuple(this PNLUnitImmutable pnl_unit, SecurityImmutable security) {
// TODO: actual impl
double calced_pnl = PNLICEHelper.get_cpp(pnl_unit,security);
double calced_pnl = bts.api.ExchangeSpecificUitlities.get_cpp(pnl_unit,security);
return (calced_pnl, pnl_unit.HasCppOverride ? pnl_unit.CppOverride.Value : (double?)null);
}
}
@@ -1,75 +0,0 @@
using bts.api;
using bts.api.orderman;
using bts.api.position;
using bts.utils;
using mbts;
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
namespace bts.shell.pnl {
static class PNLICEHelper{
public static bool is_spread(SecurityImmutable sec){
return sec.HasSpread && sec.Ice.StripName.Contains("/");
}
public static bool is_strip(SecurityImmutable sec){
return sec.HasSpread && !is_spread(sec);
}
public static bool is_ice_lots(SecurityImmutable sec){
return sec.Ice.MarketDescription.Contains("ICE Lots");
}
public static double get_future_cpp (SecurityImmutable sec){
if(sec.Symbol == "HHD")
return 1;
if(is_ice_lots(sec))
return 2500;
else{
var days_in_month = DateTime.DaysInMonth(Convert.ToInt32(sec.Future.ExpirationYear),
Convert.ToInt32(sec.Future.ExpirationMonth));
return days_in_month; // because the size is multiplied by 2500 already
}
}
public static double get_spread_cpp(SecurityImmutable sec){
return 0;
}
public static double get_strip_cpp(PNLUnitImmutable pnl,
SecurityImmutable sec){
double rv = 0;
foreach (var l in sec.Spread.Legs)
if(l.Security.HasFuture)
rv += get_cpp(pnl,l.Security);
return rv;
}
public static string get_symbol (SecurityImmutable sec){
var rv = String.Format("{0}{1}{2}",
sec.Symbol,
is_ice_lots(sec)? " ICE LOTS" : String.Empty,
is_spread(sec)? " Spread" : String.Empty);
return rv;
}
public static double get_cpp (PNLUnitImmutable pnl,
SecurityImmutable sec){
if (sec.HasGlbx)
return pnl.CashValue;
else if (sec.HasIce){
if(sec.HasFuture)
return get_future_cpp(sec);
else if(is_spread(sec))
return get_spread_cpp(sec);
else if(is_strip(sec))
return get_strip_cpp(pnl,sec);
}
return Double.NaN;
}
}}
@@ -118,15 +118,7 @@ namespace bts.shell.pnl {
).FilterWithRaw(filter_src, PNLRowData.key_fn, null)
.OnDispatcher()
.Subscribe(_update),
// Note: the symbol filter items don't really need to be an IObservable
// (the symbol items used to be extracted from the IObservable pnl_source).
// It is only an IObservable now due to legacy; the filter was designed to observe incremental updates.
// This is can be easily changed if need be.
Observable.Return(
TableUpdate<string, string>.mk_empty(sym => sym).with_added(
SecurityHelper.get_all_display_symbols()))
.OnDispatcher()
.Subscribe(symbol_filter_items_observer),
_symbols_source(pnl_source).OnDispatcher().Subscribe(symbol_filter_items_observer),
Disposable.Create(() => {
foreach (var f in _all_filters)
f.filter_changed -= _refresh_filter.execute;
+2 -2
View File
@@ -337,8 +337,8 @@ namespace mbts.spark {
public IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_algos(SecurityImmutable security, bool attachable_only = false) =>
(attachable_only ? _attachable_algo_provider : _algo_provider).subscribe_algos(security);
public IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_complete_algos(bool attachable_only = false) =>
(attachable_only ? _attachable_algo_provider : _algo_provider).subscribe_complete_algos();
public IObservable<TableUpdate<AlgoData, AlgoKey>> subscribe_complete_algos(bool attachable_only = false, bool allow_undefined_child = false) =>
(attachable_only ? _attachable_algo_provider : _algo_provider).subscribe_complete_algos(allow_undefined_child);
#endregion
#region Input
@@ -18,7 +18,6 @@ namespace bts.settings {
interface IPAndLGridSettingsMigrationRule : ISettingsMigrationRule<GridSettings> { }
interface IPNLSettingsMigrationRule : ISettingsMigrationRule<PNLSettings> { }
interface ISyntheticManagementMigrationRule : ISettingsMigrationRule<GridSettings> { }
interface ITimeAndSalesGridSettingsMigrationRule : ISettingsMigrationRule<GridSettings> { }
static class SettingsMigrationHelper {
public static bool apply<T>(T settings, IEnumerable<ISettingsMigrationRule<T>> migration_rules, out T new_settings) {
@@ -26,8 +26,7 @@ namespace mbts.persist {
_fills_propi = ImmutableProperty.create<UserSettings, GridSettings>(us => us.fills_grid, (us, gs) => us.with(provided_fills_gridsettings: gs)),
_mview_propi = ImmutableProperty.create<UserSettings, GridSettings>(us => us.market_view_grid, (us, gs) => us.with(provided_market_view_gridsettings: gs)),
_pandl_propi = ImmutableProperty.create<UserSettings, GridSettings>(us => us.pnl_grid, (us, gs) => us.with(provided_pnl_gridsettings: gs)),
_synth_propi = ImmutableProperty.create<UserSettings, GridSettings>(us => us.synthetic_management_grid, (us, gs) => us.with(provided_synthetic_management_gridsettings: gs)),
_time_and_sales_propi = ImmutableProperty.create<UserSettings, GridSettings>(us => us.time_and_sales_grid, (us, gs) => us.with(time_and_sales: gs));
_synth_propi = ImmutableProperty.create<UserSettings, GridSettings>(us => us.synthetic_management_grid, (us, gs) => us.with(provided_synthetic_management_gridsettings: gs));
static readonly IImmutableProperty<UserSettings, PNLSettings>
_pnl_propi = ImmutableProperty.create<UserSettings, PNLSettings>(us => us.pnl, (us, pnl) => us.with(pnl: pnl));
@@ -39,7 +38,6 @@ namespace mbts.persist {
readonly IEnumerable<IPAndLGridSettingsMigrationRule> _pandl_migration_rules;
readonly IEnumerable<IPNLSettingsMigrationRule> _pnl_migration_rules;
readonly IEnumerable<ISyntheticManagementMigrationRule> _synth_migration_rules;
readonly IEnumerable<ITimeAndSalesGridSettingsMigrationRule> _time_and_sales_migration_rules;
readonly IUserPersist _persist;
UserSettings _default_settings;
@@ -51,8 +49,7 @@ namespace mbts.persist {
[ImportMany]IEnumerable<IMarketViewGridSettingsMigrationRule> mview_migration_rules,
[ImportMany]IEnumerable<IPAndLGridSettingsMigrationRule> pandl_migration_rules,
[ImportMany]IEnumerable<IPNLSettingsMigrationRule> pnl_migration_rules,
[ImportMany]IEnumerable<ISyntheticManagementMigrationRule> synth_migration_rules,
[ImportMany]IEnumerable<ITimeAndSalesGridSettingsMigrationRule> time_and_sales_migration_rules) {
[ImportMany]IEnumerable<ISyntheticManagementMigrationRule> synth_migration_rules) {
//
_persist = persist;
_fills_migration_rules = fills_migration_rules ?? Enumerable.Empty<IFillsGridSettingsMigrationRule>();
@@ -60,7 +57,6 @@ namespace mbts.persist {
_pandl_migration_rules = pandl_migration_rules ?? Enumerable.Empty<IPAndLGridSettingsMigrationRule>();
_pnl_migration_rules = pnl_migration_rules ?? Enumerable.Empty<IPNLSettingsMigrationRule>();
_synth_migration_rules = synth_migration_rules ?? Enumerable.Empty<ISyntheticManagementMigrationRule>();
_time_and_sales_migration_rules = time_and_sales_migration_rules ?? Enumerable.Empty<ITimeAndSalesGridSettingsMigrationRule>();
init();
async void init() {
UserSettings s;
@@ -152,12 +148,6 @@ namespace mbts.persist {
_synth_migration_rules,
_synth_propi);
public IWindowSettingsManager<GridSettings> create_time_and_sales_settings_manager(GridSettings instance_settings) =>
_create_settings_manager(
instance_settings,
_time_and_sales_migration_rules,
_time_and_sales_propi);
public async Task save_default(UserSettings value) {
// notify observers before persisting so the changes are reflected immediately in the ui
lock (_default_settings_lock) {
@@ -402,10 +402,8 @@ namespace mbts.spark.filters {
public static IDataGridColumnPersistentFilter<TRow, Venue[]> exchange<TRow>(Func<TRow, Venue> get_row_exchange, string prop_name) =>
Create(EnumHelper.get_values_except(Venue.Unset).ToList(), get_row_exchange, prop_name);
// Symbol
public static IDataGridColumnPersistentFilter<TRow, string[]> symbol<TRow>(Func<TRow, string> get_row_symbol, string prop_name, bool use_display_symbols = true) {
var symbols = use_display_symbols ? SecurityHelper.get_all_display_symbols() : co2.UserSymbol.get_all().Select(u => u.symbol);
return Create(symbols.Distinct().Ordered().ToList(), get_row_symbol, prop_name);
}
public static IDataGridColumnPersistentFilter<TRow, string[]> symbol<TRow>(Func<TRow, string> get_row_symbol, string prop_name) =>
Create(co2.UserSymbol.get_all().Select(u => u.symbol).Distinct().ToList(), get_row_symbol, prop_name);
// Security Type
public static IDataGridColumnPersistentFilter<TRow, Security.ContractOneofCase[]> security_type<TRow>(Func<TRow, Security.ContractOneofCase> get_row_security_type, string prop_name) =>
Create(EnumHelper.get_values_except(
@@ -1,5 +1,4 @@
using bts.utils;
using Google.Protobuf;
using ProtoBuf;
using System;
@@ -10,7 +9,7 @@ namespace mbts.spark.filters {
}
[ProtoContract]
class DoubleStatus : INullableStructProto<double>, IDeepCloneable<DoubleStatus>, IEquatable<DoubleStatus> {
class DoubleStatus : INullableStructProto<double>, IEquatable<DoubleStatus> {
[ProtoMember(1)]
bool _active;
[ProtoMember(2)]
@@ -22,12 +21,6 @@ namespace mbts.spark.filters {
set { if (_active = value.HasValue && !double.IsNaN(value.Value)) _value = value.Value; }
}
public DoubleStatus Clone() =>
new DoubleStatus {
_active = _active,
_value = _value
};
public bool Equals(DoubleStatus other) =>
other != null &&
_active == other._active &&
@@ -39,25 +32,20 @@ namespace mbts.spark.filters {
}
[ProtoContract]
class DoubleMinMaxState : IMinMaxState<double>, IDeepCloneable<DoubleMinMaxState>, IEquatable<DoubleMinMaxState> {
class DoubleMinMaxState : IMinMaxState<double>, IEquatable<DoubleMinMaxState> {
[ProtoMember(1)]
readonly DoubleStatus _minimum;
[ProtoMember(2)]
readonly DoubleStatus _maximum;
public DoubleMinMaxState() : this(new DoubleStatus(), new DoubleStatus()) { }
DoubleMinMaxState(DoubleStatus minimum, DoubleStatus maximum) {
_minimum = minimum;
_maximum = maximum;
public DoubleMinMaxState() {
_minimum = new DoubleStatus();
_maximum = new DoubleStatus();
}
public INullableStructProto<double> minimum => _minimum;
public INullableStructProto<double> maximum => _maximum;
public DoubleMinMaxState Clone() =>
new DoubleMinMaxState(_minimum?.Clone(), _maximum?.Clone());
public bool Equals(DoubleMinMaxState other) =>
other != null &&
_minimum.safe_equals(other._minimum) &&
@@ -1,5 +1,4 @@
using bts.utils;
using Google.Protobuf;
using ProtoBuf;
using System;
@@ -10,7 +9,7 @@ namespace mbts.spark.filters {
}
[ProtoContract]
class UIntStatus : INullableStructProto<uint>, IDeepCloneable<UIntStatus>, IEquatable<UIntStatus> {
class UIntStatus : INullableStructProto<uint>, IEquatable<UIntStatus> {
[ProtoMember(1)]
bool _active;
[ProtoMember(2)]
@@ -22,12 +21,6 @@ namespace mbts.spark.filters {
set { if (_active = value.HasValue) _value = value.Value; }
}
public UIntStatus Clone() =>
new UIntStatus {
_active = _active,
_value = _value
};
public bool Equals(UIntStatus other) =>
other != null &&
_active == other._active &&
@@ -39,25 +32,20 @@ namespace mbts.spark.filters {
}
[ProtoContract]
class UIntMinMaxState : IMinMaxState<uint>, IDeepCloneable<UIntMinMaxState>, IEquatable<UIntMinMaxState> {
class UIntMinMaxState : IMinMaxState<uint>, IEquatable<UIntMinMaxState> {
[ProtoMember(1)]
readonly UIntStatus _minimum;
[ProtoMember(2)]
readonly UIntStatus _maximum;
public UIntMinMaxState() : this(new UIntStatus(), new UIntStatus()) { }
UIntMinMaxState(UIntStatus minimum, UIntStatus maximum) {
_minimum = minimum;
_maximum = maximum;
public UIntMinMaxState() {
_minimum = new UIntStatus();
_maximum = new UIntStatus();
}
public INullableStructProto<uint> minimum => _minimum;
public INullableStructProto<uint> maximum => _maximum;
public UIntMinMaxState Clone() =>
new UIntMinMaxState(_minimum?.Clone(), _maximum?.Clone());
public bool Equals(UIntMinMaxState other) =>
other != null &&
_minimum.safe_equals(other._minimum) &&
@@ -1,12 +1,11 @@
using Google.Protobuf;
using mbts.ui.controls;
using mbts.ui.controls;
using ProtoBuf;
using System;
using System.Globalization;
namespace mbts.ui {
[ProtoContract]
public struct TimeFilterTuple : IDeepCloneable<TimeFilterTuple>, IEquatable<TimeFilterTuple> {
public struct TimeFilterTuple : IEquatable<TimeFilterTuple> {
[ProtoMember(1)]
readonly TimeFilterMode _mode;
[ProtoMember(2)]
@@ -28,8 +27,6 @@ namespace mbts.ui {
_mode == TimeFilterMode.CMEOpen ||
(_mode == TimeFilterMode.Period && (_begin.result.HasValue || _end.result.HasValue));
public TimeFilterTuple Clone() =>
new TimeFilterTuple(_mode, _begin.Clone(), _end.Clone());
public bool Equals(TimeFilterTuple other) =>
other._mode == _mode && other._begin.Equals(_begin) && other._end.Equals(_end);
@@ -45,7 +42,7 @@ namespace mbts.ui {
}
[ProtoContract]
public struct TimeTextTuple : IDeepCloneable<TimeTextTuple>, IEquatable<TimeTextTuple> {
public struct TimeTextTuple : IEquatable<TimeTextTuple> {
[ProtoMember(1)]
readonly string _month_text;
[ProtoMember(2)]
@@ -89,9 +86,6 @@ namespace mbts.ui {
}
}
public TimeTextTuple Clone() =>
new TimeTextTuple(_month_text, _day_text, _year_text, _time_text);
public bool Equals(TimeTextTuple other) =>
other._month_text == month_text &&
other._day_text == _day_text &&
@@ -1,6 +1,5 @@
using bts.api;
using bts.utils;
using Google.Protobuf;
using ProtoBuf;
using System;
using System.Collections.Generic;
@@ -9,7 +8,7 @@ using System.Reactive.Linq;
namespace mbts.spark {
[ProtoContract]
class SecurityCriteria : IDeepCloneable<SecurityCriteria>, IEquatable<SecurityCriteria> {
class SecurityCriteria : IEquatable<SecurityCriteria> {
public SecurityCriteria()
{
clauses = new SecurityCriteriaClause[] { };
@@ -41,16 +40,13 @@ namespace mbts.spark {
internal static IObservable<IEnumerable<SecurityImmutable>> to_securities(SecurityCriteria security_criteria, IBus bus) =>
security_criteria == null ? Observable.Empty<IEnumerable<SecurityImmutable>>() : security_criteria.subscribe(bus);
public SecurityCriteria Clone() =>
new SecurityCriteria(ArrayHelper.convert(clauses, c => c.Clone()));
public bool Equals(SecurityCriteria other) => EqualityComparer.safe_sequence_equals(clauses, other?.clauses);
public override bool Equals(object obj) => Equals(obj as SecurityCriteria);
public override int GetHashCode() => EqualityComparer.safe_aggregate_hash(clauses);
}
[ProtoContract]
readonly struct SecurityCriteriaClause : IDeepCloneable<SecurityCriteriaClause>, IEquatable<SecurityCriteriaClause> {
readonly struct SecurityCriteriaClause : IEquatable<SecurityCriteriaClause> {
public SecurityCriteriaClause(
Venue[] exchanges = null,
SecurityType[] security_types = null,
@@ -90,14 +86,6 @@ namespace mbts.spark {
!symbols.is_null_or_empty() ||
!security_keys.is_null_or_empty();
public SecurityCriteriaClause Clone() =>
new SecurityCriteriaClause(
ArrayHelper.safe_clone(exchanges),
ArrayHelper.safe_clone(security_types),
ArrayHelper.safe_clone(symbols),
ArrayHelper.safe_clone(security_keys),
pretty_name == null ? null : string.Copy(pretty_name));
public override string ToString()
{
return pretty_name;
@@ -1,30 +1,26 @@
using bts.view;
using Google.Protobuf;
using ProtoBuf;
using System;
using System.Windows.Controls;
namespace bts.shell {
[ProtoContract]
struct SparkMenuDisplaySettings : IDeepCloneable<SparkMenuDisplaySettings>, IEquatable<SparkMenuDisplaySettings> {
[ProtoMember(1)]
public readonly SparkMenuDisplayState display_state;
[ProtoMember(2)]
public readonly Dock dock;
public SparkMenuDisplaySettings(SparkMenuDisplayState display_state, Dock dock) {
this.display_state = display_state;
this.dock = dock;
}
public static SparkMenuDisplaySettings Default =>
new SparkMenuDisplaySettings(SparkMenuDisplayState.Collapsed, Dock.Left);
public SparkMenuDisplaySettings Clone() =>
new SparkMenuDisplaySettings(display_state, dock);
public bool Equals(SparkMenuDisplaySettings other) =>
display_state == other.display_state &&
dock == other.dock;
}
}
using bts.view;
using ProtoBuf;
using System;
using System.Windows.Controls;
namespace bts.shell {
[ProtoContract]
struct SparkMenuDisplaySettings : IEquatable<SparkMenuDisplaySettings> {
[ProtoMember(1)]
public readonly SparkMenuDisplayState display_state;
[ProtoMember(2)]
public readonly Dock dock;
public SparkMenuDisplaySettings(SparkMenuDisplayState display_state, Dock dock) {
this.display_state = display_state;
this.dock = dock;
}
public static SparkMenuDisplaySettings Default =>
new SparkMenuDisplaySettings(SparkMenuDisplayState.Collapsed, Dock.Left);
public bool Equals(SparkMenuDisplaySettings other) =>
display_state == other.display_state &&
dock == other.dock;
}
}
@@ -1,5 +1,4 @@
using bts.utils;
using Google.Protobuf;
using ProtoBuf;
using System;
using System.ComponentModel;
@@ -7,7 +6,7 @@ using System.ComponentModel;
namespace mbts.ui {
[Serializable]
[ProtoContract]
class SortingCriteria : IDeepCloneable<SortingCriteria>, IEquatable<SortingCriteria> {
class SortingCriteria : IEquatable<SortingCriteria> {
public SortingCriteria() { }
public SortingCriteria(SortDescription sort_description) : this(sort_description.PropertyName, sort_description.Direction == ListSortDirection.Ascending) { }
public SortingCriteria(string col_name, bool is_ascending) {
@@ -20,9 +19,6 @@ namespace mbts.ui {
[ProtoMember(2)]
public bool is_ascending;
public SortingCriteria Clone() =>
new SortingCriteria(col_name?.Copy(), is_ascending);
public bool Equals(SortingCriteria other) =>
other != null &&
col_name == other.col_name &&
@@ -259,6 +259,7 @@ namespace mbts.co2 {
}
}
// If Size is wrong remember to update from
[StructLayout(LayoutKind.Sequential, Size = 704)]
internal struct ColdEntry {
public int exchange_status;
@@ -0,0 +1,128 @@
using bts.api;
using bts.api.orderman;
using bts.api.position;
using bts.utils;
using mbts;
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
namespace bts.api {
static class ExchangeSpecificUitlities{
private static string _default_hub = "Henry";
static class ICE{
#region Contract Type
public static bool is_spread(SecurityImmutable sec){
return sec.HasSpread && sec.Ice.StripName.Contains("/");
}
public static bool is_strip(SecurityImmutable sec){
return sec.HasSpread && !is_spread(sec);
}
public static bool is_ice_lots(SecurityImmutable sec){
return sec.Ice.MarketDescription.Contains("ICE Lots");
}
public static bool is_tas(SecurityImmutable sec){
return sec.Ice.MarketDescription.Contains("TAS");
}
#endregion
#region CPP
public static double get_future_cpp (SecurityImmutable sec){
if(sec.Symbol == "HHD")
// only futures in HHD are the dailies, everything
// else is a strip or spread
return 1;
if(is_ice_lots(sec))
return 2500;
else{
var days_in_month = DateTime.DaysInMonth(Convert.ToInt32(sec.Future.ExpirationYear),
Convert.ToInt32(sec.Future.ExpirationMonth));
return days_in_month; // because the size is multiplied by 2500 already
}
}
public static double get_spread_cpp(SecurityImmutable sec){
return 0;
}
public static double get_strip_cpp(PNLUnitImmutable pnl,
SecurityImmutable sec){
// add up all the legs
double rv = 0;
foreach (var l in sec.Spread.Legs)
if(l.Security.HasFuture)
rv += get_cpp(pnl,l.Security);
return rv;
}
public static double get_cpp (PNLUnitImmutable pnl,
SecurityImmutable sec){
if(sec.HasFuture)
return get_future_cpp(sec);
else if(is_spread(sec))
return get_spread_cpp(sec);
else if(is_strip(sec))
return get_strip_cpp(pnl,sec);
return Double.NaN;
}
#endregion
#region Symbol
public static string get_pnl_symbol (SecurityImmutable sec){
// for pnl purposes, for a symbol, the division is :
// Futures and Strips
// ICE Lots
// Spreads
var rv = String.Format("{0}{1}{2}",
sec.Symbol,
is_ice_lots(sec)? " ICE LOTS" : String.Empty,
is_spread(sec)? " Spread" : String.Empty);
return rv;
}
#endregion
#region Formatting
public static string format_security (SecurityImmutable? security){
if(security.HasValue && security.Value.HasIce){
bool ice_lots = is_ice_lots(security.Value),
tas = is_tas(security.Value);
return String.Format("{0}{1}{2}{3}{4}",
security?.Symbol,
ice_lots? " ICE Lots" : String.Empty,
tas ? " TAS" : String.Empty,
String.Compare(security.Value.Ice.HubAlias, _default_hub,true)==0 ? String.Empty :
" " + security.Value.Ice.HubAlias,
" " + security?.Ice.StripName);
}
return String.Empty;
}
#endregion
}
public static double get_cpp (PNLUnitImmutable pnl,
SecurityImmutable sec){
if(sec.HasIce)
return ICE.get_cpp(pnl,sec);
return pnl.CashValue;
}
public static string get_pnl_symbol (SecurityImmutable sec){
if(sec.HasIce)
return ICE.get_pnl_symbol(sec);
return sec.Symbol;
}
public static string format_security(SecurityImmutable? security){
if(security?.HasIce == true)
return ICE.format_security(security);
return security?.ToString();
}
}}
@@ -11,8 +11,5 @@ namespace bts.api {
default: return SecurityType.Unset;
}
}
public static string display_symbol(this SecurityImmutable security) =>
SecurityHelper.get_display_symbol(security);
}
}
@@ -41,7 +41,7 @@ namespace bts.api {
switch (security?.ContractCase) {
case ContractOneofCase.Future:
if(security?.HasIce == true)
return _format_ICE(security);
return bts.api.ExchangeSpecificUitlities.format_security(security);
return format_future(security?.Symbol, security?.security_desc(), security.Value.Future);
case ContractOneofCase.Spread:
if (security?.HasGlbx == true) {
@@ -50,7 +50,7 @@ namespace bts.api {
return security?.Glbx.SecurityDesc;
}
else if(security?.HasIce == true)
return _format_ICE(security);
return bts.api.ExchangeSpecificUitlities.format_security(security);
return string.Join("", _format_future_legs(SecurityHelper.get_flat_future_legs(security?.Spread.Legs)));
case ContractOneofCase.Synthetic: return security?.Synthetic.Name;
default: return "Unknown-security";
@@ -84,28 +84,5 @@ namespace bts.api {
default: return $"{fl.quantity:+0;-0}{sym_exp_name}";
}
}
static string _format_ICE (SecurityImmutable? security){
string symbol = security?.Symbol,
strip = security?.Ice.StripName,
md_desc = security?.Ice.MarketDescription,
type_append = "";
if(md_desc.Contains("25k ICE Lots")){
if(md_desc.Contains("TAS"))
type_append = "TAS";
}
else if(md_desc.Contains("ICE Lots")){
if(md_desc.Contains("TAS"))
type_append = "ICE Lots TAS";
else
type_append = "ICE Lots";
}
else if(md_desc.Contains("TAS"))
type_append = "TAS";
type_append = string.IsNullOrEmpty(type_append) ?
" " : $" {type_append} ";
return $"{symbol}{type_append}{strip}";
}
}
}
+31 -34
View File
@@ -1,24 +1,10 @@
using mbts.co2;
using System;
using System;
using System.Collections.Generic;
using System.Linq;
using static bts.api.Security;
namespace bts.api {
internal static class SecurityHelper {
public static IEnumerable<string> get_all_display_symbols() =>
get_all_display_symbols(UserSymbol.get_all().Select(usym => usym.symbol));
public static IEnumerable<string> get_all_display_symbols(IEnumerable<string> system_symbols) {
// todo: actual impl
return system_symbols;
}
public static string get_display_symbol(SecurityImmutable security) {
// todo: actual impl
return security.Symbol;
}
public static IEnumerable<FutureLeg> get_flat_future_legs(SecurityImmutable sec) {
switch (sec.ContractCase) {
case ContractOneofCase.Spread: return _get_flat_future_legs(sec.Spread.Legs, 1);
@@ -97,26 +83,37 @@ namespace bts.api {
}
public static int Compare(SecurityImmutable x, SecurityImmutable y) {
if (x.ContractCase == ContractOneofCase.Future) {
return y.ContractCase == ContractOneofCase.Future ? Compare(x.Symbol, x.Future, y.Symbol, y.Future) : -1;
} else if (y.ContractCase == ContractOneofCase.Future) {
return 1;
int c = non_deterministic_impl();
if (c == 0) {
c = x.ToString().CompareTo(y.ToString());
if (c == 0)
c = x.Id.CompareTo(y.Id);
}
var xlegs = get_flat_future_legs(x).ToList();
var ylegs = get_flat_future_legs(y).ToList();
int max = Math.Min(xlegs.Count, ylegs.Count);
for (int i = 0; i < max; i++) {
int rv = Compare(xlegs[i].symbol, xlegs[i].future, ylegs[i].symbol, ylegs[i].future);
if (rv != 0)
return rv;
rv = xlegs[i].quantity.CompareTo(ylegs[i].quantity);
if (rv != 0)
return rv;
rv = xlegs[i].exchange.CompareTo(ylegs[i].exchange);
if (rv != 0)
return rv;
return c;
// this has been observed to be non-determinisitc with certain ICE securities
// (it is determenistic with other exchanges..as far as we know).
int non_deterministic_impl() {
if (x.ContractCase == ContractOneofCase.Future) {
return y.ContractCase == ContractOneofCase.Future ? Compare(x.Symbol, x.Future, y.Symbol, y.Future) : -1;
} else if (y.ContractCase == ContractOneofCase.Future) {
return 1;
}
var xlegs = get_flat_future_legs(x).ToList();
var ylegs = get_flat_future_legs(y).ToList();
int max = Math.Min(xlegs.Count, ylegs.Count);
for (int i = 0; i < max; i++) {
int rv = Compare(xlegs[i].symbol, xlegs[i].future, ylegs[i].symbol, ylegs[i].future);
if (rv != 0)
return rv;
rv = xlegs[i].quantity.CompareTo(ylegs[i].quantity);
if (rv != 0)
return rv;
rv = xlegs[i].exchange.CompareTo(ylegs[i].exchange);
if (rv != 0)
return rv;
}
return xlegs.Count.CompareTo(ylegs.Count);
}
return xlegs.Count.CompareTo(ylegs.Count);
}
public static int Compare(string x_sym, FutureContract x, string y_sym, FutureContract y) {
@@ -130,4 +127,4 @@ namespace bts.api {
}
}
}
}
}
+1
View File
@@ -36,6 +36,7 @@ namespace bts {
Plugin,
Sdk,
Http,
iLink3
// Please keep this list in sync with logging.h/logging.cpp
}
@@ -7,6 +7,22 @@ using System.Reactive.Linq;
namespace System.Collections.Generic {
public static class EnumerableExtensions {
/// <summary>
/// Determines whether all elements of a sequence are true.
/// </summary>
/// <param name="source">An <see cref="IEnumerable{bool}"/> that contains the elements to which to apply the predicate.</param>
/// <returns>True if every element of the source sequence is true, or if the sequence is empty; otherwise false.</returns>
/// <exception cref="ArgumentNullException"><paramref name="source"/> is null</exception>
public static bool All(this IEnumerable<bool> source) {
if (source is null)
throw new ArgumentNullException(nameof(source));
foreach (var v in source) {
if (!v)
return false;
}
return true;
}
/// <summary>
/// Similar to <see cref="Enumerable.All{TSource}(IEnumerable{TSource}, Func{TSource, bool})"/>, but returns false if the <paramref name="source"/> is empty.
/// </summary>
+4 -12
View File
@@ -1,6 +1,4 @@
using bts.api.test;
using Google.Protobuf;
using mbts.utils;
using mbts.utils;
using System;
using System.Collections;
using System.Collections.Generic;
@@ -32,6 +30,9 @@ namespace bts.utils {
}
}
public static TResult[] safe_convert<T, TResult>(T[] array, Func<T, TResult> convert) =>
array == null ? Array.Empty<TResult>() : Array.ConvertAll(array, new Converter<T, TResult>(convert));
public static TResult[] convert<T, TResult>(ICollection<T> source, Func<T, TResult> convert) {
var array = new TResult[source.Count];
int i = 0;
@@ -40,21 +41,12 @@ namespace bts.utils {
return array;
}
public static T[] safe_deep_clone<T>(T[] array) where T : IDeepCloneable<T> =>
array == null ? null : Array.ConvertAll(array, v => v.Clone());
public static TResult[] safe_convert<T, TResult>(T[] array, Func<T, TResult> convert) =>
array == null ? Array.Empty<TResult>() : Array.ConvertAll(array, new Converter<T, TResult>(convert));
public static T[] clone<T>(T[] array) {
T[] rv = new T[array.Length];
Array.Copy(array, rv, array.Length);
return rv;
}
public static T[] safe_clone<T>(T[] array) =>
array == null ? Array.Empty<T>() : clone(array);
/// <summary>
/// Attemps to get an exact item count (to initialize the resultant array with an exact length, thus avoiding unneccessary allocations) by checking if the specified source implements <see cref="ICollection{T}"/>, <see cref="IReadOnlyCollection{T}"/>, or <see cref="ICollection"/>.
/// </summary>
@@ -16,8 +16,5 @@ namespace bts.utils {
public static bool SafeContains(this string str, string value) =>
str != null && str.Contains(value);
public static string Copy(this string str) =>
string.Copy(str);
}
}
+24 -13
View File
@@ -1,4 +1,5 @@
using bts.api;
using bts.collections;
using bts.resources;
using bts.test;
using bts.utils.proto;
@@ -10,7 +11,6 @@ namespace bts.mcore {
class SecurityTests {
[Test]
public void test_security_equal() {
SecurityImmutable a = new SecurityImmutable(new Security() { Id = 42 });
SecurityImmutable b = new SecurityImmutable(new Security() { Id = 42 });
SecurityImmutable c = new SecurityImmutable(new Security() { Id = 4 });
@@ -29,20 +29,31 @@ namespace bts.mcore {
[Test]
public void test_security_compare() {
const string securities_resource = "KrandallOrderBookCrash.security_proto";
// verify that the sample size doesn't shrink for some reason.
// if it grows a bit that's fine; the generic comparison test
// asserts against collections that are too large.
const int min_sample_size = 91;
foreach (var file_path in SecurityFiles.all())
_test_security_compare(_read_securities(file_path));
}
var securities = StreamHelper.ReadProtos<Security>(
ResourceHelper.LoadResource(securities_resource))
.Select(s => new SecurityImmutable(s))
.ToList();
Assert.IsTrue(securities.Count >= min_sample_size);
void _test_security_compare(IEnumerable<SecurityImmutable> securities) {
const int max_count = 2048;
var sec_list = securities as IList<SecurityImmutable> ?? securities.ToList();
if (sec_list.Count > max_count)
sec_list = sec_list.WrapSubRange(0, max_count);
ComparisonTests.transitive_comparison(sec_list, SecurityHelper.Comparer.InstanceImmutable);
ComparisonTests.identity_comparison(sec_list, SecurityHelper.Comparer.InstanceImmutable);
}
ComparisonTests.transitive_comparison(securities, SecurityHelper.Comparer.InstanceImmutable);
ComparisonTests.identity_comparison(securities, SecurityHelper.Comparer.InstanceImmutable);
IEnumerable<SecurityImmutable> _read_securities(string file_path) =>
StreamHelper.ReadProtos<Security>(ResourceHelper.LoadResource(file_path)).Select(s => new SecurityImmutable(s));
static class SecurityFiles {
public const string
KrandallOrderBookCrash = "KrandallOrderBookCrash.security_proto",
NonDeterministicICEFutures = "NonDeterministicICEFutures.security_proto";
public static IEnumerable<string> all() {
yield return KrandallOrderBookCrash;
yield return NonDeterministicICEFutures;
}
}
}
}
@@ -4,14 +4,14 @@ using System.Linq;
namespace bts.mcore {
static class ComparisonTests {
public static void transitive_comparison<T>(IEnumerable<T> items, IComparer<T> comparer) {
// Verify that the sample size isn't too big.
// This test compares each item to every other item
// (technically twice; lhs/rhs is flipped the second time),
// so performance is O(n^2).
const int max_sample_size = 4092;
/// <summary>
/// Compares every permutation of pairs in the specified items.
/// Be wary of use with large collections, as performance is O(n^2).
/// </summary>
public static void transitive_comparison<T>(IEnumerable<T> items, IComparer<T> comparer, int min_item_count = 2, int max_item_count = 5000) {
var sorted_items = items.ToList();
Assert.That(sorted_items.Count <= max_sample_size);
Assert.That(sorted_items.Count >= min_item_count);
Assert.That(sorted_items.Count <= max_item_count);
sorted_items.Sort(comparer);
for (int i = 0; i < sorted_items.Count - 1; i++) {
for (int j = i + 1; j < sorted_items.Count; j++) {
+1
View File
@@ -16,5 +16,6 @@
</ItemGroup>
<ItemGroup>
<EmbeddedResource Include="Resources\KrandallOrderBookCrash.security_proto" />
<EmbeddedResource Include="Resources\NonDeterministicICEFutures.security_proto" />
</ItemGroup>
</Project>
+1 -1
View File
@@ -1 +1 @@
set(sources files/cert_secure_logon.txt;files/futures.coconut.xz;files/onixs_ice/ConnectivityConfiguration.APITest.xml;files/onixs_ice/ConnectivityConfiguration.PerfTest.xml;files/onixs_ice/ConnectivityConfiguration.Production.xml;files/perf_trace/compiled.css;files/perf_trace/index.html;files/perf_trace/jquery-3.3.1.min.js;files/perf_trace/main.js;files/perf_trace/perf_log.sql;files/perf_trace/perf_trace_analysis.sql;files/perf_trace/plotly-latest.min.js)
set(sources files/audit_ilink3.py;files/audit_upload.sh;files/audit_uploader.py;files/cert_secure_logon.txt;files/example.conf;files/futures.coconut.xz;files/onixs_ice/ConnectivityConfiguration.APITest.xml;files/onixs_ice/ConnectivityConfiguration.APITest.xml.orig;files/onixs_ice/ConnectivityConfiguration.PerfTest.xml;files/onixs_ice/ConnectivityConfiguration.Production.xml;files/perf_trace/compiled.css;files/perf_trace/index.html;files/perf_trace/jquery-3.3.1.min.js;files/perf_trace/main.js;files/perf_trace/perf_log.sql;files/perf_trace/perf_trace_analysis.sql;files/perf_trace/plotly-latest.min.js)
+405
View File
@@ -0,0 +1,405 @@
import json
import csv
import datetime, time
import argparse, os
def valid_file(f):
if not os.path.exists(f):
# Argparse uses the ArgumentTypeError to give a rejection message like:
# error: argument input: x does not exist
raise argparse.ArgumentTypeError("{0} does not exist".format(f))
return f
def valid_date(s):
try:
return datetime.datetime.strptime(s, "%Y%m%d")
except ValueError:
raise argparse.ArgumentTypeError("Not a valid date: '{0}'.".format(s))
def print_headers():
# From https://www.cmegroup.com/confluence/display/EPICSANDBOX/iLink+3+-+Minimum+Acceptable+Audit+Trail+Elements+-+Data+Definitions+and+Validation+Rules
line_to_write = "Sending Timestamps,Message Direction,Operator ID,Session ID,Manual Order Indicator,Message Type,Instrument,Order Flow ID,\
CME Globex Order ID,Client Order ID,Order Request ID,Buy/Sell Indicator,Quantity,Limit Price,Stop Price,Order Type,Order Qualifier,IFM Flag,\
Display Quantity,Minimum Quantity,Country of Origin,Fill Price,Fill Quantity,Cumulative Quantity,Remaining Quantity,Aggressor Flag,Source of Cancellation,\
Reject Reason,CME Globex Message ID,Cross ID,Quote Request ID,Message Quote ID,Quote Set ID,Quote Entry ID,Bid Price,Bid Size,Offer Price,Offer Size,\
Processed Entries,Party Details List Request ID,ListUpdateAction,Self-Matching Prevention ID,Self-Match Prevention Instr,Customer Type Indicator,\
AvgPxGroupID,Average Px Ind,CmtaGiveUpCD,Origin,Clearing Trade Price Type,Customer Handling Instr,Executing Firm ID,Account Number,Take Up Firm,Take up Account"
with open (args.output_file, 'w') as the_file:
the_file.write(line_to_write)
the_file.write("\n")
def set_blanks():
return ""
def get_value(json_key):
prices = ["Price", "LastPx", "StopPx"]
if json_key in json_data:
if json_key in prices:
x = json_data[json_key]
return x['mantissa']/100000000000
else:
return json_data[json_key]
else:
return ""
# Column 1 - Sending Timestamps
def SendingTimeEpoch(epochtime): #1
#TIME = time.strftime('%Y%m%d-%H:%M:%S.{}'.format(epochtime/1000000000), time.gmtime(epochtime/1000000000))
TIME = datetime.datetime.fromtimestamp(int(epochtime) / 1e9).strftime('%Y%m%d-%H:%M:%S.%f')
return TIME
# Column 2 - Message Direction
def direction(whichway):
if whichway == '<':
DIRECTION = "FROM CME"
else:
DIRECTION = "TO CME"
return DIRECTION
# Column 3 - Operator ID
def SenderID(sender):
if "SenderID" in json_data:
SENDER = json_data["SenderID"]
return SENDER
else:
return SESSIONID
# Column 4 - Session ID
# Column 5 - Manual Order Indicator
def ManualOrderIndicator(manual):
if "ManualOrderIndicator" in json_data:
if manual == "Manual":
return "1"
else:
return "0"
# Column 6 - Message Type
def MessageType(msg_type,orderstatus):
if msg_type == "NewOrderSingle514":
return "D"
elif msg_type == "ExecutionReportNew522":
return "8--0"
if "ExecutionReportTrade" in msg_type:
if orderstatus == "PartiallyFilled":
return "8--1"
else:
return "8--2"
elif msg_type == "ExecutionReportCancel534":
return "8--4"
elif msg_type == "ExecutionReportModify531":
return "8--5"
elif msg_type == "ExecutionReportCancel534":
return "8--H"
elif msg_type == "ExecutionReportReject523":
return "8--8"
elif msg_type == "OrderCancelReplaceReject536":
return "9--2"
elif msg_type == "OrderCancelReject535":
return "9--1"
elif msg_type == "PartyDetailsDefinitionRequest518":
return "CX"
elif msg_type == "PartyDetailsDefinitionRequestAck519":
return "CY"
elif msg_type == "OrderCancelRequest516":
return "F"
elif msg_type == "OrderCancelReplaceRequest515":
return "G"
# Column 12 - Buy-Sell Indicator/Side
def Side(indicator):
if "Side" in json_data:
if indicator == "Buy":
return "1"
else:
return "2"
# Column 16 - Order Type/OrdType
def OrdType(ordertype):
if "OrdType" in json_data:
if ordertype == "MarketWithProtection" or ordertype == "MarketwithProtection":
return "1"
elif ordertype == "Limit":
return "2"
elif ordertype == "StopWithProtection" or ordertype == "StopwithProtection":
return "3"
elif ordertype == "StopLimit":
return "4"
elif ordertype == "MarketLimit":
return "K"
else:
return ""
# Column 17 - Order Qualifier/TimeInForce
def TimeInForce(orderqual):
if "TimeInForce" in json_data:
if orderqual == "Day":
return "0"
elif orderqual == "GTC":
return "1"
elif orderqual == "FAK":
return "3"
elif orderqual == "GTD":
return "6"
else:
return ""
# Column 18 IFM Flag/OFMOverride
def OFMOverride(imfflag):
if "OFMOverride" in json_data:
if imfflag == "Enabled":
return "1"
else:
return "0"
# Column 19 Display Quantity/DisplayQty
def DisplayQty(qty):
if "DisplayQty" in json_data:
if str(qty) == "0":
return ""
elif str(qty) == "4294967295":
return ""
else:
return qty
# Column 26 - Aggressor Flag/AggressorIndicator
def AggressorIndicator(aggressor):
if "AggressorIndicator" in json_data:
if aggressor == "True":
return "1"
else:
return "0"
# Column 27 - Source of Cancellation/ExecRestatementReason
def ExecRestatementReason(sourcecancel):
if "ExecRestatementReason" in json_data:
if sourcecancel == "Exchange":
return "8"
elif sourcecancel == "CancelledNotBest":
return "9"
elif sourcecancel == "CancelOnDisconnect":
return "100"
elif sourcecancel == "CancelRestingDueToSelfMatchPrevention":
return "103"
elif sourcecancel == "CancelFromCMEGlobexCreditControlsViolation":
return "104"
elif sourcecancel == "CancelFromCMEOne":
return "105"
elif sourcecancel == "CancelFromRiskManagementAPI":
return "106"
elif sourcecancel == "CancelAggressingDueToSelfMatchPrevention":
return "107"
elif sourcecancel == "CancelDueToVolatilityQuoteOptionLessThanMinOrderSize":
return "108"
else:
return ""
# Column 43 - Self-Match Prevention Instr/SelfMatchPreventionInstruction
def SelfMatchPreventionInstruction(instr):
if "SelfMatchPreventionInstruction" in json_data:
if instr == "CancelOldest":
return "O"
elif instr == "CancelNewest":
return "N"
else:
return ""
# Column 44 - Customer Type Indicator/CustOrderCapacity
def CustOrderCapacity(custord):
if "CustOrderCapacity" in json_data:
if custord == "NULL_VALUE":
return ""
elif custord == "Membertradingfortheirownaccount":
return "1"
elif custord == "ClearingFirmtradingforitsproprietaryaccount":
return "2"
elif custord == "Membertradingforanothermember":
return "3"
else:
return "4"
# Column 46 - Average Px Ind/AvgPxInd
def AvgPxInd(avgpxindicator):
if "AvgPxInd" in json_data:
if avgpxindicator == "NULL_VALUE":
return ""
else:
return avgpxindicator
# Column 48 - Origin/ClearingAccountType
def ClearingAccountType(accounttype):
if "ClearingAccountType" in json_data:
if accounttype == "Customer":
return "0"
elif accounttype == "Firm":
return "1"
else:
return ""
# Column 49 - Clearing Trade Price Type/ClearingTradePriceType
def ClearingTradePriceType(clrtrdprice):
if "ClearingTradePriceType" in json_data:
if clrtrdprice == "NULL_VALUE":
return "0"
else:
return "3"
# Column 50 - Customer Handling Instr/CustOrderHandlingInst
def CustOrderHandlingInst(custorderinst):
if "CustOrderHandlingInst" in json_data:
if custorderinst == "ClientElectronic":
return "C"
elif custorderinst == "TEST":
return "D"
else:
return ""
# Column 51 and 52 - Getting the Firm and Account Number
def partyid(partydetailid,column):
for x in partydetailid:
role = x['PartyDetailRole']
id = x['PartyDetailID']
if column == "51" and role == "ExecutingFirm":
return id
if column == "52" and role == "CustomerAccount":
return id
parser = argparse.ArgumentParser()
parser.add_argument("json_log", help="the json log[.gz] you wish to process", type=valid_file)
parser.add_argument("output_file", help="the destination csv file you wish to write")
parser.add_argument("-d", "--date", help="the date you wish to parse from the log, specified as YYYYmmdd",
default=datetime.datetime.today().strftime("%Y%m%d"), type=valid_date)
args = parser.parse_args()
print_headers()
list_types = ['NewOrderSingle514', 'OrderCancelReplaceRequest515', 'OrderCancelRequest516', 'PartyDetailsDefinitionRequest518', 'PartyDetailsDefinitionRequestAck519', \
'ExecutionReportNew522', 'ExecutionReportReject523', 'ExecutionReportTradeOutright525', 'ExecutionReportTradeSpread526', 'ExecutionReportTradeSpreadLeg527', \
'ExecutionReportModify531', 'ExecutionReportCancel534', 'OrderCancelReject535', 'OrderCancelReplaceReject536', ]
# Note that the timedeltas specificed in trade_day_start and trade_day_end
# are based on Chicago times; as such, this script should run on a machine
# set to a Chicago timezone. A future impromovement would be to determine
# the current machine's timezone and determine the appropriate offset
# versus Chicago time, but for now we assume this will run on a Chicago
# tz machine.
input_as_datetime = args.date
trade_day_start = input_as_datetime - datetime.timedelta(days=1) + datetime.timedelta(hours=16, minutes=30)
trade_day_utc_start = time.gmtime((time.mktime(trade_day_start.timetuple())))
trade_day_end = input_as_datetime + datetime.timedelta(hours=16)
trade_day_utc_end = time.gmtime((time.mktime(trade_day_end.timetuple())))
file_opener = None
if args.json_log.endswith(".gz"):
import gzip
file_opener = gzip.open
else:
file_opener = open
#inputFile = open(fileInput) #open json file
#outputFile = open(fileOutput, 'w') #load csv file
# Get Session
with file_opener(args.json_log) as json_file:
for line in json_file:
data = json.loads(line)
json_data = data['msg_json']
if "Session" in json_data:
SESSIONID = json_data["Session"]
#inputFile = open(fileInput) #open json file
#outputFile = open(fileOutput, 'w') #load csv file
with file_opener(args.json_log) as json_file:
for line in json_file:
data = json.loads(line)
line_timestamp = time.strptime(data['time'], "%Y%m%d-%H:%M:%S.%f")
if trade_day_utc_start <= line_timestamp <= trade_day_utc_end:
json_data = data['msg_json']
NAME = json_data["Name"]
if NAME in list_types:
EXECUTING_FIRM_ID = ""
ACCOUNT_NUMBER = ""
DIRECTION = direction(data['direction'])
# Set Blanks
blank_json = {'QuoteReqID': '', 'QuoteID': '', 'QuoteSetID': '', 'QuoteEntryID': '', 'BidPx': '', 'BidSize': '', 'OfferPx': '', \
'OfferSize': '', 'NoProcessedEntries': '', 'ListUpdateAction': '', 'MinQty': '', 'AvgPxGroupID': '', \
'CmtaGiveUpCD': '', 'TAKE_UP_FIRM': '', 'TAKE_UP_ACCOUNT': '',}
# No Data Processing
straight_json = {'SecurityID': '', 'ClOrdID': '', 'OrderID': '', 'OrderRequestID': '', 'OrderQty': '', \
'Location': '', 'LastQty': '', 'CumQty': '', 'LeavesQty': '', 'Text': '', 'SideTradeID': '', 'CrossID': '', 'SelfMatchPreventionID': '', \
'Price': '', 'StopPx': '', 'LastPx': '', 'PartyDetailsListReqID': '', 'OrdStatus': ''}
# Processing Data
process_json = {'SendingTimeEpoch': '', 'ManualOrderIndicator': '', 'SenderID': '', 'Side': '', 'OrdType': '', 'TimeInForce': '', \
'OFMOverride': '', 'AggressorIndicator': '', 'ExecRestatementReason': '', 'SelfMatchPreventionInstruction': '', 'CustOrderCapacity': '', \
'AvgPxInd': '', 'ClearingAccountType': '', 'ClearingTradePriceType': '', 'CustOrderHandlingInst': '', 'DisplayQty': ''}
for d in straight_json:
test = get_value(d)
if test != "None" or test != "":
if "," in str(test):
test = test.replace(",",";")
if "4294967295" in str(test):
test = ""
if "us;" in str(test):
test = test.upper()
if d == "SelfMatchPreventionID" and len(str(test)) > 8:
test = ""
straight_json[d] = test
else:
straight_json[d] = ''
for d in process_json:
if d in json_data:
function = locals()[d]
test = function(json_data[d])
if test != "None":
process_json[d] = test
else:
process_json[d] = ''
elif d == "SenderID":
function = locals()[d]
test = function(SenderID)
if test != "None":
process_json[d] = test
# Party Message Fields
# Column 51 - Executing Firm ID
if "NoPartyDetails" in json_data:
EXECUTING_FIRM_ID = partyid(json_data['NoPartyDetails'], "51")
# Column 52 - Party Account Number
ACCOUNT_NUMBER = partyid(json_data['NoPartyDetails'], "52")
line_to_write = process_json['SendingTimeEpoch'] + "," + DIRECTION + "," + process_json['SenderID'] + "," + SESSIONID + "," + process_json['ManualOrderIndicator'] + "," + \
MessageType(NAME,straight_json['OrdStatus']) + "," + str(straight_json['SecurityID']) + "," + str(straight_json['ClOrdID']) + "," + str(straight_json['OrderID']) + "," + str(straight_json['ClOrdID']) + "," + \
str(straight_json['OrderRequestID']) + "," + process_json['Side'] + "," + str(straight_json['OrderQty']) + "," + str(straight_json['Price']) + "," + \
str(straight_json['StopPx']) + "," + process_json['OrdType'] + "," + process_json['TimeInForce'] + "," + process_json['OFMOverride'] + "," + str(process_json['DisplayQty']) + "," + \
blank_json['MinQty'] + "," + straight_json['Location'] + "," + str(straight_json['LastPx']) + "," + str(straight_json['LastQty']) + "," + str(straight_json['CumQty']) + "," + \
str(straight_json['LeavesQty']) + "," + process_json['AggressorIndicator'] + "," + process_json['ExecRestatementReason'] + "," + str(straight_json['Text']) + "," + \
str(straight_json['SideTradeID']) + "," + str(straight_json['CrossID']) + "," + blank_json['QuoteReqID'] + "," + blank_json['QuoteID'] + "," + blank_json['QuoteSetID'] + "," + \
blank_json['QuoteEntryID'] + "," + blank_json['BidPx'] + "," + blank_json['BidSize'] + "," + blank_json['OfferPx'] + "," + blank_json['OfferSize'] + "," + \
blank_json['NoProcessedEntries'] + "," + str(straight_json['PartyDetailsListReqID']) + "," + blank_json['ListUpdateAction'] + "," + str(straight_json['SelfMatchPreventionID']) + "," + \
process_json['SelfMatchPreventionInstruction'] + "," + process_json['CustOrderCapacity'] + "," + blank_json['AvgPxGroupID'] + "," + process_json['AvgPxInd'] + "," + \
blank_json['CmtaGiveUpCD'] + "," + str(process_json['ClearingAccountType']) + "," + process_json['ClearingTradePriceType'] + "," + process_json['CustOrderHandlingInst'] + "," + \
EXECUTING_FIRM_ID + "," + str(ACCOUNT_NUMBER) + "," + blank_json['TAKE_UP_FIRM'] + "," + blank_json['TAKE_UP_ACCOUNT']
#print(line_to_write)
with open (args.output_file, 'a') as the_file:
the_file.write(line_to_write)
the_file.write("\n")
+73
View File
@@ -0,0 +1,73 @@
#/bin/bash
function help_and_exit()
{
echo
echo Usage: $0 [option]... config_file
echo
echo Can also be used in cron like the following
echo 40 16 * * 1-5 /usr/bin/docker exec spark_10000 bash /sparkbin/data_files/audit_upload.sh /sparkdata/data/audit_uploads/example.conf
echo example.conf included. The location of the conf file should be relative to the docker container, not the host
echo
echo Options are as follows:
echo " -d|--date YYYYMMDD Example is 20180731"
echo " Help: "
echo " -h|--help Display this message."
echo
exit 1
}
DATE=$(date +%Y%m%d)
ORIG_DIR=$(pwd)
ERROR_LEVEL="0"
while getopts "d:h" option; do
case $option in
d) DATE=$OPTARG ;;
h) help_and_exit;;
\?) echo "Invalid option: -$OPTARG" >&2;
exit 1;;
esac
done
shift $((OPTIND-1))
CONFIG_FILE=$1
declare -A conf_vars; while read line; do conf_vars[${line%=*}]=${line#*=}; done <$CONFIG_FILE
SESSION_ID=${conf_vars[session_id]}
FCM=${conf_vars[fcm]}
CLIENT=${conf_vars[client]}
cd /sparkbin/data_files
for SESSION in $(ls /sparkdata/data/ilink3/); do
if [[ $SESSION == *$SESSION_ID* ]]; then
echo "Session is $SESSION"
SESSION_DAY=$(date +%Y-%m-%d)
if [[ $SESSION == "cme_"* ]]; then
SESSION_DAY=$(date -d "$DATE -$(date -d $DATE +%u) days" +"%Y-%m-%d")
AUDIT_SCRIPT="audit_ilink3.py"
fi
echo $SESSION_DAY
if [[ ! -d /sparkdata/data/ilink3/$SESSION/audit_logs ]]; then
mkdir -p /sparkdata/data/ilink3/$SESSION/audit_logs
fi
if [[ -d /sparkdata/data/ilink3/$SESSION/$SESSION_DAY ]]; then
echo "Starting the creation of the audit log"
python $AUDIT_SCRIPT -d $DATE /sparkdata/data/ilink3/$SESSION/$SESSION_DAY/log /sparkdata/data/ilink3/$SESSION/audit_logs/"$FCM"_audit_"$SESSION"_"$DATE".csv
fi
if [[ ! -d /sparkdata/data/md_audit ]]; then
mkdir /sparkdata/data/md_audit
fi
if [[ -d /sparkdata/data/md_audit ]]; then
# Running the MD Audit csv generation
cd /sparkbin
./sparkd adminreport md > /sparkdata/data/md_audit/"$FCM"_md_"$CLIENT"_"$DATE".csv
cd /sparkbin/data_files
fi
echo "Uploading to the (s)FTP Server"
python audit_uploader.py --config_file="$CONFIG_FILE" --datestamp $DATE --session="$SESSION" --client="$CLIENT" --fcm="$FCM"
fi
done
+150
View File
@@ -0,0 +1,150 @@
import getopt, sys, datetime
import paramiko, ftplib
#########################################################
# Commandline Args #
#########################################################
def usage():
print "-------------------------------------------------------"
print "-Usage: -"
print "-------------------------------------------------------"
print "- --config_file= : Path to upload config file -"
print "- -if no path is specified look -"
print "- in the working directory. -"
print "- -"
print "- --help : Print this message -"
print "- -"
print "- --datestamp= : Optional, defaults to today -"
print "- -replaces *datestamp in the -"
print "- the upload config file -"
print "- -"
print "- --fcm : Optional, spark input -"
print "- -"
print "- --session : Optional, spark jf session -"
print "-------------------------------------------------------"
fcm = ''
session = ''
try:
options, arguments = getopt.getopt(sys.argv[1:],"h",['config_file=', 'datestamp=', 'help', 'fcm=', 'client=', 'session='])
except getopt.GetoptError, err:
print str(err)
usage()
sys.exit(2)
for option, value in options:
if option in ("-h","--h","-help","--help"):
usage()
sys.exit()
elif option in ("-config_file","--config_file"):
config_file = value
elif option in ("-datestamp","--datestamp"):
datestamp = value
elif option in ("-fcm","--fcm"):
fcm = value
elif option in ("-client","--client"):
client = value
elif option in ("-session","--session"):
session = value
for argument in arguments:
if argument in ("h", "help"):
usage()
sys.exit()
else:
print "Unrecognized commandline argument. Type help for usage."
sys.exit()
if len(options) <= 0:
print "Requires a config file to run."
usage()
sys.exit()
#########################################################
# Read Config Function #
#########################################################
def read_config_file(config_file, datestamp=datetime.datetime.today().strftime('%Y%m%d'), fcm=fcm, session=session):
'''
Parse our config file that gets passed in as a commandline parameter.
All config keys must be of the form key=value. White space is ignored.
All config keys must be seperated by a newline character.
Called during initialization.
'''
upload_config = {}
config_file = open(config_file, 'r')
config_file = config_file.read()
for config in config_file.split("\n"):
if '=' in config:
cleaned_key = config.split('=')[0].strip().upper()
cleaned_value = config.split('=')[1].strip()
if '*datestamp*' in cleaned_value:
cleaned_value = cleaned_value.replace('*datestamp*',datestamp)
if '*fcm*' in cleaned_value:
cleaned_value = cleaned_value.replace('*fcm*',fcm)
if '*client*' in cleaned_value:
cleaned_value = cleaned_value.replace('*client*',client)
if '*session*' in cleaned_value:
cleaned_value = cleaned_value.replace('*session*',session)
upload_config[cleaned_key] = cleaned_value
else:
pass
return upload_config
#########################################################
# FTP File Transfer Function #
#########################################################
def putfile_ftp(file, put_path, site, user, password, port=21, verbose=True):
if verbose: print 'Uploading', file
local = open(file, 'rb')
remote = ftplib.FTP(site)
remote.login(user, password)
remote.storbinary(put_path, local, 1024)
remote.quit()
local.close()
if verbose: print 'Upload done.'
#########################################################
# SFTP File Transfer Function #
#########################################################
def putfile_sftp(file, put_path, site, username, password, port=22, verbose=True):
transport = paramiko.Transport((site, port))
transport.connect(username = username, password = password)
sftp = paramiko.SFTPClient.from_transport(transport)
sftp.put(file, put_path)
sftp.close()
transport.close()
#########################################################
# Main #
#########################################################
#if datestamp != None:
# upload_config = read_config_file(config_file, datestamp)
#else:
# upload_config = read_config_file(config_file)
try:
datestamp
except NameError:
upload_config = read_config_file(config_file)
else:
upload_config = read_config_file(config_file, datestamp)
print upload_config
if upload_config["PROTOCOL_PUT"] in ['FTP','ftp']:
print "Protocol = FTP"
putfile_ftp(upload_config["GET_ILINK3_FILE_PATH"] + upload_config["GET_ILINK3_FILE_NAME"], upload_config["PUT_ILINK3_FILE_PATH"] + upload_config["PUT_ILINK3_FILE_NAME"], upload_config["PUT_HOST"], upload_config["USERNAME_PUT"], upload_config["PASSWORD_PUT"], upload_config["PORT_PUT"] )
putfile_ftp(upload_config["GET_MD_FILE_PATH"] + upload_config["GET_MD_FILE_NAME"], upload_config["PUT_MD_FILE_PATH"] + upload_config["PUT_MD_FILE_NAME"], upload_config["PUT_HOST"], upload_config["USERNAME_PUT"], upload_config["PASSWORD_PUT"], upload_config["PORT_PUT"] )
elif upload_config["PROTOCOL_PUT"] in ['SFTP','sftp']:
print "Protocol = SFTP"
putfile_sftp(upload_config["GET_ILINK3_FILE_PATH"] + upload_config["GET_ILINK3_FILE_NAME"], upload_config["PUT_ILINK3_FILE_PATH"] + upload_config["PUT_ILINK3_FILE_NAME"], upload_config["PUT_HOST"], upload_config["USERNAME_PUT"], upload_config["PASSWORD_PUT"], int(upload_config["PORT_PUT"]) )
putfile_sftp(upload_config["GET_MD_FILE_PATH"] + upload_config["GET_MD_FILE_NAME"], upload_config["PUT_MD_FILE_PATH"] + upload_config["PUT_MD_FILE_NAME"], upload_config["PUT_HOST"], upload_config["USERNAME_PUT"], upload_config["PASSWORD_PUT"], int(upload_config["PORT_PUT"]) )
+19 -12
View File
@@ -1,12 +1,19 @@
Session ID Access ID Secret Key Creation Date Expiration Date Environment
M4D lLtjH74y1Kh5XUJTLyH2 f1K_tROQp2lTIplVp34WxKa7xDUgfFnIteolfik061o 2019-09-03 2020-09-03 CERT
MC1 VEMlkDuLz74OVHSuacQB IuHyYT3b-9KUo3WsuXykt-fXsjzp7L-JOrdCSQZLNl0 2019-09-03 2020-09-03 CERT
MC5 77lzSUwPLlM9obaXh2mK z6aA9qfASDKfZkYTVKFqJd-78i0HZ2Q9ec3CKCYIqHg 2019-09-03 2020-09-03 CERT
MC7 RphzI3rn71oGUEiYnX9e y6OvvPPyoyj2S8YAnE1gFduXT8v29HIMjO5EngYxSNg 2019-09-03 2020-09-03 CERT
MC8 PYBIm2TI9gFPIxmHbuYl 6o-ZolGH2DdeQOlGtJ5Zqoh_kg7VdbxkmjeOHmigKwU 2019-09-03 2020-09-03 CERT
Session ID Access ID Secret Key Creation Date Expiration Date Environment
M4D lLtjH74y1Kh5XUJTLyH2 f1K_tROQp2lTIplVp34WxKa7xDUgfFnIteolfik061o 2019-09-03 2020-09-03 CERT
MC1 VEMlkDuLz74OVHSuacQB IuHyYT3b-9KUo3WsuXykt-fXsjzp7L-JOrdCSQZLNl0 2019-09-03 2020-09-03 CERT
MC5 77lzSUwPLlM9obaXh2mK z6aA9qfASDKfZkYTVKFqJd-78i0HZ2Q9ec3CKCYIqHg 2019-09-03 2020-09-03 CERT
MC7 RphzI3rn71oGUEiYnX9e y6OvvPPyoyj2S8YAnE1gFduXT8v29HIMjO5EngYxSNg 2019-09-03 2020-09-03 CERT
MC8 PYBIm2TI9gFPIxmHbuYl 6o-ZolGH2DdeQOlGtJ5Zqoh_kg7VdbxkmjeOHmigKwU 2019-09-03 2020-09-03 CERT
NGG RYcigz3bSLIIBBF2A2Hq qB1dm9VkTKeXcdQpFzla8B3_jwTgya_-Fk5iyEJvr2E 2020-06-15 2021-06-15 NEWRELEASE
NGF o4OTmmTLQ7Lv4g6bCm26 HbD3FneMyb9Fu-FkCDLscCczHqFcFE2PyWv978RXGyA 2019-12-20 2020-12-20 NEWRELEASE
UZE EeFdZ1FPepk6U4k3xBZA 2LjEtdLvBLLB1QuDJ4KiJW6M4dF6IHWd_UKmo7p3_Us 2020-06-24 2021-06-24 NEWRELEASE
VAE Y8iqatTewuoHX16gghw8 FNcCB3Ks1qleS56quYzH4V7zAu78YY2bCdLRi9tmU9E 2020-06-24 2021-06-24 NEWRELEASE
+16
View File
@@ -0,0 +1,16 @@
fcm=FCM
client=NAME
session_id=ses
put_host=ftpc.example.com
port_put=2222
username_put=user
password_put=password
protocol_put=sftp
get_ilink3_file_path=/sparkdata/data/ilink3/*session*/audit_logs/
put_ilink3_file_path=/
get_ilink3_file_name=*fcm*_audit_*session*_*datestamp*.csv
put_ilink3_file_name=*fcm*_audit_*session*_*datestamp*.csv
get_md_file_path=/sparkdata/data/md_audit/
put_md_file_path=/
get_md_file_name=*fcm*_md_*client*_*datestamp*.csv
put_md_file_name=*fcm*_md_*client*_*datestamp*.csv
+1 -1
View File
@@ -1 +1 @@
set(sources api/apicore.proto;api/apigateway.proto;api/apioptions.proto;api/core/ApiClientInterop.cpp;api/core/ApiServerInterop.cpp;api/public/ApiServer.cpp;api/public/ClientChannel.cpp;api/public/QuickbusClientChannel.cpp;api/public/ServerChannel.cpp;api/public/ServiceContext.cpp;api/service_helpers/AdminServiceUtils.cpp;api/service_helpers/AlgoServiceUtils.cpp;api/service_helpers/PositionUtilities.cpp;api/services/AdminService.proto;api/services/AlgoDefinitionService.proto;api/services/AlgoService.proto;api/services/ApiUnitTestService.proto;api/services/AuthService.proto;api/services/BasePositionService.proto;api/services/ErrorReportService.proto;api/services/HelloService.proto;api/services/MBOService.proto;api/services/MarginService.proto;api/services/MarketDataAuxiliaryService.proto;api/services/MarketDataRelay.proto;api/services/MarketDataService.proto;api/services/MoneyFileService.proto;api/services/OrderService.proto;api/services/OutOfProcessURLListener.proto;api/services/PersistenceService.proto;api/services/PositionService.proto;api/services/ShellDiagnosticService.proto;api/services/ShellLatencyService.proto;api/services/SyntheticSecurityService.proto;api/test/ApiTests.cpp;api/test/HelloServiceServer.cpp;config/ConfigCheck.cpp;config/ConfigDownloader.cpp;config/ConfigErrorCollector.cpp;config/ConfigLog.cpp;config/ConfigPropertyEvaluator.cpp;config/ConfigValueQueue.cpp;config/ConfigurationException.cpp;config/ConfigurationState.cpp;config/PreprocessInputStream.cpp;config/config.cpp;config/defaults/AlgoTypes.conf;config/defaults/MarketData.conf;config/defaults/Ports.conf;config/defaults/Symbols.conf;config/defaults/cme_reference_data.conf;config/defaults/edge.conf;config/defaults/jetfix.conf;config/defaults/keycode.conf;config/defaults/logging.conf;config/defaults/orderman_cfg.conf;config/defaults/orders_fills_cleanup.conf;config/defaults/position_cfg.conf;config/defaults/span.conf;config/defaults/spreader.conf;config/defaults/testkeys.conf;config/defaults/threads.conf;config/defaults/toggles.conf;config/extensions/extensions.proto;config/invalid/invalid.proto;config/protos/ExchangeSimCfg.proto;config/protos/MarketData.proto;config/protos/Ports.proto;config/protos/Symbols.proto;config/protos/UI.proto;config/protos/cme_reference_data.proto;config/protos/jetfix.proto;config/protos/keycode.proto;config/protos/logging.proto;config/protos/mercury_cfg.proto;config/protos/orderman_cfg.proto;config/protos/orders_fills_cleanup.proto;config/protos/perflog_cfg.proto;config/protos/position_cfg.proto;config/protos/quickbus_cfg.proto;config/protos/span.proto;config/protos/spreader.proto;config/protos/subprocess.proto;config/protos/system.proto;config/protos/testkeys.proto;config/protos/threads.proto;config/protos/toggles.proto;config/tests/HTTPServer.cpp;config/tests/config_ref_tests.cpp;config/tests/config_tests.cpp;config/text_format/text_format.cpp;domain_model/Security.cpp;domain_model/Security.proto;domain_model/SecurityTests.cpp;domain_model/sod.cpp;domain_model/tests/domain_tests.cpp;expression/BooleanGrammar.cpp;expression/ExpressionEvaluator.cpp;expression/ParserExceptions.cpp;expression/Value.cpp;expression/tests/expression_parser_tests.cpp;jetfix/codec/FIXArray.cpp;jetfix/common/OutFIX.cpp;jetfix/connection/AsyncReader.cpp;jetfix/connection/IConnection.cpp;jetfix/connection/SocketConnection.cpp;jetfix/gateway/CoreGateway.cpp;jetfix/gateway/JetFIXGateway.cpp;jetfix/gateway/Session.cpp;jetfix/store/FIXFileStore.cpp;jetfix/store/session_name.cpp;jetfix/tests/async_reader_tests.cpp;jetfix/tests/decoding_tests.cpp;jetfix/tests/fix_encoding_stream_tests.cpp;jetfix/tests/fix_fragment_tests.cpp;jetfix/tests/jetfix_performance_tests.cpp;jetfix/tests/sample_fix.cpp;jetfix/tests/store_tests.cpp;logging/LoggerHub.cpp;logging/Sink.cpp;logging/logging.cpp;logging/tests/logging_tests.cpp;md/Diag/MDPeriodicDumper.cpp;md/Diag/ShmDumper.cpp;md/MBOProcessor.cpp;md/MBOServiceServer.cpp;md/MDClient.cpp;md/MDConsumer.cpp;md/MDConsumerHandle.cpp;md/MDConsumerInterop.cpp;md/MDProducer.cpp;md/MDProducerHandle.cpp;md/MemorySegmentUtils.cpp;md/PlatformLoader.cpp;md/PriceSize.cpp;md/Product.cpp;md/SecurityCache.cpp;md/SecurityDB.cpp;md/SecurityIDRecorder.cpp;md/SecurityMapper.cpp;md/SecurityOracle.cpp;md/SettlementRecorder.cpp;md/Shm/FixedSecurity.cpp;md/SyntheticSecurityService.cpp;md/cme/MDP3Utils.cpp;md/common.cpp;md/ice/ICEUtils.cpp;md/protos/co2.proto;md/relay/QueryServer.cpp;md/relay/RelayClient.cpp;md/relay/RelayRecorder.cpp;md/service/MarketDataServiceServer.cpp;md/test/Co2ManagedTestHelper.cpp;md/test/Co2TestInstrument.cpp;md/test/mbo_tests.cpp;md/test/md2_tests.cpp;md/test/md_test_utils.cpp;md/test/synthetic_book_merge_tests.cpp;md/vendor/OnixsUtils.cpp;mercury/actors/IPollableActor.cpp;mercury/broker/Broker.cpp;mercury/broker/BrokerProxy.cpp;mercury/broker/ILocalBroker.cpp;mercury/broker/IRemoteBroker.cpp;mercury/common/mercury_common.cpp;mercury/events/TimerEvent.cpp;mercury/executor/Executor.cpp;mercury/topics/Topic.Line truncated
set(sources api/apicore.proto;api/apigateway.proto;api/apioptions.proto;api/core/ApiClientInterop.cpp;api/core/ApiServerInterop.cpp;api/public/ApiServer.cpp;api/public/ClientChannel.cpp;api/public/QuickbusClientChannel.cpp;api/public/ServerChannel.cpp;api/public/ServiceContext.cpp;api/service_helpers/AdminServiceUtils.cpp;api/service_helpers/AlgoServiceUtils.cpp;api/service_helpers/PositionUtilities.cpp;api/services/AdminService.proto;api/services/AlgoDefinitionService.proto;api/services/AlgoService.proto;api/services/ApiUnitTestService.proto;api/services/AuthService.proto;api/services/BasePositionService.proto;api/services/ErrorReportService.proto;api/services/HelloService.proto;api/services/MBOService.proto;api/services/MarginService.proto;api/services/MarketDataAuxiliaryService.proto;api/services/MarketDataRelay.proto;api/services/MarketDataService.proto;api/services/MoneyFileService.proto;api/services/OrderService.proto;api/services/OutOfProcessURLListener.proto;api/services/PersistenceService.proto;api/services/PositionService.proto;api/services/ShellDiagnosticService.proto;api/services/ShellLatencyService.proto;api/services/SyntheticSecurityService.proto;api/test/ApiTests.cpp;api/test/HelloServiceServer.cpp;config/ConfigCheck.cpp;config/ConfigDownloader.cpp;config/ConfigErrorCollector.cpp;config/ConfigLog.cpp;config/ConfigPropertyEvaluator.cpp;config/ConfigValueQueue.cpp;config/ConfigurationException.cpp;config/ConfigurationState.cpp;config/PreprocessInputStream.cpp;config/config.cpp;config/defaults/MarketData.conf;config/defaults/Ports.conf;config/defaults/Symbols.conf;config/defaults/cme_reference_data.conf;config/defaults/edge.conf;config/defaults/jetfix.conf;config/defaults/keycode.conf;config/defaults/logging.conf;config/defaults/orderman_cfg.conf;config/defaults/orders_fills_cleanup.conf;config/defaults/position_cfg.conf;config/defaults/span.conf;config/defaults/spreader.conf;config/defaults/testkeys.conf;config/defaults/threads.conf;config/defaults/toggles.conf;config/extensions/extensions.proto;config/invalid/invalid.proto;config/protos/AlgoTypes.proto;config/protos/ExchangeSimCfg.proto;config/protos/MarketData.proto;config/protos/Ports.proto;config/protos/Symbols.proto;config/protos/UI.proto;config/protos/cme_reference_data.proto;config/protos/jetfix.proto;config/protos/keycode.proto;config/protos/logging.proto;config/protos/mercury_cfg.proto;config/protos/orderman_cfg.proto;config/protos/orders_fills_cleanup.proto;config/protos/perflog_cfg.proto;config/protos/position_cfg.proto;config/protos/quickbus_cfg.proto;config/protos/span.proto;config/protos/spreader.proto;config/protos/subprocess.proto;config/protos/system.proto;config/protos/testkeys.proto;config/protos/threads.proto;config/protos/toggles.proto;config/tests/HTTPServer.cpp;config/tests/config_ref_tests.cpp;config/tests/config_tests.cpp;config/text_format/text_format.cpp;domain_model/Security.cpp;domain_model/Security.proto;domain_model/SecurityTests.cpp;domain_model/sod.cpp;domain_model/tests/domain_tests.cpp;expression/BooleanGrammar.cpp;expression/ExpressionEvaluator.cpp;expression/ParserExceptions.cpp;expression/Value.cpp;expression/tests/expression_parser_tests.cpp;jetfix/codec/FIXArray.cpp;jetfix/common/OutFIX.cpp;jetfix/connection/AsyncReader.cpp;jetfix/connection/IConnection.cpp;jetfix/connection/SocketConnection.cpp;jetfix/gateway/CoreGateway.cpp;jetfix/gateway/JetFIXGateway.cpp;jetfix/gateway/Session.cpp;jetfix/store/FIXFileStore.cpp;jetfix/store/session_name.cpp;jetfix/tests/async_reader_tests.cpp;jetfix/tests/decoding_tests.cpp;jetfix/tests/fix_encoding_stream_tests.cpp;jetfix/tests/fix_fragment_tests.cpp;jetfix/tests/jetfix_performance_tests.cpp;jetfix/tests/sample_fix.cpp;jetfix/tests/store_tests.cpp;logging/LoggerHub.cpp;logging/Sink.cpp;logging/logging.cpp;logging/tests/logging_tests.cpp;md/Diag/MDPeriodicDumper.cpp;md/Diag/ShmDumper.cpp;md/MBOProcessor.cpp;md/MBOServiceServer.cpp;md/MDClient.cpp;md/MDConsumer.cpp;md/MDConsumerHandle.cpp;md/MDConsumerInterop.cpp;md/MDProducer.cpp;md/MDProducerHandle.cpp;md/MemorySegmentUtils.cpp;md/PlatformLoader.cpp;md/PriceSize.cpp;md/Product.cpp;md/SecurityCache.cpp;md/SecurityDB.cpp;md/SecurityIDRecorder.cpp;md/SecurityMapper.cpp;md/SecurityOracle.cpp;md/SettlementRecorder.cpp;md/Shm/FixedSecurity.cpp;md/SyntheticSecurityService.cpp;md/cme/MDP3Utils.cpp;md/common.cpp;md/ice/ICEUtils.cpp;md/protos/co2.proto;md/relay/QueryServer.cpp;md/relay/RelayClient.cpp;md/relay/RelayRecorder.cpp;md/service/MarketDataServiceServer.cpp;md/test/Co2ManagedTestHelper.cpp;md/test/Co2TestInstrument.cpp;md/test/mbo_tests.cpp;md/test/md2_tests.cpp;md/test/md_test_utils.cpp;md/test/synthetic_book_merge_tests.cpp;md/vendor/OnixsUtils.cpp;mercury/actors/IPollableActor.cpp;mercury/broker/Broker.cpp;mercury/broker/BrokerProxy.cpp;mercury/broker/ILocalBroker.cpp;mercury/broker/IRemoteBroker.cpp;mercury/common/mercury_common.cpp;mercury/events/TimerEvent.cpp;mercury/executor/Executor.cpp;mercury/topics/Topic.cLine truncated
+1 -1
View File
@@ -319,7 +319,7 @@ if __name__ == "__main__":
if DEBUG:
for f in response.file:
print f.content
print (f.content)
else:
output = response.SerializeToString()
sys.stdout.write(output)
@@ -580,7 +580,9 @@ bts::api::orderman::RoutingRecord AdminServiceUtils::get_routing_record(const bt
rv.set_session(conn_ptr->name());
rv.set_operator_id(cme_user_data->operator_id());
rv.set_location(cme_user_data->location());
rv.set_smp_id(cme_user_data->smp_id());
if (cme_user_data->smp_instruction()== "O")
rv.set_smp_instruction(api::orderman::SelfMatchPreventionInstruction::SELF_MATCH_PREVENTION_INSTRUCTION_CANCEL_RESTING);
else if (cme_user_data->smp_instruction()== "N")
+4 -1
View File
@@ -386,7 +386,9 @@ message NativeOrderState {
RoutingRecord routing_record = 5;
NativeOrderType native_type = 6;
string last_transact_time = 7;
uint64 last_transact_time_unix_ns = 9;
double stop_px = 8;
uint64 exchange_order_id_u64 = 10;
}
// Custom order states
@@ -455,7 +457,8 @@ enum TimeInForce {
TIME_IN_FORCE_UNDEFINED = 0;
TIME_IN_FORCE_DAY = 48;
TIME_IN_FORCE_GTC = 49;
TIME_IN_FORCE_IOC = 51;
TIME_IN_FORCE_IOC = 51; // FAK
TIME_IN_FORCE_DATE = 52;
}
enum ManualOrderIndicator {
+6
View File
@@ -45,6 +45,12 @@ CMEOptions {
}
}
ICEPublisherOptions {
value {
network_interface: "eth1"
}
}
#to make this work in dev clone the operations repo
#go here: c:\\code\\bts\\operations\\ice\\ICE_iMpactMulticastFeed_GettingStarted-1_1_36\\bin\\easyTunnel>
#modify the easyTunnelProxy.properties file to replace
File diff suppressed because it is too large. Load diff
+5
View File
@@ -103,6 +103,7 @@ message CMEOptions
// Defines amount of time Feed Engine keeps cycling before going to sleep when no useful activity can be done
optional uint32 spin_time = 4 [default = 1];
optional uint32 channels_per_thread = 5 [default = 3];
// rmem_max
optional uint32 udp_socket_buffer_size = 11 [default = 268435456]; //256mb
}
message CMEChannel {
@@ -110,9 +111,13 @@ message CMEOptions
optional string feed_engine = 3 [default = "default"];
// for re-orders
optional uint32 max_gap_recovery_cache_size = 11 [default = 6];
// for re-orders
optional uint32 lost_packet_wait_timeout_ms = 12 [default = 10];
optional OnixSLogMode onixs_log_mode = 13 [default = Important];
// If using both feeds prefer the B feed for snapshots
optional bool prefer_backup_feeds = 15 [default = false];
optional bool use_tcp_replay = 20 [default = true];
+5 -4
View File
@@ -9,9 +9,9 @@ message JetFIX {
enum JetFIXPlatform {
GLBX = 1;
CBOE = 6;
SIM = 2;
ICE = 3;
SIMICE = 4;
SIM = 2;
ICE = 3;
SIMICE = 4;
}
optional JetFIXPlatform platform = 15 [default = GLBX];
@@ -87,6 +87,7 @@ message JetFIX {
optional uint32 max_connection_attempts = 346 [default=10];
optional bool auto_fix_seq_num = 347 [default=false];
optional bool use_ilink3 = 348 [default=false] ;
}
repeated Connection connection = 10;
@@ -116,4 +117,4 @@ message CMECannedSecureLogon {
"MC8 AbEEtBn63WIAuItPDz1c n3ndjYb03Hx6wBO3me35HoONyDgO5TZKWk0tDSaX9KA 2018-04-05 2019-04-05 CERT \n"
"T39 UWggBgHWAe07aqW3ps2I tbb3it0rUQSSTa0TC9QcRwkrHEMDLWhaJH4VQ0FqGB4 2018-04-05 2019-04-05 NEWRELEASE \n"
];
}
}
+1
View File
@@ -73,4 +73,5 @@ message DynamicAlgos{
optional bool enable_tas = 3 [default = false];
optional bool enable_tascounter = 4 [default = false];
optional bool enable_limit_cover = 5 [default = false];
optional bool enable_onsignal = 6 [default = false];
}
+8 -19
View File
@@ -212,31 +212,20 @@ inline bool gr_than(double p1, double p2, double tick_size) {
return gr_than_eq(p1, p2, tick_size) &&
!test_price_equality(p1, p2, tick_size);
}
inline bool test_price_equality(double price_1, double price_2) {
return std::fabs(price_2 - price_1) < market_epsilon();
}
// Works only if ICEExchangeData is part of the a spread security
// inline bool is_ice_strip(const bts::api::ICEExchangeData& ice_data){
// return ice_data.is_strip();
// }
// ice specialization /////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////
inline bool is_ice_strip(const bts::api::ICEExchangeData& ice_data){
// TODO, put this in the security
inline bool is_ice_spread(const bts::api::ICEExchangeData& ice_data){
const auto& strip = ice_data.strip_name();
auto find = [&](const std::string& sub) -> bool {
return strip.find(sub) != std::string::npos;
};
if(!find("/"))
return find("-") ||
find("Q1") || find("Q2") || find("Q3") || find("Q4") ||
find("Cal");
else
return false;
return strip.find("/") != std::string::npos;
}
inline bool is_strip(const bts::api::Security& sec){
if(sec.has_spread())
if (sec.has_ice())
return is_ice_strip(sec.ice());
return false;
return sec.has_ice() && sec.has_spread() && !is_ice_spread(sec.ice());
}
inline bool is_spread_of_strips(const bts::api::Security& sec){
+1 -1
View File
@@ -52,7 +52,7 @@ class FIXEncodingStream {
auto msg_offset = _buffer.size();
_current_msg = FIXFragment(_buffer);
_begin_string_bm = _current_msg.add<fix::BeginString>("FIX.4.2");
_begin_string_bm = _current_msg.add<fix::BeginString>(std::string_view {"FIX.4.2"});
_body_length_bm = _current_msg.add<fix::BodyLength>(0, 6);
char mt_encoding[4];
+1
View File
@@ -85,6 +85,7 @@ typedef StringField<307> UnderlyingSecurityDesc;
typedef StringField<336> TradingSessionID;
typedef StringField<35> MsgType;
typedef StringField<37> OrderID;
typedef BigIntField<37> OrderIDu64;
typedef StringField<41> OrigClOrdID;
typedef StringField<440> ClearingAccount;
typedef StringField<48> SecurityID;
+9
View File
@@ -2,6 +2,8 @@
#include <memory>
#include <boost/date_time/posix_time/posix_time.hpp>
#include <string_view>
#include "lib/jetfix/codec/ReaderCursor.h"
#include "lib/utils/meta_programming.h"
#include "lib/utils/datetime.h"
@@ -203,6 +205,13 @@ public:
static std::pair<uint32_t,uint32_t> write (char*buffer,std::string const &str)
{return write (buffer,str.data (),str.size ());}
#if __cplusplus >= 201703L
static uint32_t max_value_size (std::string_view const&str) {return str.size ();}
static std::pair<uint32_t,uint32_t> write (char*buffer,std::string_view const &str)
{return write (buffer,str.data (),str.size ());}
#endif
static std::pair<uint32_t,uint32_t> write (char*buffer,const char*str,uint32_t sz)
{
memcpy (buffer,str,sz);
@@ -23,9 +23,9 @@ SocketConnection::SocketConnection(bts::config::JetFIX::Connection const &cfg,
: _fd(-1), _cfg(cfg), _log(cfg.log_fix_stream()), _in_log(), _out_log(),
_perf_log(perf_log) {
if (_log) {
_in_log.open(std::string("/tmp/fix_raw_read_log_") + session_name(cfg),
_in_log.open(std::string("/tmp/fix_raw_read_log_") + session_name(cfg).as_string,
std::ios_base::app);
_out_log.open(std::string("/tmp/fix_raw_write_log_") + session_name(cfg),
_out_log.open(std::string("/tmp/fix_raw_write_log_") + session_name(cfg).as_string,
std::ios_base::app);
}
}
+22 -23
View File
@@ -786,9 +786,10 @@ OutFIXPtr make_logon(uint32_t seq_num, uint32_t last_seq_num_processed,
if(session_cfg.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_ICE ||
session_cfg.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_SIMICE)
{
buffer.add<ICEApplicationSystemName>("Katallage");
buffer.add<ICEApplicationSystemVersion>("1.0");
buffer.add<ICEApplicationSystemVendor>("Katallage");
std::string sender_comp_id = session_cfg.sender_subid();
buffer.add<ICEApplicationSystemName>(sender_comp_id);
buffer.add<ICEApplicationSystemVersion>(std::string_view{"1.0"});
buffer.add<ICEApplicationSystemVendor>(sender_comp_id);
buffer.add<UserName>(session_cfg.user_name());
buffer.add<RawDataLength>(session_cfg.raw_data().size());
@@ -796,19 +797,18 @@ OutFIXPtr make_logon(uint32_t seq_num, uint32_t last_seq_num_processed,
buffer.add<EncryptMethod>(0);
buffer.add<ICEStrategyPreference>(1);
buffer.add<ICETimeStampPreference>(1);
}
else
{
buffer.add<ApplicationSystemName>(application);
buffer.add<ApplicationSystemVersion>("1.0");
buffer.add<ApplicationSystemVendor>("BTS");
} else {
buffer.add<ApplicationSystemName>(std::string_view{application});
buffer.add<ApplicationSystemVersion>(std::string_view{"1.0"});
buffer.add<ApplicationSystemVendor>(std::string_view{"BTS"});
if (session_cfg.has_trading_session_id()) {
buffer.add<fix::NoTradingSessions>(1);
buffer.add<fix::TradingSessionID>(session_cfg.trading_session_id());
}
if (session_cfg.has_udf_support_indicator())
buffer.add<fix::UDFSupportIndicator>(session_cfg.udf_support_indicator());
buffer.add<fix::UDFSupportIndicator>(
session_cfg.udf_support_indicator());
if (session_cfg.use_cme_secure_logon()) {
bts_assert(session_cfg.sender_compid().size() >= 3);
@@ -819,25 +819,24 @@ OutFIXPtr make_logon(uint32_t seq_num, uint32_t last_seq_num_processed,
std::string last_seq_num_str(9, ' ');
i32_toa(last_seq_num_processed, &last_seq_num_str[0], 9);
auto ld = glbx::calculate_hmac(
session_id, seq_num_str, session_cfg.sender_compid(),
session_cfg.sender_subid(), time_str, session_cfg.target_subid(),
std::to_string(session_cfg.seconds_between_hbs()),
session_cfg.sender_locationid(), last_seq_num_str, application, "1.0",
"BTS");
#ifndef BTS_EDGE
LOG(Jetfix, Info, "Canonical string for secure logon: {}", ld.canonical);
#endif
session_id, seq_num_str, session_cfg.sender_compid(),
session_cfg.sender_subid(), time_str, session_cfg.target_subid(),
std::to_string(session_cfg.seconds_between_hbs()),
session_cfg.sender_locationid(), last_seq_num_str, application,
"1.0", "BTS");
#ifndef BTS_EDGE
LOG(Jetfix, Info, "Canonical string for secure logon: {}",
ld.canonical);
#endif
buffer.add<EncodedTextLen>(ld.access_key.size());
buffer.add<EncodedText>(ld.access_key);
buffer.add<EncryptedPasswordMethod>("CME-1-SHA-256");
buffer.add<EncryptedPasswordMethod>(std::string_view{"CME-1-SHA-256"});
buffer.add<EncryptedPasswordLen>(ld.signature.size());
buffer.add<EncryptedPassword>(ld.signature);
}
else
{
} else {
buffer.add<RawDataLength>(session_cfg.raw_data().size());
buffer.add<RawData>(session_cfg.raw_data());
if(session_cfg.user_name() != "")
if (session_cfg.user_name() != "")
buffer.add<UserName>(session_cfg.user_name());
}
}
+1 -1
View File
@@ -185,7 +185,7 @@ void load_seq_nums(std::string const &path, uint32_t *last_seq_num,
FIXFileStore::FIXFileStore(config::JetFIX::Connection const &cfg,
std::string base_path)
: me(new PImpl(cfg)) {
auto sn = session_name(cfg);
auto sn = session_name(cfg).as_string;
me->active = sn.size() > 0;
if (me->active) {
+19 -14
View File
@@ -75,49 +75,55 @@ bool in_session (datetime::PTime const &ref_date,
return closest_earlier_end<closest_earlier_start;
}
std::string session_name_from_ptime (datetime::PTime t,
SessionName session_name_from_ptime (datetime::PTime t,
boost::posix_time::time_duration time_of_day_start,
boost::posix_time::time_duration time_of_day_end)
{
std::string result (10,' ');
SessionName result;
result.as_string = std::string(10,' ');
auto start_of_session = t;
if (time_of_day_start > time_of_day_end &&
start_of_session.time_of_day () <= time_of_day_end)
start_of_session -= boost::gregorian::date_duration (1);
sprintf (&result[0], "%04d-%02d-%02d",
using namespace boost::posix_time;
ptime epoch(boost::gregorian::date(1970,1,1));
ptime session_start_date(start_of_session.date());
result.as_uint64_t = (session_start_date - epoch).total_milliseconds();
sprintf (&(result.as_string[0]), "%04d-%02d-%02d",
(unsigned short) start_of_session.date ().year (),
start_of_session.date ().month ().as_number (),
start_of_session.date ().day ().as_number ());
return result;
}
std::string session_name (datetime::PTime const &ref_date,
boost::gregorian::greg_weekday weekday_start,
boost::posix_time::time_duration time_of_day_start,
boost::gregorian::greg_weekday weekday_end,
boost::posix_time::time_duration time_of_day_end)
SessionName session_name (datetime::PTime const &ref_date,
boost::gregorian::greg_weekday weekday_start,
boost::posix_time::time_duration time_of_day_start,
boost::gregorian::greg_weekday weekday_end,
boost::posix_time::time_duration time_of_day_end)
{
if (in_session (ref_date,weekday_start,time_of_day_start,weekday_end,time_of_day_end))
return session_name_from_ptime (closest_before (ref_date,weekday_start,time_of_day_start),
boost::posix_time::time_duration(0, 0, 0),
boost::posix_time::time_duration(0, 0, 1));
else
return std::string ();
return {};
}
std::string session_name (datetime::PTime const &ref_date,
SessionName session_name (datetime::PTime const &ref_date,
boost::posix_time::time_duration time_of_day_start,
boost::posix_time::time_duration time_of_day_end)
{
if (in_session (ref_date,time_of_day_start,time_of_day_end))
return session_name_from_ptime (ref_date, time_of_day_start, time_of_day_end);
else
return "";
return {};
}
std::string session_name (config::JetFIX::Connection const &session_cfg)
SessionName session_name (config::JetFIX::Connection const &session_cfg)
{
auto now = datetime::local_now ();
@@ -134,7 +140,6 @@ std::string session_name (config::JetFIX::Connection const &session_cfg)
parse_time (session_cfg.end_time ()));
}
}}
+34 -28
View File
@@ -1,9 +1,9 @@
#pragma once
#include "lib/config/protos/jetfix.pb.h"
#include "lib/utils/datetime.h"
#include <memory>
#include <string>
#include "lib/utils/datetime.h"
#include "lib/config/protos/jetfix.pb.h"
namespace boost {
namespace gregorian {
@@ -11,39 +11,45 @@ class greg_weekday;
}
namespace posix_time {
class time_duration;
}}
}
} // namespace boost
namespace bts {
namespace jetfix {
boost::gregorian::greg_weekday parse_weekday (std::string weekday_str);
bool parse_time (std::string const &time_str,boost::posix_time::time_duration *out);
boost::gregorian::greg_weekday parse_weekday(std::string weekday_str);
bool parse_time(std::string const &time_str,
boost::posix_time::time_duration *out);
datetime::PTime closest_before (datetime::PTime const &ref_date,
boost::gregorian::greg_weekday weekday,
boost::posix_time::time_duration time_of_day);
datetime::PTime closest_before(datetime::PTime const &ref_date,
boost::gregorian::greg_weekday weekday,
boost::posix_time::time_duration time_of_day);
bool in_session (datetime::PTime const &ref_date,
boost::gregorian::greg_weekday weekday_start,
boost::posix_time::time_duration time_of_day_start,
boost::gregorian::greg_weekday weekday_end,
boost::posix_time::time_duration time_of_day_end);
bool in_session(datetime::PTime const &ref_date,
boost::gregorian::greg_weekday weekday_start,
boost::posix_time::time_duration time_of_day_start,
boost::gregorian::greg_weekday weekday_end,
boost::posix_time::time_duration time_of_day_end);
bool in_session (datetime::PTime const &ref_date,
boost::posix_time::time_duration time_of_day_start,
boost::posix_time::time_duration time_of_day_end);
bool in_session(datetime::PTime const &ref_date,
boost::posix_time::time_duration time_of_day_start,
boost::posix_time::time_duration time_of_day_end);
std::string session_name (datetime::PTime const &ref_date,
boost::gregorian::greg_weekday weekday_start,
boost::posix_time::time_duration time_of_day_start,
boost::gregorian::greg_weekday weekday_end,
boost::posix_time::time_duration time_of_day_end);
struct SessionName {
std::string as_string;
uint64_t as_uint64_t;
bool empty() const {return as_string.empty();};
};
std::string session_name (datetime::PTime const &ref_date,
boost::posix_time::time_duration time_of_day_start,
boost::posix_time::time_duration time_of_day_end);
SessionName session_name(datetime::PTime const &ref_date,
boost::gregorian::greg_weekday weekday_start,
boost::posix_time::time_duration time_of_day_start,
boost::gregorian::greg_weekday weekday_end,
boost::posix_time::time_duration time_of_day_end);
SessionName session_name(datetime::PTime const &ref_date,
boost::posix_time::time_duration time_of_day_start,
boost::posix_time::time_duration time_of_day_end);
SessionName session_name(config::JetFIX::Connection const &session_cfg);
std::string session_name (config::JetFIX::Connection const &session_cfg);
}}
} // namespace jetfix
} // namespace bts
+7 -7
View File
@@ -81,9 +81,9 @@ TEST_CASE ("JetFIX Field2Tests", "[jetfix]")
std::string out (20, ' ');
typedef StringField<123> Field;
REQUIRE(Field::max_value_size ("hello")==5);
REQUIRE(Field::max_value_size (std::string_view{"hello"})==5);
auto result = Field::write (&out[0],"hello");
auto result = Field::write (&out[0],std::string_view{"hello"});
REQUIRE (result.first==5);
REQUIRE (result.second=='h'+'e'+'l'+'l'+'o');
REQUIRE (out.compare (0,5,"hello") == 0);
@@ -112,7 +112,7 @@ TEST_CASE ("JetFIX FIXFragmentTests" , "[jetfix]")
FIXFragment g (0);
g.add<CharField<1>> ('a');
g.add<FloatField<2>> (1.2,-1,6);
g.add<StringField<3>> ("hello");
g.add<StringField<3>> (std::string_view{"hello"});
g.add<IntField<4>> (123,3);
const char*t_s = "20150427-19:05:18.922";
g.add<TimeField<5>> (datetime::parse_FIX_timestamp (t_s));
@@ -141,14 +141,14 @@ TEST_CASE ("JetFIX FIXFragmentTests" , "[jetfix]")
FIXFragment g;
auto c_bm = g.add<CharField<1>> ('a');
auto f_bm = g.add<FloatField<2>> (1.2,-1,6);
auto s_bm = g.add<StringField<3>> ("hello");
auto s_bm = g.add<StringField<3>> (std::string_view{"hello"});
auto i_bm = g.add<IntField<4>> (123,3);
auto t_bm = g.add<TimeField<5>> (datetime::parse_FIX_timestamp (t_s1));
auto b_bm = g.add<BoolField<6>> (true);
g.overwrite (c_bm,'b');
g.overwrite (f_bm,1.3,-1,6);
g.overwrite (s_bm,"olleh");
g.overwrite (s_bm,std::string_view {"olleh"});
g.overwrite (i_bm,321,3);
g.overwrite (t_bm,(datetime::parse_FIX_timestamp (t_s2)));
g.overwrite (b_bm,false);
@@ -172,11 +172,11 @@ TEST_CASE ("JetFIX FIXFragmentTests" , "[jetfix]")
FIXFragment g1;;
g1.add<CharField<1>> ('a');
g1.add<FloatField<2>> (1.2,-1,6);
g1.add<StringField<3>> ("hello");
g1.add<StringField<3>> (std::string_view{"hello"});
auto chk1 = g1.checksum ();
FIXFragment g2;
g2.add<StringField<4>> ("goodbye");
g2.add<StringField<4>> (std::string_view{"goodbye"});
g2.add<IntField<5>> (321,3);
auto chk2 = g2.checksum ();
+202 -211
View File
@@ -6,279 +6,270 @@
#include "lib/jetfix/common/OutFIXPtr.h"
#include "lib/jetfix/store/FIXFileStore.h"
#include "lib/jetfix/store/session_name.h"
#include "lib/logging/logging.h"
#include "lib/utils/TempDir.h"
#include "lib/utils/TimeStamp.h"
#include "lib/utils/datetime.h"
#include "lib/logging/logging.h"
#include <boost/date_time.hpp>
#include <boost/date_time/gregorian/gregorian.hpp>
#include <boost/filesystem.hpp>
#include <boost/date_time.hpp>
namespace bts {
namespace jetfix {
TEST_CASE ("JetFIX SessionNameTests", "[jetfix]")
{
SECTION("greg_weekday_works")
{
const char*days [] = {"sun","Monday","tuesday","Wedn","Thursday","Friday","sat"};
TEST_CASE("JetFIX SessionNameTests", "[jetfix]") {
SECTION("greg_weekday_works") {
const char *days[] = {"sun", "Monday", "tuesday", "Wedn",
"Thursday", "Friday", "sat"};
int i =0;
int i = 0;
for (auto d : days)
REQUIRE (parse_weekday (d)==boost::gregorian::greg_weekday (i++));
}
SECTION("closest_before_works")
{
auto same_day_before = closest_before (datetime::make_ptime (2015,7,17,17,0,0,0),
boost::gregorian::Friday,
boost::posix_time::time_duration(16, 59, 59));
REQUIRE (datetime::FIX_date_time_string (same_day_before)=="20150717-16:59:59");
auto same_day_after = closest_before (datetime::make_ptime (2015,7,17,17,0,0,0),
boost::gregorian::Friday,
boost::posix_time::time_duration(17,0,1));
REQUIRE (datetime::FIX_date_time_string (same_day_after)=="20150710-17:00:01");
auto another_day_same_week = closest_before (datetime::make_ptime (2015,7,17,17,0,0,0),
boost::gregorian::Thursday,
boost::posix_time::time_duration(16, 59, 59));
REQUIRE (datetime::FIX_date_time_string (another_day_same_week)=="20150716-16:59:59");
auto another_day_previous_week = closest_before (datetime::make_ptime (2015,7,17,17,0,0,0),
boost::gregorian::Saturday,
boost::posix_time::time_duration(16, 59, 59));
REQUIRE (datetime::FIX_date_time_string (another_day_previous_week)=="20150711-16:59:59");
REQUIRE(parse_weekday(d) == boost::gregorian::greg_weekday(i++));
}
SECTION("in_session_works")
{
REQUIRE (in_session (datetime::make_ptime (2015,7,17,17,0,0,0),
boost::gregorian::Sunday,
boost::posix_time::time_duration(16, 59, 59),
boost::gregorian::Friday,
boost::posix_time::time_duration(17, 0, 1)));
SECTION("closest_before_works") {
auto same_day_before =
closest_before(datetime::make_ptime(2015, 7, 17, 17, 0, 0, 0),
boost::gregorian::Friday,
boost::posix_time::time_duration(16, 59, 59));
REQUIRE(datetime::FIX_date_time_string(same_day_before) ==
"20150717-16:59:59");
REQUIRE (!in_session (datetime::make_ptime (2015,7,17,17,0,0,0),
boost::gregorian::Sunday,
boost::posix_time::time_duration(16, 59, 59),
boost::gregorian::Friday,
boost::posix_time::time_duration(16, 59, 59)));
auto same_day_after = closest_before(
datetime::make_ptime(2015, 7, 17, 17, 0, 0, 0),
boost::gregorian::Friday, boost::posix_time::time_duration(17, 0, 1));
REQUIRE(datetime::FIX_date_time_string(same_day_after) ==
"20150710-17:00:01");
REQUIRE (in_session (datetime::make_ptime (2015,7,17,17,0,0,0),
boost::posix_time::time_duration(16, 59, 59),
boost::posix_time::time_duration(17, 59, 59)));
auto another_day_same_week =
closest_before(datetime::make_ptime(2015, 7, 17, 17, 0, 0, 0),
boost::gregorian::Thursday,
boost::posix_time::time_duration(16, 59, 59));
REQUIRE(datetime::FIX_date_time_string(another_day_same_week) ==
"20150716-16:59:59");
REQUIRE (!in_session (datetime::make_ptime (2015,7,17,17,0,0,0),
boost::posix_time::time_duration(0,0,0),
boost::posix_time::time_duration(16, 0,0)));
REQUIRE (in_session (datetime::make_ptime (2015,7,17,17,0,0,0),
boost::posix_time::time_duration(17, 50, 59),
boost::posix_time::time_duration(17, 45, 59)));
REQUIRE (!in_session (datetime::make_ptime (2015,7,17,17,47,0,0),
boost::posix_time::time_duration(17, 50, 59),
boost::posix_time::time_duration(17, 45, 59)));
auto another_day_previous_week =
closest_before(datetime::make_ptime(2015, 7, 17, 17, 0, 0, 0),
boost::gregorian::Saturday,
boost::posix_time::time_duration(16, 59, 59));
REQUIRE(datetime::FIX_date_time_string(another_day_previous_week) ==
"20150711-16:59:59");
}
SECTION("session_name_works")
{
REQUIRE (session_name (datetime::make_ptime (2015,7,17,17,0,0,0),
boost::gregorian::Sunday,
boost::posix_time::time_duration(16, 59, 59),
boost::gregorian::Friday,
boost::posix_time::time_duration(17, 0, 1)) ==
"2015-07-12");
SECTION("in_session_works") {
REQUIRE(in_session(datetime::make_ptime(2015, 7, 17, 17, 0, 0, 0),
boost::gregorian::Sunday,
boost::posix_time::time_duration(16, 59, 59),
boost::gregorian::Friday,
boost::posix_time::time_duration(17, 0, 1)));
REQUIRE (session_name (datetime::make_ptime (2015,7,17,17,0,0,0),
boost::gregorian::Sunday,
boost::posix_time::time_duration(16, 59, 59),
boost::gregorian::Friday,
boost::posix_time::time_duration(16, 59, 59)) == "");
REQUIRE(!in_session(datetime::make_ptime(2015, 7, 17, 17, 0, 0, 0),
boost::gregorian::Sunday,
boost::posix_time::time_duration(16, 59, 59),
REQUIRE (session_name (datetime::make_ptime (2015,7,17,17,0,0,0),
boost::posix_time::time_duration(16, 59, 59),
boost::posix_time::time_duration(17, 59, 59)) ==
"2015-07-17");
boost::gregorian::Friday,
boost::posix_time::time_duration(16, 59, 59)));
REQUIRE (session_name (datetime::make_ptime (2015,7,17,18,0,0,0),
boost::posix_time::time_duration(16, 59, 59),
boost::posix_time::time_duration(17, 59, 59)) ==
"");
REQUIRE(in_session(datetime::make_ptime(2015, 7, 17, 17, 0, 0, 0),
boost::posix_time::time_duration(16, 59, 59),
boost::posix_time::time_duration(17, 59, 59)));
REQUIRE (session_name (datetime::make_ptime (2015,7,17,17,0,0,0),
boost::posix_time::time_duration(17, 50, 59),
boost::posix_time::time_duration(17, 45, 59)) ==
"2015-07-16");
REQUIRE(!in_session(datetime::make_ptime(2015, 7, 17, 17, 0, 0, 0),
boost::posix_time::time_duration(0, 0, 0),
boost::posix_time::time_duration(16, 0, 0)));
REQUIRE (session_name (datetime::make_ptime (2015,7,17,17,51,0,0),
boost::posix_time::time_duration(17, 50, 59),
boost::posix_time::time_duration(17, 45, 59)) ==
"2015-07-17");
REQUIRE (session_name (datetime::make_ptime (2015,7,17,17,47,0,0),
boost::posix_time::time_duration(17, 50, 59),
boost::posix_time::time_duration(17, 45, 59)) ==
"");
REQUIRE(in_session(datetime::make_ptime(2015, 7, 17, 17, 0, 0, 0),
boost::posix_time::time_duration(17, 50, 59),
boost::posix_time::time_duration(17, 45, 59)));
REQUIRE(!in_session(datetime::make_ptime(2015, 7, 17, 17, 47, 0, 0),
boost::posix_time::time_duration(17, 50, 59),
boost::posix_time::time_duration(17, 45, 59)));
}
SECTION("session_name_works") {
REQUIRE(session_name(datetime::make_ptime(2015, 7, 17, 17, 0, 0, 0),
boost::gregorian::Sunday,
boost::posix_time::time_duration(16, 59, 59),
boost::gregorian::Friday,
boost::posix_time::time_duration(17, 0, 1)).as_string ==
"2015-07-12");
REQUIRE(session_name(datetime::make_ptime(2015, 7, 17, 17, 0, 0, 0),
boost::gregorian::Sunday,
boost::posix_time::time_duration(16, 59, 59),
boost::gregorian::Friday,
boost::posix_time::time_duration(16, 59, 59)).as_string == "");
REQUIRE(session_name(datetime::make_ptime(2015, 7, 17, 17, 0, 0, 0),
boost::posix_time::time_duration(16, 59, 59),
boost::posix_time::time_duration(17, 59, 59)).as_string ==
"2015-07-17");
REQUIRE(session_name(datetime::make_ptime(2015, 7, 17, 18, 0, 0, 0),
boost::posix_time::time_duration(16, 59, 59),
boost::posix_time::time_duration(17, 59, 59)).as_string == "");
REQUIRE(session_name(datetime::make_ptime(2015, 7, 17, 17, 0, 0, 0),
boost::posix_time::time_duration(17, 50, 59),
boost::posix_time::time_duration(17, 45, 59)).as_string ==
"2015-07-16");
REQUIRE(session_name(datetime::make_ptime(2015, 7, 17, 17, 51, 0, 0),
boost::posix_time::time_duration(17, 50, 59),
boost::posix_time::time_duration(17, 45, 59)).as_string ==
"2015-07-17");
REQUIRE(session_name(datetime::make_ptime(2015, 7, 17, 17, 47, 0, 0),
boost::posix_time::time_duration(17, 50, 59),
boost::posix_time::time_duration(17, 45, 59)).as_string == "");
}
}
TEST_CASE ("JetFIX FileStoreTests", "[jetfix]")
{
SECTION("empty_store_works")
{
TEST_CASE("JetFIX FileStoreTests", "[jetfix]") {
SECTION("empty_store_works") {
config::JetFIX::Connection c;
c.set_name ("test");
c.set_sender_compid ("CME");
c.set_sender_subid ("G");
c.set_target_compid ("1U7001N");
c.set_target_subid ("BLUESTREAM");
c.set_name("test");
c.set_sender_compid("CME");
c.set_sender_subid("G");
c.set_target_compid("1U7001N");
c.set_target_subid("BLUESTREAM");
c.set_start_time ("00:00:00");
c.set_end_time ("23:59:59");
c.set_start_time("00:00:00");
c.set_end_time("23:59:59");
utils::TempDir td;
FIXFileStore fs (c,td.name ());
REQUIRE (fs.session_active ());
REQUIRE (fs.replay ().size ()==0);
REQUIRE (fs.last_seq_num ()==0);
REQUIRE (fs.last_seq_num_processed ()==0);
FIXFileStore fs(c, td.name());
REQUIRE(fs.session_active());
REQUIRE(fs.replay().size() == 0);
REQUIRE(fs.last_seq_num() == 0);
REQUIRE(fs.last_seq_num_processed() == 0);
}
SECTION("in_msg_is_stored")
{
auto td = std::make_shared<utils::TempDir> ();
SECTION("in_msg_is_stored") {
auto td = std::make_shared<utils::TempDir>();
config::JetFIX::Connection c;
c.set_name ("test");
c.set_sender_compid ("CME");
c.set_sender_subid ("G");
c.set_target_compid ("1U7001N");
c.set_target_subid ("BLUESTREAM");
c.set_name("test");
c.set_sender_compid("CME");
c.set_sender_subid("G");
c.set_target_compid("1U7001N");
c.set_target_subid("BLUESTREAM");
c.set_start_time ("00:00:00");
c.set_end_time ("23:59:59");
c.set_start_time("00:00:00");
c.set_end_time("23:59:59");
FIXEncodingStream stream (c);
auto &buf = stream.new_msg (MessageTypes::Logon);
buf.add<fix::TargetLocationID> ("US,NC");
buf.add<fix::EncryptMethod> (0);
FIXEncodingStream stream(c);
auto &buf = stream.new_msg(MessageTypes::Logon);
buf.add<fix::TargetLocationID>(std::string_view{"US,NC"});
buf.add<fix::EncryptMethod>(0);
buf.add<fix::HeartBtInt>(30);
buf.add<fix::ApplicationSystemName> ("O2");
buf.add<fix::ApplicationSystemVersion> ("1.0");
buf.add<fix::ApplicationSystemVendor> ("BTS");
stream.close ();
stream.stamp (2,3,datetime::FIX_timestamp (datetime::now ()));
auto fix = stream.to_str ();
buf.add<fix::ApplicationSystemName>(std::string_view{"O2"});
buf.add<fix::ApplicationSystemVersion>(std::string_view{"1.0"});
buf.add<fix::ApplicationSystemVendor>(std::string_view{"BTS"});
stream.close();
stream.stamp(2, 3, datetime::FIX_timestamp(datetime::now()));
auto fix = stream.to_str();
{
FIXFileStore fs (c,td->name ());
fs.store (InMsgBuffer (fix));
fs.set_last_msg_processed (3);
FIXFileStore fs(c, td->name());
fs.store(InMsgBuffer(fix));
fs.set_last_msg_processed(3);
}
FIXFileStore fs (c,td->name ());
auto replay = fs.replay ();
REQUIRE (replay.size ()==1);
REQUIRE (replay[0].type ()==MessageTypes::Logon);
REQUIRE (replay[0].to_str ()==fix);
REQUIRE (fs.last_seq_num ()==0);
REQUIRE (fs.last_seq_num_processed ()==3);
FIXFileStore fs(c, td->name());
auto replay = fs.replay();
REQUIRE(replay.size() == 1);
REQUIRE(replay[0].type() == MessageTypes::Logon);
REQUIRE(replay[0].to_str() == fix);
REQUIRE(fs.last_seq_num() == 0);
REQUIRE(fs.last_seq_num_processed() == 3);
}
SECTION("out_msg_is_stored")
{
auto td = std::make_shared<utils::TempDir> ();
SECTION("out_msg_is_stored") {
auto td = std::make_shared<utils::TempDir>();
config::JetFIX::Connection c;
c.set_name ("test");
c.set_sender_compid ("CME");
c.set_sender_subid ("G");
c.set_target_compid ("1U7001N");
c.set_target_subid ("BLUESTREAM");
c.set_name("test");
c.set_sender_compid("CME");
c.set_sender_subid("G");
c.set_target_compid("1U7001N");
c.set_target_subid("BLUESTREAM");
c.set_start_time ("00:00:00");
c.set_end_time ("23:59:59");
c.set_start_time("00:00:00");
c.set_end_time("23:59:59");
auto msg_vec = std::make_shared<OutFIX>(c);
auto &stream = msg_vec->fix.back ();
auto &buf = stream.new_msg (MessageTypes::Logon);
buf.add<fix::TargetLocationID> ("US,NC");
buf.add<fix::EncryptMethod> (0);
auto &stream = msg_vec->fix.back();
auto &buf = stream.new_msg(MessageTypes::Logon);
buf.add<fix::TargetLocationID>(std::string_view{"US,NC"});
buf.add<fix::EncryptMethod>(0);
buf.add<fix::HeartBtInt>(30);
buf.add<fix::ApplicationSystemName> ("O2");
buf.add<fix::ApplicationSystemVersion> ("1.0");
buf.add<fix::ApplicationSystemVendor> ("BTS");
stream.close ();
stream.stamp (2,3,datetime::FIX_timestamp (datetime::now ()));
buf.add<fix::ApplicationSystemName>(std::string_view{"O2"});
buf.add<fix::ApplicationSystemVersion>(std::string_view{"1.0"});
buf.add<fix::ApplicationSystemVendor>(std::string_view{"BTS"});
stream.close();
stream.stamp(2, 3, datetime::FIX_timestamp(datetime::now()));
{
FIXFileStore fs (c,td->name ());
fs.store (msg_vec);
FIXFileStore fs(c, td->name());
fs.store(msg_vec);
}
FIXFileStore fs (c,td->name ());
auto replay = fs.replay ();
REQUIRE (replay.size ()==0);
REQUIRE (fs.last_seq_num ()==3);
REQUIRE (fs.last_seq_num_processed ()==0);
FIXFileStore fs(c, td->name());
auto replay = fs.replay();
REQUIRE(replay.size() == 0);
REQUIRE(fs.last_seq_num() == 3);
REQUIRE(fs.last_seq_num_processed() == 0);
}
SECTION("test_performance")
{
auto td = std::make_shared<utils::TempDir> ();
SECTION("test_performance") {
auto td = std::make_shared<utils::TempDir>();
config::JetFIX::Connection c;
c.set_name ("test");
c.set_sender_compid ("CME");
c.set_sender_subid ("G");
c.set_target_compid ("1U7001N");
c.set_target_subid ("BLUESTREAM");
c.set_name("test");
c.set_sender_compid("CME");
c.set_sender_subid("G");
c.set_target_compid("1U7001N");
c.set_target_subid("BLUESTREAM");
c.set_start_time ("00:00:00");
c.set_end_time ("23:59:59");
c.set_start_time("00:00:00");
c.set_end_time("23:59:59");
auto start = utils::TimeStamp::now ();
auto start = utils::TimeStamp::now();
{
FIXFileStore fs (c,td->name ());
for (int i = 0; i < 10000;i++) {
FIXFileStore fs(c, td->name());
for (int i = 0; i < 10000; i++) {
auto msg_vec = std::make_shared<std::vector<FIXEncodingStream>>();
msg_vec->emplace_back (c);
auto &stream = msg_vec->back ();
auto &buf = stream.new_msg (MessageTypes::Logon);
buf.add<fix::TargetLocationID> ("US,NC");
buf.add<fix::EncryptMethod> (0);
msg_vec->emplace_back(c);
auto &stream = msg_vec->back();
auto &buf = stream.new_msg(MessageTypes::Logon);
buf.add<fix::TargetLocationID>(std::string_view{"US,NC"});
buf.add<fix::EncryptMethod>(0);
buf.add<fix::HeartBtInt>(30);
buf.add<fix::ApplicationSystemName> ("O2");
buf.add<fix::ApplicationSystemVersion> ("1.0");
buf.add<fix::ApplicationSystemVendor> ("BTS");
stream.close ();
stream.stamp (i,3,datetime::FIX_timestamp (datetime::now () - boost::gregorian::days (1)));
fs.store (InMsgBuffer (stream.to_str ()));
buf.add<fix::ApplicationSystemName>(std::string_view{"O2"});
buf.add<fix::ApplicationSystemVersion>(std::string_view{"1.0"});
buf.add<fix::ApplicationSystemVendor>(std::string_view{"BTS"});
stream.close();
stream.stamp(i, 3,
datetime::FIX_timestamp(datetime::now() -
boost::gregorian::days(1)));
fs.store(InMsgBuffer(stream.to_str()));
}
}
auto stop = utils::TimeStamp::now ();
LOG(Testing,Info,"jetfix store gen: {} ms", (stop-start).milliseconds () );
start = utils::TimeStamp::now ();
FIXFileStore fs (c,td->name ());
stop = utils::TimeStamp::now ();
LOG(Testing,Info,"jetfix store load: {} ms", (stop-start).milliseconds () );
auto stop = utils::TimeStamp::now();
LOG(Testing, Info, "jetfix store gen: {} ms",
(stop - start).milliseconds());
start = utils::TimeStamp::now();
FIXFileStore fs(c, td->name());
stop = utils::TimeStamp::now();
LOG(Testing, Info, "jetfix store load: {} ms",
(stop - start).milliseconds());
}
}
}}
} // namespace jetfix
} // namespace bts
+1
View File
@@ -120,6 +120,7 @@ void configure_logging(bts::config::Logging const&cfg, bts::config::ThreadGroupC
{log::Plugin,"Plugin"},
{log::Sdk,"Sdk"},
{log::Http,"Http"},
{log::iLink3,"iLink3"},
};
real_cfg.sinks["console"] =
+1
View File
@@ -46,6 +46,7 @@ constexpr log::Subsystem DataFlow {Latency.ordinal+1 };
constexpr log::Subsystem Plugin {DataFlow.ordinal+1 };
constexpr log::Subsystem Sdk {Plugin.ordinal+1 };
constexpr log::Subsystem Http {Sdk.ordinal+1 };
constexpr log::Subsystem iLink3 {Http.ordinal+1 };
struct LoggerHandle {
LoggerHandle(LoggerHandle const&) = delete;
+5
View File
@@ -0,0 +1,5 @@
#pragma once
#include "lib/utils/ValueWrapper.h"
namespace bts {
using ExchangeSecurityKey = utils::ValueWrapper<struct ExchangeSecurityKeyTag,uint64_t>;
}
+28 -3
View File
@@ -60,10 +60,13 @@ class IntPrice {
return delta.mantissa() < 0;
}
}
bool operator <=(const IntPrice& other) {
bool operator <=(const IntPrice& other) const {
return *this == other || *this < other;
}
bool operator >=(const IntPrice& other) {
bool operator >(const IntPrice& other) const {
return other < *this;
}
bool operator >=(const IntPrice& other) const {
return !(*this < other);
}
IntPrice operator - (const IntPrice& o) const {
@@ -72,9 +75,31 @@ class IntPrice {
IntPrice operator + (const IntPrice& o) const {
return op(o, [](int64_t m1, int64_t m2) { return m1 + m2; });
}
IntPrice& operator +=(const IntPrice& o) {
auto temp = *this + o;
_mantissa = temp._mantissa;
_exponent = temp._exponent;
return *this;
}
IntPrice& operator -=(const IntPrice& o) {
auto temp = *this - o;
_mantissa = temp._mantissa;
_exponent = temp._exponent;
return *this;
}
IntPrice operator -() {
return IntPrice(-_mantissa, _exponent);
}
IntPrice operator * (const IntPrice& other) const {
return op(other, [](int64_t m1, int64_t m2) { return m1 * m2; });
}
IntPrice operator * (const int& other) const {
return IntPrice(_mantissa * other, _exponent);
}
IntPrice& operator *=(const int& other) {
_mantissa *= other;
return *this;
}
IntPrice operator / (double v) const {
return { (int64_t)(_mantissa / v), exponent() };
}
@@ -183,4 +208,4 @@ inline std::ostream& operator<<(std::ostream& os, const IntPrice& p) {
return os;
}
}}
}}
+172 -166
View File
@@ -1,71 +1,72 @@
#ifndef _WIN32
#include "MBOServiceServer.h"
#include "MBOServiceServer.h"
#include "lib/api/services/OrderService.client.pb.h"
#include "lib/utils/Dispatcher.h"
#include "lib/utils/TimeStamp.h"
#include "lib/domain_model/Security.h"
#include "MDClient.h"
#include "ExchangeOrderKey.h"
#include "MBOProcessor.h"
#include "MDClient.h"
#include "lib/api/services/OrderService.client.pb.h"
#include "lib/domain_model/Security.h"
#include "lib/orderman/orderman_utils/orderman_common.h"
#include "lib/utils/Dispatcher.h"
#include "lib/utils/TimeStamp.h"
#include "lib/utils/LockBox.h"
#include <tuple>
#include <vector>
#include <unordered_map>
#include <vector>
using namespace bts::api::orderman;
namespace bts {
namespace md {
namespace bts {
namespace md {
//
// Simple class to handle MD recovery and maintain an index of SecurityKey=>MDConsumerHandle
// Simple class to handle MD recovery and maintain an index of
// SecurityKey=>MDConsumerHandle
class HandleIndex {
MDClient& _client;
MDClient &_client;
int _sub;
std::optional<MDConsumer> _consumer;
public:
utils::LockBox<std::unordered_map<SecurityKey,MDConsumerHandle>> index;
public:
utils::LockBox<std::unordered_map<SecurityKey, MDConsumerHandle>> index;
HandleIndex(md::MDClient& client):
_client(client),
_sub(0),
_consumer(std::nullopt),
index() {
_sub = _client.on_consumer().subscribe([=](const MDConsumer& c) { on_consumer(c); });
on_consumer(_client.current_consumer());
}
HandleIndex(md::MDClient &client)
: _client(client), _sub(0), _consumer(std::nullopt), index() {
_sub = _client.on_consumer().subscribe(
[=](const MDConsumer &c) { on_consumer(c); });
on_consumer(_client.current_consumer());
}
~HandleIndex() { _client.on_consumer().unsubscribe(_sub); }
private:
void on_consumer(const MDConsumer& c) {
private:
void on_consumer(const MDConsumer &c) {
MDConsumer temp(c);
if (temp.all_heartbeats_ok()) {
auto existing = _consumer ? _consumer->id() : 0;
LOG(md,Info, "Updateing MBOService MDConsumer. Old {} new {}", existing, temp.id());
LOG(md, Info, "Updateing MBOService MDConsumer. Old {} new {}",
existing, temp.id());
// clear the index
index.lock([](auto& index) { index.clear(); });
index.lock([](auto &index) { index.clear(); });
// Swap to new consumer
if (_consumer) {
_consumer->swap(temp);
} else {
_consumer = temp;
}
else { _consumer = temp; }
// Subscribe to on_security
auto id = _consumer->id();
_consumer->on_security([this,id](const SecurityPtr&, MDConsumerHandle handle) {
if (_consumer->id() == id) {
index.lock([handle](auto& index) {
index[SecurityKey(handle.key())] = handle;
});
}
});
_consumer->on_security(
[this, id](const SecurityPtr &, MDConsumerHandle handle) {
if (_consumer->id() == id) {
index.lock([handle](auto &index) {
index[SecurityKey(handle.key())] = handle;
});
}
});
}
}
};
@@ -73,21 +74,20 @@ private:
class ActiveOrder {
SecurityKey _security;
SparkOrderKey _id;
ExchangeOrderKey _ex_id;
ExchangeOrderKey _ex_id;
int32_t _position;
public:
public:
// empty constructor needed for unordered map operator[]
// ActiveOrder(): _security(), _id(), _ex_id(), _position(-1) { bts_assert(false); }
ActiveOrder(SecurityKey sec, SparkOrderKey s, ExchangeOrderKey ex): _security(sec), _id(s), _ex_id(ex), _position(-1) {}
ActiveOrder(SecurityKey sec, SparkOrderKey s, const std::string& ex): _security(sec), _id(s), _ex_id(), _position(-1) {
_ex_id = ExchangeOrderKey(std::stoul(ex));
}
// ActiveOrder(): _security(), _id(), _ex_id(), _position(-1) {
// bts_assert(false); }
ActiveOrder(SecurityKey sec, SparkOrderKey s, ExchangeOrderKey ex)
: _security(sec), _id(s), _ex_id(ex), _position(-1) {}
SecurityKey security_id() const { return _security; }
SparkOrderKey id() const { return _id; }
ExchangeOrderKey exchange_id() const { return _ex_id; }
int32_t& position() { return _position; };
int32_t &position() { return _position; };
int32_t position() const { return _position; };
bool set_position(int32_t pos) {
if (pos != _position) {
@@ -108,23 +108,14 @@ public:
// MBOService
struct MBOServiceImpl {
MBOServiceImpl(
api::ServiceContext& ctx,
utils::Dispatcher& d,
md::MDClient& c,
MBOServiceBase::PiqPublisher&& pub
):
dispatcher(d),
client(c),
orders(ctx.locate<api::orderman::OrderService>()),
handle_index(c),
publish(pub),
timer(d, 200, [&]() {timer_tick();}),
orders_sub()
{}
MBOServiceImpl(api::ServiceContext &ctx, utils::Dispatcher &d,
md::MDClient &c, MBOServiceBase::PiqPublisher &&pub)
: dispatcher(d), client(c),
orders(ctx.locate<api::orderman::OrderService>()), handle_index(c),
publish(pub), timer(d, 200, [&]() { timer_tick(); }), orders_sub() {}
utils::Dispatcher& dispatcher;
md::MDClient& client;
utils::Dispatcher &dispatcher;
md::MDClient &client;
OrderService orders;
HandleIndex handle_index;
@@ -138,23 +129,23 @@ struct MBOServiceImpl {
void flush();
};
///////////////////////
std::optional<uint32_t> search_orders(const ExchangeOrderKey oid, const std::vector<SnapshotOrder>& orders) {
std::optional<uint32_t>
search_orders(const ExchangeOrderKey oid,
const std::vector<SnapshotOrder> &orders) {
IntPrice price = {};
size_t position = 0;
for (size_t x = 0; x < orders.size(); ++x) {
const auto& oo = orders[x];
const auto &oo = orders[x];
if (oo.id == oid) {
if (oo.price == price) return { position };
else return 0;
}
else if (price == oo.price) {
if (oo.price == price)
return {position};
else
return 0;
} else if (price == oo.price) {
position += oo.size;
}
else {
} else {
price = oo.price;
position = oo.size;
}
@@ -162,11 +153,11 @@ std::optional<uint32_t> search_orders(const ExchangeOrderKey oid, const std::vec
return std::nullopt;
}
bool update_position(const HandleIndex& handles, ActiveOrder& order) {
const auto& index = handles.index;
auto handle = index.lock([id=order.security_id()](const auto& index) {
bool update_position(const HandleIndex &handles, ActiveOrder &order) {
const auto &index = handles.index;
auto handle = index.lock([id = order.security_id()](const auto &index) {
auto iter = index.find(id);
return iter != index.end() ? iter->second : MDConsumerHandle {};
return iter != index.end() ? iter->second : MDConsumerHandle{};
});
if (handle.is_initialized()) {
@@ -185,9 +176,9 @@ bool update_position(const HandleIndex& handles, ActiveOrder& order) {
}
void MBOServiceImpl::timer_tick() {
//Dispatcher thread
// Dispatcher thread
std::vector<OrderPIQ> rows;
for (auto& pair : active) {
for (auto &pair : active) {
if (update_position(handle_index, pair.second)) {
rows.push_back(pair.second.to_piq());
}
@@ -197,29 +188,30 @@ void MBOServiceImpl::timer_tick() {
}
}
const NativeOrderState* native_order_state(const OrderRecord& record) {
switch(record.state().custom_case()) {
case OrderState::kMarket:
return &record.state().market().native_state();
case OrderState::kLimit:
return &record.state().limit().native_state();
case OrderState::kStop:
return &record.state().stop().native_state();
case OrderState::kStopLimit:
return &record.state().stop_limit().native_state();
case OrderState::kGlbxIceberg:
return &record.state().glbx_iceberg().native_state();
default:
const NativeOrderState *native_order_state(const OrderRecord &record) {
switch (record.state().custom_case()) {
case OrderState::kMarket:
return &record.state().market().native_state();
case OrderState::kLimit:
return &record.state().limit().native_state();
case OrderState::kStop:
return &record.state().stop().native_state();
case OrderState::kStopLimit:
return &record.state().stop_limit().native_state();
case OrderState::kGlbxIceberg:
return &record.state().glbx_iceberg().native_state();
default:
return nullptr;
}
}
bool is_active_exchange_order(const OrderRecord& record) {
bool is_active_exchange_order(const OrderRecord &record) {
return record.state().status() == ORDER_STATUS_WORKING &&
native_order_state(record) != nullptr;
}
void try_remove(MBOServiceImpl& me, std::set<uint32_t>& to_remove, const OrderRecord& record) {
void try_remove(MBOServiceImpl &me, std::set<uint32_t> &to_remove,
const OrderRecord &record) {
auto key = SparkOrderKey(record.key());
auto iter = me.active.find(key);
if (iter != me.active.end()) {
@@ -227,103 +219,113 @@ void try_remove(MBOServiceImpl& me, std::set<uint32_t>& to_remove, const OrderRe
to_remove.insert(key.value());
}
}
void try_add(MBOServiceImpl& me, std::vector<OrderPIQ>& to_add, const OrderRecord& record) {
void try_add(MBOServiceImpl &me, std::vector<OrderPIQ> &to_add,
const OrderRecord &record) {
if (is_active_exchange_order(record)) {
auto native = native_order_state(record);
auto ex_key = orderman::get_exchange_order_id(*native);
auto u64_key =
std::visit(overloaded{[](orderman::ExchangeOrderID const &str) {
return std::stoul(str.value().data());
},
[](orderman::ExchangeOrderID64 oid) {
return oid.value();
}},
ex_key);
bts_assert(native);
ActiveOrder order(
SecurityKey(record.def().security_key()),
SparkOrderKey(record.key()),
native->exchange_order_id());
ActiveOrder order(SecurityKey(record.def().security_key()),
SparkOrderKey(record.key()), ExchangeOrderKey{u64_key});
update_position(me.handle_index, order);
to_add.push_back(order.to_piq());
me.active.insert(std::make_pair(order.id(), order));
}
}
MBOServiceServer::MBOServiceServer(
api::ServiceContext& ctx,
utils::Dispatcher& d,
md::MDClient& c
) :
MBOServiceBase(ctx),
_me(new MBOServiceImpl(ctx,d,c,make_piq_publisher()))
{
// subscribe to mdclient ✔
// make publishers ✔
// subscribe to orders => stash active orders dict
_me->orders_sub =
_me->orders.subscribe_orders().subscribe([=](const auto& update) {
// api thread
const auto& added = update.added();
const auto& changed = update.changed();
const auto& removed = update.removed();
_me->dispatcher.post([this,added,changed,removed]() {
// dispatcher thread
std::vector<OrderPIQ> to_add, to_change;
std::set<uint32_t> to_remove;
MBOServiceServer::MBOServiceServer(api::ServiceContext &ctx,
utils::Dispatcher &d, md::MDClient &c)
: MBOServiceBase(ctx),
_me(new MBOServiceImpl(ctx, d, c, make_piq_publisher())) {
// subscribe to mdclient ✔
// make publishers ✔
// subscribe to orders => stash active orders dict
_me->orders_sub =
_me->orders.subscribe_orders().subscribe([=](const auto &update) {
// api thread
const auto &added = update.added();
const auto &changed = update.changed();
const auto &removed = update.removed();
_me->dispatcher.post([this, added, changed, removed]() {
// dispatcher thread
std::vector<OrderPIQ> to_add, to_change;
std::set<uint32_t> to_remove;
// Remove
for(const auto& r : removed) {
// If it was active remove it and add to to_remove
try_remove(*_me, to_remove, *r);
}
// Added
for(const auto& add : added) {
// If active add look up the position and add to the state
try_add(*_me, to_add, *add);
}
// Changed
for(const auto& ch : changed) {
// new order is active and old is not look up piq and add to state
bool is_active = is_active_exchange_order(*ch.current());
bool was_active = is_active_exchange_order(*ch.previous());
if (is_active && !was_active) {
try_add(*_me, to_add, *ch.current());
}
else if (was_active && !is_active) {
try_remove(*_me, to_remove, *ch.current());
}
// is_active && was_active => don't care
// !is_active && !was_active => don't care
}
// Remove
for (const auto &r : removed) {
// If it was active remove it and add to to_remove
try_remove(*_me, to_remove, *r);
}
// Added
for (const auto &add : added) {
// If active add look up the position and add to the state
try_add(*_me, to_add, *add);
}
// Changed
for (const auto &ch : changed) {
// new order is active and old is not look up piq and add to
// state
bool is_active = is_active_exchange_order(*ch.current());
bool was_active = is_active_exchange_order(*ch.previous());
if (is_active && !was_active) {
try_add(*_me, to_add, *ch.current());
} else if (was_active && !is_active) {
try_remove(*_me, to_remove, *ch.current());
}
// is_active && was_active => don't care
// !is_active && !was_active => don't care
}
_me->publish(&to_add, &to_change, &to_remove);
if (!_me->timer.running()) _me->timer.start();
});
});
}
_me->publish(&to_add, &to_change, &to_remove);
if (!_me->timer.running())
_me->timer.start();
});
});
}
MBOServiceServer::~MBOServiceServer() { }
MBOServiceServer::~MBOServiceServer() {}
void MBOServiceServer::get_orders_for_security(const MBOSecurityQuery& request, const api::Response<PositionData>& reply) {
void MBOServiceServer::get_orders_for_security(
const MBOSecurityQuery &request, const api::Response<PositionData> &reply) {
PositionData response;
response.set_security_id(request.security_id());
auto handle = _me->handle_index.index.lock([id=request.security_id()](const auto& index) {
auto iter = index.find(SecurityKey(id));
return iter != index.end() ? iter->second : MDConsumerHandle {};
});
auto handle = _me->handle_index.index.lock(
[id = request.security_id()](const auto &index) {
auto iter = index.find(SecurityKey(id));
return iter != index.end() ? iter->second : MDConsumerHandle{};
});
std::unordered_map<ExchangeOrderKey, SparkOrderKey> reverse;
for (const auto& pair : _me->active) {
for (const auto &pair : _me->active) {
reverse.insert(std::make_pair(pair.second.exchange_id(), pair.first));
}
size_t count = 0;
auto tick = handle.get_minimum_tick();
if(handle.is_initialized()) {
if (handle.is_initialized()) {
auto mbo = handle.get_mbo();
if (mbo) {
for (const auto& o : mbo->bids) {
for (const auto &o : mbo->bids) {
double price = o.price.price();
bool isnan = os::is_nan(request.price_level());
if (isnan || securities::test_price_equality(request.price_level(), price, tick)) {
if (isnan || securities::test_price_equality(request.price_level(),
price, tick)) {
++count;
if (request.limit() && count > request.limit()) break;
auto* entry = response.add_bid();
if (request.limit() && count > request.limit())
break;
auto *entry = response.add_bid();
entry->set_exchange_id(o.id.value());
if (isnan) entry->set_price(price);
if (isnan)
entry->set_price(price);
entry->set_size(o.size);
auto iter = reverse.find(o.id);
if (iter != reverse.end()) {
@@ -331,15 +333,18 @@ void MBOServiceServer::get_orders_for_security(const MBOSecurityQuery& request,
}
}
}
for (const auto& o : mbo->asks) {
for (const auto &o : mbo->asks) {
double price = o.price.price();
bool isnan = os::is_nan(request.price_level());
if (isnan || securities::test_price_equality(request.price_level(), price, tick)) {
if (isnan || securities::test_price_equality(request.price_level(),
price, tick)) {
++count;
if (request.limit() && count > request.limit()) break;
auto* entry = response.add_ask();
if (request.limit() && count > request.limit())
break;
auto *entry = response.add_ask();
entry->set_exchange_id(o.id.value());
if (isnan) entry->set_price(price);
if (isnan)
entry->set_price(price);
entry->set_size(o.size);
auto iter = reverse.find(o.id);
if (iter != reverse.end()) {
@@ -349,9 +354,10 @@ void MBOServiceServer::get_orders_for_security(const MBOSecurityQuery& request,
}
}
}
reply(response);
}
}} //namespace
} // namespace md
} // namespace bts
#endif
+1 -1
View File
@@ -92,7 +92,7 @@ MDConsumer& MDClient::current_consumer() const {
return _self->_consumer;
}
SecurityOracle MDClient::oracle() const {
return SecurityOracle(*this);
return SecurityOracle(_self->_cache);
}
std::set<UserSymbol> MDClient::symbols() const {
return _self->_symbols;
+2 -2
View File
@@ -897,7 +897,7 @@ public:
scan_securities(resolve_legs);
}
SecurityPtr get_security(SecurityKey key) const {
const Security* get_security(SecurityKey key) const {
return _cache.get_security(key);
}
@@ -979,7 +979,7 @@ size_t MDConsumer::walk_security_list(SecurityCallback cb, const string& symbol,
size_t MDConsumer::scan_securities(bool resolve_legs) { return _pimpl->scan_securities(resolve_legs); }
void MDConsumer::on_security(SecurityCallback cb, bool resolve_legs) { _pimpl->on_security(cb, resolve_legs); }
const api::Security* MDConsumer::get_security(SecurityKey key) const { return _pimpl->get_security(key).get(); }
const api::Security* MDConsumer::get_security(SecurityKey key) const { return _pimpl->get_security(key); }
std::vector<api::Security const*> MDConsumer::get_securities() const {return _pimpl->get_securities();}
MDConsumerChannel MDConsumer::get_channel(const std::string& symbol, SecurityType type, Venue exchange) const {
+66 -42
View File
@@ -1,8 +1,10 @@
#include "SecurityCache.h"
#include "SecurityDB.h"
#include "lib/utils/Dispatcher.h"
#include "lib/utils/LockBox.h"
#include <unordered_map>
#include <google/btree/btree_map.h>
#include <tbb/concurrent_queue.h>
#include <mutex>
@@ -10,8 +12,8 @@ namespace bts {
namespace md {
struct SecurityCacheImpl {
std::mutex cache_mutex;
std::unordered_map<SecurityKey, SecurityPtr> cache;
utils::LockBox<btree::btree_map<SecurityKey, SecurityPtr>> cache;
utils::LockBox<btree::btree_map<ExchangeSecurityKey, SecurityPtr>> ex_cache;
tbb::concurrent_queue<SecurityPtr> update_queue;
void enqueue_update_db(const SecurityPtr&);
void flush();
@@ -42,61 +44,83 @@ void SecurityCacheImpl::flush() {
SecurityCache::SecurityCache() : _me(std::make_shared<SecurityCacheImpl>()) {}
SecurityCache::~SecurityCache() {}
SecurityPtr SecurityCache::try_get_security(SecurityKey key) const {
std::lock_guard<std::mutex> lock(_me->cache_mutex);
auto iter = _me->cache.find(key);
if (iter != _me->cache.end()) {
return iter->second;
}
return nullptr;
const Security* SecurityCache::try_get_security(SecurityKey key) const {
return _me->cache.lock([key](btree::btree_map<SecurityKey, SecurityPtr>& t) {
auto iter = t.find(key);
if (iter != t.end()) {
return iter->second.get();
}
return (const api::Security*)nullptr;
});
}
const Security* SecurityCache::try_get_security(ExchangeSecurityKey key) const {
return _me->ex_cache.lock([key](btree::btree_map<ExchangeSecurityKey,SecurityPtr>& t) {
auto iter = t.find(key);
if (iter != t.end()) {
return iter->second.get();
}
return (const api::Security*)nullptr;
});
}
SecurityPtr SecurityCache::get_security(SecurityKey key) {
std::unique_lock<std::mutex> lock(_me->cache_mutex);
auto iter = _me->cache.find(key);
if (iter != _me->cache.end()) {
return iter->second;
}
else {
// unlock for db read
lock.unlock();
const Security* SecurityCache::get_security(SecurityKey key) {
const Security* result = try_get_security(key);
if (result == nullptr) {
// outside lock
auto security = db::find_security(key);
if (security) {
LOG(md,Info, "Loaded security from database: {} {}", security->symbol(), security->id());
return add_db_security(security);
}
return security;
return nullptr;
}
}
std::vector<api::Security const*> SecurityCache::get_securities()
{
std::vector<api::Security const*> result;
result.reserve(100000);
std::unique_lock<std::mutex> lock(_me->cache_mutex);
for (auto const&p : _me->cache)
result.push_back(p.second.get());
return result;
}
SecurityPtr SecurityCache::add_db_security(const SecurityPtr& security) {
std::vector<api::Security const*> SecurityCache::get_securities() {
std::vector<api::Security const*> result;
result.reserve(100000);
_me->cache.lock([&result](auto& cache) {
for (auto const&p : cache) {
result.push_back(p.second.get());
}
});
return result;
}
const Security* SecurityCache::add_db_security(const SecurityPtr& security) {
bts_assert(security);
std::lock_guard<std::mutex> lock(_me->cache_mutex);
SecurityKey key(security->id());
auto result = _me->cache.insert(std::make_pair(SecurityKey(security->id()), security));
// If result was inserted result.first->second is equal to security
// If the value was already present it must be a recently inserted 'live' security and we should use it
return result.first->second;
if (security->has_glbx()) {
_me->ex_cache.lock([&security](auto& cache) {
ExchangeSecurityKey key(security->id());
cache.insert(std::make_pair(key, security));
});
}
return _me->cache.lock([&security](auto& cache) {
SecurityKey key(security->id());
auto result = cache.insert(std::make_pair(key, security));
// If result was inserted result.first->second is equal to security
// If the value was already present it must be a recently inserted 'live' security and we should use it
return result.first->second.get();
});
}
bool SecurityCache::add_live_security(const SecurityPtr& security) {
bts_assert(security);
std::lock_guard<std::mutex> lock(_me->cache_mutex);
SecurityKey key(security->id());
// Insert but don't overwrite existing
auto result = _me->cache.insert(std::make_pair(SecurityKey(security->id()), security));
_me->enqueue_update_db(security);
return result.second;
if (security->has_glbx()) {
_me->ex_cache.lock([&security](auto& cache) {
ExchangeSecurityKey key(security->id());
cache.insert(std::make_pair(key, security));
});
}
return _me->cache.lock([&](auto& cache) {
SecurityKey key(security->id());
// Insert but don't overwrite existing
auto result = cache.insert(std::make_pair(SecurityKey(security->id()), security));
_me->enqueue_update_db(security);
return result.second;
});
}
void SecurityCache::flush_to_disk() { _me->flush(); }
+5 -3
View File
@@ -3,6 +3,7 @@
#include "lib/domain_model/Security.pb.h"
#include "MDConsumerHandle.h"
#include "lib/domain_model/SecurityKey.h"
#include "lib/md/ExchangeSecurityKey.h"
#include "SecurityPtr.h"
#include <memory>
@@ -20,17 +21,18 @@ public:
~SecurityCache();
//Load security from cache with fallback to database
SecurityPtr get_security(SecurityKey key);
const Security* get_security(SecurityKey key);
// Load all securities from cache, no fallback to database
std::vector<api::Security const*> get_securities();
// Get the security if it's in the cache otherwise return nullptr. Does not query security database.
SecurityPtr try_get_security(SecurityKey key) const;
const Security* try_get_security(SecurityKey key) const;
const Security* try_get_security(ExchangeSecurityKey key) const;
// returns true if the cache was updated
bool add_live_security(const SecurityPtr& security);
SecurityPtr add_db_security(const SecurityPtr& security);
const Security* add_db_security(const SecurityPtr& security);
void flush_to_disk();
-56
View File
@@ -1,56 +0,0 @@
#pragma once
#include "core/bts_core/utils/bts_no_copy.h"
#include "Shm/FixedSecurity.h"
#include "../bts_conn_exports.h"
#include <memory>
#include <functional>
#include <atomic>
namespace bts {
namespace md {
class BTS_EXPORT SecurityConstructor {
// An object "pool" optimized for the case where only one instance is ever used at a time
template <class T>
class SingleObjectPool<T> {
public:
using ptr_type = std::unique_ptr<T, std::function<void(T*)>>;
ptr_type acquire() {
T* pt = _inst ? _inst.release() : new T();
return ptr_type(pt, [this](T* ptr) {
return_inst(std::unique_ptr<T>(ptr));
});
}
private:
void return_inst(std::unique_ptr<T> t) {
if (!_inst) {
reset(t);
_inst = std::move(t);
}
}
unique_ptr<T> _inst;
};
public:
SecurityConstructor() : _future(), _option(), _spread() {}
SingleObjectPool<FixedFutureSecurity>& future() { return _future; }
SingleObjectPool<FixedOptionSecurity>& option() { return _option; }
SingleObjectPool<FixedSpreadSecurityL>& spread() { return _spread; }
private:
template <typename T>
void reset(T& t) {
new (&t)T();
}
SingleObjectPool<FixedFutureSecurity> _future;
SingleObjectPool<FixedOptionSecurity> _option;
SingleObjectPool<FixedSpreadSecurityL> _spread;
};
}}//namespace
+7 -3
View File
@@ -4,14 +4,18 @@
namespace bts {
namespace md {
SecurityOracle::SecurityOracle(const MDClient& client): _client(client) {}
SecurityOracle::SecurityOracle(SecurityCache& cache): _cache(cache) {}
SecurityOracle::~SecurityOracle() { }
const api::Security* SecurityOracle::get_security(SecurityKey key) const {
return _client.current_consumer().get_security(key);
return _cache.get_security(key);
}
const api::Security* SecurityOracle::get_security(ExchangeSecurityKey key) const {
return _cache.try_get_security(key);
}
std::vector<api::Security const*> SecurityOracle::get_securities() const {
return _client.current_consumer().get_securities();
return _cache.get_securities();
}
}}//namespace
+5 -2
View File
@@ -1,6 +1,8 @@
#pragma once
#include "lib/domain_model/SecurityKey.h"
#include "lib/md/ExchangeSecurityKey.h"
#include "lib/md/SecurityCache.h"
#include <memory>
#include <vector>
@@ -9,11 +11,12 @@ namespace api {class Security;}
namespace md {
class MDClient;
class SecurityOracle {
const MDClient& _client;
SecurityCache& _cache;
public:
SecurityOracle(const MDClient&);
SecurityOracle(SecurityCache&);
~SecurityOracle();
const api::Security* get_security(SecurityKey key) const;
const api::Security* get_security(ExchangeSecurityKey key) const;
std::vector<api::Security const*> get_securities() const;
};
+209
View File
@@ -0,0 +1,209 @@
# Overview
The core abstraction that everything else is built on is the MDProducer+MDProducerHandle / MDConsumer+MDConsumerHandle.
A new market data source constructs an MDProducer and adds securities to it. When a security is added you get back an MDProducerHandle representing that security in shared memory. The memory is fully allocated and books can be written immidiately but the security will not be visible to the MDConsumer until the security key is stored on disk (SecurityIDRecorder). The MDProducerHandle assumes a **single writer** for most operations and *may break silently if this constraint is violated*.
The MDConsumer is the dual of the MDProducer. It provides an on_security() callback which is driven by the call to scan_securities(). No threads are created by MDConsumer/MDProducer instead they get cpu time from periodic calls to scan_securities (consumer), SecurityIDRecorder::do_work (producer). When a security is discovered by scan_securities the callback is called with the Security and the MDConsumerHandle. The MDConsumerHandle is thread safe and can be safely copied and read from concurrently.
## Single Writer Principle
- https://mechanical-sympathy.blogspot.com/2011/09/single-writer-principle.html
# Core Classes
'*' = New in spark
* MBOProcessor*
* MBOServiceServer*
* MDClient*
* MDConsumer
* MDConsumerHandle
* MDProducer
* MDProducerHandle
* PlatformLoader
* PriceSize/IntPrice
* SecurityCache*
* SecurityOracle*
* SecurityDB*
* SecurityIDRecorder
* SecurityMapper*
* SettlementRecorder
* SyntheticSecurityService
* FixedSecurity.h / MDStructure.h
* SharedList.h
## MBOProcessor*
This class is an optional dependency of MDProducer. It takes the low level incremental updates to the MBO book and constructs the book on a different thread. This is optimized for the UI use case so it tries to get the mbo messages out of the critical path as soon as possible. This uses regular heap memory so is not compatible with out of process operation. It exposes a reader and writer interface.
Each update goes into a queue and gets processed by MBOProcessorImpl::process_incremental() on a single background thread. This thread drains the queue and constructs the books in place. Then it copies snapshots of the books and publishes them as immutable snapshots which are consumed by the MBOServiceServer/UI. If mbo load gets very high this function will stop publishing snapshots and only update books until the load reduces.
## MBOServiceServer*
Reads the snapshots from the MBOProcessor/MBOReader and serves them for the UI.
## MDClient*
New in spark.
This class handles constructing the MDConsumer, calling scan_securities on it and checking for bad heartbeats and re-constructing the MDConsumer if there was a failure. In edge those tasks were the responsibility of each consumer of the market data but it was mitigated because most things used streams rather than MDConsumer directly. MDConsumer is reference counted internally so when the consuming code gets an on_consumer() callback with a new MDConsumer it should keep a copy of the old MDConsumer until it has finished any asyncronous processes involving the MDConsumerHandles from that first consumer. Once an MDConsumer is destroyed the MDConsumerHandles it produced will cause an **Access Violation** on reading because the memory has been unmapped.
Spark has always used MD in process so, while based on existing functionality, this recovery process hasn't been well tested.
## MDConsumer
An MDConsumer maps the shared memory sections and produces MDConsumerHandles. These handles are only valid as long as the MDConsumer exists. It would make some sense to store the MDConsumer reference in the MDConsumerHandle to avoid this but MDConsumerHandle is designed to be *very* small for fast iteration. The MDConsumer also holds a reference to the ISyntheticSecurityService and makes synthetic securities 'look like' regular shared memory securities using polymorphism in the MDConsumerHandle.
Internally MDConsumer is a collection of MDConsumerInst which each handle a single symbol+type+exchange and the corresponding shared memory blocks.
* scan_securities - in spark this is always called by MDClient
* get_handle - Low level way to get a handle. Does a linear walk though shared memory and can return a void handle (MDConsumerHandle::is_initialize() == false).
* get_managed_handle - returns a 'managed handle' this is a shared_ptr to an MDConsumerhandle. The only difference is that this is tracked by the MDConsumer and may transition from uninitialized to initialized. Storing these in a vector and walking them is almost *50% slower* than doing the same thing with an unmanaged handle. If you use these it's probably best to store the uninitialized handles in a nursery and check them periodically. You can dereference a managed handle into a regular handle at any time.
* get_channel - Gets a MDConsumerChannel representing common data for a symbol+type+exchange. This is mostly for the trade ticker / rfq and also exposes various counters which get published to dev-stats.
On Linux MDConsumer maps the shared memory as read-only pages. On Windows it cannot do this because std::atomic uses a different instruction which is considered a write by the CPU so we map it read/write.
If we have multiple platforms that can produce md for an exchange (i.e. onixs vs emumba) the MDConsumer consults the PlatformMap config key to determine which to read from. The md replayer impersonates the producer it was recorded from.
### MDConsumerChannel
read_trades() and read_rfqs() read from a ring buffer stored in the header section of the shared memory. They read directly into a google::protobuf::RepeatedPtrField to avoid extra copies in the QueryServer but can just as easily read into a std::vector. They expose a 'cursor' representing the position in the ring buffer and this cursor is sent all the way to the client so it can request updates.
## MDConsumerHandle
An MDConsumerHandle 4 states and would proably be impemented as an std::variant<> if done today. If not for the high performance requirements it could also be a virtual class but I didn't want to pay for a vtable. It is the size of 4 pointers plus the enum tag.
* VOID - an uninitialized handle
* BBO - Legacy from Edge. This represents an option book where we only kept the top to save space. I think this has rotted by now and wont work but I kept it as an example of how to add polymorphism to the shared memory types.
* DEPTH - Used for all exchange securities in Spark.
* SYNTHETIC - A Synthetic handle. These are bigger so they get stored as a pointer.
Reading from a BBO/DEPTH handle is done with an optimistic copy. If you inrease the size of the book data this copy can fail more often and get significantly worse perf. The result of the read can be changed, unchanged or busy (timeout). In edge 'busy' happened 1-2 times per day across all reads and gets logged by QueryServer.
## MDProducer
Handle to writeable shared memory. Each market data producer "platform" (onixs_cme, onixs_ice, emumba, etc) constructs one.
## MDProducerHandle
All updates to a single security go through here. Assumes a single writer thread.
## PlatformLoader
In Edge this was the entry point to the run() for the market data producer process. It loaded the bts_md_*.so so each producer process only loaded the third party objects for a particular producer. In spark this is all in process so this is defunct. It looks like it still exists because it has some utility functions that probably belong in another file.
## PriceSize/IntPrice
Market data uses integer prices to avoid double problems.
## SecurityCache*
Created/Owned by MDClient this owns the Securities.
It has methods to get a security from the live collection or load them from the SecurityDB on disk if someone asks for one that hasn't been seen yet.
* If a security is loaded from disk it 'freezes' it so when the live security arrives it will continue to serve the old security so we don't end up with 2 copies of a single security with different properties.
* This class is also responsible for sending the live securities to the SecurityDB where we *always* overwrite stored securities with what comes from the exchange.
* Survives beyond the MDConsumer in case of crash/recovery of out of process md.
## SecurityOracle*
Interface on top of SecurityCache exposing the read get_ operations.
## SecurityDB*
* Just a collection of methods for save/load of securities to sqlite.
* Stored in binary format so we can use the protobuf versioning
## SecurityIDRecorder
* Manages security to unique 32 bit id mapping across multiple processes
* Provides a callback when ids are safely persisted.
## SecurityMapper*
* Happens *before* the security gets stored in shared memory.
* Replaces the mapping.h from edge with a more organized mechanism.
* Owned by the MDProducer each security is passed though SecurityMapper::apply() before it is first stored in shared memory
* SecurityMapper is made up of multiple ISecurityMapper instances so each mapping can be in it's own class so it's easier to read each mapping. Currently all mappings are defined in SecurityMaper.cpp
* The mappings each have an opportunity to mutate the FixedSecurity.
## SettlementRecorder
In Edge SettlementRecorder stored the settlements for a security so we could preserve yesterday/day before settlement. In Spark this broke and nobody seemed to care so it was never fixed.
## SyntheticSecurityService
Each synthetic security is a api::Security and a chunk of metadata. This service handles API requests to create/update synthetics and fires callbacks for MDConsumer which does the work of producing synethic books.
## MDStructure.h
### Hot vs Cold
## FixedSecurity.h
* variable sized structure in shared mem
* For FixedSpreadSecurity we stack allocate a lot of legs then memcpy() into shared memory only the correct number of legs
* When accessing FixedSpreadSecurity it is cast back to a large number of legs. Iterate legs based on leg_count() not the static leg count.
## SharedList.h
* Primitive list-like abstraction over shared memory
* Allows variable sized entries so always accessed in O(N)
* Data never moves so we point into it for O(1)
# Market Data Relay / Recorder
* QueryServer
- Stateless request/response
- Uses multiple threads for throughput
- Uses change timestamps per channel instead of per security
- Will over-send books that change around the time of the request
* RelayClient
- Runs in the shell (or outside)
- Replicates books from the server (QueryServer) to local shared memory
* RelayRecorder
- Records Protobufs from QueryServer to a file
# Diagnostic tools
* sparkd md dump [symbol] [type] [exchange] [hot?]
- **Primary diagnostic tool**
- ShmDumper.cpp
- Shows current contents of shared memory
- append 'hot' to the end to see hot data i.e. books
- If you get a "cannot open" check that the file exists in /run/shm/
* dumpsecurities
- Inspect security database
* sparkd md tail
- MDPeriodicDumper.cpp
- for validating new market data
- prints books at given sequence number interval
* rectool
- print recording to text / convert back to futures.coconut
- This is most useful to checking if a particular symbol is in a replay file
- Occasionally useful for testing very specific market data transitions
# Tuning / Dealing with Loss
## Packet loss
#### If a small number 1-5 is lost it's upstream
- Use tcpdump to capture
- Check the tcpdump for gaps
- Ask Pete to contact hosting provider
#### If large gap probably overflow
- make sure socket buffers sizes, rmem_max, spin_time, data_wait are all set
- check affinity is set and working
- check ethtool
#### Show ethernet ring buffer sizes
ethtool -g eth0
#### change ethernet ring buffer rx size
sudo ethtool -G [device] rx 4096
#### interface statistics
ethtool -S eth0
- particularly look at:
ethtool -S p1p1 | egrep "(bad)|(nodesc)"
### If increasing the ring buffer doesn't work you can tweak interrupt coalescing using:
- /proc/sys/net/core/dev_weight
- This is the maximum number of packets that can be dequeued in a single interrupt
- https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/s-network-common-queue-issues.html
- **NOTE**: We've never used this and it's unlikely it will fix anything
-83
View File
@@ -1,83 +0,0 @@
********************
classes
********************
btscmd md => PlatformLoader
PlatformLoader:
* Loads the platform specific dll
* Calls ->start_marketdata() with a full vector<security_info>
SharedList<T> / SharedListWriter
* Shared memory list object which spans across shared memory segments as necessary
* O(N) walk
* atomic add()
* Single adder / multiple readers
* no delete
* Custom header
SecurityIDRecorder
* add_security(security, lambda callback) -> int returns security id instantly
* security_persisted callback when security id is safely persisted
SecurityIndex
* Platform Implementation helper
* Maps exhange_id to (ProducerHandle + Data)
*
CacheEntry / CacheEntryDepth (CacheStructure.h)
* CacheEntry(symbol, depth, # legs) -- size determined in constructor
* Simple inheritance tree for optional depth
* Has FixedSecurity
* Fits in SharedList<T>
* CRITICAL fields are at the front cache aligned. bbo/depth. Security goes at the end
* size, is_public(), is_persisted, is_resolved (spread)
* statistics like bbo/trade counters
* is_status_ok
* bbo, depth, trade, status, open, close
FixedSecurity / FixsdSpreadSecurity
* Normalized version of our protobuf security
* Has methods to convert to pb Security
* Size known at construction w/ # legs - spread legs of type security_key
* Can use a union for option/future fields
MDProducer(symbol, type)
* manages lifetime of SharedList instances
* Hands out MDProducerHandle - entry for one security
MDProducerHandle
* pointers to shared memory
* update methods
MDConsumer
* list of securities
* find_security_id(symbol, type, etc)
* get_cache_handle(security id#)
* Hands out MDCacheHandle
- if security is null add MDCacheHandle to a list to be updated
next time securities are polled
MDCacheHandle
* If security is not present add this is "null"
* has pointers to relevant shared memory
********************
* Cache Structure
********************
2 Shared memory segment for each (SYMBOL, TYPE) - md_hot (bbo,depth,status), md_cold (others)
When a new security is discovered
1. id is allocated to the security
2. New CacheEntry is allocated in appropriate arena (is_public=false)
3. md_cache entry count is incremented
4. When id is persisted is_public=true, increment version on cache arena
+30 -19
View File
@@ -10,6 +10,7 @@
#include "lib/persistence/SqliteStore.h"
#include "lib/utils/PBUtils.h"
#include "lib/orderman/actors/SODPosition.h"
#include "lib/utils/overloaded.h"
#include <boost/date_time/posix_time/posix_time.hpp>
#include <sys/time.h>
@@ -39,6 +40,7 @@ struct FillStore::PImpl {
FillStore::FillStore(md::SecurityOracle o) : me(new PImpl(o)) {}
// returns the sod type of a non sod type fill
auto sod_fill_kind (const api::orderman::Fill& fill){
auto rv = Fill::FILL_KIND_SOD;
switch (fill.fill_kind()){
@@ -52,6 +54,7 @@ auto sod_fill_kind (const api::orderman::Fill& fill){
rv = Fill::FILL_KIND_SOD_STRIP_LEG;
break;
default:
LOG(persist, Error, "Stored fill kinds can not be SOD");
rv = Fill::FILL_KIND_SOD;
break;
}
@@ -60,24 +63,19 @@ auto sod_fill_kind (const api::orderman::Fill& fill){
auto leg_fill_kind (const api::Security& parent,
const api::Security& child){
if (parent.has_ice()){
if(securities::is_strip(parent))
return Fill::FILL_KIND_STRIP_LEG;
else{
// parent is a spread
if(securities::is_spread_of_strips(parent)){
// this has 3 levels strip spread -> strip -> outright
// first level is an ordinary leg and second is a strip leg.
// since we dont get the level passed we have to
// use the following heurestics
if(securities::is_strip(child))
return Fill::FILL_KIND_LEG;
else
return Fill::FILL_KIND_STRIP_LEG;
}
else
// regular spread
if(securities::is_strip(parent))
return Fill::FILL_KIND_STRIP_LEG;
else{
// parent is a spread
if(securities::is_spread_of_strips(parent)){
// this has 3 levels strip spread -> strip -> outright
// first level is an ordinary leg and second is a strip leg.
// since we dont get the level passed we have to
// use the following heurestics
if(securities::is_strip(child))
return Fill::FILL_KIND_LEG;
else
return Fill::FILL_KIND_STRIP_LEG;
}
}
return Fill::FILL_KIND_LEG;
@@ -167,8 +165,21 @@ void FillStore::on_execution_report(
fill.set_fill_status(api::orderman::Fill::FILL_STATUS_CORRECTED);
fill.set_fill_key(fk);
auto exchange_ts = utils::pb::to_timestamp(
datetime::parse_FIX_timestamp(to_str(er.transact_time)));
auto exchange_ts =
std::visit(overloaded {
[](const orderman::ExchangeTimeStampStr& ts) {
return utils::pb::to_timestamp(
datetime::parse_FIX_timestamp(to_str(ts)));
},
[](ExchangeTimeStamp64 ts) {
google::protobuf::Timestamp gts;
auto seconds = ts.value() / 1E9;
gts.set_seconds(seconds);
gts.set_nanos(ts.value() - (seconds * 1E9));
return gts;
}
}, er.transact_time);
*fill.mutable_timestamp() = exchange_ts;
auto it = me->known_orders.find(er.order_key);
@@ -222,7 +222,9 @@ void GLBXExecutionGateway::on_exe_report(
msg.get<fix::ManualOrderIndicator>()
? api::orderman::MANUAL_ORDER_INDICATOR_YES
: api::orderman::MANUAL_ORDER_INDICATOR_YES;
str_to(msg.get<fix::OrderID>(), &data->exchange_order_id);
str_to(msg.get<fix::OrderID>(),
& std::get<ExchangeOrderID>(
data->exchange_order_id));
if (msg.has<fix::Price>())
data->price = securities::decode_exchange_price(
@@ -261,7 +263,9 @@ void GLBXExecutionGateway::on_exe_report(
: Price(0);
if (msg.has<fix::ExecID>())
str_to(msg.get<fix::ExecID>(), &data->exchange_fill_id);
str_to(msg.get<fix::TransactTimeStr>(), &data->transact_time);
orderman::ExchangeTimeStampStr t;
str_to(msg.get<fix::TransactTimeStr>(), &t);
data->transact_time = t;
if (msg.has<fix::CustOrderHandlingInst>())
data->cust_order_handling_inst = msg.get<fix::CustOrderHandlingInst>();
@@ -280,12 +284,16 @@ void GLBXExecutionGateway::on_cancel_reject(
decode_clorid(msg.get<fix::ClOrdID>(), &ok, &sk);
std::string reject_msg;
CancelRejectReason reason = CancelRejectReason::Other;
if (msg.has<fix::Text>())
reject_msg = msg.get<fix::Text>();
else if (msg.has<fix::CXlRejReason>())
else if (msg.has<fix::CXlRejReason>()) {
if (msg.get<fix::CXlRejReason>() == 2045)
reason = CancelRejectReason::UnknownOrder;
reject_msg = fmt::format("Order cancel was rejected with error code {}",
msg.get<fix::CXlRejReason>());
me->on_cancel_reject(ok, reject_msg);
}
me->on_cancel_reject({ok, reason,reject_msg});
}
void GLBXExecutionGateway::on_connection_status_change(bool is_connected) {
@@ -18,6 +18,7 @@ class GLBXExecutionGateway : public IExecutionGateway {
void on_securities(const mercury::Update<NewSecuritiesEvent> &) override;
void on_admin_data(const mercury::Update<NewAdminDataEvent> &) override;
void send_fix(jetfix::OutFIXPtr const &fix, trace::CorID trace) override;
void send_msgs(std::vector<OrderMsg> const&, trace::CorID) override {bts_assert(false);}
OnExeReport& on_exe_report() override;
OnGwStatusChange& on_gw_status_change() override;
@@ -0,0 +1,305 @@
#include "lib/orderman/actors/GLBXiLink3ExecutionGateway.h"
#include "lib/config/config.h"
#include "lib/config/protos/orderman_cfg.pb.h"
#include "lib/domain_model/Security.h"
#include "lib/jetfix/common/OutFIX.h"
#include "lib/jetfix/store/session_name.h"
#include "lib/orderman/actors/GLBXExecutionGateway.h"
#include "lib/orderman/actors/ilink3/IL3GW.h"
#include "lib/orderman/actors/ilink3/IL3MessageFramer.h"
#include "lib/orderman/actors/ilink3/IL3Reactor.h"
#include "lib/orderman/actors/ilink3/IL3SecDB.h"
#include "lib/orderman/actors/ilink3/SessionStore.h"
#include "lib/orderman/events/NewSecuritiesEvent.h"
#include "lib/orderman/orderman_utils/fix_order_codec.h"
namespace bts {
namespace orderman {
namespace ilink3 {
enum class GLBXiLink3ExecutionGateway::State {
WAITING_FOR_SECURITIES,
PAUSED,
CONNECTING,
READY
};
struct GLBXiLink3ExecutionGateway::PImpl {
PImpl(config::JetFIX::Connection const &cfg, trace::PerfLog &log,
md::SecurityOracle o)
: state(), cfg(cfg), reactor(), start_of_pause(), on_exe_report(),
on_gw_status_change(), on_cancel_reject(), perf_log(log), db(o),
gw_name(cfg.name()), session_store(), gw(), connection(nullptr),
connection_failures(0), started(false), replay_messages(),
has_replayed(false) {
TCPEndpoint endpoint{cfg.socket_connect_host_ip(),
(ushort)cfg.socket_connect_port()};
connection = &reactor.register_connection({endpoint});
}
State state;
config::JetFIX::Connection const cfg;
IL3Reactor reactor;
IL3MessageFramer framer;
TimeStamp start_of_pause;
OnExeReport on_exe_report;
OnGwStatusChange on_gw_status_change;
orderman::OnCancelReject on_cancel_reject;
trace::PerfLog &perf_log;
IL3SecDB db;
GatewayName gw_name;
std::optional<SessionStore> session_store;
std::optional<IL3GW> gw;
IL3Connection *connection;
size_t connection_failures;
bool started;
// We need to keep this collection alive for the duration of the
// program because we will be passing pointers to ExecutionReport
// to other threads
std::vector<ExecutionReport> replay_messages;
bool has_replayed;
};
GLBXiLink3ExecutionGateway::GLBXiLink3ExecutionGateway(
config::JetFIX::Connection const &cfg, trace::PerfLog &log,
md::SecurityOracle o)
: me(new PImpl(cfg, log, o)) {
me->state = State::WAITING_FOR_SECURITIES;
}
GLBXiLink3ExecutionGateway::~GLBXiLink3ExecutionGateway() {}
void GLBXiLink3ExecutionGateway::start() {
bts_assert(!me->started);
me->started = true;
}
void GLBXiLink3ExecutionGateway::dispatch_incoming_fix_messages() {
if (me->started) {
switch (me->state) {
case State::WAITING_FOR_SECURITIES:
break;
case State::PAUSED:
on_time();
break;
case State::READY:
case State::CONNECTING:
me->reactor.do_work();
me->gw->on_time();
break;
}
}
}
void GLBXiLink3ExecutionGateway::on_time() {
bts_assert(me->state == State::PAUSED);
// If we are still paused bail early, this avoids logging too much
// or spending too much CPU
if (pause_in_effect())
return;
me->start_of_pause = Timer::now();
// If we alread exceed the maximum number of failures per run
// bail. This is a safety measure to avoid overwhelming the
// exchange in the event of a bug or operational problem that
// results in repeated connection failures.
if (max_failures_reached()) {
LOG(iLink3, Warn,
"Maximum number of connection failures reached for {}. Reboot to try "
"again",
me->gw_name.value());
return;
}
// If we reach this point we are no longer in the pause window and
// we have not exceed our connection failure limit. The only thing
// to check now is wheter we are inside the exchange's active
// hours.
auto current_session = jetfix::session_name(me->cfg);
if (current_session.empty()) {
LOG(iLink3, Info, "Outside of session in ", me->gw_name.value());
return;
}
// Yep, the stars are all aligned, lets try to connect
connect(current_session);
}
void GLBXiLink3ExecutionGateway::connect(jetfix::SessionName const &sn) {
bts_assert(me->state == State::PAUSED);
transition(State::CONNECTING);
Config c;
c.cme_session = me->cfg.sender_compid().substr(0, 3);
c.firm = me->cfg.sender_compid().substr(3);
c.failover_on_disconnect = false;
me->session_store.emplace(me->db, sn, &me->cfg);
if (!me->has_replayed) {
me->has_replayed = true;
me->replay_messages = me->session_store.value().replay();
for (auto const &er : me->replay_messages)
me->on_exe_report(er, {});
}
c.raw_message_logger =
[store = &me->session_store.value()](auto d, auto sn, auto const &msg) {
store->async_store_raw_msg(d, sn, msg);
};
me->gw.emplace(*me->connection, me->framer, &me->db,
me->session_store->starting_session_descriptor(), c,
&me->cfg);
me->gw->subscribe_cancel_reject([this](auto &cr) {
bts_assert(me->state == State::READY || me->state == State::CONNECTING);
me->on_cancel_reject(cr);
});
me->gw->subscribe_execution_reports([this](auto er) {
bts_assert(me->state == State::READY || me->state == State::CONNECTING);
me->on_exe_report(*er, {});
});
me->gw->subscribe_session(
[this](auto const &s) { me->session_store->async_update_session(s); });
me->gw->subscribe_state([this](auto gw_state) {
LOG(iLink3, Info, "Internal gateway state change in {}. State is now {}",
me->gw_name.value(),
log::CharPtrWrapper{ilink3::state_string(gw_state)});
switch (me->state) {
case State::CONNECTING:
if (gw_state == ilink3::State::READY) {
transition(State::READY);
me->on_gw_status_change(true);
} else if (gw_state == ilink3::State::ERROR) {
me->connection_failures++;
pause();
}
break;
case State::READY:
if (gw_state == ilink3::State::ERROR) {
me->connection_failures++;
pause();
me->on_gw_status_change(false);
}
break;
default:
bts_assert(false);
break;
}
});
me->gw->connect();
}
void GLBXiLink3ExecutionGateway::pause() {
me->start_of_pause = Timer::now();
me->gw.reset();
transition(State::PAUSED);
}
bool GLBXiLink3ExecutionGateway::pause_in_effect() const {
return sec_diff(Timer::now(), me->start_of_pause) <=
me->cfg.seconds_to_pause();
}
bool GLBXiLink3ExecutionGateway::max_failures_reached() const {
return me->connection_failures > me->cfg.max_connection_attempts();
}
void GLBXiLink3ExecutionGateway::on_securities(
const mercury::Update<NewSecuritiesEvent> &update) {
if (update.events) {
for (auto const &ev : *(update.events)) {
auto sec_ptr = ev.security;
me->db.add_security(*sec_ptr);
}
if (me->state == State::WAITING_FOR_SECURITIES) {
LOG(iLink3, Warn,
"Received first set of {} securities. Proceeding "
"with initialization of execution gateway {}",
update.events->size(), me->gw_name.value());
// got our security synopsis, ready to start the GW
transition(State::PAUSED);
}
}
}
void GLBXiLink3ExecutionGateway::on_admin_data(
const mercury::Update<NewAdminDataEvent> &) {}
void GLBXiLink3ExecutionGateway::send_msgs(std::vector<OrderMsg> const &msgs,
trace::CorID corid) {
if (me->state == State::READY) {
for (auto &m : msgs) {
// NewOrderSingle
auto create_ptr = std::get_if<CreateOrderMsg>(&m);
if (create_ptr) {
me->gw->send_order(*create_ptr);
continue;
}
// CancelReplace
auto cancel_replace_ptr = std::get_if<CancelReplaceOrderMsg>(&m);
if (cancel_replace_ptr) {
LOG(iLink3,Info,"Cancel Replace. CorID:{}.{} Order:{}",
log::CharPtrWrapper{corid.channel},
corid.sn,
cancel_replace_ptr->exchange_order_id);
me->gw->send_cancel_replace(*cancel_replace_ptr);
continue;
}
// Cancel
auto cancel_ptr = std::get_if<CancelOrderMsg>(&m);
if (cancel_ptr) {
me->gw->send_cancel(*cancel_ptr);
continue;
}
}
me->gw->flush();
}
}
OnExeReport &GLBXiLink3ExecutionGateway::on_exe_report() {
return me->on_exe_report;
}
OnGwStatusChange &GLBXiLink3ExecutionGateway::on_gw_status_change() {
return me->on_gw_status_change;
}
orderman::OnCancelReject &GLBXiLink3ExecutionGateway::on_cancel_reject() {
return me->on_cancel_reject;
}
log::CharPtrWrapper GLBXiLink3ExecutionGateway::state_name(State st) {
switch (st) {
case State::WAITING_FOR_SECURITIES:
return {"WAITING_FOR_SECURITIES"};
case State::PAUSED:
return {"PAUSED"};
case State::CONNECTING:
return {"CONNECTING"};
case State::READY:
return {"READY"};
}
bts_assert(false);
return {""};
}
void GLBXiLink3ExecutionGateway::transition(State new_state) {
LOG(iLink3, Info, "Transitioning {} from {} to {}", me->gw_name.value(),
state_name(me->state), state_name(new_state));
me->state = new_state;
}
} // namespace ilink3
} // namespace orderman
} // namespace bts
@@ -0,0 +1,48 @@
#pragma once
#include "IExecutionGateway.h"
#include "lib/logging/logging.h"
namespace bts {
namespace jetfix {struct SessionName;}
namespace orderman {
namespace ilink3 {
struct Session;
class GLBXiLink3ExecutionGateway : public IExecutionGateway {
public:
GLBXiLink3ExecutionGateway(config::JetFIX::Connection const &, trace::PerfLog &,
md::SecurityOracle);
virtual ~GLBXiLink3ExecutionGateway();
void start() override;
void dispatch_incoming_fix_messages() override;
void on_securities(const mercury::Update<NewSecuritiesEvent> &) override;
void on_admin_data(const mercury::Update<NewAdminDataEvent> &) override;
void send_fix(jetfix::OutFIXPtr const &, trace::CorID ) override {bts_assert(false);}
void send_msgs(std::vector<OrderMsg> const&, trace::CorID) override;
OnExeReport& on_exe_report() override;
OnGwStatusChange& on_gw_status_change() override;
OnCancelReject &on_cancel_reject() override;
private:
enum class State;
struct PImpl;
std::unique_ptr<PImpl> me;
// Time passing
void on_time();
bool pause_in_effect() const;
bool max_failures_reached()const;
static log::CharPtrWrapper state_name(State);
void transition(State);
void pause();
void connect(jetfix::SessionName const&);
};
} // namespace orderman
} // namespace bts
}
@@ -390,7 +390,9 @@ void ICEExecutionGateway::on_exe_report(const jetfix::fix::ICEExecutionReportMsg
data->manual_order_indicator = api::orderman::MANUAL_ORDER_INDICATOR_NO;
std::string oid_string = get_ice_order_id_string(msg);
str_to(oid_string, &data->exchange_order_id);
str_to(oid_string,
& std::get<ExchangeOrderID>(
data->exchange_order_id));
data->ice_cl_ord_id = 0;
if(msg.has<fix::Int_ClOrdID>())
data->ice_cl_ord_id = msg.get<fix::Int_ClOrdID>();
@@ -438,8 +440,10 @@ void ICEExecutionGateway::on_exe_report(const jetfix::fix::ICEExecutionReportMsg
if (msg.has<fix::TransactTimeStr>())
{
std::string stamp = msg.get<fix::TransactTimeStr>();
stamp.resize(sizeof(mercury::TimeStampStr));
str_to(stamp, &data->transact_time);
stamp.resize(sizeof(orderman::ExchangeTimeStampStr));
orderman::ExchangeTimeStampStr t;
str_to(stamp, &t);
data->transact_time = t;
}
data->cfg = &me->cfg;
data->msg = text_msg;
@@ -458,7 +462,7 @@ void ICEExecutionGateway::on_cancel_reject(const jetfix::fix::ICECancelRejectMsg
else if (msg.has<fix::CXlRejReason>())
reject_msg = fmt::format("Order cancel was rejected with error code {}",
msg.get<fix::CXlRejReason>());
me->on_cancel_reject(ok, reject_msg);
me->on_cancel_reject(CancelReject {ok, CancelRejectReason::Other,reject_msg});
}
@@ -17,6 +17,7 @@ class ICEExecutionGateway : public IExecutionGateway {
void on_securities(const mercury::Update<NewSecuritiesEvent> &) override;
void on_admin_data(const mercury::Update<NewAdminDataEvent> &) override;
void send_fix(const jetfix::OutFIXPtr& fix, trace::CorID trace) override;
void send_msgs(std::vector<OrderMsg> const&, trace::CorID) override {bts_assert(false);}
OnExeReport& on_exe_report() override;
OnGwStatusChange& on_gw_status_change() override;
+12 -5
View File
@@ -1,17 +1,24 @@
#include "IExecutionGateway.h"
#include "ICEExecutionGateway.h"
#include "GLBXExecutionGateway.h"
#include "ICEExecutionGateway.h"
#include "lib/utils/ValueWrapper.h"
#include "lib/orderman/actors/GLBXiLink3ExecutionGateway.h"
#include <variant>
namespace bts {
namespace orderman {
std::unique_ptr<IExecutionGateway>
ExecutionGatewayFactory::create_gateway(const config::JetFIX::Connection& conn, trace::PerfLog& pl, md::SecurityOracle o)
{
if(conn.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_ICE ||
conn.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_SIMICE)
ExecutionGatewayFactory::create_gateway(const config::JetFIX::Connection &conn,
trace::PerfLog &pl,
md::SecurityOracle o) {
if (conn.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_ICE ||
conn.platform() == bts::config::JetFIX_Connection_JetFIXPlatform_SIMICE)
return std::make_unique<ICEExecutionGateway>(conn, pl, o);
else if (conn.use_ilink3())
return std::make_unique<ilink3::GLBXiLink3ExecutionGateway>(conn, pl, o);
else
return std::make_unique<GLBXExecutionGateway>(conn, pl, o);
}
+34 -2
View File
@@ -9,6 +9,7 @@
#include "lib/utils/bts_no_copy.h"
#include <memory>
#include <variant>
namespace bts {
namespace trace {struct CorID;class PerfLog;}
@@ -16,11 +17,42 @@ namespace orderman {
class NewSecuritiesEvent;
class NewAdminDataEvent;
class ExecutionReport;
class CancelReject;
struct CreateOrderMsg {
OrderKey key;
Price price;
Quantity quantity;
api::Security const *security;
api::Side side;
Price stop_px;
Quantity display_quantity;
api::orderman::NativeOrderType order_type;
api::orderman::TimeInForce time_in_force;
api::orderman::ManualOrderIndicator manual_order_indicator;
api::orderman::RoutingRecord const *routing_record;
};
struct CancelReplaceOrderMsg {
CreateOrderMsg create_fields;
ExchangeOrderID64 exchange_order_id;
};
struct CancelOrderMsg {
OrderKey key;
ExchangeOrderID64 exchange_order_id;
api::orderman::ManualOrderIndicator manual_order_indicator;
api::Security const *security;
api::orderman::RoutingRecord const *routing_record;
api::Side side;
};
using OrderMsg =
std::variant<CreateOrderMsg, CancelReplaceOrderMsg, CancelOrderMsg>;
using OnGwStatusChange = std::function<void(bool)>;
using OnExeReport = std::function<void(ExecutionReport const &, trace::CorID const &)>;
using OnCancelReject = std::function<void(OrderKey, std::string const &)>;
using OnCancelReject = std::function<void(CancelReject const&)>;
// fix execution gateway for a single FIX session
class IExecutionGateway {
@@ -33,7 +65,7 @@ class IExecutionGateway {
virtual void on_securities(const mercury::Update<NewSecuritiesEvent> &) = 0;
virtual void on_admin_data(const mercury::Update<NewAdminDataEvent> &) = 0;
virtual void send_fix(const jetfix::OutFIXPtr&, trace::CorID) = 0;
virtual void send_msgs(std::vector<OrderMsg> const&, trace::CorID) = 0;
virtual OnExeReport& on_exe_report() = 0;
virtual OnGwStatusChange& on_gw_status_change() = 0;
virtual OnCancelReject &on_cancel_reject() = 0;
+88 -73
View File
@@ -1,5 +1,6 @@
#pragma once
#include "lib/domain_model/Security.h"
#include "lib/logging/logging.h"
#include "lib/md/IntPrice.h"
#include "lib/md/MDConsumer.h"
@@ -10,12 +11,11 @@
#include "lib/mercury/events/TimerEvent.h"
#include "lib/orderman/events/BookUpdateEvent.h"
#include "lib/orderman/events/MDRequestEvent.h"
#include "lib/orderman/events/TickerUpdateEvent.h"
#include "lib/orderman/events/StatusUpdateEvent.h"
#include "lib/orderman/events/TickerUpdateEvent.h"
#include "lib/tracing/PerfLog.h"
#include "lib/tracing/bts_trace.h"
#include "lib/utils/TimeStamp.h"
#include "lib/domain_model/Security.h"
#include <date/date.h>
@@ -31,9 +31,9 @@ class MDSource : public mercury::Actor, public mercury::IPollableActor {
public:
MDSource(md::MDConsumer &consumer, metrics::MetricsContainer &stats,
trace::PerfLog &log)
: _handles(), _pending_handles(), _status_subs(), _md_consumer(consumer),
_book_update(), _ticker_update(), _status_update(),
_perf_log(log), _stats(stats), _loop_timer(), _counter(0) {
: _handles(), _pending_handles(), _status_subs(), _md_consumer(consumer),
_book_update(), _ticker_update(), _status_update(), _perf_log(log),
_stats(stats), _loop_timer(), _counter(0) {
_trades_buffer.Reserve(1000);
auto cme_channels = md::get_configured_cme_channels();
@@ -67,9 +67,8 @@ class MDSource : public mercury::Actor, public mercury::IPollableActor {
auto ticker_channel_cout_metric =
std::make_shared<metrics::CounterMetric<int>>(
"spark_mdsource_ticker_channel_count");
auto status_count_metric =
std::make_shared<metrics::CounterMetric<int>>(
"spark_mdsource_status_count");
auto status_count_metric = std::make_shared<metrics::CounterMetric<int>>(
"spark_mdsource_status_count");
_stats.publish(security_count_metric);
_stats.publish(synthetic_count_metric);
_stats.publish(ticker_channel_cout_metric);
@@ -78,37 +77,40 @@ class MDSource : public mercury::Actor, public mercury::IPollableActor {
_book_update = make_stream<BookUpdateEvent>();
_ticker_update = make_stream<TickerUpdateEvent>();
_status_update = make_stream<StatusUpdateEvent>();
subscribe<MDRequestEvent>(
[this, security_count_metric, synthetic_count_metric,
ticker_channel_cout_metric, status_count_metric](auto const &update) {
if (update.events) {
for (auto const &ev : *update.events) {
auto k = ev.security_key;
subscribe<MDRequestEvent>([this, security_count_metric,
synthetic_count_metric,
ticker_channel_cout_metric,
status_count_metric](auto const &update) {
if (update.events) {
for (auto const &ev : *update.events) {
auto k = ev.security_key;
switch (ev.md_type) {
case MDRequestType::BOOK:
on_book_sub(k, *synthetic_count_metric);
security_count_metric->set(_handles.size());
break;
case MDRequestType::TICKER:
on_ticker_sub(k, *ticker_channel_cout_metric);
break;
case MDRequestType::STATUS:
on_status_sub(k);
status_count_metric->set(_status_subs.size());
break;
}
}
}
});
switch (ev.md_type) {
case MDRequestType::BOOK:
on_book_sub(k, *synthetic_count_metric);
security_count_metric->set(_handles.size());
break;
case MDRequestType::TICKER:
on_ticker_sub(k, *ticker_channel_cout_metric);
break;
case MDRequestType::STATUS:
on_status_sub(k);
status_count_metric->set(_status_subs.size());
break;
}
}
}
});
security_count_metric->set(_handles.size());
}
private:
void on_poll(trace::PerfLog& log) override {
void on_poll(trace::PerfLog &log) override {
auto scoped_do_work = trace::log_scope(log, {}, "MDSource::on_poll", utils::TimeStamp::Interval::from_microseconds(2));
auto scoped_do_work =
trace::log_scope(log, {}, "MDSource::on_poll",
utils::TimeStamp::Interval::from_microseconds(2));
// Book Updates
Book book;
@@ -158,11 +160,11 @@ class MDSource : public mercury::Actor, public mercury::IPollableActor {
}
// Status updates
for (auto& s : _status_subs) {
for (auto &s : _status_subs) {
auto status = s.handle.trading_status();
if (status != s.status) {
s.status = status;
publish(_status_update, SecurityKey { s.handle.key() }, status);
publish(_status_update, SecurityKey{s.handle.key()}, status);
}
}
@@ -171,7 +173,7 @@ class MDSource : public mercury::Actor, public mercury::IPollableActor {
_loop_timer->observe((start - stop).nanoseconds());
_counter += 1;
if (_counter % (2<<20) == 0) {
if (_counter % (2 << 20) == 0) {
// Invoke periodic work every 2M calls on on_poll
// This is approximately 20-30 seconds
do_periodic_work();
@@ -180,35 +182,39 @@ class MDSource : public mercury::Actor, public mercury::IPollableActor {
void do_periodic_work() {
if (!_pending_handles.empty()) {
for (auto iter = _pending_handles.begin(); iter != _pending_handles.end(); ++iter) {
for (auto iter = _pending_handles.begin();
iter != _pending_handles.end(); ++iter) {
if (iter->handle->is_initialized()) {
auto handle = *(iter->handle);
LOG(OrderMan, Info, "Found md for {}. Pending handle removed.", handle.key());
switch(iter->type) {
case MDRequestType::BOOK:
_handles.push_back({utils::TimeStamp(), handle,
get_external_event_channel(handle.get_usersymbol()),
std::make_unique<std::string>(securities::descriptor(
handle.get_normalized_security()))});
break;
case MDRequestType::STATUS:
for (auto& h : _status_subs) {
// no need to reset status
if (h.handle.key() == handle.key()) break;
}
_status_subs.push_back(
StatusRecord{ handle, co2::TRADINGSTATUS_UNKNOWN });
case MDRequestType::TICKER:
LOG(OrderMan, Info, "Found md for {}. Pending handle removed.",
handle.key());
switch (iter->type) {
case MDRequestType::BOOK:
_handles.push_back(
{utils::TimeStamp(), handle,
get_external_event_channel(handle.get_usersymbol()),
std::make_unique<std::string>(securities::descriptor(
handle.get_normalized_security()))});
break;
case MDRequestType::STATUS:
for (auto &h : _status_subs) {
// no need to reset status
if (h.handle.key() == handle.key())
break;
}
_status_subs.push_back(
StatusRecord{handle, co2::TRADINGSTATUS_UNKNOWN});
case MDRequestType::TICKER:
// We aren't currently monitoring for pending ticker requests
break;
}
}
}
_pending_handles.erase(
std::remove_if(_pending_handles.begin(),
_pending_handles.end(),
[](const auto& h) { return h.handle->is_initialized(); }),
_pending_handles.end());
std::remove_if(
_pending_handles.begin(), _pending_handles.end(),
[](const auto &h) { return h.handle->is_initialized(); }),
_pending_handles.end());
}
}
@@ -229,18 +235,18 @@ class MDSource : public mercury::Actor, public mercury::IPollableActor {
auto mhandle = _md_consumer.get_managed_handle(k);
if (mhandle->is_initialized()) {
auto handle = *mhandle;
_handles.push_back({utils::TimeStamp(), handle,
get_external_event_channel(handle.get_usersymbol()),
std::make_unique<std::string>(securities::descriptor(
handle.get_normalized_security()))});
_handles.push_back(
{utils::TimeStamp(), handle,
get_external_event_channel(handle.get_usersymbol()),
std::make_unique<std::string>(
securities::descriptor(handle.get_normalized_security()))});
if (handle.is_synthetic())
synthetic_count_metric.increment();
}
else {
// A pending handle is a handle for security we are reasonbly sure exists
// but we haven't yet found live market data for.
} else {
// A pending handle is a handle for security we are reasonbly sure
// exists but we haven't yet found live market data for.
LOG(OrderMan, Warn, "No market data for {}. Pending handle added.", k);
_pending_handles.push_back({ mhandle, MDRequestType::BOOK });
_pending_handles.push_back({mhandle, MDRequestType::BOOK});
}
}
@@ -253,8 +259,17 @@ class MDSource : public mercury::Actor, public mercury::IPollableActor {
void on_ticker_sub(SecurityKey sk,
metrics::CounterMetric<int> &channel_count_metric) {
auto handle = _md_consumer.get_managed_handle(sk.value());
auto user_symbol = _md_consumer.get_handle(sk.value()).get_usersymbol();
if (!handle->is_initialized()) {
LOG(OrderMan, Warn,
"Received ticker subscription for security {} which does not have "
"valid market data. Ignoring",
sk.value());
return;
}
auto user_symbol = handle->get_usersymbol();
for (auto &ch : _channels) {
if (ch.user_symbol == user_symbol) {
auto it =
@@ -262,7 +277,7 @@ class MDSource : public mercury::Actor, public mercury::IPollableActor {
[sk](auto const &ev) { return ev.key() == sk; });
if (it == ch.securities.end()) {
// first time we are asked for this security, we will
// publish the trades for it next time next round
// publish the trades for it next time around
LOG(OrderMan, Warn,
"Starting to publish ticker updates on user symbol {} for "
"security key: {} (sec count for the symbol: {} )",
@@ -294,7 +309,7 @@ class MDSource : public mercury::Actor, public mercury::IPollableActor {
void on_status_sub(SecurityKey sk) {
auto k = sk.value();
for (auto& h : _status_subs) {
for (auto &h : _status_subs) {
if (h.handle.key() == k) {
// reset status so it get resent next on_poll
h.status = co2::TRADINGSTATUS_UNKNOWN;
@@ -303,11 +318,11 @@ class MDSource : public mercury::Actor, public mercury::IPollableActor {
}
auto handle = _md_consumer.get_managed_handle(k);
if (handle->is_initialized()) {
_status_subs.push_back(StatusRecord{ *handle, co2::TRADINGSTATUS_UNKNOWN });
}
else {
_status_subs.push_back(
StatusRecord{*handle, co2::TRADINGSTATUS_UNKNOWN});
} else {
LOG(OrderMan, Warn, "No market data for {}. Pending handle added.", k);
_pending_handles.push_back({ handle, MDRequestType::STATUS });
_pending_handles.push_back({handle, MDRequestType::STATUS});
}
}
@@ -337,7 +352,7 @@ class MDSource : public mercury::Actor, public mercury::IPollableActor {
md::MDConsumerHandle handle;
co2::TradingStatus status;
};
boost::container::small_vector<StatusRecord,20> _status_subs;
boost::container::small_vector<StatusRecord, 20> _status_subs;
md::MDConsumer _md_consumer;
mercury::Stream<BookUpdateEvent> _book_update;
+4
View File
@@ -101,6 +101,8 @@ void OrderStore::on_order_state_update(
for (auto const &ev : *u.events) {
auto &rec_info = me->known_orders[ev.key];
LOG(iLink3,Info, "Received order state update for {}",
ev.key.value());
auto &rec = rec_info.rec;
bts_assert(rec.has_def());
rec.set_key(ev.key.value());
@@ -123,6 +125,8 @@ void OrderStore::on_order_definition_update(
std::set<OrderKey> modified_orders;
if (u.events)
for (auto const &ev : *u.events) {
LOG(iLink3,Info, "Received order definition update for {}",
ev.key.value());
auto &rec_info = me->known_orders[ev.key];
auto &rec = rec_info.rec;
rec.set_key(ev.key.value());
@@ -0,0 +1,20 @@
#pragma once
#include "IL3ConnectionHandler.h"
#include <string>
#include <vector>
namespace bts {
namespace orderman {
namespace ilink3 {
struct IL3Connection {
virtual ~IL3Connection() = default;
virtual void connect(IL3ConnectionHandler&) = 0;
virtual void disconnect() = 0;
// Switch connection but do not connect()
virtual void failover() = 0;
virtual void send(const std::string_view) = 0;
virtual void send(const std::vector<std::string_view>&) = 0;
};
}}}//namespace
@@ -0,0 +1,10 @@
#include "IL3ConnectionHandler.h"
namespace bts {
namespace orderman {
namespace ilink3 {
IL3ConnectionHandler::~IL3ConnectionHandler() = default;
}}}
@@ -0,0 +1,17 @@
#pragma once
#include <cstddef>
#include <string>
namespace bts {
namespace orderman {
namespace ilink3 {
struct IL3ConnectionHandler {
virtual ~IL3ConnectionHandler() ;
virtual void on_connect_success() = 0;
virtual void on_disconnect() = 0;
// Send available data and length. Handler will return what it consumed
virtual size_t on_data(const std::string_view&) = 0;
};
}}}//namespace
@@ -0,0 +1,140 @@
#pragma once
#include <cstdint>
#include "IL3GeneratedFull.h"
#include "IL3MessageFramer.h"
namespace bts {
namespace orderman {
namespace ilink3 {
template <class Handler, class T>
void handle(Handler& h, const char* data, uint16_t len) {
const FramingHeader* fheader = reinterpret_cast<const FramingHeader*>(data);
iLinkBinary::MessageHeader mheader {
(char*)data, sizeof(FramingHeader), fheader->message_size,
iLinkBinary::MessageHeader::sbeSchemaVersion() };
T t {
const_cast<char*>(data),
sizeof(FramingHeader) + iLinkBinary::MessageHeader::encodedLength(),
len,
mheader.blockLength(),
mheader.version()};
h.handle(t);
}
template <class H>
void dispatch_message(H& handler, const FramingHeader* header, const char* data) {
using namespace iLinkBinary;
bts_assert_d(header->encoding_type == 0XCAFE);
MessageHeader mheader {
(char*)data, sizeof(FramingHeader), header->message_size,
MessageHeader::sbeSchemaVersion() };
switch(mheader.templateId()) {
case 501: handle<H, NegotiationResponse501>(handler, data, header->message_size); break;
case 502: handle<H, NegotiationReject502>(handler, data, header->message_size); break;
case 504: handle<H, EstablishmentAck504>(handler, data, header->message_size); break;
case 505: handle<H, EstablishmentReject505>(handler, data, header->message_size); break;
case 506: handle<H, Sequence506>(handler, data, header->message_size); break;
case 507: handle<H, Terminate507>(handler, data, header->message_size); break;
case 509: handle<H, Retransmission509>(handler, data, header->message_size); break;
case 510: handle<H, RetransmitReject510>(handler, data, header->message_size); break;
case 513: handle<H, NotApplied513>(handler, data, header->message_size); break;
case 519: handle<H, PartyDetailsDefinitionRequestAck519>(handler, data, header->message_size); break;
case 521: handle<H, BusinessReject521>(handler, data, header->message_size); break;
case 522: handle<H, ExecutionReportNew522>(handler, data, header->message_size); break;
case 523: handle<H, ExecutionReportReject523>(handler, data, header->message_size); break;
case 524: handle<H, ExecutionReportElimination524>(handler, data, header->message_size); break;
case 525: handle<H, ExecutionReportTradeOutright525>(handler, data, header->message_size); break;
case 526: handle<H, ExecutionReportTradeSpread526>(handler, data, header->message_size); break;
case 527: handle<H, ExecutionReportTradeSpreadLeg527>(handler, data, header->message_size); break;
case 531: handle<H, ExecutionReportModify531>(handler, data, header->message_size); break;
case 532: handle<H, ExecutionReportStatus532>(handler, data, header->message_size); break;
case 534: handle<H, ExecutionReportCancel534>(handler, data, header->message_size); break;
case 535: handle<H, OrderCancelReject535>(handler, data, header->message_size); break;
case 536: handle<H, OrderCancelReplaceReject536>(handler, data, header->message_size); break;
// case 538: handle<H, PartyDetailsListReport538>(handler, data, header->message_size); break;
// The Execution Acknowledgment message notifies CME that a given fill has either been accepted or rejected by a counterparty for EU bilateral trades.
// case 539: handle<H, ExecutionAck539>(handler, data, header->message_size); break;
// case 544: handle<H, NewOrderCross544>(handler, data, header->message_size); break;
// case 545: handle<H, MassQuoteAck545>(handler, data, header->message_size); break;
// case 546: handle<H, RequestForQuoteAck546>(handler, data, header->message_size); break;
// case 547: handle<H, QuoteCancelAck547>(handler, data, header->message_size); break;
case 548: handle<H, ExecutionReportTradeAddendumOutright548>(handler, data, header->message_size); break;
case 549: handle<H, ExecutionReportTradeAddendumSpread549>(handler, data, header->message_size); break;
case 550: handle<H, ExecutionReportTradeAddendumSpreadLeg550>(handler, data, header->message_size); break;
// case 558: handle<H, iLinkBinary::OrderMassActionReport558>(handler, data, header->message_size); break;
// case 561: handle<H, iLinkBinary::SecurityDefinitionResponse561>(handler, data, header->message_size); break;
}
}
template <class H>
void dispatch_execution_reports(H& handler, const FramingHeader* header, const char* data) {
using namespace iLinkBinary;
bts_assert_d(header->encoding_type == 0XCAFE);
MessageHeader mheader {
(char*)data, sizeof(FramingHeader), header->message_size,
MessageHeader::sbeSchemaVersion() };
switch(mheader.templateId()) {
case 522: handle<H, ExecutionReportNew522>(handler, data, header->message_size); break;
case 523: handle<H, ExecutionReportReject523>(handler, data, header->message_size); break;
case 524: handle<H, ExecutionReportElimination524>(handler, data, header->message_size); break;
case 525: handle<H, ExecutionReportTradeOutright525>(handler, data, header->message_size); break;
case 526: handle<H, ExecutionReportTradeSpread526>(handler, data, header->message_size); break;
case 527: handle<H, ExecutionReportTradeSpreadLeg527>(handler, data, header->message_size); break;
case 531: handle<H, ExecutionReportModify531>(handler, data, header->message_size); break;
case 532: handle<H, ExecutionReportStatus532>(handler, data, header->message_size); break;
case 534: handle<H, ExecutionReportCancel534>(handler, data, header->message_size); break;
}
}
// This handler handles exchange messages for testing
template <class H>
void dispatch_exchange_message(H& handler, const FramingHeader* header, const char* data) {
using namespace iLinkBinary;
bts_assert_d(header->encoding_type == 0XCAFE);
MessageHeader mheader {
(char*)data, sizeof(FramingHeader), header->message_size,
MessageHeader::sbeSchemaVersion() };
// Rather than trying to be exhaustive like the client
// we will only add handlers for the messages we use for testing
switch(mheader.templateId()) {
case 500: handle<H, Negotiate500>(handler, data, header->message_size); break;
case 503: handle<H, Establish503>(handler, data, header->message_size); break;
case 506: handle<H, Sequence506>(handler, data, header->message_size); break;
case 508: handle<H, RetransmitRequest508>(handler, data, header->message_size); break;
case 518: handle<H, PartyDetailsDefinitionRequest518>(handler, data, header->message_size); break;
case 514: handle<H, NewOrderSingle514>(handler, data, header->message_size); break;
case 515: handle<H, OrderCancelReplaceRequest515>(handler, data, header->message_size); break;
case 516: handle<H, OrderCancelRequest516>(handler, data, header->message_size); break;
}
}
template <class Handler, class Dispatcher>
size_t consume_messages(
Handler& handler, const char* data, size_t len, Dispatcher dispatch) {
// void(*dispatch)(Handler&, const FramingHeader*, const char*)) {
size_t read = 0;
while(len > sizeof(FramingHeader)) {
// read the FramingHeader
const FramingHeader* header = reinterpret_cast<const FramingHeader*>(data);
// Check if we can consume the message
auto msize = header->message_size;
if (msize <= len) {
read += msize;
dispatch(handler, header, data);
data += msize;
len -= msize;
}
else {
return read;
}
}
return read;
}
}}}//namespace
+662
View File
@@ -0,0 +1,662 @@
#include "IL3GW.h"
#include "IL3Dispatcher.h"
#include "IL3GeneratedFull.h"
#include "IL3MessageFramer.h"
#include "ILink3Utils.h"
#include "lib/orderman/orderman_utils/fix_order_codec.h"
#include "lib/utils/SecureLogon.h"
#include "lib/utils/bts_assert.h"
#include "lib/utils/datetime.h"
/*
# Some notes about session binding and initialization: #
## Official documentation:
https://www.cmegroup.com/confluence/display/EPICSANDBOX/iLink+3+Binary+Order+Entry+-+Session+Layer#iLink3BinaryOrderEntry-SessionLayer-WeeklyReset
## Begining of week (new session?)
- config.session = X
- config.outgoing_sequence = 1
-> Negotiate session=X
<- NegotiateResponse session=X, oldsession=Y, oldSeq=N
if oldSeq > 0 -> ReplayRequest
-> Establish seq=1
<- EstablishAck oldsession=Y, oldSeq=N
## Middle of week (same session)
- config.session = X
- config.outgoing_sequence > 1
-> Establish seq=N
*/
namespace bts {
namespace orderman {
namespace ilink3 {
log::CharPtrWrapper wrap(const char *str) { return log::CharPtrWrapper{str}; }
IL3GW::IL3GW(IL3Connection &con, IL3MessageFramer &buffer, IL3SecDB *db,
const Session &session, const Config &conf,
config::JetFIX::Connection const *cfg)
: _config(conf), _x_state(State::INIT), _buffer(buffer),
_on_state_change([](auto) {}), _on_exe_report([](auto *) {}),
_on_session_change([](auto) {}), _connection(con), _session(session),
_msg_count(0), _time(), _pool(), _exe_parser(cfg, db), _retransmit() {
_clorid_buffer.resize(clordid_sz());
}
void IL3GW::connect() {
bts_assert(state() == State::INIT || state() == State::ERROR);
set_state(State::CONNECTING);
_connection.connect(*this); // -> on_connect_success
}
// This determines the session uuid
void IL3GW::send_negotiate() {
// add 2 bytes for the empty 'credentials' (39008) length which must be
// present
auto msg = _buffer.frame<iLinkBinary::Negotiate500>(
iLinkBinary::Negotiate500::credentialsHeaderLength());
msg.uUID(_session.uuid);
msg.requestTimestamp(_time.now());
msg.putSession(_config.cme_session);
// We need this call to set the length of the credential block to 0
// This behavior is specific to "data" in the xml template
msg.putCredentials("", 0);
msg.putFirm(_config.firm);
const auto ld = glbx::calculate_hmac(
_config.cme_session, std::to_string(msg.requestTimestamp()),
std::to_string(msg.uUID()), _config.cme_session, _config.firm);
msg.putAccessKeyID(ld.access_key);
msg.putHMACSignature(ld.hmac);
std::string temp = ld.canonical;
for (size_t x = 0; x < temp.size(); ++x) {
if (temp[x] == '\n')
temp[x] = '$';
}
flush(false);
// clear HMAC for json logging
msg.putHMACSignature(std::string_view{nullptr, 0});
log_out(msg, 0);
// expect -> NegotiationResponse501 or NegotiationReject502
}
void IL3GW::send_establish() {
// add 2 bytes for the empty 'credentials' (39008) length which must be
// present
auto msg = _buffer.frame<iLinkBinary::Establish503>(2);
msg.uUID(_session.uuid);
msg.requestTimestamp(_time.now());
msg.putSession(_config.cme_session);
msg.putCredentials("", 0);
msg.putFirm(_config.firm);
msg.putTradingSystemName(std::string_view("SPARK"));
msg.putTradingSystemVersion(std::string_view("1"));
msg.putTradingSystemVendor(std::string_view("BTS"));
// On primary connection, the expected message sequence number of the next
// business message from customer to CME
msg.nextSeqNo(_session.to_cme);
msg.keepAliveInterval(30000);
const auto ld =
glbx::calculate_hmac(_config.cme_session,
// RequestTimestamp
std::to_string(msg.requestTimestamp()),
std::to_string(msg.uUID()), _config.cme_session,
// Firm ID
_config.firm,
// TradingSystemName
msg.getTradingSystemNameAsString(),
// TradingSystemVersion
msg.getTradingSystemVersionAsString(),
// TradingSystemVendor
msg.getTradingSystemVendorAsString(),
// NextSeqNo
std::to_string(msg.nextSeqNo()),
// KeepAliveInterval
std::to_string(msg.keepAliveInterval()));
msg.putAccessKeyID(ld.access_key);
msg.putHMACSignature(ld.hmac);
flush(false);
// clear HMAC for json logging
msg.putHMACSignature(std::string_view{nullptr, 0});
log_out(msg, 0);
// expect -> EstablishmentAck504 or EstablishmentReject505
}
void IL3GW::send_sequence(bool lapsed) {
auto msg = _buffer.frame<iLinkBinary::Sequence506>();
msg.uUID(_session.uuid);
msg.nextSeqNo(_session.to_cme);
msg.faultToleranceIndicator(iLinkBinary::FTI::NULL_VALUE);
msg.keepAliveIntervalLapsed(lapsed
? iLinkBinary::KeepAliveLapsed::Lapsed
: iLinkBinary::KeepAliveLapsed::NotLapsed);
log_out(msg, 0);
}
void IL3GW::send_retransmit_request(uint64_t uuid, uint32_t from,
size_t count) {
bts_assert(from > 0);
using namespace iLinkBinary;
auto msg = _buffer.frame<RetransmitRequest508>();
msg.uUID(_session.uuid);
msg.lastUUID(uuid);
msg.requestTimestamp(_time.now());
msg.fromSeqNo(from);
msg.msgCount(count);
log_out(msg, 0);
}
uint64_t IL3GW::send_party_details(const api::orderman::RoutingRecord &route) {
using namespace iLinkBinary;
auto msg = _buffer.frame<PartyDetailsDefinitionRequest518>(
// 2 repeating group header.
// 1 for party details and 1 for trade reg
// This seems unsafe but the api doesn't seem to want you to know how to
// size buffers ahead of time
2 * PartyDetailsDefinitionRequest518::NoPartyDetails::sbeHeaderSize() +
// N * size of each block
3 * PartyDetailsDefinitionRequest518::NoPartyDetails::sbeBlockLength() +
0 * PartyDetailsDefinitionRequest518::NoTrdRegPublications::
sbeBlockLength());
msg.partyDetailsListReqID(0);
msg.sendingTimeEpoch(_time.now());
msg.listUpdateAction(ListUpdAct::Add);
msg.seqNum(next_seq());
// msg.putMemo("") -- not supported
if (route.smp_id().empty()) {
msg.selfMatchPreventionID(
PartyDetailsDefinitionRequest518::selfMatchPreventionIDNullValue());
} else {
auto smp_id = std::atoi(route.smp_id().c_str());
msg.selfMatchPreventionID(smp_id);
}
msg.cmtaGiveupCD(CmtaGiveUpCD::NULL_VALUE);
msg.custOrderCapacity(map_cust_cti(route.cti_code()));
msg.clearingAccountType(map_customer_or_firm(route.customer_or_firm()));
msg.selfMatchPreventionInstruction(
map_smp_instruction(route.smp_instruction()));
msg.avgPxIndicator(AvgPxInd::NULL_VALUE);
msg.clearingTradePriceType(SLEDS::NULL_VALUE);
msg.custOrderHandlingInst(static_cast<iLinkBinary::CustOrdHandlInst::Value>(
route.cust_order_handing_inst()[0]));
msg.executor(PartyDetailsDefinitionRequest518::executorNullValue());
msg.iDMShortCode(PartyDetailsDefinitionRequest518::iDMShortCodeNullValue());
auto details = msg.noPartyDetailsCount(3);
details.next();
details.partyDetailRole(PartyDetailRole::CustomerAccount);
details.putPartyDetailID(route.account());
details.next();
details.partyDetailRole(PartyDetailRole::ExecutingFirm);
// 004 in cert
details.putPartyDetailID(
std::string_view(route.sender_compid().c_str() + 3, 3));
details.next();
details.partyDetailRole(PartyDetailRole::Operator);
details.putPartyDetailID(std::string_view(route.sender_compid().c_str(), 3));
msg.noTrdRegPublicationsCount(0);
// In the future we may check for existing party details definitions and
// return that id
log_out(msg, msg.seqNum());
return 0;
}
bool needs_stop_px(api::orderman::NativeOrderType order_type) {
return order_type ==
api::orderman::NativeOrderType::NATIVE_ORDER_TYPE_STOP ||
order_type ==
api::orderman::NativeOrderType::NATIVE_ORDER_TYPE_STOP_LIMIT;
}
bool need_px(api::orderman::NativeOrderType order_type) {
return order_type !=
api::orderman::NativeOrderType::NATIVE_ORDER_TYPE_MARKET &&
order_type !=
api::orderman::NativeOrderType::NATIVE_ORDER_TYPE_MARKET_LIMIT &&
order_type != api::orderman::NativeOrderType::NATIVE_ORDER_TYPE_STOP;
}
void IL3GW::send_order(const CreateOrderMsg &create) {
bts_assert_d(create.routing_record);
bts_assert_d(create.security);
bts_assert_d(create.security->has_glbx());
auto party = send_party_details(*create.routing_record);
auto msg = _buffer.frame<iLinkBinary::NewOrderSingle514>();
if (need_px(create.order_type)) {
msg.price().mantissa(to_price9(create.price));
} else {
msg.price().mantissa(iLinkBinary::PRICENULL9::mantissaNullValue());
}
msg.orderQty(create.quantity.value());
msg.securityID(create.security->glbx().exchange_id());
msg.side(map_side(create.side));
msg.seqNum(next_seq());
msg.putSenderID(create.routing_record->operator_id());
encode_clorid(create.key, SecurityKey(create.security->id()),
&_clorid_buffer);
msg.putClOrdID(_clorid_buffer);
msg.partyDetailsListReqID(party);
msg.orderRequestID(++_msg_count);
msg.sendingTimeEpoch(_time.now());
if (needs_stop_px(create.order_type)) {
msg.stopPx().mantissa(to_price9(create.stop_px));
} else {
msg.stopPx().mantissa(iLinkBinary::PRICENULL9::mantissaNullValue());
}
msg.putLocation(create.routing_record->location());
msg.minQty(iLinkBinary::NewOrderSingle514::minQtyNullValue());
if (create.time_in_force != api::orderman::TimeInForce::TIME_IN_FORCE_IOC) {
msg.displayQty(create.display_quantity.value());
} else {
msg.displayQty(iLinkBinary::NewOrderSingle514::displayQtyNullValue());
}
if (create.time_in_force == api::orderman::TimeInForce::TIME_IN_FORCE_DATE) {
// TODO: # of days since jan 1 1970
msg.expireDate(18424);
} else {
msg.expireDate(iLinkBinary::NewOrderSingle514::expireDateNullValue());
}
msg.ordType(map_order_type(create.order_type));
msg.timeInForce(map_time_in_force(create.time_in_force));
msg.manualOrderIndicator(
map_manual_order_indicator(create.manual_order_indicator));
msg.execInst().clear();
msg.executionMode(iLinkBinary::ExecMode::NULL_VALUE);
msg.liquidityFlag(iLinkBinary::BooleanNULL::NULL_VALUE);
msg.managedOrder(iLinkBinary::BooleanNULL::NULL_VALUE);
msg.shortSaleType(iLinkBinary::ShortSaleType::NULL_VALUE);
log_out(msg, msg.seqNum());
}
void IL3GW::send_cancel_replace(const CancelReplaceOrderMsg &replace) {
const auto &create = replace.create_fields;
bts_assert_d(create.routing_record);
bts_assert_d(create.security);
bts_assert_d(create.security->has_glbx());
auto party = send_party_details(*create.routing_record);
auto msg = _buffer.frame<iLinkBinary::OrderCancelReplaceRequest515>();
msg.orderID(replace.exchange_order_id.value());
if (need_px(create.order_type)) {
msg.price().mantissa(to_price9(create.price));
} else {
msg.price().mantissa(iLinkBinary::PRICENULL9::mantissaNullValue());
}
msg.orderQty(create.quantity.value());
msg.securityID(create.security->glbx().exchange_id());
msg.side(map_side(create.side));
msg.seqNum(next_seq());
msg.putSenderID(create.routing_record->operator_id());
encode_clorid(create.key, SecurityKey(create.security->id()),
&_clorid_buffer);
msg.putClOrdID(_clorid_buffer);
msg.partyDetailsListReqID(party);
msg.orderRequestID(++_msg_count);
msg.sendingTimeEpoch(_time.now());
if (needs_stop_px(create.order_type)) {
msg.stopPx().mantissa(to_price9(create.stop_px));
} else {
msg.stopPx().mantissa(iLinkBinary::PRICENULL9::mantissaNullValue());
}
msg.putLocation(create.routing_record->location());
msg.minQty(iLinkBinary::NewOrderSingle514::minQtyNullValue());
msg.displayQty(create.display_quantity.value());
msg.expireDate(iLinkBinary::NewOrderSingle514::expireDateNullValue());
msg.ordType(map_order_type(create.order_type));
msg.timeInForce(map_time_in_force(create.time_in_force));
msg.manualOrderIndicator(
map_manual_order_indicator(create.manual_order_indicator));
msg.oFMOverride(iLinkBinary::OFMOverrideReq::Enabled);
msg.execInst().clear();
msg.executionMode(iLinkBinary::ExecMode::NULL_VALUE);
msg.liquidityFlag(iLinkBinary::BooleanNULL::NULL_VALUE);
msg.managedOrder(iLinkBinary::BooleanNULL::NULL_VALUE);
msg.shortSaleType(iLinkBinary::ShortSaleType::NULL_VALUE);
log_out(msg, msg.seqNum());
}
void IL3GW::send_cancel(const CancelOrderMsg &cancel) {
bts_assert_d(cancel.routing_record);
bts_assert_d(cancel.security);
bts_assert_d(cancel.security->has_glbx());
auto party = send_party_details(*cancel.routing_record);
auto msg = _buffer.frame<iLinkBinary::OrderCancelRequest516>();
msg.orderID(cancel.exchange_order_id.value());
msg.partyDetailsListReqID(party);
msg.manualOrderIndicator(
map_manual_order_indicator(cancel.manual_order_indicator));
msg.seqNum(next_seq());
msg.putSenderID(cancel.routing_record->operator_id());
encode_clorid(cancel.key, SecurityKey(cancel.security->id()),
&_clorid_buffer);
msg.putClOrdID(_clorid_buffer);
msg.orderRequestID(++_msg_count);
msg.sendingTimeEpoch(_time.now());
msg.putLocation(cancel.routing_record->location());
msg.securityID(cancel.security->glbx().exchange_id());
msg.side(map_side(cancel.side));
msg.liquidityFlag(iLinkBinary::BooleanNULL::NULL_VALUE);
log_out(msg, msg.seqNum());
}
void IL3GW::send_terminate(uint16_t code) {
LOG(iLink3, Info, "Terminate");
auto msg = _buffer.frame<iLinkBinary::Terminate507>();
msg.uUID(_session.uuid);
msg.requestTimestamp(_time.now());
msg.errorCodes(code);
log_out(msg, 0);
}
void IL3GW::flush(bool publish) {
_buffer.flush(_connection);
if (publish)
publish_session_update();
}
const char *state_string(State s) {
switch (s) {
case State::INIT:
return "INIT";
case State::CONNECTING:
return "CONNECTING";
case State::NEGOTIATING:
return "NEGOTIATING";
case State::ESTABLISHING:
return "ESTABLISHING";
case State::RETRANSMITTING:
return "RETRANSMITTING";
case State::READY:
return "READY";
case State::ERROR:
return "ERROR";
}
return "UNKNOWN";
}
log::CharPtrWrapper wstate_string(State s) { return wrap(state_string(s)); }
void IL3GW::set_state(State s) {
if (s != _x_state) {
LOG(iLink3, Info, "ILinkGW {}: {}->{}", _session.uuid,
wstate_string(_x_state), wstate_string(s));
_x_state = s;
if (_on_state_change)
_on_state_change(s);
}
}
//
// IConnectionHandler
void IL3GW::on_connect_success() {
if (!_session) {
// If we have a 0 cme_seq this is a new session and we need to negotiate
// it
LOG(iLink3, Info, "Negotiating new iLink session. {}", _session.uuid);
set_state(State::NEGOTIATING);
send_negotiate();
} else {
// We only need to establish the current sequence number
LOG(iLink3, Info, "re-establishing existing iLink session. {}",
_session.uuid);
_session.to_cme = std::max(_session.to_cme, 1u);
set_state(State::ESTABLISHING);
send_establish();
}
}
void IL3GW::on_disconnect() {
LOG(iLink3, Error, "Disconnected.");
if (_config.failover_on_disconnect) {
LOG(iLink3, Error, "Attempting failover");
_connection.failover();
connect();
} else {
set_state(State::ERROR);
}
}
void IL3GW::enqueue_retransmit(uint32_t from, uint32_t to, uint64_t uuid) {
LOG(iLink3, Warn, "Prepare retransmit from: {} to: {}", from, to);
bts_assert(to >= from);
do {
// +1 because to/from are using an inclusive range
auto count = std::min(to - from + 1, Retransmit::MAX_RECOVERY);
_retransmit.push(Retransmit{uuid, from, from + count - 1});
from += count;
} while (from < to);
}
void IL3GW::begin_retransmit() {
if (_retransmit.empty())
return;
auto &working = _retransmit.front();
// Don't pop the queue because we need to know when we're done
auto count = working.to - working.from + 1;
LOG(iLink3, Warn,
"Sending retransmit for missing packets: from: {} to: {} count: {}",
working.from, working.to, count);
bts_assert_d(count <= Retransmit::MAX_RECOVERY);
send_retransmit_request(working.uuid, working.from, count);
}
void IL3GW::on_retransmit_complete() {
// Replay is complete. Advance to go. Collect $200
_retransmit.pop();
LOG(iLink3, Warn, "Retransmit complete");
if (!_retransmit.empty()) {
// More replay to do...
begin_retransmit();
flush();
} else {
// Replay is finished, transition to ready state.
// Since we're already established we dont' need to send anything
set_state(State::READY);
}
}
size_t IL3GW::on_data(const std::string_view &data) {
return consume_messages(*this, data.data(), data.size(),
dispatch_message<IL3GW>);
}
void IL3GW::handle(const iLinkBinary::NegotiationResponse501 &msg) {
// If we're receiving a NegotiateResponse we just created a session
log_in(msg, 0);
// NegotiationResponse indicates the session was created. CME sequence will
// be 1
_session.from_cme = 1;
publish_session_update();
// Even though the CME sends a previous uuid+sequence we can't act on
// that data right now since we aren't established. We don't have to remember
// it, though, because exactly the same data comes on the EstablishmentAck
// msg.previousUUID(), msg.previousSeqNo();
set_state(State::ESTABLISHING);
send_establish();
}
void IL3GW::handle(const iLinkBinary::NegotiationReject502 &msg) {
LOG(iLink3, Error, "Negotiate Reject: {} '{}'.", msg.errorCodes(),
std::string(translate_negotiation_reject(msg.errorCodes())));
log_in(msg, 0);
_connection.disconnect();
set_state(State::ERROR);
}
void IL3GW::handle(const iLinkBinary::EstablishmentAck504 &msg) {
// previous uuid+sequence
auto cme_prev_uuid = msg.previousUUID();
auto cme_prev_seq = msg.previousSeqNo();
if (cme_prev_seq != 0 && cme_prev_uuid != _session.uuid) {
// There was a previous session and we need to replay messages from it
// We *intend* to only have a single session each week except
// in the case of catastrophic failure. So If we log in and
// there is a previous UUID we can just replay it entirely.
// The other possibility is a session was created automatically
// to handle a fill from a GTC order from the previous week
enqueue_retransmit(1, cme_prev_seq, cme_prev_uuid);
}
// next sequence number from the current session may also be ahead
auto cme_current_seq = msg.nextSeqNo() - 1;
if (_session.from_cme < cme_current_seq) {
// If we're behind it indicates we missed messages while disconnected
enqueue_retransmit(_session.from_cme + 1, cme_current_seq, msg.uUID());
}
_session.from_cme = cme_current_seq;
log_in(msg, 0);
if (!_retransmit.empty()) {
send_sequence();
begin_retransmit();
flush();
set_state(State::RETRANSMITTING);
} else {
send_sequence();
flush();
set_state(State::READY);
}
}
void IL3GW::handle(const iLinkBinary::EstablishmentReject505 &msg) {
LOG(iLink3, Error, "Establishment Reject: {} '{}'. NextSeq: {}.",
msg.errorCodes(),
std::string(translate_establish_reject(msg.errorCodes())),
msg.nextSeqNo());
_connection.disconnect();
log_in(msg, 0);
set_state(State::ERROR);
}
void IL3GW::handle(const iLinkBinary::Sequence506 &msg) {
log_in(msg, 0);
_session.from_cme = msg.nextSeqNo() - 1;
publish_session_update();
send_sequence();
flush();
}
void IL3GW::handle(const iLinkBinary::Terminate507 &msg) {
// https://www.cmegroup.com/confluence/display/EPICSANDBOX/Terminate
LOG(iLink3, Error, "Terminated by exchange. Code: {} Reason: {} Text: {}",
msg.errorCodes(), wrap(msg.reason()),
wrap(translate_terminate(msg.errorCodes())));
if (msg.errorCodes() == 11 || msg.errorCodes() == 10) {
// This indicates our sequence number is too low. We will jump
// forward one at a time. That should eventually fix it.
LOG(iLink3, Error,
"Will attempt to recover by advancing our sequence number");
_session.to_cme++;
publish_session_update();
}
log_in(msg, 0);
_connection.disconnect();
set_state(State::ERROR);
}
void IL3GW::handle(const iLinkBinary::Retransmission509 &msg) {
// Accepted Retransmit
log_in(msg, 0);
}
void IL3GW::handle(const iLinkBinary::RetransmitReject510 &msg) {
LOG(iLink3, Error, "Retransmit requeste rejected. {}",
msg.getReasonAsString());
log_in(msg, 0);
set_state(State::ERROR);
}
void IL3GW::handle(const iLinkBinary::NotApplied513 &msg) {
log_in(msg, 0);
send_sequence();
}
void IL3GW::handle(
const iLinkBinary::PartyDetailsDefinitionRequestAck519 &msg) {
_exe_parser.set_account(msg);
handle_common(msg);
}
void IL3GW::handle(const iLinkBinary::BusinessReject521 &msg) {
LOG(iLink3, Error, "Business Reject: {} '{}'.", msg.businessRejectReason(),
wrap(translate_business_reject_reason(msg.businessRejectReason())));
handle_common(msg);
// A business reject can de-sync the sequence number because it rejects
// the sequence number of the rejected message. Send a sequence message
// to tell the cme to advance.
send_sequence();
flush();
}
void IL3GW::handle(const iLinkBinary::ExecutionReportNew522 &msg) {
handle_exe_report(msg);
}
void IL3GW::handle(const iLinkBinary::ExecutionReportReject523 &msg) {
LOG(iLink3, Error, "Execution report reject: {}", wrap(msg.text()));
handle_exe_report(msg);
}
void IL3GW::handle(const iLinkBinary::ExecutionReportElimination524 &msg) {
handle_exe_report(msg);
}
void IL3GW::handle(const iLinkBinary::ExecutionReportTradeOutright525 &msg) {
handle_exe_report(msg);
}
void IL3GW::handle(const iLinkBinary::ExecutionReportTradeSpread526 &msg) {
handle_exe_report(msg);
}
void IL3GW::handle(const iLinkBinary::ExecutionReportTradeSpreadLeg527 &msg) {
handle_exe_report(msg);
}
void IL3GW::handle(const iLinkBinary::ExecutionReportModify531 &msg) {
handle_exe_report(msg);
}
void IL3GW::handle(const iLinkBinary::ExecutionReportStatus532 &msg) {
handle_exe_report(msg);
}
void IL3GW::handle(const iLinkBinary::ExecutionReportCancel534 &msg) {
handle_exe_report(msg);
}
void IL3GW::handle(const iLinkBinary::OrderCancelReject535 &msg) {
CancelReject cr;
bool result = _exe_parser.parse(msg, &cr);
if (result) {
_on_cancel_reject(cr);
} else {
LOG(iLink3, Error, "Failed to parse cancel reject.");
}
handle_common(msg);
}
void IL3GW::handle(const iLinkBinary::OrderCancelReplaceReject536 &msg) {
handle_common(msg);
}
void IL3GW::handle(
const iLinkBinary::ExecutionReportTradeAddendumOutright548 &msg) {
// Ignore trade correction/addendum messages for now.
// See GLBXExecutionGateway.cpp:139 for further details
handle_common(msg);
}
void IL3GW::handle(
const iLinkBinary::ExecutionReportTradeAddendumSpread549 &msg) {
// Ignore trade correction/addendum messages for now.
// See GLBXExecutionGateway.cpp:139 for further details
handle_common(msg);
}
void IL3GW::handle(
const iLinkBinary::ExecutionReportTradeAddendumSpreadLeg550 &msg) {
// Ignore trade correction/addendum messages for now.
// See GLBXExecutionGateway.cpp:139 for further details
handle_common(msg);
}
} // namespace ilink3
} // namespace orderman
} // namespace bts
Loaded 100 of 284 files, more files were not shown because too many files have changed in this diff. Show more