Compare commits

..
Author SHA1 Message Date
John Colvin b3566155a3 fix: Fix rounding error when creating tick factors 2025-06-16 15:52:09 +00:00
John Colvin a4e39afbb9 ui: Add cash-per-point col to mview 2025-06-16 15:52:09 +00:00
John Colvin efce01d28e vscode: Replace net7 w/ net8 in tasks.json 2025-06-16 15:52:09 +00:00
John Colvin d362022b89 Add c# SyntheticBookCalculator 2025-06-16 15:52:09 +00:00
John Colvin e7d751fb68 refactor: Remove unnecessary constraint on MathUtils.lcm params
Not sure why T ever needed to implement IMultiplyOperators<T, int, T>
2025-06-16 15:52:09 +00:00
John Colvin 50eefc65d4 cleanup: Remove unnecessary py_module.cpp files
We used to have separate modules for md and execution. I forgot
to remove these after unifying them to a single module.
2025-06-16 15:52:09 +00:00
John Colvin 8770de4a87 diag: Reduce book log noise in md plugins
They're too noisy for Info level.
2025-06-16 15:52:09 +00:00
John Colvin e799b6e24b diag: Optimize logging in bybit exe gw plugin
* Format params as json (not python pseudo json)
* Format params lazily
2025-06-16 15:52:09 +00:00
John Colvin 69e338c1ca fix: Use lock_guard instead of lock/unlock calls
I just noticed it was technically possible for the "work thread" (in py
plugin manager) to be started twice, due to improper lock/unlock calls.
2025-06-16 11:08:59 -04:00
John Colvin e8bb64a80f Increase sleep time in bybit_md_provider keep-alive loop
This thread's only job is to stay alive until "stop" is called. It
doesn't need to be particularly fast/responsive. Longer sleep intervals
reduce contention with the "hot" thread that receives market data.
2025-06-16 11:08:59 -04:00
John Colvin 8e2ede596f diag: Add security to assert msg when qty tick deduction fails 2025-06-16 11:08:59 -04:00
John Colvin 9772d0eb77 deploy: Add alphaworx_zen_hk_live-preview configuration 2025-06-16 11:08:59 -04:00
John Colvin 687e2477c0 fix: Assert that acc name length < max_size (not size)
Since we changed the account field from std::array -> bts::core::SmallString
2025-06-16 11:08:59 -04:00
John Colvin 1bd0ba9950 deploy: Install python requirements in docker
No point in using a virtual environment. These containers have one
purpose: run sparkd with a constant, predetermined set of py plugins.
2025-06-16 11:08:59 -04:00
John Colvin ddabc744f2 chore: Update alphaworx_zen_hk_live init.py to query new secret
I decided to put all alphaworx api keys in one secret. It's easier this way.
2025-06-16 11:08:58 -04:00
John Colvin 74b88b003e fix: Fix config dir env var in bybit_dev profile 2025-06-16 11:08:58 -04:00
John Colvin aaf1be09c2 fix: Fix SmallString numeric parsing, make more robust
* Fix by parsing numbers from left to right instead of right to left
  (previously, would parse "17" to 71)
* Support leading +/- signs
* Throw if unexpected char encountered
* Unit tests
2025-06-16 11:08:58 -04:00
John Colvin e3bd9a457c fix: Change thread_id type to size_t (not int64_t) 2025-06-16 11:08:58 -04:00
John Colvin 59f62b6ec9 cleanup: Remove unnecessary signal handler in bybit_md_provider.py
We handle them properly in sparkd now
2025-06-16 11:08:58 -04:00
John Colvin b46e75f7d7 cleanup: Sort gateio symbols in symbols.conf 2025-06-16 11:08:58 -04:00
John Colvin 3f100feaca chore: Add config json for py plugins
Support overrides per venue in a separate file, so symbols can be
changed after deployment using arham's widget.
2025-06-16 11:08:58 -04:00
John Colvin 093da2a938 fix: Init shared_ptr w/ default obj (not null)
Prob a typo
2025-06-16 11:08:58 -04:00
John Colvin 2f17c2b8c5 chore: Fix path in .env file (server/sparkd -> app_sparkd) 2025-06-16 11:08:58 -04:00
John Colvin 33bfdc517a fix build: Remove include of non-existent header
I created the file locally, but ended up not using it and forgot to remove an
include before I pushed.
2025-06-16 11:08:58 -04:00
John Colvin d17be48e6d fix build: Always return value in non-void fn 2025-06-16 11:08:58 -04:00
John Colvin cbaa1df641 fix: Don't compare size_t to -1 in SmallString num parsers 2025-06-16 11:08:58 -04:00
John Colvin 8c23665d65 refactor: Manage all py plugins (md and exe) in one class
All calls to/from python are performed on one thread via a simple
message queue (necessary because of the global interpreter lock).
2025-06-16 11:08:58 -04:00
John Colvin c7db105533 chore: Add pybit to py venv 2025-06-16 11:08:58 -04:00
John Colvin 3f9923239e diag: Log incoming books at Info level in bybit_md_provider 2025-06-16 11:08:58 -04:00
John Colvin 9ae9451359 chore: Add bybit execution config to bybit_dev profile 2025-06-16 11:08:58 -04:00
John Colvin c33e5ddb02 chore: Add numpy to py venv
This is needed for md plugins
2025-06-16 11:08:58 -04:00
John Colvin 54388d3b92 fix: Include fill id in format str for ExecutionReport 2025-06-16 11:08:58 -04:00
John Colvin 17ada3c57c dev qol: Bind PySecurity __format__ to fmt::formatter impl 2025-06-16 11:08:58 -04:00
John Colvin 046f412ec4 refactor: Change floats to doubles in py bridge types
Getting rounding errors with floats
2025-06-16 11:08:58 -04:00
John Colvin c94d11f525 refactor: Use SmallString instead of std::array in ExchangeFillID 2025-06-16 11:08:57 -04:00
John Colvin 2b69846381 Add py bybit exe config to bybit_dev profile 2025-06-16 11:08:57 -04:00
John Colvin bdd5e08b60 refactor: Consolidate all py interop into single lib
It was just too hard trying to make it work with separate libs. The
main issue was getting the python md and execution modules to
recognize PySecurityList and other types defined in the core module.
2025-06-16 11:08:57 -04:00
John Colvin 7c9330650d refactor: Move tick fns into gen-purpose tick_utils.h 2025-06-16 11:08:57 -04:00
John Colvin 98fca5ad7f Change bybit_dev symbols.conf to list bybit futs (not spots) 2025-06-16 11:08:57 -04:00
John Colvin 8a0fcce90d fix: Actually use configured logger in bybit_md_provider.py 2025-06-16 11:08:57 -04:00
John Colvin 112527f9ed dev qol: Reduce noisy logs in PositionServicePNLHelper 2025-06-16 11:08:57 -04:00
John Colvin fc1e20538d build: Add cmake script to sym link py module and plugins
We have several of these (python_core, python_md, python_execution),
so the repetative boilerplate code was starting to pile up.
2025-06-16 11:08:57 -04:00
John Colvin 83acbecf55 deploy: Install/activate py venv in sparkd container 2025-06-16 11:08:57 -04:00
John Colvin 06c494291c dev qol: Add pyrightconfig.json for type hints in py/c++ bridge 2025-06-16 11:08:57 -04:00
John Colvin fe74aed048 Tweak py to get rid of false positive type hint warns 2025-06-16 11:08:57 -04:00
John Colvin b94cdb561e feat: Support python execution plugins 2025-06-16 11:08:57 -04:00
John Colvin 06bd8791c0 refactor: Move py security (and related types) to python_core lib
So the execution gw plugins can use it (as well as the md plugins).
2025-06-16 11:08:57 -04:00
John Colvin b5850cdb02 fix: Add missing switch cases for bybit & gateio 2025-06-16 11:08:57 -04:00
John Colvin 545d76142f fix: Initialize bybit secs as perp futs (not spots) 2025-06-16 11:08:57 -04:00
John Colvin edc2cea60e dev qol: Add wait/notify functionality to ConcurrentQueue 2025-06-16 11:08:56 -04:00
John Colvin e6466f8309 chore: Add CANCEL_ORDER op to binance.env 2025-06-16 11:08:56 -04:00
John Colvin 501be8d6a5 dev qol: Add ConcurrentValueStore, with notify/wait functionality
Uses std::condition_variable
2025-06-16 11:08:56 -04:00
John Colvin 7dea9b3792 fix: Don't raise ex if var isn't defined in req path 2025-06-16 11:08:56 -04:00
John Colvin a3f49b1ce8 dev qol: Add gen-purpose validation fns for execution gws 2025-06-16 11:08:56 -04:00
John Colvin 765237d701 build: Use new gen-purpose python cmake fns in py md proj
Significantly simplifies CMakeLists.txt
2025-06-16 11:08:56 -04:00
John Colvin 97c52410a2 build: Add gen-purpose InitPythonModule, InitPythonPlugin cmake scripts 2025-06-16 11:08:56 -04:00
John Colvin 8b43d93e26 dev qol: Add fn to make Stopwatch and start it at once 2025-06-16 11:08:56 -04:00
John Colvin 14baa7d62e chore: Use same log format as sparkd in py md plugins 2025-06-16 11:08:56 -04:00
John Colvin a054f4cc63 improvement: Set PyBook data in single call to reduce interop overhead 2025-06-16 11:08:56 -04:00
John Colvin 5311128607 Reduce log noise in py md providers 2025-06-16 11:08:56 -04:00
John Colvin 075acdc8eb doc: Improve py md readme
* Put examples in a table
* Show command to create symlinks for intellisense
2025-06-16 11:08:56 -04:00
John Colvin e1fdf6306b refactor: Redesign py md so it doesn't lock the global interpreter
We can only embed one interpreter in the sparkd process, so all py
plugins need to share it.

* Move core py interop code to new lib_python_core. We're about to add a
  new lib for py execution, so it needs to be accesible from both
  execution and md libs
* Fix typo for type hint in py_md_utils.py
* Run py md providers on background thread
* Rename MdSink -> MdBridge
2025-06-16 11:08:56 -04:00
John Colvin 3b9fb2759b build: Add lib_spark subdir first (all others depend on it) 2025-06-16 11:08:56 -04:00
John Colvin 3f89d3b7f7 dev qol: Add to_chrono fns to TimeSpan 2025-06-16 11:08:56 -04:00
John Colvin a6dd0d842d dev qol: Add -> operators to ValueWrapper for convenience
So we can type "foo->" instead of "foo.value()" (much shorter!)
2025-06-16 11:08:56 -04:00
John Colvin d050d2d19a build: Link against system installed ssl to support py interop
Unless we want to build python packages ourselves, the underlying clibs
they use link to the system installed ssl. Notably, the requests module.
2025-06-16 11:08:56 -04:00
John Colvin 942d00a8b1 chore: Add pybind11, pybind11-stubgen to py venv 2025-06-16 11:08:56 -04:00
John Colvin 31cb25ac2e improvement: Generate py stubs for md providers
Provides intellisense for the python_md_module
2025-06-16 11:08:56 -04:00
John Colvin cbb43a22de cleanup: Get rid of unecessary namespace prefixes 2025-06-16 11:08:55 -04:00
John Colvin 9e7b4849f0 cleanup: Get rid of "PyList" classes, bind to stl vector instead
I wasn't aware this was possible until recently.
2025-06-16 11:08:55 -04:00
John Colvin 6e72890153 refactor: Move SimpleOrderKeyMap from binance_execution to lib_spark
It provides basic functionality that will be useful in other execution gateways.
2025-06-16 11:08:55 -04:00
John Colvin c238fce057 chore: Add smart parse and pretty name fns for proto enums
Mainly to ease the pain of representing enums as strings in python md
and execution plugins
2025-06-16 11:08:55 -04:00
John Colvin b8f5db4ee2 refactor: Move ErrorList from binance_execution to lib_spark
It provides basic functionality that will be useful elsewhere
2025-06-16 11:08:55 -04:00
John Colvin 87c8b9862a refactor: Use SmallString std::array in AccountName 2025-06-16 11:08:55 -04:00
John Colvin fbc2d8633d refactor: Use SmallString instead of std::array in ExchangeOrderID 2025-06-16 11:08:55 -04:00
John Colvin d12d798fa5 Add our own "SmallString" class for fixed cap strings
This will replace the std::array<char, N> we use in ExecutionReport, etc.
among other things
2025-06-16 11:08:55 -04:00
John Colvin e75be2da2f refactor: Move ExecutionReportStore to lib_spark
From lib_binance_execution
2025-06-16 11:08:55 -04:00
Adeel Khan 3ae3a7094e CLEANUP: BYBIT Symbols.conf. 2025-06-16 11:08:55 -04:00
Adeel Khan bafa2f71ba ADD: BYBIT MD Working, NaN's also Added. 2025-06-16 11:08:55 -04:00
Adeel Khan 63bd29a49e ADD: BYBIT Securities showing up on frontend. Default Contract Spec 2025-06-16 11:08:55 -04:00
Adeel Khan 5fdc9ca48b ADD: GATEIO md_provider with dynamic contract specs using Batch REST call 2025-06-16 11:08:55 -04:00
Adeel Khan a19e7bc7b4 ADD: GATEIO md_provider with dynamic contract specs 1 REST call per Contract 2025-06-16 11:08:55 -04:00
Adeel Khan 04b0a89644 ADD: GATEIO md_provider with default contract specs 2025-06-16 11:08:55 -04:00
John Colvin 363a43cb8b Add gateio_dev config profile for Adeel 2025-06-16 11:08:55 -04:00
John Colvin c9f9c5f0f6 cleanup: Remove unused spark.conf file
Originally, this spark.conf was hardcoded into compose.yaml, and copied
into the sparkd container on each deploy. It hasn't been used since
config profiles were introduced.
2025-06-16 11:08:54 -04:00
John Colvin ab84a05644 Add py_dummy_md config profile to test py md plugin 2025-06-16 11:08:54 -04:00
John Colvin 4681f22e71 deploy: Move configurations to new stutter dir "app_sparkd" 2025-06-16 11:08:54 -04:00
John Colvin a25e055e35 Add protos for gateio 2025-06-16 11:08:54 -04:00
John Colvin 2386612b51 Add protos for bybit 2025-06-16 11:08:54 -04:00
John Colvin 3606dac145 build: Move onix fix adapter to its own lib
We should keep order execution dependencies out of lib_spark.
2025-06-16 11:08:54 -04:00
John Colvin bf936c9d13 fix build: Move OnixsUtils to from lib_spark to app_sparkd
They're only needed MDP3Main.cpp and ICEMain.cpp. We're one step closer
to removing lib_spark's dependency on onixs.
2025-06-16 11:08:54 -04:00
John Colvin 1aeeb7170e cleanup: Remove unnecessary onixs include from TrafixOrderStateHandler.h 2025-06-16 11:08:54 -04:00
John Colvin 78ad8a3aed feat: Support python md plugins 2025-06-16 11:08:54 -04:00
John Colvin 5ca8dbafaf Add "clear" fn to Book & BookSide
Prelude to python market data bindings
2025-06-16 11:08:54 -04:00
John Colvin b464b9466e Add "smart" parse fns for a couple proto enums
Prelude to python market data bindings. Unlike the normal parse fns, the
"smart" variants aren't case sensitive, and can match strings with or
without TYPE_NAME_ prefixes.
2025-06-16 11:08:54 -04:00
John Colvin 7a5c676e99 build: Move okx execution to its own lib 2025-06-16 11:08:54 -04:00
John Colvin 1e8d699dd1 cleanup: Remove unnecessary onixs include from IDGenerator.h 2025-06-16 11:08:53 -04:00
John Colvin aadd2dd594 cleanup: Remove unnecessary includes from MDP3Listener.h 2025-06-16 11:08:53 -04:00
John Colvin 5163ca4cae refactor: Organize libspark proj w/ stuttering include
This new structure provides several benefits:

* Allows us to install multiple CMake projects in the same destination
  dir (without include path conflicts).
  - This makes shipping the trading algo sdk a trivial excercise. Simply
    copy the lib and include dirs from the install dest. That's it!
* Puts us in a position to break up the monolithic libspark into smaller
  modules, which are easier to manage.

Windows specific notes:

* Get rid of unnecessary script vs_filters.cmake
* Define WebsocketConnectionGateway directly in cs (instead of proto)
  - The proto would no longer be in the include path passed to protoc.
    Moving it to c# because there wasn't a compelling reason for it to
    be a proto in the first place.
* Set spark output prefix to "lib" on windows
  - So we get "libspark.dll" instead of "spark.dll", which conflicts
    with the csharp lib, "Spark.dll".
* Copy libspark.dll to net8.0 & net8.0-windows in a single cmake build
  target

fix build: Copy libspark to both dirs net8.0, net8.0-windows
2025-06-16 11:08:53 -04:00
John Colvin b5d520d568 cleanup: Move TradeSummarySignalDispatcher to libspark
It is needed by TradeSummarySignal, which is in libspark. This prevents a
circular dependency from libspark -> sparkd.
2025-06-16 11:08:51 -04:00
John Colvin 83e4465a3b build: Clean up protobuf cmake scripts
Make them more flexible, reusable. This is a precursor to porting over
python md provider code from another branch.
2025-06-16 11:08:51 -04:00
John Colvin 54cfb343a8 build: Rename external path vars w/ "EXTERNAL_" prefix in cmake
The old "SPARK_" prefix felt like a misnomer.
2025-06-16 11:08:51 -04:00
John Colvin 4e9b360e23 Add a few arithmetic ops to TimeSpan 2025-06-16 11:08:51 -04:00
John Colvin 16bc93b1a0 diag: Log time it takes to start md sources
So we can detect bottlenecks
2025-06-16 11:08:50 -04:00
John Colvin f56d4d9062 refactor: Split MarketDataGateway up into providers/sources
MarketDataGateway still exists, but is greatly simplified. It creates
several `IMarketDataProvider`s, each of which generates a list of
`IMarketDataSource`s. The market data sources provide market data to an
`IMarketDataListener`.

* EmumbaMarketDataProvider
* CCApiMarketDataProvider
* PyMarketDataProvider
* OnixsMarketDataProvider
* ReplayMarketDataProvider

As a side effect, individual py market data sources are now explicitly
limited to a single venue (it probably should have been this way all
along, since the shared memory writers do not expect updates to involve
multiple venues).
2025-06-16 11:08:50 -04:00
John Colvin 36cc186626 cleanup: Remove dependency on quickfix for string to_lower
We shouldn't depend on external libs for trivial things like converting
strings to lower case.
2025-06-16 11:08:50 -04:00
John Colvin 2e78af1756 fix build: Upgrade IronPython to 3.4.2
All csprojs need to be on same version (Spark.csproj was on 3.4.1, while
DataStream.csproj was on 3.4.2)
2025-06-16 11:08:50 -04:00
13 changed files with 323 additions and 28 deletions

No files matched your search

+17 -17
View File
@@ -363,7 +363,7 @@
"${workspaceFolder}/src/DataStreamSuite/DataStreamQueryAnalyzer/Configurations/DataStreamQueryAnalyzer.json",
],
"options": {
"cwd": "${workspaceFolder}/build/${input:buildConfiguration}/bin/net7.0-windows"
"cwd": "${workspaceFolder}/build/${input:buildConfiguration}/bin/net8.0-windows"
},
},
"icon": {
@@ -383,7 +383,7 @@
"${workspaceFolder}/src/DataStreamSuite/DataStreamQueryAnalyzer/Configurations/DataStreamQueryAnalyzer.json",
],
"options": {
"cwd": "${workspaceFolder}/build/Debug/bin/net7.0-windows"
"cwd": "${workspaceFolder}/build/Debug/bin/net8.0-windows"
},
},
"icon": {
@@ -403,7 +403,7 @@
"${workspaceFolder}/src/DataStreamSuite/DataStreamQueryAnalyzer/Configurations/DataStreamQueryAnalyzer.json",
],
"options": {
"cwd": "${workspaceFolder}/build/Release/bin/net7.0-windows"
"cwd": "${workspaceFolder}/build/Release/bin/net8.0-windows"
},
},
"icon": {
@@ -427,7 +427,7 @@
"${workspaceFolder}/src/DataStreamSuite/DataStreamPublisher/Configurations/${input:dataStreamPublisherConfig}.json",
],
"options": {
"cwd": "${workspaceFolder}/build/${input:buildConfiguration}/bin/net7.0"
"cwd": "${workspaceFolder}/build/${input:buildConfiguration}/bin/net8.0"
},
"icon": {
"id": "run",
@@ -450,7 +450,7 @@
"${workspaceFolder}/src/DataStreamSuite/DataStreamPublisher/Configurations/${input:dataStreamPublisherConfig}.json",
],
"options": {
"cwd": "${workspaceFolder}/build/Debug/bin/net7.0"
"cwd": "${workspaceFolder}/build/Debug/bin/net8.0"
},
"icon": {
"id": "run",
@@ -473,7 +473,7 @@
"${workspaceFolder}/src/DataStreamSuite/DataStreamPublisher/Configurations/${input:dataStreamPublisherConfig}.json",
],
"options": {
"cwd": "${workspaceFolder}/build/Release/bin/net7.0"
"cwd": "${workspaceFolder}/build/Release/bin/net8.0"
},
"icon": {
"id": "run",
@@ -496,7 +496,7 @@
"${workspaceFolder}/src/DataStreamSuite/PubSubServer/Configurations/${input:pubSubServerConfig}.json",
],
"options": {
"cwd": "${workspaceFolder}/build/${input:buildConfiguration}/bin/net7.0"
"cwd": "${workspaceFolder}/build/${input:buildConfiguration}/bin/net8.0"
},
"icon": {
"id": "run",
@@ -519,7 +519,7 @@
"${workspaceFolder}/src/DataStreamSuite/PubSubServer/Configurations/${input:pubSubServerConfig}.json",
],
"options": {
"cwd": "${workspaceFolder}/build/Debug/bin/net7.0"
"cwd": "${workspaceFolder}/build/Debug/bin/net8.0"
},
"icon": {
"id": "run",
@@ -541,7 +541,7 @@
"${workspaceFolder}/src/DataStreamSuite/PubSubServer/Configurations/${input:pubSubServerConfig}.json",
],
"options": {
"cwd": "${workspaceFolder}/build/Debug/bin/net7.0"
"cwd": "${workspaceFolder}/build/Debug/bin/net8.0"
},
"icon": {
"id": "run",
@@ -564,7 +564,7 @@
"${workspaceFolder}/src/DataStreamSuite/PubSubServer/Configurations/${input:pubSubServerConfig}.json",
],
"options": {
"cwd": "${workspaceFolder}/build/Release/bin/net7.0"
"cwd": "${workspaceFolder}/build/Release/bin/net8.0"
},
"icon": {
"id": "run",
@@ -586,7 +586,7 @@
"${workspaceFolder}/src/DataStreamSuite/PubSubServer/Configurations/${input:pubSubServerConfig}.json",
],
"options": {
"cwd": "${workspaceFolder}/build/Release/bin/net7.0"
"cwd": "${workspaceFolder}/build/Release/bin/net8.0"
},
"icon": {
"id": "run",
@@ -601,7 +601,7 @@
"windows": {
"command": "Spark.exe",
"options": {
"cwd": "${workspaceFolder}/build/${input:buildConfiguration}/bin/net7.0-windows"
"cwd": "${workspaceFolder}/build/${input:buildConfiguration}/bin/net8.0-windows"
},
},
"icon": {
@@ -617,7 +617,7 @@
"windows": {
"command": "Spark.exe",
"options": {
"cwd": "${workspaceFolder}/build/Debug/bin/net7.0-windows"
"cwd": "${workspaceFolder}/build/Debug/bin/net8.0-windows"
},
},
"icon": {
@@ -633,7 +633,7 @@
"windows": {
"command": "Spark.exe",
"options": {
"cwd": "${workspaceFolder}/build/Release/bin/net7.0-windows"
"cwd": "${workspaceFolder}/build/Release/bin/net8.0-windows"
},
},
"icon": {
@@ -648,7 +648,7 @@
"windows": {
"command": "Spark.exe",
"options": {
"cwd": "${workspaceFolder}/build/${input:buildConfiguration}/bin/net7.0-windows"
"cwd": "${workspaceFolder}/build/${input:buildConfiguration}/bin/net8.0-windows"
},
},
"icon": {
@@ -673,7 +673,7 @@
"${input:sparkConfigQuery}",
],
"options": {
"cwd": "${workspaceFolder}/build/${input:buildConfiguration}/bin/net7.0"
"cwd": "${workspaceFolder}/build/${input:buildConfiguration}/bin/net8.0"
},
"icon": {
"id": "run",
@@ -754,7 +754,7 @@
"${input:sparkTest}"
],
"options": {
"cwd": "${workspaceFolder}/build/Debug/bin/net7.0",
"cwd": "${workspaceFolder}/build/Debug/bin/net8.0",
}
},
},
@@ -38,6 +38,7 @@ namespace mbts.spark {
Vega,
Theta,
Rho,
Currency
Currency,
CashPerPoint,
}
}
@@ -128,6 +128,8 @@ namespace bts.shell.market_view {
public DoubleMinMaxState theta { get; set; }
[ProtoMember(36)]
public DoubleMinMaxState rho { get; set; }
[ProtoMember(39)]
public DoubleMinMaxState cash_per_point { get; set; }
public Func<MarketViewVM.Row, bool> build_predicate() {
var pb = new ColumnPredicateBuilder<MarketViewVM.Row>();
@@ -185,6 +187,7 @@ namespace bts.shell.market_view {
pb.add_numeric(vega, r => r.vega);
pb.add_numeric(theta, r => r.theta);
pb.add_numeric(rho, r => r.rho);
pb.add_numeric(cash_per_point, r => r.cash_per_point);
return pb.to_predicate();
}
}
@@ -209,6 +209,7 @@ namespace mbts.spark {
public bts.api.Venue venue => security_md.security.Exchange;
public uint security_key => security_md.security.Id;
public string currency => security_md.security.Currency;
public double cash_per_point => security_md.security.ContractSpec.CashPerPoint;
public bool inside_bids => _inside_bids.value;
public bool inside_asks => _inside_asks.value;
@@ -146,6 +146,7 @@ namespace bts.shell.market_view {
public static GridColumnSettings vega => _mk(Columns.Vega, "Vega", ColWidth.Greeks, false);
public static GridColumnSettings theta => _mk(Columns.Theta, "Theta", ColWidth.Greeks, false);
public static GridColumnSettings rho => _mk(Columns.Rho, "Rho", ColWidth.Greeks, false);
public static GridColumnSettings cash_per_point => _mk(Columns.CashPerPoint, "CPP", ColWidth.Greeks, false);
public static IEnumerable<SortingCriteria> default_sorting_criteria {
get {
@@ -198,6 +198,7 @@ namespace mbts.spark {
add_col(ColKey.Vega, r => fmt_greek(r.vega), nameof(Row.vega), vega_filter, FilterType.DoubleMinMax, ColWidth.Greeks);
add_col(ColKey.Theta, r => fmt_greek(r.theta), nameof(Row.theta), theta_filter, FilterType.DoubleMinMax, ColWidth.Greeks);
add_col(ColKey.Rho, r => fmt_greek(r.rho), nameof(Row.rho), rho_filter, FilterType.DoubleMinMax, ColWidth.Greeks);
add_col(ColKey.CashPerPoint, r => r.cash_per_point.ToString("0.00#####"), nameof(Row.cash_per_point), currency_filter, FilterType.DoubleMinMax, ColWidth.Greeks);
// row vm filter
// Althoguh securities are filtered at the row 'model' level (SecurityMD), the column base filters have to be applied at the row vm level,
@@ -436,6 +437,7 @@ namespace mbts.spark {
public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> gamma_filter { get; } = new DoubleMinMax<Row>(r => r.gamma, nameof(Row.gamma));
public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> theta_filter { get; } = new DoubleMinMax<Row>(r => r.theta, nameof(Row.theta));
public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> rho_filter { get; } = new DoubleMinMax<Row>(r => r.rho, nameof(Row.rho));
public IDataGridColumnPersistentFilter<Row, DoubleMinMaxState> cash_per_point_filter { get; } = new DoubleMinMax<Row>(r => r.cash_per_point, nameof(Row.cash_per_point));
IEnumerable<IDataGridColumnFilter<Row>> _all_filters =>
_filter_getters.Select(g => g(this));
@@ -477,6 +479,7 @@ namespace mbts.spark {
ColKey.Vega => wrap(r => r.vega),
ColKey.Theta => wrap(r => r.theta),
ColKey.Rho => wrap(r => r.rho),
ColKey.CashPerPoint => wrap(r => r.cash_per_point),
_ => null,
};
@@ -583,6 +586,7 @@ namespace mbts.spark {
vega = vega_filter.get_state(),
theta = theta_filter.get_state(),
rho = rho_filter.get_state(),
cash_per_point = cash_per_point_filter.get_state(),
};
public void load_filters(MarketViewFilterStates filters) {
@@ -625,6 +629,7 @@ namespace mbts.spark {
vega_filter.load_state(filters.vega);
theta_filter.load_state(filters.theta);
rho_filter.load_state(filters.rho);
cash_per_point_filter.load_state(filters.cash_per_point);
}
static (string desc, Color fg) _fills_title(ColKey key) =>
+1 -1
View File
@@ -30,7 +30,7 @@
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Google.Protobuf" Version="3.24.3" />
<PackageReference Include="IronPython" Version="3.4.1" />
<PackageReference Include="IronPython" Version="3.4.2" />
<PackageReference Include="MathNet.Numerics" Version="5.0.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.2.0" />
<PackageReference Include="NAudio" Version="2.0.1" />
@@ -15,6 +15,10 @@ namespace bts.md {
implied_size = implied_sz;
price = prx;
}
public PriceSize(Size sz, double prx)
: this(sz, sz, 0, prx) { }
public override string ToString() {
return $"{price} {size} {implied_size} ";
}
@@ -0,0 +1,184 @@
#nullable enable
namespace Spark.Api.MarketData;
using bts;
using bts.api;
using bts.md;
using bts.utils;
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
public static class SyntheticBookCalculator {
public static Book GetSyntheticBook(uint securityId, IEnumerable<(ContractLegImmutable, Book)> legDepths) {
return new(
securityId,
GetSyntheticDepth(legDepths.Select(GetBidLegParams).ToList()).Select(ToPriceSize).ToList(),
GetSyntheticDepth(legDepths.Select(GetAskLegParams).ToList()).Select(ToPriceSize).ToList());
PriceSize ToPriceSize(DepthLevel level) =>
new((float)level.Quantity, level.Price);
(double Quantity, double TickSize, Book Book) GetBidLegParams((ContractLegImmutable, Book) ld) =>
(ld.Item1.signed_quantity(), ld.Item1.Security.ContractSpec.LowTick, ld.Item2);
(double Quantity, double TickSize, Book Book) GetAskLegParams((ContractLegImmutable, Book) ld) =>
(-ld.Item1.signed_quantity(), ld.Item1.Security.ContractSpec.LowTick, ld.Item2);
}
public static List<DepthLevel> GetSyntheticDepth(IEnumerable<(ContractLegImmutable, Book)> legDepths) {
return GetSyntheticDepth(
legDepths.Select(getLegDepthConsumer).ToList());
LegDepthConsumer getLegDepthConsumer((ContractLegImmutable, Book) legDepth) {
var (leg, book) = legDepth;
Log.assert(leg.Quantity.is_real(), "Leg quantity must be real");
Log.assert(leg.Security.HasContractSpec && leg.Security.ContractSpec.LowTick.is_real_positive(),
"Leg security LowTick must be real and positive");
return LegDepthConsumer.Create(leg.Quantity, leg.Security.ContractSpec.LowTick);
}
}
public static List<DepthLevel> GetSyntheticDepth(IEnumerable<(double Quantity, double TickSize, Book Book)> legDepths) {
return GetSyntheticDepth(
legDepths.Select(getLegDepthConsumer).ToList());
LegDepthConsumer getLegDepthConsumer((double Quantity, double TickSize, Book Book) legDepth) {
var (qty, tickSz, bk) = legDepth;
Log.assert(qty.is_real(), "Leg quantity must be real");
Log.assert(tickSz.is_real_positive(),
"Leg tickSz must be real and positive");
return LegDepthConsumer.Create(qty, tickSz);
}
}
public static List<DepthLevel> GetSyntheticDepth(List<LegDepthConsumer> legs) {
if (legs.Count == 0) {
return new(0);
}
var synthLeg = legs.Skip(1).Aggregate(
legs[0],
(l1, l2) => new LegDepthConsumer(new(getSyntheticDepth(l1, l2).ToList())) {
LegQuantity = 1,
QuantityTick = MathUtils.lcm(l1.QuantityTick, l2.QuantityTick),
});
return synthLeg.Depth.Levels;
}
static IEnumerable<DepthLevel> getSyntheticDepth(LegDepthConsumer leg1, LegDepthConsumer leg2) {
while (!leg1.Depth.IsDone && !leg2.Depth.IsDone) {
if (!leg1.LegQuantity.is_real() || !leg2.LegQuantity.is_real()) {
throw new ArgumentException("Invalid leg quantities");
}
double leg2PerLeg1 = (leg2.LegQuantity / leg1.LegQuantity).abs();
double leg1PerLeg2 = (leg1.LegQuantity / leg2.LegQuantity).abs();
double avail1 = leg1.Depth.AvailableQuantity(1);
double avail2 = leg2.Depth.AvailableQuantity(1);
(double take1, double take2) = (avail1 / leg1PerLeg2) < (avail2 / leg2PerLeg1)
? (avail1, avail1 * leg2PerLeg1)
: (avail2 * leg1PerLeg2, avail2);
var (qty1, cost1) = leg1.Depth.ConsumeQuantity(take1);
var (qty2, cost2) = leg2.Depth.ConsumeQuantity(take2);
Log.assert(qty2 / qty1 == leg2PerLeg1);
var price1 = cost1 / qty1;
var price2 = cost2 / qty2;
price1 *= leg1.LegQuantity;
price2 *= leg2.LegQuantity;
yield return new(Price: price1 + price2, Quantity: qty1);
}
}
public record DepthLevel(double Price, double Quantity) {
public override string ToString() {
return $"{Price} ({Quantity})";
}
}
public class DepthLevelComparer : IEqualityComparer<DepthLevel> {
readonly IEqualityComparer<double> priceComparer;
readonly IEqualityComparer<double> quantityComparer;
public DepthLevelComparer(IEqualityComparer<double>? priceComparer = null, IEqualityComparer<double>? quantityComparer = null) {
this.priceComparer = priceComparer ?? EqualityComparer<double>.Default;
this.quantityComparer = quantityComparer ?? EqualityComparer<double>.Default;
}
public bool Equals(DepthLevel? x, DepthLevel? y) {
if (x is null) {
return y is null;
}
return y is not null
&& priceComparer.Equals(x.Price, y.Price)
&& quantityComparer.Equals(x.Quantity, y.Quantity);
}
public int GetHashCode([DisallowNull] DepthLevel x) =>
HashCode.Combine(
priceComparer.GetHashCode(x.Price),
quantityComparer.GetHashCode(x.Quantity));
}
public class DepthConsumer {
public DepthConsumer() { }
public DepthConsumer(params DepthLevel[] levels) : this(levels.AsEnumerable()) { }
public DepthConsumer(IEnumerable<DepthLevel> levels) =>
Levels.AddRange(levels);
public int Pos { get; private set; } = 0;
public double CurrLevelConsumed { get; private set; } = 0;
public List<DepthLevel> Levels { get; } = new();
public bool IsDone =>
Pos == Levels.Count || (Pos == Levels.Count - 1 && Levels.Last().Quantity == CurrLevelConsumed);
public double AvailableQuantity(int levelLimit) {
int end = Math.Min(Pos + levelLimit, Levels.Count);
double rv = 0;
for (int i = Pos; i < end; i++) {
double levelQty = Levels[i].Quantity;
if (i == Pos) {
levelQty -= CurrLevelConsumed;
}
rv += levelQty;
}
return rv;
}
public (double Qty, double Price) ConsumeQuantity(double quantity) {
double consumed = 0;
double price = 0;
for (; Pos < Levels.Count; Pos++) {
double levelQty = Levels[Pos].Quantity - CurrLevelConsumed;
var rem = quantity - consumed;
if (levelQty > rem) {
CurrLevelConsumed += rem;
consumed = quantity;
price += rem * Levels[Pos].Price;
break;
}
consumed += levelQty;
price += levelQty * Levels[Pos].Price;
CurrLevelConsumed = 0;
}
return (consumed, price);
}
}
public class LegDepthConsumer {
public LegDepthConsumer() : this(new()) { }
public LegDepthConsumer(DepthConsumer depth) =>
Depth = depth;
public double LegQuantity { get; set; } = 1;
public double QuantityTick { get; set; } = 1;
public DepthConsumer Depth { get; } = new();
public static LegDepthConsumer Create(double legQuantity, double quantityTick, params DepthLevel[] levels) =>
new(new(levels)) {
LegQuantity = legQuantity,
QuantityTick = quantityTick
};
}
}
@@ -124,8 +124,9 @@ namespace bts.data.ticks {
do {
// Note that the modulus operator is not used here due to rounding errors.
// Such a rounding errror was the cause of the bug https://bluetradesys.atlassian.net/browse/SP-283.
if (MathUtils.is_int(factored_base / raw_base))
return new TickFactors(factored_base, factored_base / raw_base);
if (MathUtils.is_int(factored_base / raw_base, MathUtils.API_DECIMAL_PRECISION)) {
return new TickFactors(factored_base, Math.Round(factored_base / raw_base));
}
factored_base *= 10;
} while (++count < 10);
return null;
+2 -3
View File
@@ -122,8 +122,7 @@ namespace bts.utils {
return 0;
}
public static T lcm<T>(IEnumerable<T> nums)
where T : INumber<T>, IMultiplyOperators<T, int, T> {
public static T lcm<T>(IEnumerable<T> nums) where T : INumber<T> {
using var en = nums.Where(n => n != T.Zero).GetEnumerator();
if (!en.MoveNext()) {
return T.Zero;
@@ -135,7 +134,7 @@ namespace bts.utils {
return res;
}
public static T lcm<T>(T a, T b) where T : INumber<T>, IMultiplyOperators<T, int, T> {
public static T lcm<T>(T a, T b) where T : INumber<T> {
T num1, num2;
if (a > b) {
num1 = a;
@@ -0,0 +1,96 @@
#nullable enable
namespace bts.tests.security;
using bts.test;
using bts.utils;
using mbts;
using System;
using System.Collections.Generic;
using System.Linq;
using static Spark.Api.MarketData.SyntheticBookCalculator;
[TestGroup]
class SyntheticBookTests {
[Test]
public void TestFractionalLegs() {
var legs = new List<LegDepthConsumer>() {
LegDepthConsumer.Create(1, 0.001, new(40, 0.205), new(38, 0.195)),
LegDepthConsumer.Create(-0.5, 0.001, new(50, 0.365), new(55, 0.100)),
};
var synthBook = getSyntheticBook(legs);
var expectedBook = new List<DepthLevel>() {
new(15, 0.205),
new(13, 0.195),
};
Assert.That(synthBook.SequenceEqual(expectedBook));
legs = new List<LegDepthConsumer>() {
LegDepthConsumer.Create(1, 0.001, new(50, 0.365), new(55, 0.100)),
LegDepthConsumer.Create(-0.5, 0.001, new(40, 0.205), new(38, 0.195)),
};
synthBook = getSyntheticBook(legs);
expectedBook = new() {
new(30, 0.365),
new(35, 0.045),
new(36, 0.055),
};
var tf = new mbts.user.settings.TickFactors(1000, 1);
var priceComp = tf.tick_equality_comparer();
var qtyComp = new mbts.user.settings.TickFactors(1000000, 1).tick_equality_comparer();
var levelComp = new DepthLevelComparer(priceComp, qtyComp);
Assert.That(synthBook.SequenceEqual(expectedBook, levelComp));
legs = new List<LegDepthConsumer>() {
LegDepthConsumer.Create(1, 0.001, new(50, 0.365), new(45, 0.100)),
LegDepthConsumer.Create(1, 0.001, new(40, 0.205), new(38, 0.195)),
LegDepthConsumer.Create(-2, 0.001, new(40, 0.118), new(42.5, 0.068), new(45, 1.1)),
};
synthBook = getSyntheticBook(legs);
expectedBook = new() {
new(10, 0.059),
new(5, 0.034),
new(0, 0.112),
new(-2, 0.16),
new(-7, 0.035),
};
Assert.That(synthBook.SequenceEqual(expectedBook, levelComp));
}
static IEnumerable<DepthLevel> getSyntheticBook(LegDepthConsumer leg1, LegDepthConsumer leg2) {
while (!leg1.Depth.IsDone && !leg2.Depth.IsDone) {
if (!leg1.LegQuantity.is_real() || !leg2.LegQuantity.is_real()) {
throw new ArgumentException("Invalid leg quantities");
}
double leg2PerLeg1 = (leg2.LegQuantity / leg1.LegQuantity).abs();
double leg1PerLeg2 = (leg1.LegQuantity / leg2.LegQuantity).abs();
double avail1 = leg1.Depth.AvailableQuantity(1);
double avail2 = leg2.Depth.AvailableQuantity(1);
(double take1, double take2) = (avail1 / leg1PerLeg2) < (avail2 / leg2PerLeg1)
? (avail1, avail1 * leg2PerLeg1)
: (avail2 * leg1PerLeg2, avail2);
var (qty1, cost1) = leg1.Depth.ConsumeQuantity(take1);
var (qty2, cost2) = leg2.Depth.ConsumeQuantity(take2);
Log.assert(qty2 / qty1 == leg2PerLeg1);
var price1 = cost1 / qty1;
var price2 = cost2 / qty2;
price1 *= leg1.LegQuantity;
price2 *= leg2.LegQuantity;
yield return new(Price: price1 + price2, Quantity: qty1);
}
}
List<DepthLevel> getSyntheticBook(List<LegDepthConsumer> legs) {
if (legs.Count == 0) {
return new(0);
}
var synthLeg = legs.Skip(1).Aggregate(
legs[0],
(l1, l2) => new LegDepthConsumer(new(getSyntheticBook(l1, l2).ToList())) {
LegQuantity = 1,
QuantityTick = MathUtils.lcm(l1.QuantityTick, l2.QuantityTick),
});
return synthLeg.Depth.Levels;
}
}
@@ -254,7 +254,7 @@ class BybitExecutionGateway(ExecutionGateway):
self.stop_event.set()
def create_order(self, req: PyCreateOrderRequest):
self.logger.info("[bybit_gw_crt_ord.json]%s" % SLog(req=req))
self.logger.info("[bybit_gw_crt_ord.json]%s" % req)
if not self.stream_cli or not self.trade_cli or not self.bridge:
self._on_abort(
req.key(),
@@ -287,7 +287,7 @@ class BybitExecutionGateway(ExecutionGateway):
self.trade_cli.place_order(lambda msg: self._on_order_ack("crt", msg), **params)
def cancel_replace_order(self, req: PyCancelReplaceOrderRequest):
self.logger.info("[bybit_gw_cxlrep_ord.json]%s" % SLog(req=req))
self.logger.info("[bybit_gw_cxlrep_ord.json]%s" % req)
if not self.stream_cli or not self.trade_cli:
self._on_abort(req.key(), "cxlrep", "WebSocket client not created.")
return
@@ -313,7 +313,7 @@ class BybitExecutionGateway(ExecutionGateway):
)
def cancel_order(self, req: PyCancelOrderRequest):
self.logger.info("[bybit_gw_cxl_ord.json]%s" % SLog(req=req))
self.logger.info("[bybit_gw_cxl_ord.json]%s" % req)
if not self.stream_cli or not self.trade_cli:
self._on_abort(req.key(), "cxl", "WebSocket client not created.")
return
@@ -333,5 +333,5 @@ class BybitExecutionGateway(ExecutionGateway):
)
def on_securities(self, securities: PySecurityList):
self.logger.info("[bybit_gw_sec.json]%s" % SLog(secs=securities))
self.logger.info("[bybit_gw_sec.json]%s" % securities)
self.securities = {sec.name(): sec for sec in securities}