Commit Graph
12628 Commits
Author SHA1 Message Date
John Colvin 687e2477c0 fix: Assert that acc name length < max_size (not size)
Since we changed the account field from std::array -> bts::core::SmallString
2025-06-16 11:08:59 -04:00
John Colvin 1bd0ba9950 deploy: Install python requirements in docker
No point in using a virtual environment. These containers have one
purpose: run sparkd with a constant, predetermined set of py plugins.
2025-06-16 11:08:59 -04:00
John Colvin ddabc744f2 chore: Update alphaworx_zen_hk_live init.py to query new secret
I decided to put all alphaworx api keys in one secret. It's easier this way.
2025-06-16 11:08:58 -04:00
John Colvin 74b88b003e fix: Fix config dir env var in bybit_dev profile 2025-06-16 11:08:58 -04:00
John Colvin aaf1be09c2 fix: Fix SmallString numeric parsing, make more robust
* Fix by parsing numbers from left to right instead of right to left
  (previously, would parse "17" to 71)
* Support leading +/- signs
* Throw if unexpected char encountered
* Unit tests
2025-06-16 11:08:58 -04:00
John Colvin e3bd9a457c fix: Change thread_id type to size_t (not int64_t) 2025-06-16 11:08:58 -04:00
John Colvin 59f62b6ec9 cleanup: Remove unnecessary signal handler in bybit_md_provider.py
We handle them properly in sparkd now
2025-06-16 11:08:58 -04:00
John Colvin b46e75f7d7 cleanup: Sort gateio symbols in symbols.conf 2025-06-16 11:08:58 -04:00
John Colvin 3f100feaca chore: Add config json for py plugins
Support overrides per venue in a separate file, so symbols can be
changed after deployment using arham's widget.
2025-06-16 11:08:58 -04:00
John Colvin 093da2a938 fix: Init shared_ptr w/ default obj (not null)
Prob a typo
2025-06-16 11:08:58 -04:00
John Colvin 33bfdc517a fix build: Remove include of non-existent header
I created the file locally, but ended up not using it and forgot to remove an
include before I pushed.
2025-06-16 11:08:58 -04:00
John Colvin d17be48e6d fix build: Always return value in non-void fn 2025-06-16 11:08:58 -04:00
John Colvin cbaa1df641 fix: Don't compare size_t to -1 in SmallString num parsers 2025-06-16 11:08:58 -04:00
John Colvin 8c23665d65 refactor: Manage all py plugins (md and exe) in one class
All calls to/from python are performed on one thread via a simple
message queue (necessary because of the global interpreter lock).
2025-06-16 11:08:58 -04:00
John Colvin 3f9923239e diag: Log incoming books at Info level in bybit_md_provider 2025-06-16 11:08:58 -04:00
John Colvin 9ae9451359 chore: Add bybit execution config to bybit_dev profile 2025-06-16 11:08:58 -04:00
John Colvin 54388d3b92 fix: Include fill id in format str for ExecutionReport 2025-06-16 11:08:58 -04:00
John Colvin 17ada3c57c dev qol: Bind PySecurity __format__ to fmt::formatter impl 2025-06-16 11:08:58 -04:00
John Colvin 046f412ec4 refactor: Change floats to doubles in py bridge types
Getting rounding errors with floats
2025-06-16 11:08:58 -04:00
John Colvin c94d11f525 refactor: Use SmallString instead of std::array in ExchangeFillID 2025-06-16 11:08:57 -04:00
John Colvin 2b69846381 Add py bybit exe config to bybit_dev profile 2025-06-16 11:08:57 -04:00
John Colvin bdd5e08b60 refactor: Consolidate all py interop into single lib
It was just too hard trying to make it work with separate libs. The
main issue was getting the python md and execution modules to
recognize PySecurityList and other types defined in the core module.
2025-06-16 11:08:57 -04:00
John Colvin 7c9330650d refactor: Move tick fns into gen-purpose tick_utils.h 2025-06-16 11:08:57 -04:00
John Colvin 98fca5ad7f Change bybit_dev symbols.conf to list bybit futs (not spots) 2025-06-16 11:08:57 -04:00
John Colvin 8a0fcce90d fix: Actually use configured logger in bybit_md_provider.py 2025-06-16 11:08:57 -04:00
John Colvin 112527f9ed dev qol: Reduce noisy logs in PositionServicePNLHelper 2025-06-16 11:08:57 -04:00
John Colvin fc1e20538d build: Add cmake script to sym link py module and plugins
We have several of these (python_core, python_md, python_execution),
so the repetative boilerplate code was starting to pile up.
2025-06-16 11:08:57 -04:00
John Colvin 06c494291c dev qol: Add pyrightconfig.json for type hints in py/c++ bridge 2025-06-16 11:08:57 -04:00
John Colvin fe74aed048 Tweak py to get rid of false positive type hint warns 2025-06-16 11:08:57 -04:00
John Colvin b94cdb561e feat: Support python execution plugins 2025-06-16 11:08:57 -04:00
John Colvin 06bd8791c0 refactor: Move py security (and related types) to python_core lib
So the execution gw plugins can use it (as well as the md plugins).
2025-06-16 11:08:57 -04:00
John Colvin b5850cdb02 fix: Add missing switch cases for bybit & gateio 2025-06-16 11:08:57 -04:00
John Colvin 545d76142f fix: Initialize bybit secs as perp futs (not spots) 2025-06-16 11:08:57 -04:00
John Colvin edc2cea60e dev qol: Add wait/notify functionality to ConcurrentQueue 2025-06-16 11:08:56 -04:00
John Colvin 501be8d6a5 dev qol: Add ConcurrentValueStore, with notify/wait functionality
Uses std::condition_variable
2025-06-16 11:08:56 -04:00
John Colvin a3f49b1ce8 dev qol: Add gen-purpose validation fns for execution gws 2025-06-16 11:08:56 -04:00
John Colvin 765237d701 build: Use new gen-purpose python cmake fns in py md proj
Significantly simplifies CMakeLists.txt
2025-06-16 11:08:56 -04:00
John Colvin 8b43d93e26 dev qol: Add fn to make Stopwatch and start it at once 2025-06-16 11:08:56 -04:00
John Colvin 14baa7d62e chore: Use same log format as sparkd in py md plugins 2025-06-16 11:08:56 -04:00
John Colvin a054f4cc63 improvement: Set PyBook data in single call to reduce interop overhead 2025-06-16 11:08:56 -04:00
John Colvin 5311128607 Reduce log noise in py md providers 2025-06-16 11:08:56 -04:00
John Colvin 075acdc8eb doc: Improve py md readme
* Put examples in a table
* Show command to create symlinks for intellisense
2025-06-16 11:08:56 -04:00
John Colvin e1fdf6306b refactor: Redesign py md so it doesn't lock the global interpreter
We can only embed one interpreter in the sparkd process, so all py
plugins need to share it.

* Move core py interop code to new lib_python_core. We're about to add a
  new lib for py execution, so it needs to be accesible from both
  execution and md libs
* Fix typo for type hint in py_md_utils.py
* Run py md providers on background thread
* Rename MdSink -> MdBridge
2025-06-16 11:08:56 -04:00
John Colvin 3f89d3b7f7 dev qol: Add to_chrono fns to TimeSpan 2025-06-16 11:08:56 -04:00
John Colvin a6dd0d842d dev qol: Add -> operators to ValueWrapper for convenience
So we can type "foo->" instead of "foo.value()" (much shorter!)
2025-06-16 11:08:56 -04:00
John Colvin d050d2d19a build: Link against system installed ssl to support py interop
Unless we want to build python packages ourselves, the underlying clibs
they use link to the system installed ssl. Notably, the requests module.
2025-06-16 11:08:56 -04:00
John Colvin 31cb25ac2e improvement: Generate py stubs for md providers
Provides intellisense for the python_md_module
2025-06-16 11:08:56 -04:00
John Colvin cbb43a22de cleanup: Get rid of unecessary namespace prefixes 2025-06-16 11:08:55 -04:00
John Colvin 9e7b4849f0 cleanup: Get rid of "PyList" classes, bind to stl vector instead
I wasn't aware this was possible until recently.
2025-06-16 11:08:55 -04:00
John Colvin 6e72890153 refactor: Move SimpleOrderKeyMap from binance_execution to lib_spark
It provides basic functionality that will be useful in other execution gateways.
2025-06-16 11:08:55 -04:00