John Colvin
f88915a940
fix build: Add --spark-version opt in build_data_stream.py
2026-01-07 02:34:56 -05:00
John Colvin
549529aa60
tool: Add script to add BTS github nuget source
...
Where we host the BTS sdk nugets
2026-01-04 21:19:00 +00:00
John Colvin
8c458d48bb
feat: Add server side csharp sdk
...
For Ophir
2026-01-04 21:19:00 +00:00
John Colvin
bb1e7c30c8
build: Add msbuild flag to reference projs or local nugets
...
So we can use project refs for dev builds, and package refs for releases.
2026-01-04 21:19:00 +00:00
John Colvin
1f68f47b42
build: Propagate py exit codes back to the shell
...
* publish_spark.cmd
2026-01-04 21:19:00 +00:00
John Colvin
4bc1a4c833
build (qol): Add opts to only build cpp or cs
...
Usually, we're only working in one language or the other, and don't need
to rebuild both.
2026-01-04 21:19:00 +00:00
John Colvin
69e4ed84c1
build: Generate version info via git cli in build scripts
...
Instead of GitVersion.MsBuild nuget
2026-01-04 21:18:58 +00:00
John Colvin
4d5d0f8573
build: Consolidate cmake presets
2026-01-04 21:08:13 +00:00
John Colvin
76fa35382e
refactor: Add DataStream service modules for Dependency Injection
...
Namely, in DataStreamPublisher and PubSubServer. This is part of a broader
paradigm shift towards Dependency Injection and Factory Pattern.
https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection-guidelines
* Use interfaces to decouple services:
- Decouple ManagedSubscriber from ISubscriberCore impl
- Decouple PubTableBroker from IPublisherCore impl
- Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
* Model config profiles after dotnet appsettings
- Rename Configurations/<name>.json -> appsettings.<name>.json
- Make json fields match corresponding csharp props verbatim
* The Dependency Injection framework doesn't support JsonPropertyName
attributes. Config class props must match json fields verbatim.
* Move reusable startup code into new Bts.Extensions.Hosting proj
* Add pubsub and md-publisher services to compose.yaml
2026-01-04 21:08:11 +00:00
John Colvin
8d6e94384c
cleanup: Fix benign typos in build scripts
...
Change BUILD_SPARK_PY var name to BUILD_PY (these scripts build other
projects than 'spark').
2025-12-30 11:38:20 -05:00
John Colvin
0b3fa813f1
tool: Add script to dump structured logs to console
2025-07-17 16:25:23 +00:00
John Colvin
83f1b5b567
dev qol: In envsource.fish, split PATH vars into list
...
Assume they are semi-colon delimited.
2025-06-24 10:48:42 -04:00
John Colvin
e6466f8309
chore: Add CANCEL_ORDER op to binance.env
2025-06-16 11:08:56 -04:00
John Colvin
7dea9b3792
fix: Don't raise ex if var isn't defined in req path
2025-06-16 11:08:56 -04:00
John Colvin
4681f22e71
deploy: Move configurations to new stutter dir "app_sparkd"
2025-06-16 11:08:54 -04:00
John Colvin
5163ca4cae
refactor: Organize libspark proj w/ stuttering include
...
This new structure provides several benefits:
* Allows us to install multiple CMake projects in the same destination
dir (without include path conflicts).
- This makes shipping the trading algo sdk a trivial excercise. Simply
copy the lib and include dirs from the install dest. That's it!
* Puts us in a position to break up the monolithic libspark into smaller
modules, which are easier to manage.
Windows specific notes:
* Get rid of unnecessary script vs_filters.cmake
* Define WebsocketConnectionGateway directly in cs (instead of proto)
- The proto would no longer be in the include path passed to protoc.
Moving it to c# because there wasn't a compelling reason for it to
be a proto in the first place.
* Set spark output prefix to "lib" on windows
- So we get "libspark.dll" instead of "spark.dll", which conflicts
with the csharp lib, "Spark.dll".
* Copy libspark.dll to net8.0 & net8.0-windows in a single cmake build
target
fix build: Copy libspark to both dirs net8.0, net8.0-windows
2025-06-16 11:08:53 -04:00
John Colvin
5ad7755b28
tool: Make open orders symbol param optional in binance.env
2025-05-20 20:08:34 +00:00
John Colvin
df4ca124c3
cleanup: Remove redundant script get_binance_md.sh
...
I should have removed this when I added binance_rest_client.sh
2025-05-20 20:08:34 +00:00
John Colvin
2c8b9ee288
tool: Add envsource.fish to load .env files in fish
2025-05-20 20:08:34 +00:00
John Colvin
f02f28e347
tool: Add a few vars to binance.env
2025-05-20 20:08:34 +00:00
John Colvin
c8a89c1e8d
cleanup: Rename get_binance_md.py -> binance_client.py
...
This is a more appropriate name (it's not just md anymore).
2025-05-20 20:08:34 +00:00
John Colvin
7ae9e07a30
tool: Use binance.env file in python client
...
Like the bash impl (binance_rest_client.sh), it gets urls, params
(templates), and api credentials from env vars.
2025-05-20 20:08:34 +00:00
John Colvin
af9eb670c1
build: Update build scripts to target dotnet 8
2025-04-30 12:06:28 -04:00
John Colvin
61f04c4605
fix: Implement state machine for binance cxl/replace
2025-04-28 17:12:59 +00:00
John Colvin
043efab5a2
build: Upgrade c# to net8.0
2025-04-28 17:12:58 +00:00
John Colvin
54cb5a2ab2
tool: Add script to configure aws sso
2025-04-28 17:12:58 +00:00
John Colvin
10436f0f3b
tool: Rename binance_orders_client.sh -> binance_rest_client.sh
...
It supports md and order/trade requests
2025-04-28 17:12:58 +00:00
John Colvin
eeffb735ae
deploy (fix): Override existing values when loading env files
2025-04-16 13:48:24 +00:00
John Colvin
b602997f3a
deploy: Build sparkd if any service starts with "sparkd"
...
Previously, we only built sparkd for the "sparkd" service (i.e., exact match)
2025-04-16 13:48:24 +00:00
arham
14e3eee0bc
Add: config changes to make server controller work and set up hk zen server
2025-04-09 09:42:47 -04:00
John Colvin
716c4dd674
tool: Add script to dump sparkd logs to sqlite db
2025-03-19 14:41:23 +00:00
John Colvin
10b2d38de8
tool: Add cmd to tail sparkd log file using docker compose
2025-03-17 00:51:54 +00:00
John Colvin
3f58c2ce84
Add script to install aws cli
...
The aws cli can be used to configure credentials in the shell in order
to retrieve secrets (e.g., production api keys for alphaworx, etc.).
2025-03-10 05:24:49 +00:00
John Colvin
b555850015
build: Package & deploy sparkd with docker compose
2025-03-10 03:05:59 +00:00
John Colvin
aa602b3fd6
tool: Add cross margin reqs to binance.env
2025-03-05 18:04:04 +00:00
John Colvin
b4b8cabdae
tool: Add listen key reqs to binance.env
2025-03-05 18:04:04 +00:00
John Colvin
cfe06869b3
tool: Support empty query params in binance_orders_client.sh
2025-03-05 18:04:04 +00:00
John Colvin
71f7e64ccb
Move config profiles dir to server/sparkd
2025-03-05 16:00:43 +00:00
John Colvin
19efb77e07
tool: Add some usds futures requests to binance.env
2025-02-24 15:12:45 +00:00
John Colvin
fb030742d3
Clean up binance_orders_client.sh a bit, add docs
2025-02-24 06:06:15 +00:00
John Colvin
f0651a5341
tool: Add flag to roll logs without running sparkd
2025-02-23 15:36:10 +00:00
John Colvin
9d81354c3a
Add binance_orders_client.sh for quick/easy troubleshooting
2025-02-23 15:36:10 +00:00
John Colvin
0f37be89a4
Fix typos in binance.env
2025-02-23 15:36:10 +00:00
John Colvin
4dcd5d63a1
tool: Add script to easily switch between spark.conf profiles
2025-02-23 06:10:36 +00:00
John Colvin
284aa53ad4
Add script to run sparkd and roll over log w/ timestamp suffix
...
So each run gets its own log file
2025-02-19 22:49:30 +00:00
John Colvin
582fc10dce
Add get_binance_md.sh for quick/easy troubleshooting
2025-02-12 20:34:47 +00:00
John Colvin
c8fc649b6b
fix build: Pass config arg to cmake --build
...
For Ninja Multi-Config builds (i.e., api server and data streams)
2025-01-28 19:59:09 +00:00
John Colvin
b1f4654b3c
Fix benign typo in publish_spark.py
2025-01-21 17:28:50 -05:00
John Colvin
9d350334c0
fix build (installer): Fix install path typo in publish_spark.py
2025-01-21 17:28:05 -05:00
John Colvin
4fdc866485
build: Propagate py script exit codes back to the shell
...
I noticed the build cmds were broken on Windows because the debugger
would still start even when the preLaunchTask (build) failed!
2025-01-09 01:03:32 -05:00