John Colvin
bef84d6dc3
build: Install boost via vcpkg on linux, update ccapi
...
Old ccapi wasn't compatible with new boost
2026-03-20 17:02:44 +00:00
John Colvin
b061509126
build: Compile spark-windows with clang
...
Made a couple minor changes to cpp to address new werrors
2026-03-20 17:02:44 +00:00
John Colvin
5c0487358e
build: Install boost via vcpkg on windows
2026-03-20 17:02:44 +00:00
John Colvin
5c7eedbf7b
build: Don't activate venv when generating individual protos
...
We activate the venv for the whole build
2026-03-20 17:02:44 +00:00
John Colvin
74475cb68f
deploy: Resolve profile dirs with regex patterns
2026-03-20 17:02:44 +00:00
John Colvin
24978f76b5
improve: Wait until py work requested before loading py modules
...
So we don't crash due to missing py modules when no py plugins are running.
2026-03-20 17:02:44 +00:00
John Colvin
0698567f53
deploy: Add sparkd-shell service to compose.yaml for troubleshooting
2026-03-20 17:02:44 +00:00
John Colvin
ec78e35733
cleanup: Remove unused code
2026-03-20 17:02:44 +00:00
John Colvin
4801ce6b83
refactor: Wrap md run logic in class MarketDataSession
...
Start in the constructor, gracefully exit in the destructor.
2026-03-20 17:02:44 +00:00
John Colvin
7eb8cd5ead
Allow md recording dir to be mounted as volume in docker compose
2026-03-20 17:02:44 +00:00
John Colvin
d7b42e3cee
refactor: Split MDRelay and QueryServer up into smaller files
...
This code is inherently complicated. It's hard to follow when everything
is in one giant file.
2026-03-20 17:02:44 +00:00
John Colvin
35d65f107b
deploy: Configure timezone in entrypoint script
2026-03-20 17:02:44 +00:00
John Colvin
e335a2d827
deploy: Move compose entrypoint to dedicated script
2026-03-20 17:02:44 +00:00
John Colvin
abc7d44ce2
deploy: Fix typo in sparkd conf dir path in compose.yaml
2026-03-20 17:02:44 +00:00
John Colvin
85ff7f412a
cleanup: Fix a few more build warnings
2026-03-20 17:02:44 +00:00
John Colvin
8a3ad08456
refactor: Replace DataPacket with IDataPacket in more places
2026-03-20 17:02:44 +00:00
John Colvin
f181d34cbe
build: Check if venv already active before activating
2026-03-20 17:02:44 +00:00
John Colvin
002087f62a
refactor: Replace TableFormatter with table.h
...
Also move to "format" dir
2026-03-20 17:02:44 +00:00
John Colvin
e37ae056c0
refactor: Replace StringFinder/Joiner.h with find/join.h
...
Also move to "algorithm" dir
2026-03-20 17:02:44 +00:00
John Colvin
b8a7f398c9
refactor: Rename a few TitleCase .h files snake_case
...
* EnumMacros.h -> enum_macros.h
* TimeSpan.h ---> time_span.h
* TimeStamp.h --> time_stamp.h
2026-03-20 17:02:44 +00:00
John Colvin
2038d747d8
diag: Use structured logs in MDP3Listener::onOption
2026-03-20 17:02:44 +00:00
John Colvin
edf127069f
deploy: Install tzdata in linux_dotnet8 Dockerfile
...
So we can configure timezone in compose services. Specifically, we need this
to support live cme md (so onixs doesn't think messages are "too old").
2026-03-20 17:02:44 +00:00
John Colvin
07448cede9
cleanup: Remove unused code
2026-03-20 17:02:44 +00:00
John Colvin
9aef42cb21
refactor: Add new glbx variable ticks to security (shm and proto)
2026-03-20 17:02:44 +00:00
John Colvin
baa43cbecf
refactor: Enable nullability in Bts.Spark.Api
2026-03-08 16:00:50 +00:00
John Colvin
de3e491843
build: Upgrade cs LangVersion 11 -> 12
2026-03-08 16:00:50 +00:00
John Colvin
7cc09350a0
refactor: Rename SparkApiCredentials -> SparkCredentials
2026-03-08 16:00:50 +00:00
John Colvin
059e1202e1
refactor: Rename SparkdLoginService -> SparkLoginService
...
This makes it consistent with other type names (we don't use "Sparkd"
anywhere else).
2026-03-08 16:00:50 +00:00
John Colvin
9093f56498
refactor: Rename SparkdCredentials.cs -> SparkApiCredentials.cs
...
Matches the class name
2026-03-08 16:00:50 +00:00
John Colvin
649b4560ff
cleanup: Remove unnecessary subscription wrapper in cs sdk
2026-03-08 16:00:50 +00:00
John Colvin
6c060f6171
cleanup: Fix indentation in TopStream
...
Whitespace changes
2026-03-08 16:00:50 +00:00
John Colvin
581b9259b8
cleanup: Fix build warnings in EqualityComparer
...
Nullability issues
2026-03-08 16:00:50 +00:00
John Colvin
e7bd57964e
refactor: Rename Optional members PascalCase
...
Also reformat code according to new style guidlines
2026-03-08 16:00:50 +00:00
John Colvin
6ea955aacd
refactor: Replace ToDictionaryCrushed/Dupes w/ DuplicateKeyPolicy enum
...
Overload ToDictionary with this enum instead.
2026-03-08 16:00:49 +00:00
John Colvin
29d028fee2
refactor: Replace find_first_index with FindIndex
2026-03-08 16:00:49 +00:00
John Colvin
dc5da87a65
cleanup: Fix cs build warnings
2026-03-08 16:00:49 +00:00
John Colvin
7de98a313f
refactor: Break up Bts.Spark.Core into a few diff projs
...
This is in anticipation of decoupling BlueOrb from Spark.
* Bts.Collections
* Bts.Core
* Bts.Disposables
* Bts.Numerics
2026-03-08 16:00:49 +00:00
John Colvin
7873e76ba0
build: Clean up spark_client.sln
...
* Remove unused csprojs related to old sdk
* Remove erroneous build type config
2026-03-08 16:00:49 +00:00
John Colvin
5b8a8a8882
build: Simplify target to include native assets in cs
2026-03-08 16:00:49 +00:00
John Colvin
21052090a2
fix (pre-emptive): Format dotnet flag opts correctly
...
In pack_spark_sdk.py, the dotnet pack cmd is constructed with a dict of
opts. If the value part is None, the opt should be formatted as a flag,
"--key" instead of "--key None".
2026-03-07 13:32:03 -05:00
John Colvin
9bd32b8225
fix build (pack cs sdk): Fix --(no-)cs toggle behavior
...
* Always run "dotnet pack", regardless of csharp toggle opt
* Run "dotnet publish" if csharp toggled on
2026-03-07 13:20:56 -05:00
John Colvin
e8e234f420
fix: Don't fire empty updates in Books IObservable
2026-03-07 01:19:38 -05:00
John Colvin
be23237a45
fix: Update last val in IObservable FlattenSeries ext
...
The bootstrap delegate that stitches the sequences together wasn't
getting properly invoked because of this.
2026-03-07 01:19:28 -05:00
arham
a9d92650f8
Add: Spread name to synthspreadbbo
2026-03-03 04:33:44 +00:00
arham
c03fb13126
Fix: rename EnterSpreaderOrderAsync from CreateSynthAsync
2026-03-03 01:16:10 +00:00
arham
01e6e37db6
Add: Config for ophir live data sim exe
2026-03-03 01:10:34 +00:00
arham
786b0849a7
FIX: removed errenous code that stopped mdp3 from working
2026-03-03 01:09:06 +00:00
John Colvin
4826fc9237
build: Remove dependency on Newtonsoft.Json (in DataStreamCore)
...
Replace with System.Text.Json (official dotnet)
2026-02-23 13:03:49 -05:00
John Colvin
c3341ec29f
refactor: Replace a few more DataPacket refs with IDataPacket
2026-02-23 12:56:27 -05:00
John Colvin
eb88052c7e
chore: Add Metadata prop to IDataPacket
...
To give it parity with DataPacket
2026-02-23 12:55:23 -05:00
John Colvin
74332f9944
fix: Pass IMarketDataClient to PrimitiveManager to enable md streams
...
In DataStreamQueryAnalyzer and Spark Api Server
2026-02-23 11:24:33 -05:00
John Colvin
54c97b5610
fix: Set buffer Position to 0 when deserializing pubsub echo payload
2026-02-21 16:36:53 -05:00
John Colvin
5c05afd9b3
fix: Only fire stream update if there are actually changes
...
In other words, don't fire if addedRows and deletedRows are both empty.
2026-02-21 16:36:53 -05:00
John Colvin
dcbba39e4a
fix: Add null check in BlueDataUtility.build_dp_inc
2026-02-21 16:36:53 -05:00
John Colvin
d09f22007e
fix: Ensure PubSubTagHandlerFactory uses correct config
...
* Add this service as part of the PubSubClient services for dep
injection (it is needed by PublisherCore)
* Get rid of PublisherCore's default constructor, so dep injection
treats the tag handler factory as a required param (fails if it isn't
available)
- Replace the default ctr with a static "Create" fn, so code that
hasn't been updated to use dependency injection can still use that.
- Use PubSubConfig.Instance (runtime) instead of compile time Default
* Fix compile time Default config
- In fallback handler config, change Tag from "" to ".*" and move it
from the beginning of the list to the end (it should be the last
resort, not the other way around).
2026-02-21 16:36:53 -05:00
John Colvin
a07e7ea6b8
fix: Don't call MasterWorker/vsd doWork on DSQA ui thread
...
Now that we use dependency injection, this is handled by a dedicated
service (MasterWorkerService). As a side effect, stream updates aren't
necessarily fired on the ui thread, so we refresh the grid on a form
timer callback.
2026-02-21 16:36:47 -05:00
John Colvin
4da52ff724
fix: Replace DataPacket w/ IDataPacket in DSQA
2026-02-21 14:49:39 -05:00
John Colvin
05c7a32fc5
refactor: Add parameterless ctr to PubsubEchoTestStream
...
So it's quick/easy to check in DSQA (less typing required)
2026-02-21 14:23:51 -05:00
John Colvin
404d3073f9
ui: Unhide PubsubHeartbeatStream (pss_hb) from DSQA
2026-02-21 14:11:46 -05:00
John Colvin
7f1154d70c
cleanup: Fix indentation in PubsubHeartbeatStream.cs
2026-02-21 14:10:11 -05:00
John Colvin
5bc2e29096
fix build: Only check "dsqa" flag when building on windows
...
The --dsqa opt isn't defined when running on linux.
2026-02-21 17:55:12 +00:00
John Colvin
f5731a184c
fix: Actually disable Spark in default DSQA config
2026-02-20 11:08:33 -05:00
John Colvin
120fe8e5c6
vscode: Add DSQA debug config to skip build
2026-02-20 10:44:19 -05:00
John Colvin
137fdc537d
refactor: Move DSQA config to appsettings.json
2026-02-20 10:43:13 -05:00
John Colvin
a605e57bee
build: Add dsqa toggle to build_data_stream.py
...
Windows only
2026-02-20 02:33:38 -05:00
John Colvin
3ca4bf784c
ui: Remove bts.BlueOrb namespace filter for DSQA streams
2026-02-20 02:13:17 -05:00
John Colvin
6c3b7ebdaf
refactor: Use dependency injection in DSQA
2026-02-20 02:13:17 -05:00
John Colvin
2fbb6fe563
Disable Spark in default DSQA config
2026-02-20 02:13:17 -05:00
John Colvin
9ea33ac110
ui: Unhide PubSubServerStateStream (pss_state) from DSQA
2026-02-20 02:13:17 -05:00
John Colvin
0bc021912a
fix: Replace more DataPacket references with IDataPacket
2026-02-20 02:13:17 -05:00
John Colvin
fa017bd9ff
fix build: Remove old ViewStream.rows references from dsqa
...
The 'rows' prop no longer exists
2026-02-20 02:13:17 -05:00
John Colvin
57b59215e6
fix build: Don't echo parenthesis in .cmd if block
...
It's not allowed (because windows).
2026-02-18 02:54:25 -05:00
arham
210768e391
fix: handle badrequests in websocket api
2026-02-18 05:40:27 +00:00
arham
f01e8a578d
Add: enter synthetic spreader orders through
2026-02-18 05:36:02 +00:00
arham
528a00cf98
Add: config for okx_bin_sim
2026-02-18 05:31:18 +00:00
arham
96a04a7c84
Fix: side and status to json in orderrecodtuple
2026-02-18 05:30:28 +00:00
arham
abaf48beb4
Add: other fields to tosjon for security.cs
2026-02-18 05:29:03 +00:00
John Colvin
08fcae94c0
Update externals (moved emumba deribit deb to bitbucket cloud)
2026-02-17 18:16:22 +00:00
John Colvin
30814a2ed6
fix build: Fix Bts.Extensions.Hosting projref path
2026-02-15 16:48:02 +00:00
John Colvin
3c61de5f13
devcontainer: Install Rakurai deb
...
Download from bitbucket cloud.
2026-02-14 18:43:07 +00:00
John Colvin
771c58d4ad
devcontainer: Move emumba deb to bitbucket cloud
...
I just learned that Bitbucket offers per-repo cloud storage for arbitary
files (the "Downloads" section on the web gui). Doing this before adding
another third-party deb to our build (rakurai), which is quite large at
140MB. We don't want to pack that into our repo.
2026-02-14 18:42:03 +00:00
John Colvin
5db5d40a99
devcontainer: Fix CMake NO_PUBLIC_KEY warning
2026-02-14 18:29:28 +00:00
John Colvin
020ec9c6bf
build: Replace proj refs w/ internal nugets when packing
2026-02-03 16:57:32 +00:00
John Colvin
4e28835bbd
vscode: Add tools/py to shell PYTHONPATH
2026-02-03 16:57:32 +00:00
John Colvin
6985fbb090
refactor: Move CompressionUtilities into separate csproj
...
It was nested in PubSub. The new csproj is named Bts.IO.Compression.
2026-02-03 16:57:32 +00:00
John Colvin
2f42311220
build: Update a few python packages
...
* protobuf 4.21.12 -> 6.33.4
* pybind11 2.13.6 --> 3.0.1
* pybind11-stubgen 2.5.3 ---> 2.5.5
2026-02-03 16:57:32 +00:00
John Colvin
3842efcef1
build: Define build py requirements in separate files
...
No need to install backend build deps when building the front-end, and
vice versa. The build scripts (build_spark.cmd, build_sparkd.sh, etc.)
automatically install only the necessary requirements now.
Note: We'll keep the monolith requirements.txt file around (at least for
now), as it includes more than just build deps (e.g., numpy, pandas, and
jupyter for data analysis; pybit for bybit plugins, etc.).
2026-02-03 16:57:32 +00:00
John Colvin
635107eb68
cleanup: Remove unused py code
2026-02-03 05:20:29 +00:00
John Colvin
1b9a49b924
cleanup: Add missing files and env vars to sparkd config profiles
2026-02-03 05:20:29 +00:00
John Colvin
ead35389d5
build: Make PubSub packable
2026-02-03 05:20:29 +00:00
John Colvin
22b44845fb
build: Update Microsoft.AspNetCore.Cors to 2.3.9
2026-02-03 05:20:29 +00:00
John Colvin
8ab26f3e1d
build: Remove dependency on Newtonsoft.Json
...
Replace with System.Text.Json (official dotnet)
2026-02-03 05:16:21 +00:00
John Colvin
01c61bb16d
fix build: Publish spark_api_server csproj instead of the sln
...
There's a version conflict with System.Data.Common when publishing the sln.
2026-01-22 05:50:49 +00:00
John Colvin
e09e49ce56
fix build: Add Bts.Net to Spark.Api.Server.sln
...
It seems like the dep is automatically detected on windows, but not
linux (so we need to add it explicitly).
2026-01-21 15:06:00 -05:00
John Colvin
e6cc20858b
build: Honor "UseProjectRefs" in spark_websocket, spark_api_server
2026-01-21 14:28:48 -05:00
John Colvin
2eb80082d8
refactor: Move Bts.Extensions.Hosting, Bts.Net to cs dir
...
They aren't exclusively used by DataStream csprojs
2026-01-21 14:20:39 -05:00
John Colvin
d55ef854ba
fix build: Don't actually copy files when building with --dry-run
2026-01-21 14:03:01 -05:00
John Colvin
b1417ece5d
fix build: Fix cmake windows preset in publish_api_server.py
...
Changed from 'spark' -> 'spark-windows'
2026-01-21 14:03:01 -05:00
John Colvin
f6ae219a72
build: Add a few toggle opts to publish py scripts
...
--(no-)cpp Enable/disable cpp build
--(no-)cs Enable/disable csharp build
--(no-)csproj-refs Depend on csproj refs, or locally built nugets
2026-01-21 14:03:01 -05:00
John Colvin
1e4b9cf833
Add new StartWithOnTimeout ext fn for IObservable
...
Ophir wanted this functionality for data in the sdk
2026-01-19 10:44:51 -05:00
John Colvin
228931dd2a
cleanup: Fix indentation in launch.json
...
Also rename launch task "(gdb) Attach" -> "Sparkd Attach"
2026-01-19 10:44:51 -05:00
John Colvin
f5b4b36ea1
Move devcontainer Dockerfile to bluetradingsystems/linux_dotnet8_dev
...
We now host it on dockerhub under that name
2026-01-19 10:44:51 -05:00
arham
f19d2d695b
Fix: Updated path so that profiles work
2026-01-16 06:55:09 +00:00
John Colvin
d4b9b6bcba
fix: Dispose subscription in TopOfBookQuoter demo
2026-01-14 10:35:42 -05:00
John Colvin
84d74a7873
refactor: Actually implement IDisposable in MarketDataClient
2026-01-14 10:35:42 -05:00
John Colvin
8c34280d75
fix: Init PriceTicker Security to avoid null ref in Equals impl
2026-01-14 10:35:42 -05:00
John Colvin
474d96ced4
feat: Add PriceStats/Ticker to ISimpleSparkClient (cs sdk)
2026-01-14 10:35:42 -05:00
John Colvin
1432692a26
refactor: Make PriceStats/Ticker record structs
...
Also rename props in PascalCase
2026-01-13 13:57:47 -05:00
arham
1dde5c3ac2
add: trades aggressor side works in the recorder
2026-01-13 18:25:56 +00:00
arham
1cc31e0d2d
add: databento datahandler reads cme recorded files with time delay
2026-01-13 18:00:43 +00:00
arham
edbd0d2a7b
add: conf for es weekly
2026-01-13 17:59:25 +00:00
arham
56a4ec94a2
Fix: glbx config to use databento as marketdata src
2026-01-13 17:55:13 +00:00
arham
e3e3d42972
fix: removed double call to std::move
2026-01-13 17:52:16 +00:00
arham
1b9dff3a83
Add: low tick high tick to genereal marketdata listener
2026-01-13 17:51:43 +00:00
John Colvin
8521a98445
ui: Add Tag column to OrderBook window
2026-01-12 10:45:00 -05:00
John Colvin
10eb1ecf1e
feat: Add order tag param to ISimpleSparkClient (cs sdk)
2026-01-12 10:44:59 -05:00
John Colvin
88beb0f148
fix: Avoid null ref ex in DiffStream
2026-01-11 15:15:13 -05:00
John Colvin
0f70ae5da2
fix: Fix 'future' -> 'spread' typo in LiveFuturesContractStream
2026-01-11 15:15:13 -05:00
John Colvin
bcd207f89e
cleanup: Fix more build warnings in DataStreamSuite
...
Alot more
2026-01-11 15:15:13 -05:00
John Colvin
989d103eef
fix: Properly store and dispose md subscription in BBOStream
2026-01-11 15:15:12 -05:00
John Colvin
aacd22d496
refactor: Move Exception utils to Spark.Core.Diagnostics ns
2026-01-11 15:15:12 -05:00
John Colvin
5d318b6422
refactor: Rename PropertyNotNull -> MemberNotNull
...
Because it can be used to null check any member (including fields and
method returns), not just properties.
2026-01-11 15:15:11 -05:00
John Colvin
d89164c0ff
refactor: Move CollectionExtensions to Spark.Core.Collections namespace
2026-01-11 15:12:33 -05:00
John Colvin
cbddb3ae28
fix: Avoid potential race bug in c# OrderPool
2026-01-11 11:20:40 -05:00
John Colvin
2aa25d4085
refactor: Rename ExceptionAssert.not_null -> NotNull
2026-01-11 11:20:40 -05:00
John Colvin
cd34f9b1cb
cleanup: Tidy up SecondStream, move ThrottleStream to own file
...
There actually already was a file called "ThrottleStream", but since it
didn't have the ".cs" extension it wasn't getting built.
2026-01-11 11:20:40 -05:00
John Colvin
5f689dbd8e
cleanup: Tidy whitespace and using statements
2026-01-11 11:20:40 -05:00
John Colvin
03a8f4e9c0
refactor: Replace DataPacket w/ IDataPacket in many places
2026-01-11 11:20:40 -05:00
John Colvin
49a952fb32
refactor: Generate a few row types with new t4 templates
...
* PubSubHeartbeatRow
* PubSubRouteRow
* PubSubServerStateRow
* PubSubServerTagStatsRow
* TableMetadataRow
2026-01-10 12:59:14 -05:00
John Colvin
a1c6e0ccab
build: Add t4 templates to generate c# row types
...
Produces classes/structs that implement IBlueRow with strongly typed
properties for each column.
2026-01-10 12:59:14 -05:00
John Colvin
63a3eb968c
diag: Assert DataStream updates are incremental
...
In debug builds
2026-01-10 12:59:14 -05:00
John Colvin
9de4b27003
refactor: Ensure stream subscriptions only disposed once
...
In the debugger, I noticed a few streams were unhooking their internal
onChange callbacks unnecessarily (when they were already unhooked).
2026-01-10 12:59:14 -05:00
John Colvin
849c6c28e5
fix: Don't fire empty updates in MutableAppendStream
2026-01-10 12:59:14 -05:00
John Colvin
bf3a8f076f
fix: Change DSQA config pubsub port to 22019
...
The port PubSubServer runs on by default
2026-01-10 12:59:14 -05:00
John Colvin
44749739b9
build: Define centralized SparkVersion prop
...
All csprojs should be on the same version. At the very least, it
is a sensible default.
2026-01-10 12:59:14 -05:00
John Colvin
ddf16bcd5c
fix: Add missing LiveMarketDataStream attribute to InstrumentsStream
...
Otherwise PrimitiveManager returns null for that stream type
2026-01-10 12:59:14 -05:00
John Colvin
cfe7d43770
fix: Initialize ViewStreamDatabase.instance in DSQA
2026-01-10 12:59:14 -05:00
John Colvin
bb5ea19f69
refactor: Get rid of ViewStream.rows prop
...
It was causing weird side effects in debugger
2026-01-10 12:59:14 -05:00
John Colvin
327665b4c3
fix: Add null check in ProximityJoinStream
2026-01-08 14:24:27 -05:00
John Colvin
2843638386
fix: Only start Stopwatch when actually doing work
2026-01-08 14:24:27 -05:00
John Colvin
915322927a
fix: Dispose md subscription in InstrumentsStream
2026-01-08 14:23:09 -05:00
John Colvin
9905216131
fix: Format PubSub and DSQA config json in PascalCase
...
Matches what's in appsettings.json files
2026-01-08 14:22:57 -05:00
John Colvin
f88915a940
fix build: Add --spark-version opt in build_data_stream.py
2026-01-07 02:34:56 -05:00
John Colvin
c5ea913e27
fix: Shutdown application when user closes login dialog
...
Previously, it would hang until you manually killed the process.
2026-01-06 14:02:10 -05:00
John Colvin
7719aaa2b3
fix: Add EnvironmentVariables config provider to HostBuilder
2026-01-06 14:02:10 -05:00
John Colvin
ad3a8a7544
fix: Add json config sources before env vars and cmd line sources
...
So environment variables override json files (if there is a conflict),
and command line args override everything else.
2026-01-06 02:34:36 -05:00
John Colvin
75d3ba1d59
fix: Support cross-type numeric comparisons in ArgumentComparer
...
For example, so we can compare ints with doubles. Also adds support for
DateTime and TimeSpan cross-type comparisons.
2026-01-05 13:36:07 -05:00
John Colvin
0ebe9a876c
fix: Apply MarketDataClientConfig to MDSource
...
For throttle intervals, etc.
2026-01-05 13:26:29 -05:00
John Colvin
ac1cbaa985
deploy: Add ophir_demo config profile for sparkd
2026-01-05 00:04:57 +00:00
John Colvin
e861835690
deploy: Change sparkdata dir to ~/.local/share/bts/spark
...
In docker scripts (Dockerfile, compose.yaml)
2026-01-04 23:35:04 +00:00
John Colvin
10752257cc
vscode: Build sparkd as prelaunch task
...
Must have been removed by accident
2026-01-04 21:19:00 +00:00
John Colvin
8e67bd45d9
improve: Add 'file' and 'start_date' props to md replay config
2026-01-04 21:19:00 +00:00
John Colvin
140e5bb5ee
fix build (sparkd): Disable warnings specifically for clang >= 18
...
The build was broken for clang < 18 (and gcc) because we were blindly disabling
warnings as errors specific to clang 18, without checking the compiler version.
Namely, "-Weffc++" and "-Wenum-constexpr-conversion".
2026-01-04 21:19:00 +00:00
John Colvin
9ebc282ba5
fix: Use AtomicGate to make TableUpdate Select ext thread-safe
2026-01-04 21:19:00 +00:00
John Colvin
549529aa60
tool: Add script to add BTS github nuget source
...
Where we host the BTS sdk nugets
2026-01-04 21:19:00 +00:00
John Colvin
5ed4a5c1c5
feat: Add tag property to OrderDefinition
2026-01-04 21:19:00 +00:00
John Colvin
8c458d48bb
feat: Add server side csharp sdk
...
For Ophir
2026-01-04 21:19:00 +00:00
John Colvin
bb1e7c30c8
build: Add msbuild flag to reference projs or local nugets
...
So we can use project refs for dev builds, and package refs for releases.
2026-01-04 21:19:00 +00:00
John Colvin
6568c29ad4
cleanup: Remove unnecessary ProjectReference
2026-01-04 21:19:00 +00:00
John Colvin
e5e67e1a01
build: Optionally launch console with Spark exe
2026-01-04 21:19:00 +00:00
John Colvin
4f34e57194
refactor: Move Bus code into Bts.Spark.Core
2026-01-04 21:19:00 +00:00
John Colvin
59b1aead62
refactor: Use HostBuilder to launch spark w/ streams
2026-01-04 21:19:00 +00:00
John Colvin
49a4c65f3b
build: Define custom "main" (entry point) for Spark exe
...
This is a prelude to integrating streams into spark. The plan is to use
an ms HostBuilder to instantiate stream services/dependencies alongside
the spark Application object.
2026-01-04 21:19:00 +00:00
John Colvin
cf4158b34c
refactor: Consolidate ShellBootstrapper and base class into one
2026-01-04 21:19:00 +00:00
John Colvin
db0e0d4311
cleanup: Get rid of stunnel code
...
This looks like it hasn't been hooked up for years...it would only start
if a stunnel.conf file was detected, which hasn't been around for a very
long time.
2026-01-04 21:19:00 +00:00
John Colvin
50788f8828
fix: Only create mutex if spark app is NOT running in vs designer
...
I accidentally droped the '!' in a cleanup commit 4 years ago
2026-01-04 21:19:00 +00:00
John Colvin
a515468e37
refactor: Rename TableUpdate props to PascalCase
...
Makes the sdk look more polished, professional
2026-01-04 21:19:00 +00:00
John Colvin
1f68f47b42
build: Propagate py exit codes back to the shell
...
* publish_spark.cmd
2026-01-04 21:19:00 +00:00
John Colvin
84f6d3d9fe
cleanup: Fix build warnings pre-emptively before compiler upgrade
...
For clang>=18
* non-constant-expression cannot be narrowed from type 'uint32_t' (aka 'unsigned int')
to 'float' in initializer list [-Wc++11-narrowing-const-reference]
* -Wenum-constexpr-conversion
2026-01-04 21:19:00 +00:00
John Colvin
4bc1a4c833
build (qol): Add opts to only build cpp or cs
...
Usually, we're only working in one language or the other, and don't need
to rebuild both.
2026-01-04 21:19:00 +00:00
John Colvin
c50a006d7d
deploy: Update dotnet7 -> dotnet8 in docker images
2026-01-04 21:19:00 +00:00
John Colvin
2031f536d6
build: Pack Bts.Spark.Core with libspark
2026-01-04 21:19:00 +00:00
John Colvin
69e4ed84c1
build: Generate version info via git cli in build scripts
...
Instead of GitVersion.MsBuild nuget
2026-01-04 21:18:58 +00:00
John Colvin
2a284b1e9c
cleanup: Remove unused Protobuf.cmake script
2026-01-04 21:08:13 +00:00
John Colvin
4d5d0f8573
build: Consolidate cmake presets
2026-01-04 21:08:13 +00:00
John Colvin
c4a6b4591d
cleanup: Remove last vestiges of spark proto config from DataStreamCore
2026-01-04 21:08:13 +00:00
John Colvin
8f3fa18bca
refactor: Rename spark_api csproj Bts.Spark.Api
...
This follows the conventional naming practices, where the project shares its
name with the namespace. Doing this in anticipation of the csharp sdk we are
about to release (for Ophir).
2026-01-04 21:08:13 +00:00
John Colvin
f60782915c
refactor: Rename spark_core csproj Bts.Spark.Core
...
This follows the conventional naming practices, where the project shares its
name with the namespace. Doing this in anticipation of the csharp sdk we are
about to release (for Ophir).
2026-01-04 21:08:13 +00:00
John Colvin
3565f78be8
cleanup (build): Remove redundant csproj refs
...
For example, projects that reference spark_api don't need an explicit reference
to spark_core, since spark_api already references it. The same goes for projects
that reference DataStream and DataStreamCore.
2026-01-04 21:08:13 +00:00
John Colvin
41ad7b96f4
build: Only expose spark_core internals to spark and spark_test csprojs
2026-01-04 21:08:13 +00:00
John Colvin
c338d26b82
build: Define project specific metadata props for nuget pack
2026-01-04 21:08:13 +00:00
John Colvin
319f054a0c
build: Define global company info build props for csprojs
...
* Authors - Blue Trading Systems
* Company - Blue Trading Systems
2026-01-04 21:08:13 +00:00
John Colvin
389bf6e130
refactor: Rename DictionarySetWrapper -> DictionaryKeySet
2026-01-04 21:08:12 +00:00
John Colvin
ded28c23d7
diag: Preserve file and line number in "log_at" ext fn
2026-01-04 21:08:12 +00:00
John Colvin
04acba409f
dev qol: Add local appsettings, configurations to gitignore
2026-01-04 21:08:12 +00:00
John Colvin
67f7124ed5
build: Remove DSQA from DataStreamSuite.sln
...
Because it doesn't build on linux
2026-01-04 21:08:12 +00:00
John Colvin
322794434a
fix build: Copy libspark to "net8.0" dir (was net7.0)
2026-01-04 21:08:12 +00:00
John Colvin
fbcba118ac
fix build (api server): Fix typo DOTNET_BINARY_DIR cmake var (path)
...
This is the dir into which libspark (c++) is copied for dotnet (c#) exes.
2026-01-04 21:08:12 +00:00
John Colvin
1c579ee063
refactor: Change namespace to Bts.DataStreamCore for a few types
...
Since they're in the DataStreamCore csproj
2026-01-04 21:08:12 +00:00
John Colvin
7508fa0b7f
dev qol: Mark ExceptionAssert 'name' param w/ CallerArgumentExpression
2026-01-04 21:08:12 +00:00
John Colvin
1f18ec4198
refactor: Replace if/else with switch cases
2026-01-04 21:08:12 +00:00
John Colvin
81d7820cca
fix: Ensure IDGenerator instance is created before deref
2026-01-04 21:08:12 +00:00
John Colvin
e125be1f85
fix: Emplace MDConsumerHandle instead of copy assign
2026-01-04 21:08:12 +00:00
John Colvin
040c8cab59
cleanup: Remove unnecessary code
...
Trim includes as well
2026-01-04 21:08:12 +00:00
John Colvin
32a12cf1f7
cleanup: Remove redundant macro def
2026-01-04 21:08:12 +00:00
John Colvin
db604fbed0
improve: Abort md rec loop if nothing configured to record
2026-01-04 21:08:12 +00:00
John Colvin
cc0a3ea15f
cleanup: Trivial whitespace cleanup in Protobuf.cmake
2026-01-04 21:08:12 +00:00
John Colvin
4066cd50ce
build: Clean up data_files cmake proj (support --target clean)
2026-01-04 21:08:12 +00:00
John Colvin
f1ccff9134
build: Resolve app data dir and shared mem prefix at runtime
...
We can't define them at compile time anymore since we build multiple exes that
depend on libspark in the same dir (e.g., api-server, pubsub, sparkd, etc.),
and each one needs its own shared memory and app data filespaces.
For shared memory:
* Prefix all shm files with the proc name and an environment variable,
BTS_SHARED_MEMORY_PREFIX (if defined). The env var is only necessary
when 2 exe's with the same name run simultaneously.
For app data:
* On linux, the default is:
/home/<user_name>/.local/share/bts/<proc_name>
* On windows, the default is:
/User/<user_name>/AppData/Roaming/bts/<proc_name>
* This path can be overridden in the environment variable:
<proc_name>__APPDATA
- For example, SPARKD__APPDATA, PUBSUB__APPDATA, etc.
2026-01-04 21:08:12 +00:00
John Colvin
51ab182b2f
Don't log "initializing spark config" if it has already been initialized
2026-01-04 21:08:12 +00:00
John Colvin
856cd552eb
refactor: Add Api Server service modules for Dependency Injection
...
* Part of a broader paradigm shift towards Dependency Injection and
Factory Pattern.
* Adds "api-server" service to docker compose
* Replaces MarketDataClient w/ IMarketDataClient in a few streams
* Replaces ViewStreamDatabase w/ IWorkScheduler in a few streams
2026-01-04 21:08:12 +00:00
John Colvin
997647e917
refactor: Add handy WebsocketExtensions
...
* IsSocketStateOpen
* IsClosedProperly
2026-01-04 21:08:12 +00:00
John Colvin
687a9741db
fix: Don't assert on exception in BooleanExpression.evaluate
...
Bugs (fixed):
* DataStream WebApp Server sometimes crashes when the user enters a
bad query.
2026-01-04 21:08:12 +00:00
John Colvin
7faa0ade24
diag: Assert that local ip address is successfully obtained
2026-01-04 21:08:12 +00:00
John Colvin
4d6d68aff6
improve: Resolve hostnames when comparing ip addrs
...
So we can to configure pubsub routes by hostname or ipv4 (ipv6 hasn't
been tested yet...so we should assume it doesn't work).
2026-01-04 21:08:12 +00:00
John Colvin
dcf291d6b2
diag: Assert TcpConnection endpoint matches expected
2026-01-04 21:08:12 +00:00
John Colvin
a6b83988e7
diag: Catch exceptions on AsyncTCPConnector_AsyncWrapper bg thread
...
Since it spawns a new thread on each attempt, uncaught exceptions were
previously undetected, and caused silent failures down the pipe. This
should make it easier to track such bugs down.
2026-01-04 21:08:12 +00:00
John Colvin
2c097500f4
fix: Format PubSubRoute as json in log msgs
...
It was previously using the default ToString, which simply returns the type name
2026-01-04 21:08:12 +00:00
John Colvin
81a212e7a3
build: Upgrade packages (deps) in DataStreamTests
2026-01-04 21:08:12 +00:00
John Colvin
449453625c
vscode: Add PubSubServer debug cfg to skip build
2026-01-04 21:08:12 +00:00
John Colvin
5058187d5b
cleanup: Remove reference to spark protobuf Config in ViewStream
...
In general, we should keep BlueOrb dependencies on spark/sparkd to a minimum.
The dotnet dependency injection architecture provides a robust config framework
we can use instead, based on json files, environment variables, and command-line
arguments.
2026-01-04 21:08:12 +00:00
John Colvin
0253a0f60f
refactor: Run RemoteQueryManager as an IHostedService
...
This paradigm more accurately represents the RemoteQueryManager as a
Service that can be started/stopped.
2026-01-04 21:08:12 +00:00
John Colvin
6986cb5e8e
fix: Ensure spark config init and login are only performed once
...
As a static function that could be called from anywhere in the code
(directly or indirectly), 'Configuration.Initialize' was vulnerable to
race bugs. The "Configuration.get<T>()" method, whose calls are
ubiquitous in our code, would initialize config if it hadn't been done
so already. Without a reference to the configServer param, this param
would effectively be ignored/overridden with null.
It's important that login only be performed once, for obvious reasons
(our exes that connect to sparkd are generally designed to process data
for a single user). Furthermore, not every routine that requires a
successful login has a reference to the credentials needed to do it. To
address this, we now pass the ISparkdLoginService singleton to these
routines so they can wait until the singular login task has completed
before they begin.
2026-01-04 21:08:12 +00:00
John Colvin
35a45c35ba
refactor: Move position model to spark_api
...
So we can use it in the new sdk (c# server-side api) for Ophil
2026-01-04 21:08:12 +00:00
John Colvin
970555d2a1
diag: Assert ViewStreamDatabase.instance is only created once
...
The instance must be initialized via ViewStreamDatabase.SetGlobalInstance
2026-01-04 21:08:12 +00:00
John Colvin
1d46ea061a
cleanup: Remove unused code from ViewStreamDatabase
2026-01-04 21:08:12 +00:00
John Colvin
4bd12dc465
fix: Define BoooleanConstant before StringConstant
...
Otherwise the former masks the latter and prevents it from matching
2026-01-04 21:08:12 +00:00
John Colvin
ad1d404314
cleanup: Remove unnecessary deps from csprojs
2026-01-04 21:08:12 +00:00
John Colvin
a7b6da27ae
fix: Catch exception when creating TcpClient
2026-01-04 21:08:12 +00:00
John Colvin
de4a55b34c
fix: Make SystemSubscriptionMapPublisher threadsafe
...
The PubSubState.onTagMapChanged event may not be invoked on
the same thread as doWork.
2026-01-04 21:08:12 +00:00
John Colvin
2e14598667
fix: Add a few null checks
...
* TimeBasedFilter
* TableAccumulator2
* NonIncGroupByStream
2026-01-04 21:08:12 +00:00
John Colvin
3038d9154f
improve: Reduce task wait/timeout code in AsyncTCPConnector
2026-01-04 21:08:12 +00:00
John Colvin
5805758187
diag: Assert System publishers 'doWork' is always called on same thread
2026-01-04 21:08:12 +00:00
John Colvin
2c1b7b95a4
improve: Propagate DataStream exceptions via ErrorChanged event
2026-01-04 21:08:12 +00:00
John Colvin
60af47c4eb
cleanup: Remove unused PubSubConfig props
2026-01-04 21:08:12 +00:00
John Colvin
b0d0594e3d
fix: Always call 'handle_internal_pubs' in PubSubMessageReactor main loop
2026-01-04 21:08:12 +00:00
John Colvin
b29b2a0adb
fix: Fix a few column types in in PubSubServerTagStatsStream
...
* total_bytes (int -> long)
* last_update_time (DateTime -> long)
2026-01-04 21:08:11 +00:00
John Colvin
76fa35382e
refactor: Add DataStream service modules for Dependency Injection
...
Namely, in DataStreamPublisher and PubSubServer. This is part of a broader
paradigm shift towards Dependency Injection and Factory Pattern.
https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection-guidelines
* Use interfaces to decouple services:
- Decouple ManagedSubscriber from ISubscriberCore impl
- Decouple PubTableBroker from IPublisherCore impl
- Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
* Model config profiles after dotnet appsettings
- Rename Configurations/<name>.json -> appsettings.<name>.json
- Make json fields match corresponding csharp props verbatim
* The Dependency Injection framework doesn't support JsonPropertyName
attributes. Config class props must match json fields verbatim.
* Move reusable startup code into new Bts.Extensions.Hosting proj
* Add pubsub and md-publisher services to compose.yaml
2026-01-04 21:08:11 +00:00
John Colvin
716bc92394
refactor: Only support 1 tag type in IPubSubServerTagHandler
2025-12-30 11:42:23 -05:00
John Colvin
3a6141f3ae
refactor: Change Configuration.initialize to be async
...
Tweak SparkCoreServices to synchronize Configuration.init globally
2025-12-30 11:42:20 -05:00
John Colvin
be2368f95a
refactor: Add Spark service modules for Dependency Injection
2025-12-30 11:38:21 -05:00
John Colvin
7e32ac9e06
cleanup: Remove unused pubsub tag handler code
...
* Removed 'IPubSubServerTagHandler.register'
- All implementations of this function were empty
* Removed 'IPubSubServerTagHandler.supportedTagTypes'
- This property wasn't referenced anywhere
* Remove IWorker inheritance from IPubSubServerTagHandler
- No reason these interfaces have to be tied together
2025-12-30 11:38:21 -05:00
John Colvin
8fc4df1427
refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
...
This matches the file name
2025-12-30 11:38:21 -05:00
John Colvin
2020ebed2d
cleanup: Change pub sub schema prefix 'bluto://' -> 'bts-cloud://'
...
In config json files (not actual code change)
2025-12-30 11:38:20 -05:00
John Colvin
cc483ca4b5
diag: Fix typos in log msgs
...
* Bad string interopolation syntax
2025-12-30 11:38:20 -05:00
John Colvin
aae7356dc7
vscode: Add DataStreamPublisher debug cfg to skip build
2025-12-30 11:38:20 -05:00
John Colvin
8d6e94384c
cleanup: Fix benign typos in build scripts
...
Change BUILD_SPARK_PY var name to BUILD_PY (these scripts build other
projects than 'spark').
2025-12-30 11:38:20 -05:00
John Colvin
7d6e4efa4f
cleanup: Fix numerous build warnings in DataStreamSuite
...
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.
* Compression related files
- BZip2InputStream (disable nullability warnings)
- BZip2OutputStream (disable nullability warnings)
- DeflaterHuffman (disable nullability warnings)
- Inflater (disable nullability warnings)
- InflaterDynHeader (disable nullability warnings)
- ParallelJoinTest (disable nullability warnings)
- TarHeader
- TarInputStream (disable nullability warnings)
- ZipEntry
- ZipFile
- ZipInputStream
* AsyncTCPConnection_APM
* AsyncTCPConnection_APM
* InflaterHuffmanTree
* ManagedSubscriber
* ManagedTCPMessageHub
* MapStream
* PubSubClientMessage
- Normalize indentation/whitespace
- Fix some typos
- Address nullability issues to fix build warnings
- Modernize code
* PubSubMessageReactor
* PubSubServerManagedTCPConnection
* PubSubSingleTagStateEvent
* PubSubState
* PublisherCore
* PubsubDeserializationController
* RemoteMessage
- make 'message' nullable to resolve warnings elsewhere (in calling code)
* SharedStreamConfig
* SingleTagData
* SubscriberCore
* SuperMapStream
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SystemSubscriptionMapPublisher
* ThreadSafeQueue
* TradeBarsStream
Only use Registry on windows.
2025-12-30 11:38:20 -05:00
John Colvin
02f566a970
cleanup: Modernize PubSubRoute code, json parsing
2025-12-30 11:38:20 -05:00
John Colvin
d0d05a5c18
cleanup: Remove unused DateTime grammar from exp parser
2025-12-30 11:38:20 -05:00
John Colvin
3fafeca9ad
cleanup: Remove empty file (clearly added by accident)
2025-12-30 11:38:20 -05:00
John Colvin
f5a4e7c410
fix: Remove query from literal error stream
...
The parser can't handle colons in the expression (it throws/crashes)
2025-12-30 11:38:17 -05:00
John Colvin
a0b2a8b186
chore: Add async SQLiteEnumerator to read table as IBlueRowCollection
2025-12-30 11:36:35 -05:00
John Colvin
35a207e234
chore: Add various collection classes
...
These will be useful in stream operators, row indexing, etc:
* AggregateList
* CircularBuffer
* DeltaList
* DictionarySetWrapper
* ImmutableOrderedDictionary
* ImmutableOrderedSet
* IMutableCollectionFacade
- inherits IReadOnlyCollection<T> and ICollection<T>
- throws NotSupportedException in default impls for mutation ops
* IMutableListFacade
- inherits IReadOnlyList<T> and IList<T>
- throws NotSupportedException in default impls for mutation ops
* IndexDependentSelectorList
* MergedSortedList
* OffsetList
* OrderedDictionary
* ReverseArrayList
* SliceList
* StepList
More notes:
* Renames ExceptionAssert.is_valid ExceptionAssert.Index.IsValid.
- Like the 'ExceptionAssert.Argument' class, which provides methods to throw
ArgumentException if a condition isn't satisfied, this class throws
IndexOutOfRange exceptions.
2025-12-30 11:36:35 -05:00
John Colvin
2069c64864
chore: Support mutations in AggregateDictionary (optionally)
2025-12-30 11:36:35 -05:00
John Colvin
9a29606fc1
style: Use standard c# capitalization rules
...
* Use 'PascalCase' for public members (instead of 'snake_case')
* Use 'camelCase' for private members (instead of '_snake_case')
2025-12-30 11:36:34 -05:00
arham
e69012a5ad
fix: sanitized dsl in websocket
2025-12-03 13:35:08 +00:00
arham
4237d58f76
Add: metadata to dsl callback
2025-12-03 13:34:37 +00:00
arham
1c75f4a53f
Add: get last n lines of log for websocketserver
2025-12-03 12:24:16 +00:00
arham
aabf1490eb
Fix: Trycatch on base program.cs to prevent crash
2025-12-03 12:21:03 +00:00
arham
14a79b6352
Add:: cross exchange arb stream
2025-12-02 15:21:06 -05:00
arham
f69b517f83
add: f10 to stop dsqa
2025-12-02 13:05:39 -05:00
arham
2d0d33028a
fix: bug fix to rowbasedappendstream
2025-12-02 12:58:22 -05:00
arham
f953b3bb8a
change: change UnionStream name to AppendStream
2025-12-02 12:47:36 -05:00
arham
73672276f3
Add: add year to datetimexpander stream
2025-12-02 12:46:49 -05:00
John Colvin
10bb90c731
fix: Use IDictionary.GetEnumerator explicitly to iterate keyvals
...
We really need to replace these old non-generic collections with generic ones.
BlueRow's IDictionaryEnumerator implementation iterates over values when passed
to a foreach loop, not key value pairs.
2025-11-19 23:55:35 -05:00
John Colvin
0746b27a40
cleanup: Remove docker submodules again
...
I lost count of the number of times they have been deleted long ago.
Taha, please remove them from your repo so they don't come back :)
2025-11-19 10:33:31 -05:00
John Colvin
3a3c3d45d3
cleanup: Change default DSQA sparkd config server to 127.0.0.1
2025-11-19 10:33:31 -05:00
John Colvin
2014931653
feat: Add live bbo stream
2025-11-19 10:33:31 -05:00
John Colvin
aa740e786c
refactor: Automatically add id column in BlueTableSchemaBuilder
2025-11-19 00:36:56 -05:00
John Colvin
12c6b367f4
fix (preemptive): Avoid StackOverflow in md client securities fn
...
One of the 'securities_incremental' fns was calling back into itself instead of
the other overload. This is a 'preemptive' fix because the problematic code path
wasn't running in practice (though it was a ticking time bomb).
2025-11-19 00:36:56 -05:00
John Colvin
5080960b97
refactor: Replace DataPacket w/ IDataPacket in stream handlers
...
This commit also renames BlueTableSchemaAdaptor to BlueTableSchemaAdapter,
and makes it implement IBlueTableSchema.
2025-11-19 00:36:56 -05:00
John Colvin
770e14752a
chore: Add json facilities for DataStream websocket api
2025-11-19 00:36:56 -05:00
John Colvin
e24b877e58
Add gen purpose blue row/table utils for data streams
...
* Add BlueRowReflection<T> so ViewStream implementations can use
strong row types without needing to implement IBlueRow
* Add default implementations for IDictionary members in IBlueRow to make
implementing IBlueRow less tedious
2025-11-19 00:31:53 -05:00
Taha Afzal
09abe4a432
Added RowBasedAppendStream
2025-11-18 03:10:30 +00:00
Taha Afzal
66c0baa150
Adding meta data to DataPacket, NLStream adds nl -> dsl as a name value pair in the meta data. Also removing windows ref from DataStreamStudio.sln
2025-09-24 13:16:28 +00:00
arham
5305b32c04
Fix file path didnt work in linux
2025-09-17 04:28:25 +00:00
arham
e6304609f8
changes to make streams run on linux
2025-09-16 15:38:21 +00:00
arham
91d6f80b45
streams ws defaults to nl
2025-09-16 15:33:58 +00:00
arham
21d84d811d
simple changes to make stream work with wbesite
2025-09-02 17:17:43 +00:00
arham
77e542efa9
Fix: websocket code
2025-09-02 17:17:43 +00:00
Taha
6b28fc4779
Lot of BlueOrb stream changes
2025-09-02 17:17:43 +00:00
John Colvin
c0602ceb93
fix (api server linux): Add missing deps to fix crash on start
...
The api server likely broke when we reorganized cpp with stuttering dirs.
* Link Catch2 and ssh2
* Add a few .cpp files to the build
2025-09-02 14:06:08 +00:00
John Colvin
6f5d89cceb
fix build (api server linux): Trivial cmake changes
...
* Get rid of old 'cfg_defaults' param in get_proto_sources impl for linux client
builds
* Make libfmt dep header-only
* Use CMAKE_SHARED_LIBRARY_PREFIX if "PREFIX" is undefined when copying libspark
to dotnet bin dirs
2025-08-29 19:54:39 +00:00
arham
7baaafbeeb
sasalgo local pos working
2025-08-29 15:50:43 +00:00
Adeel Khan
8569b02f5c
cleanup: spread websocket.
2025-08-25 11:45:57 +00:00
Adeel Khan
8f0784e6c2
cleanup: logs.
2025-08-25 11:31:28 +00:00
Adeel Khan
c15968e17e
ADD: py_spread.h..
2025-08-25 07:51:04 +00:00
John Colvin
0e331229d4
fix build: Add missing constructors for SecuritiesEventLog
2025-08-25 06:07:30 +00:00
John Colvin
45c85310e7
hotfix: Remove 0 qty check in ExecutionReport validation fn
...
This (hopefully) fixes part of a compounding error where 1) the SprayAndSkewAlgo
sends 0 lot orders, which are rejected by the exchange. Then 2) the REJECT exe
report fails a misguided "validation" check due to its 0 qty, and is never
propagated to the rest of the system.
2025-08-25 05:53:36 +00:00
John Colvin
7b41dbb19a
diag: Reduce "Waiting for request" log level to diag in py bridge
...
It's too noisy for Info
2025-08-24 17:45:21 +00:00
Adeel Khan
e6a3116327
FIX: Minor recorder fixes.
2025-08-21 22:12:03 +00:00
Adeel Khan
86ce16790a
FIX: Changed log level.
2025-08-21 22:12:03 +00:00
Adeel Khan
897cf10be6
ADD: Support for bbo in MDRecorder & Config support.
2025-08-21 22:12:03 +00:00
Adeel Khan
ca27903f63
ADD: Support for Spreads on Bybit. Gateio fixes.
2025-08-21 22:12:03 +00:00
Adeel Khan
1b5f1bfb27
UPDATE: Added back Spots to Bybit MD.
2025-08-21 22:12:03 +00:00
John Colvin
217db51cbc
fix: Propagate error message to ExecutionReport in bybit gw
...
Specifically, the 'rejectReason' field in order update json.
2025-08-21 13:52:57 +00:00
John Colvin
c08a893ecf
change: Hardcode PostOnly flag for Limit/GTC orders in bybit gw
...
'PostOnly' is a TimeInForce flag that we don't have in our system. It makes it
so Limit orders that would be filled on entry are rejected. Ali's SprayAndSkew
algo needs this to avoid taker fees.
2025-08-21 13:31:37 +00:00
John Colvin
a022f08b49
fix: Don't log error when positions are empty in bybit gw
...
This particular error log was intended for failed position requests. Since an
empty list evaluates to false in python, we were incorrectly interpreting an
empty result as a failed request.
2025-08-20 15:08:25 +00:00
John Colvin
5af98658d8
fix: Fix typos in bybit_execution_gateway.py
...
* Change 'new_req' -> 'req'
* Change 'self.exe_report_callback' -> 'self.bridge.on_execution_report'
...this is what you get when you take shortcuts and copy/paste code.
2025-08-18 22:14:55 +00:00
John Colvin
933511f430
fix: Reset last filled qty to 0 on updates with no fills in bybit gw
...
For example, on status change updates.
2025-08-18 18:54:38 +00:00
John Colvin
63291d7374
fix: Buffer req on ws closed exception just like normal disconnect in bybit gw
2025-08-18 18:31:58 +00:00
John Colvin
30276066f8
cleanup: Remove redundant code
2025-08-18 18:24:05 +00:00
John Colvin
809a66323d
fix: Guarantee unique tx id for bybit exe reports
...
Use a version counter instead of 'time.time_ns' (the latter probably would have
been fine in practice, but the former is better).
2025-08-18 15:54:22 +00:00
John Colvin
5c4ee1a262
fix: Insulate algos from bybit disconnect/reconnect
...
* Buffer incoming requests while reconnecting
* Sync orders upon reconnect and fire execution reports to account for diff
2025-08-18 15:54:22 +00:00
John Colvin
504f441d2e
fix: Count fills correctly in bybit exe gw plugin
...
* Round fill price and qty to 7 digits in bybit gw plugin
* Remove erroneous line of code that reset filled_quantity to 0 on each
update (it was causing partial fills to be overcounted)
2025-08-18 15:51:11 +00:00
John Colvin
fd51025b00
diag: Increase waiting for admin data log throttle to 5s
...
This log is particularly noisy if the configured account isn't in the system. It
logs continuously until the account is added via the Risk webapp.
2025-08-18 15:51:11 +00:00
John Colvin
f4afd92db2
diag: Get sec name from py_sec instead of var that was std::moved
2025-08-18 15:51:11 +00:00
John Colvin
b1fe5b52f6
diag: Change bybit_dev exe gw console level to INFO
...
I found myself constantly changing it from WARN to INFO while developing/testing.
This is a 'dev' config profile, so it should be tuned for development.
2025-08-18 15:45:43 +00:00
John Colvin
164ca306f5
hotfix: Reject new orders when bybit gw not in READY state
2025-08-15 13:42:33 +00:00
John Colvin
a6d23ef4ef
devcontainer: Add python extension
2025-08-15 04:00:25 +00:00
John Colvin
c0f4090d70
fix: Add error_msg to PyExecutionReport json fmt
2025-08-14 03:54:33 +00:00
John Colvin
63c94e09e7
diag: Add bybit order ticket webapp for testing purposes
2025-08-13 21:39:21 +00:00
John Colvin
fc94ec4d46
feat: Automatically reconnect in bybit exe gw plugin
...
Previously, the plugin would just throw an exception and assert crashed
when this happened (the exception would bubble up to the c++ bridge).
2025-08-13 21:39:21 +00:00
John Colvin
e2fdb569c6
feat: Fire position snapshots in bybit exe gw plugin
2025-08-13 21:39:21 +00:00
John Colvin
804c27eb49
fix: Stop bybit websocket stream ping timer on disconnect
2025-08-13 21:39:21 +00:00
John Colvin
6448b4cf90
diag: Add support for custom json formatter in JsonElement
...
The JsonElement class is primarily used for structured logging, as it provides a
means to format objs as json via printf style formatting outside the "hot" path.
2025-08-13 21:39:21 +00:00
John Colvin
e01a825aff
fix: Create parent dirs if necessary for py plugin log files
2025-08-13 21:39:21 +00:00
John Colvin
00649e0c14
chore: Hook up py execution gw position callback
...
Refactor the py bridge placeholder function to accept a vector of position
objs (instead of a single one) to more accurately reflects its c++ counterpart.
It also allows us to pass the account's entire position (for all securities) in a
single call.
2025-08-13 21:39:21 +00:00
arham
ae1e9ecb29
Removing docker submodules
2025-08-11 17:03:26 -04:00
John Colvin
ff65acba05
fix: Gracefully handle missing column settings in pnl win
...
Pnl window settings are stored with the workspace on the server (like all shell
window settings), and may originate from an older version of Spark with a
different set of columns. We shouldn't crash if this happens.
2025-08-07 14:33:55 -04:00
John Colvin
dc08945719
fix: Check proto HasPrice/QuantityDisplay before accessing props
2025-08-07 14:02:10 -04:00
John Colvin
a739ca9a75
fix build: Disable warning in cs Timestamp struct
...
It is initialized from data stored in shared memory (not a normal struct). The
cs build emits a warning because it detects a field that "is never written to
and will always have a default value of 0."
This code hasn't changed for years. It suddenly becoming a warning-as-error is
likely related to a recent Visual Studio update.
2025-08-07 13:39:21 -04:00
John Colvin
ed7b187876
feat: Add ExportToCsv button to fills win
2025-08-07 13:01:06 -04:00
John Colvin
53e1e92a6a
fix: Keep existing items in SettingsPersistenceModel filtered sub
...
Bugs (fixed):
* SettingsPersistenceModel drops items after initial update in filtered
subscriptions.
Cause: The SettingsPersistenceModel subscribe fn only publishes the "added"
items in each update, and omits pre-existing items. From the observer's
perspective, pre-existing items appear deleted.
2025-08-05 16:23:02 -04:00
John Colvin
eee7af494a
fix (fills ui): Preserve sort order during incremental update
...
Binary search is used to add, remove, and change rows in response to
incremental updates. Needless to say, if the rows become unsorted,
everything breaks.
Bugs (fixed):
* The fills window crashes sometimes, possibly related to client
reconnecting to server.
Cause: This was a 2 part bug.
1) Changes to existing rows are applied BEFORE the new sort index
is determined via binary search. If the change affects a
property in the current sorting criteria, the rows could
become unsorted and thus invalidate the binary search.
2) When an existing row is changed so that its new sort index is
greater than its current one, the new index is not decremented
by one (it must be since the row is removed before it is
re-inserted).
2025-08-05 13:47:06 -04:00
Taha Afzal
91a7c35edf
Registering Pingpong algo
2025-08-05 15:59:43 +00:00
Taha Afzal
3eaef1a1b6
PingpongAlgo scaffolding
2025-08-05 15:22:53 +00:00
arham
4a70b4910d
config: test algos
2025-07-28 13:07:15 +00:00
arham
7697f73ada
Fix: change order of plugin loading
2025-07-28 13:03:45 +00:00
arham
029875e0a0
Add: SASALGO
2025-07-28 13:03:26 +00:00
John Colvin
9dc7611c0a
fix build: Forgot to add NullableWrapper.cs a few commits back
...
When I made the change to format min order qty/notional cols with sec
ticks
2025-07-21 17:07:52 -04:00
John Colvin
782cdd7430
ui (orderbook): Use dark theme font color in Algo State grid
...
It was barely legible to against the dark background (must have been
overlooked when we originally implemented dark theme).
2025-07-21 10:02:38 -04:00
John Colvin
4184278d94
fix: Fix typo in weighted_average fn
...
Was multiplying distance from x by wrong weight
Bugs (fixed):
* Pnl window avg cost is wrong when there are both start and day
positions
2025-07-18 09:54:17 -04:00
John Colvin
0b3fa813f1
tool: Add script to dump structured logs to console
2025-07-17 16:25:23 +00:00
John Colvin
e60f2159e1
fix: Prevent python plugins from intercepting kill sig
...
So the process actually terminates on ctrl+c
2025-07-17 16:25:23 +00:00
John Colvin
610861be20
diag: Shorten BookLevel json field names
...
To reduce log file size and improve readability
2025-07-17 16:25:23 +00:00
John Colvin
1787460c35
refactor: Rewrite binance usds futs md without ccapi
2025-07-17 16:25:23 +00:00
John Colvin
588b86710f
refactor: Rename lib_binance_execution lib_binance
...
Implementing md and execution for the same exchange in separate projects
has proven to be messy and impractical. I'm renaming this project in
advance of a binance md rewrite which will join md and execution
together in the same lib.
2025-07-17 16:25:23 +00:00
John Colvin
3022a25a10
Add config toggle to batch subscribe binance ufuts md
2025-07-17 16:25:23 +00:00
John Colvin
cf30649dd2
fix: Fix elapsed check in ConcurrentQueue waiter
2025-07-17 16:25:23 +00:00
John Colvin
afb8444815
diag: Use structured logging in gen md listener
2025-07-17 16:25:23 +00:00
John Colvin
fc59247958
cleanup: Remove redundant if check in gen md listener
2025-07-17 16:25:23 +00:00
John Colvin
14c417eeec
refactor: Use core::TimeStamp::to_ll in gen md listener
...
Working towards unifying TimeStamp code
2025-07-17 16:25:23 +00:00
John Colvin
62a6e4340f
chore: Add to_ll fn to core::TimeStamp
...
One step closer to unifying TimeStamp code
2025-07-17 16:25:23 +00:00
John Colvin
cce85e45b7
chore: Make a few domain types fmt formattable
...
So we can pass them directly to LOG statements
* Book
* BookSide
* TradeRecord
2025-07-17 16:25:23 +00:00
John Colvin
ff584ee19a
refactor: Make pretty enum formatting more performant
2025-07-17 16:25:23 +00:00
John Colvin
f146859677
diag: Disable frame_header/payload logs in ws client
...
They're useless noise. Worse than that, the messages are byte strings
that cause text searches to omit matches (via grep, less, etc.).
2025-07-17 16:25:23 +00:00
John Colvin
cf55e4bc1d
refactor: Make domain::Security default ctr constexpr
2025-07-17 16:25:23 +00:00
John Colvin
a0b1078572
chore: Move a few inlined fns to .cpp
2025-07-17 16:25:23 +00:00
Taha
afd229b7c7
Merging
2025-07-15 17:51:28 -04:00
John Colvin
bfc7615c9a
ui: Add "Is Aggressor" column to fills grid
2025-07-15 14:48:44 -04:00
John Colvin
0023574905
build: Add data stream projs to spark_client.sln
...
They're already built with the solution as dependencies, so it seems
appropriate to include them. This also fixes intellisense issues in
those projs when you have spark_client.sln loaded.
2025-07-15 10:22:23 -04:00
John Colvin
7e79a21cd4
Add some nifty DataPacket/TableUpdate IObservable utils
2025-07-15 10:13:53 -04:00
John Colvin
6b0030559b
fix build: Fill in transaction fee prop getter
...
TODO: Actually implement maker/taker fee. For now, just return 0.
2025-07-15 10:13:53 -04:00
John Colvin
77caef2640
fix build: Add missing name to fn arg
2025-07-15 10:13:53 -04:00
Adeel Khan
edb8a9d516
FIX: Removed Commented Out Code.
2025-07-14 22:35:36 +00:00
Adeel Khan
153631191a
FIX: Removed fallbacks(not needed).
2025-07-14 22:35:36 +00:00
Adeel Khan
026a8af893
FIX: Removed Normalize Contract Function-Not Needed.
2025-07-14 22:35:36 +00:00
Adeel Khan
3739532947
FIX: Reduced REST calls.
2025-07-14 22:35:36 +00:00
Adeel Khan
0cb3cadc11
FIX: Added filter to exclude InverseFutures--MD not freezing.
2025-07-14 22:35:36 +00:00
Adeel Khan
36db69dc67
FIX: bybit_md_provider.py works with default symbol format in py_bybit_md_provider.json.--MD Freezes on InverseFutures.
2025-07-14 22:35:36 +00:00
Adeel Khan
c7c755a964
ADD: Added support for Linear USDC channel-py_bybit_md_provider.json takes full security names.
2025-07-14 22:35:36 +00:00
Adeel Khan
0e460353a7
ADD: BYBIT MD support for USDC Margined Perps.
2025-07-14 22:35:36 +00:00
Adeel Khan
5198236b45
ADD: BYBIT MD support for Spots and Linear USDT Margined Perps/Futures.
2025-07-14 22:35:36 +00:00
Taha
767fa25d61
Adding comments
2025-07-14 15:18:03 -04:00
Taha
0858eaac23
Polishing BlueOrb streams
2025-07-13 21:21:10 -04:00
Taha Afzal
593601551d
Adding FillAggressorFlag o ExecutionReport
2025-07-13 22:43:19 +00:00
John Colvin
036df46b1c
deploy: Copy algo plugins from config to runtime dir
...
So we can deploy algos by uploading them to the config dir on the
server, then bouncing sparkd.
2025-07-07 17:24:13 +00:00
John Colvin
84697a6d65
dev qol: Add slim requirements.txt to binance_dev cfg profile
...
So it doesn't needlessly install every package in our dev venv when it
builds the container, which can take several minutes.
2025-07-07 16:34:29 +00:00
John Colvin
bb550d3f88
fix (deploy): Add PYTHON_REQUIREMENTS docker arg to base image
...
Docker args must be redeclared in each build stage that uses them.
2025-07-07 16:34:29 +00:00
John Colvin
85bd2da414
cleanup: Remove security from StoredFillEvent
...
It created an unnecessary dependency
2025-07-07 16:02:42 +00:00
John Colvin
cc55fe1acf
ui: Format min order qty/notional cols with sec ticks
2025-07-07 12:00:39 -04:00
John Colvin
bb64760c62
fix (mview): Add switch case for Venue column comparer
...
Bugs (fixed):
* In mview, venue column sorting is broken
2025-07-07 12:00:39 -04:00
John Colvin
8b9525f503
dev qol: Add python.env for plugin intellisense
2025-06-26 04:25:43 +00:00
John Colvin
eccfd1ee30
fix: Disable ccapi md provider if venue has empty symbols config
...
Previously, ccapi exchanges (e.g., binance) would subscribe to EVERYTHING if
symbols config was empty. There are way too many symbols for that.
2025-06-26 04:20:08 +00:00
John Colvin
7b933026bb
refactor: Only import fake_md_bridge if running plugin standalone
2025-06-25 16:33:30 +00:00
John Colvin
ffe859703a
fix: Forgot to add fake_md_bridge.py
...
Needed to run py md plugins as standalone apps
2025-06-25 16:30:44 +00:00
John Colvin
4756e243fa
chore: Add DUCK and MILK to bybit_dev config profile
2025-06-24 19:57:44 +00:00
John Colvin
0133159296
fix: Support integer ticks in sparkd, py plugins
...
Bugs (fixed):
* sparkd crashes if DUCK is added to bybit symbols
Cause: DUCKUSDT qty tick is 10 (previously, we only supported qty
ticks <= 1)
2025-06-24 18:09:20 +00:00
John Colvin
38afd3055f
diag: Add sec name, venue, and type to PyBook json
2025-06-24 18:08:27 +00:00
John Colvin
4f184c2566
feat: Support running bybit_md_provider as standalone app
...
HACK: I changed it so numpy is imported AFTER python_interop_module to
avoid a strange segfault that only happens when running standalone app.
2025-06-24 18:08:27 +00:00
John Colvin
f608497094
fix: Propagate min notional value from py plugins sparkd
...
It was getting lost in the translation from PySecurity ->
domain::Security -> FixedSecurity.
2025-06-24 15:27:13 +00:00
John Colvin
9035b95612
fix: Propagate newly deleted synths in MDDaemon
...
There was a typo when determining "deletes" and "undeletes" on synth
changes (i.e., when deleted changes false -> true, and true-> false
respectively).
2025-06-24 10:48:42 -04:00
John Colvin
7483665b5a
chore: Set min notional value for secs in bybit md provider
2025-06-24 10:48:42 -04:00
John Colvin
5f141a8030
chore: Add missing TimeInForce enum vals in py
2025-06-24 10:48:42 -04:00
John Colvin
83f1b5b567
dev qol: In envsource.fish, split PATH vars into list
...
Assume they are semi-colon delimited.
2025-06-24 10:48:42 -04:00
John Colvin
06ad3a1d9e
dev qol: Add trade_client_cli.py to test md/exe plugins
...
Independent of sparkd
2025-06-24 10:48:42 -04:00
John Colvin
01de13bb43
refactor: Pass sec id instead of ptr to FilledPositionEvent
...
Then we don't need to resolve the security in order to fire the event.
This removes a noisy log that would happen when the security resolution
fails as a byproduct.
2025-06-24 10:48:42 -04:00
John Colvin
155a396af5
fix: Round tick base to power of 10 in default case
...
Bugs (fixed):
* Bybit execution gw throws on MILKUSDT orders due to bad price tick.
Cause: rounding error when converting the small price tick (0.00001)
to "tick factors" (1/10000). The denominator is computed as
1/0.00001 = 99999.9...
2025-06-24 10:48:42 -04:00
John Colvin
399f9e61e2
chore: Add security type to py reqs
...
Venue and sec name alone don't always identify a unique security.
2025-06-24 10:48:42 -04:00
John Colvin
1f67456cc5
cleanup: Remove unused code
2025-06-24 10:48:42 -04:00
John Colvin
9ff7821b59
fix: Propagate bybit gw conn status change to bridge
2025-06-24 10:48:42 -04:00
John Colvin
c598f831d0
ui: Change default TimeInForce to GTC
...
The old default, "Day" orders, isn't supported at crypto exchanges, it
doesn't apply (there is no "end of day").
2025-06-24 10:48:42 -04:00
John Colvin
3ca4cedf54
ui (mview): Add min order qty/notional value columns
2025-06-24 10:48:42 -04:00
John Colvin
d919ec5474
fix (mview): Bind cpp column filter to correct prop in vm
...
Was bound to "currency_filter" (prob a typo due to lazy copy/paste job)
2025-06-21 01:24:04 -04:00
John Colvin
98c2d738b2
fix: Set timeInForce in bybit order reqs
2025-06-20 20:17:01 +00:00
John Colvin
cf3f7e6862
fix: Don't send unlisted/fake bybit securities to sparkd
...
Securities that aren't listed in the instrument-info response from the
exchange shouldn't be in our system.
2025-06-20 16:37:47 +00:00
John Colvin
ae868ab74d
fix: Use default cash-per-point of 1 at bybit
2025-06-20 16:37:47 +00:00
John Colvin
bd22f4779d
fix: Request all "pages" of instrument info (not just first)
2025-06-20 16:37:47 +00:00
John Colvin
27cbd51b1c
fix: Fire reject report if order ack has errors
...
Rejects are ONLY observable in the order ack, not the execution report
stream from which all other updates are observed. While the pybit sdk
does expose ack callbacks, it doesn't invoke them on rejects/errors.
2025-06-20 16:37:47 +00:00
John Colvin
02626ce35c
chore: Copy a few pybit modules into our code for tweaking
...
No tweaks have been made (yet). I'm copying the files verbatim first
to give us a frame of reference. Then, when the tweaks are committed,
we'll be able to view them in terms of the diff with the sdk.
2025-06-18 18:39:18 +00:00
John Colvin
d219804caf
fix: Prepend plugin dir to py path, rather than append
...
So our modules win import-name conflicts with system/third party ones.
This became a problem when I copied a couple files from the pybit sdk
into our plugins dir in order to apply tweaks.
2025-06-18 18:18:35 +00:00
John Colvin
1e0ec5bb46
chore: Bind __str__ fns for py interop list types
...
The default __str__ for these list types shows item addresses (not the
items themselves)...a bit of a headscratcher considering the lists'
element types have proper __str__ functions.
2025-06-18 18:14:45 +00:00
John Colvin
c281bc49b5
improvement: Set bybit exe gw websocket timeout to 5s
...
The default (1s) is too strict in practice.
2025-06-17 17:49:52 +00:00
John Colvin
b3566155a3
fix: Fix rounding error when creating tick factors
2025-06-16 15:52:09 +00:00
John Colvin
a4e39afbb9
ui: Add cash-per-point col to mview
2025-06-16 15:52:09 +00:00
John Colvin
efce01d28e
vscode: Replace net7 w/ net8 in tasks.json
2025-06-16 15:52:09 +00:00
John Colvin
d362022b89
Add c# SyntheticBookCalculator
2025-06-16 15:52:09 +00:00
John Colvin
e7d751fb68
refactor: Remove unnecessary constraint on MathUtils.lcm params
...
Not sure why T ever needed to implement IMultiplyOperators<T, int, T>
2025-06-16 15:52:09 +00:00
John Colvin
50eefc65d4
cleanup: Remove unnecessary py_module.cpp files
...
We used to have separate modules for md and execution. I forgot
to remove these after unifying them to a single module.
2025-06-16 15:52:09 +00:00
John Colvin
8770de4a87
diag: Reduce book log noise in md plugins
...
They're too noisy for Info level.
2025-06-16 15:52:09 +00:00
John Colvin
e799b6e24b
diag: Optimize logging in bybit exe gw plugin
...
* Format params as json (not python pseudo json)
* Format params lazily
2025-06-16 15:52:09 +00:00
John Colvin
69e338c1ca
fix: Use lock_guard instead of lock/unlock calls
...
I just noticed it was technically possible for the "work thread" (in py
plugin manager) to be started twice, due to improper lock/unlock calls.
2025-06-16 11:08:59 -04:00
John Colvin
e8bb64a80f
Increase sleep time in bybit_md_provider keep-alive loop
...
This thread's only job is to stay alive until "stop" is called. It
doesn't need to be particularly fast/responsive. Longer sleep intervals
reduce contention with the "hot" thread that receives market data.
2025-06-16 11:08:59 -04:00
John Colvin
8e2ede596f
diag: Add security to assert msg when qty tick deduction fails
2025-06-16 11:08:59 -04:00
John Colvin
9772d0eb77
deploy: Add alphaworx_zen_hk_live-preview configuration
2025-06-16 11:08:59 -04:00
John Colvin
687e2477c0
fix: Assert that acc name length < max_size (not size)
...
Since we changed the account field from std::array -> bts::core::SmallString
2025-06-16 11:08:59 -04:00
John Colvin
1bd0ba9950
deploy: Install python requirements in docker
...
No point in using a virtual environment. These containers have one
purpose: run sparkd with a constant, predetermined set of py plugins.
2025-06-16 11:08:59 -04:00
John Colvin
ddabc744f2
chore: Update alphaworx_zen_hk_live init.py to query new secret
...
I decided to put all alphaworx api keys in one secret. It's easier this way.
2025-06-16 11:08:58 -04:00
John Colvin
74b88b003e
fix: Fix config dir env var in bybit_dev profile
2025-06-16 11:08:58 -04:00
John Colvin
aaf1be09c2
fix: Fix SmallString numeric parsing, make more robust
...
* Fix by parsing numbers from left to right instead of right to left
(previously, would parse "17" to 71)
* Support leading +/- signs
* Throw if unexpected char encountered
* Unit tests
2025-06-16 11:08:58 -04:00
John Colvin
e3bd9a457c
fix: Change thread_id type to size_t (not int64_t)
2025-06-16 11:08:58 -04:00
John Colvin
59f62b6ec9
cleanup: Remove unnecessary signal handler in bybit_md_provider.py
...
We handle them properly in sparkd now
2025-06-16 11:08:58 -04:00
John Colvin
b46e75f7d7
cleanup: Sort gateio symbols in symbols.conf
2025-06-16 11:08:58 -04:00
John Colvin
3f100feaca
chore: Add config json for py plugins
...
Support overrides per venue in a separate file, so symbols can be
changed after deployment using arham's widget.
2025-06-16 11:08:58 -04:00
John Colvin
093da2a938
fix: Init shared_ptr w/ default obj (not null)
...
Prob a typo
2025-06-16 11:08:58 -04:00
John Colvin
2f17c2b8c5
chore: Fix path in .env file (server/sparkd -> app_sparkd)
2025-06-16 11:08:58 -04:00
John Colvin
33bfdc517a
fix build: Remove include of non-existent header
...
I created the file locally, but ended up not using it and forgot to remove an
include before I pushed.
2025-06-16 11:08:58 -04:00
John Colvin
d17be48e6d
fix build: Always return value in non-void fn
2025-06-16 11:08:58 -04:00
John Colvin
cbaa1df641
fix: Don't compare size_t to -1 in SmallString num parsers
2025-06-16 11:08:58 -04:00
John Colvin
8c23665d65
refactor: Manage all py plugins (md and exe) in one class
...
All calls to/from python are performed on one thread via a simple
message queue (necessary because of the global interpreter lock).
2025-06-16 11:08:58 -04:00
John Colvin
c7db105533
chore: Add pybit to py venv
2025-06-16 11:08:58 -04:00
John Colvin
3f9923239e
diag: Log incoming books at Info level in bybit_md_provider
2025-06-16 11:08:58 -04:00
John Colvin
9ae9451359
chore: Add bybit execution config to bybit_dev profile
2025-06-16 11:08:58 -04:00
John Colvin
c33e5ddb02
chore: Add numpy to py venv
...
This is needed for md plugins
2025-06-16 11:08:58 -04:00
John Colvin
54388d3b92
fix: Include fill id in format str for ExecutionReport
2025-06-16 11:08:58 -04:00
John Colvin
17ada3c57c
dev qol: Bind PySecurity __format__ to fmt::formatter impl
2025-06-16 11:08:58 -04:00
John Colvin
046f412ec4
refactor: Change floats to doubles in py bridge types
...
Getting rounding errors with floats
2025-06-16 11:08:58 -04:00
John Colvin
c94d11f525
refactor: Use SmallString instead of std::array in ExchangeFillID
2025-06-16 11:08:57 -04:00
John Colvin
2b69846381
Add py bybit exe config to bybit_dev profile
2025-06-16 11:08:57 -04:00
John Colvin
bdd5e08b60
refactor: Consolidate all py interop into single lib
...
It was just too hard trying to make it work with separate libs. The
main issue was getting the python md and execution modules to
recognize PySecurityList and other types defined in the core module.
2025-06-16 11:08:57 -04:00
John Colvin
7c9330650d
refactor: Move tick fns into gen-purpose tick_utils.h
2025-06-16 11:08:57 -04:00
John Colvin
98fca5ad7f
Change bybit_dev symbols.conf to list bybit futs (not spots)
2025-06-16 11:08:57 -04:00
John Colvin
8a0fcce90d
fix: Actually use configured logger in bybit_md_provider.py
2025-06-16 11:08:57 -04:00
John Colvin
112527f9ed
dev qol: Reduce noisy logs in PositionServicePNLHelper
2025-06-16 11:08:57 -04:00
John Colvin
fc1e20538d
build: Add cmake script to sym link py module and plugins
...
We have several of these (python_core, python_md, python_execution),
so the repetative boilerplate code was starting to pile up.
2025-06-16 11:08:57 -04:00
John Colvin
83acbecf55
deploy: Install/activate py venv in sparkd container
2025-06-16 11:08:57 -04:00
John Colvin
06c494291c
dev qol: Add pyrightconfig.json for type hints in py/c++ bridge
2025-06-16 11:08:57 -04:00
John Colvin
fe74aed048
Tweak py to get rid of false positive type hint warns
2025-06-16 11:08:57 -04:00
John Colvin
b94cdb561e
feat: Support python execution plugins
2025-06-16 11:08:57 -04:00
John Colvin
06bd8791c0
refactor: Move py security (and related types) to python_core lib
...
So the execution gw plugins can use it (as well as the md plugins).
2025-06-16 11:08:57 -04:00
John Colvin
b5850cdb02
fix: Add missing switch cases for bybit & gateio
2025-06-16 11:08:57 -04:00
John Colvin
545d76142f
fix: Initialize bybit secs as perp futs (not spots)
2025-06-16 11:08:57 -04:00
John Colvin
edc2cea60e
dev qol: Add wait/notify functionality to ConcurrentQueue
2025-06-16 11:08:56 -04:00
John Colvin
e6466f8309
chore: Add CANCEL_ORDER op to binance.env
2025-06-16 11:08:56 -04:00
John Colvin
501be8d6a5
dev qol: Add ConcurrentValueStore, with notify/wait functionality
...
Uses std::condition_variable
2025-06-16 11:08:56 -04:00
John Colvin
7dea9b3792
fix: Don't raise ex if var isn't defined in req path
2025-06-16 11:08:56 -04:00
John Colvin
a3f49b1ce8
dev qol: Add gen-purpose validation fns for execution gws
2025-06-16 11:08:56 -04:00
John Colvin
765237d701
build: Use new gen-purpose python cmake fns in py md proj
...
Significantly simplifies CMakeLists.txt
2025-06-16 11:08:56 -04:00
John Colvin
97c52410a2
build: Add gen-purpose InitPythonModule, InitPythonPlugin cmake scripts
2025-06-16 11:08:56 -04:00
John Colvin
8b43d93e26
dev qol: Add fn to make Stopwatch and start it at once
2025-06-16 11:08:56 -04:00
John Colvin
14baa7d62e
chore: Use same log format as sparkd in py md plugins
2025-06-16 11:08:56 -04:00
John Colvin
a054f4cc63
improvement: Set PyBook data in single call to reduce interop overhead
2025-06-16 11:08:56 -04:00
John Colvin
5311128607
Reduce log noise in py md providers
2025-06-16 11:08:56 -04:00
John Colvin
075acdc8eb
doc: Improve py md readme
...
* Put examples in a table
* Show command to create symlinks for intellisense
2025-06-16 11:08:56 -04:00
John Colvin
e1fdf6306b
refactor: Redesign py md so it doesn't lock the global interpreter
...
We can only embed one interpreter in the sparkd process, so all py
plugins need to share it.
* Move core py interop code to new lib_python_core. We're about to add a
new lib for py execution, so it needs to be accesible from both
execution and md libs
* Fix typo for type hint in py_md_utils.py
* Run py md providers on background thread
* Rename MdSink -> MdBridge
2025-06-16 11:08:56 -04:00
John Colvin
3b9fb2759b
build: Add lib_spark subdir first (all others depend on it)
2025-06-16 11:08:56 -04:00
John Colvin
3f89d3b7f7
dev qol: Add to_chrono fns to TimeSpan
2025-06-16 11:08:56 -04:00
John Colvin
a6dd0d842d
dev qol: Add -> operators to ValueWrapper for convenience
...
So we can type "foo->" instead of "foo.value()" (much shorter!)
2025-06-16 11:08:56 -04:00
John Colvin
d050d2d19a
build: Link against system installed ssl to support py interop
...
Unless we want to build python packages ourselves, the underlying clibs
they use link to the system installed ssl. Notably, the requests module.
2025-06-16 11:08:56 -04:00
John Colvin
942d00a8b1
chore: Add pybind11, pybind11-stubgen to py venv
2025-06-16 11:08:56 -04:00
John Colvin
31cb25ac2e
improvement: Generate py stubs for md providers
...
Provides intellisense for the python_md_module
2025-06-16 11:08:56 -04:00
John Colvin
cbb43a22de
cleanup: Get rid of unecessary namespace prefixes
2025-06-16 11:08:55 -04:00
John Colvin
9e7b4849f0
cleanup: Get rid of "PyList" classes, bind to stl vector instead
...
I wasn't aware this was possible until recently.
2025-06-16 11:08:55 -04:00
John Colvin
6e72890153
refactor: Move SimpleOrderKeyMap from binance_execution to lib_spark
...
It provides basic functionality that will be useful in other execution gateways.
2025-06-16 11:08:55 -04:00
John Colvin
c238fce057
chore: Add smart parse and pretty name fns for proto enums
...
Mainly to ease the pain of representing enums as strings in python md
and execution plugins
2025-06-16 11:08:55 -04:00
John Colvin
b8f5db4ee2
refactor: Move ErrorList from binance_execution to lib_spark
...
It provides basic functionality that will be useful elsewhere
2025-06-16 11:08:55 -04:00
John Colvin
87c8b9862a
refactor: Use SmallString std::array in AccountName
2025-06-16 11:08:55 -04:00
John Colvin
fbc2d8633d
refactor: Use SmallString instead of std::array in ExchangeOrderID
2025-06-16 11:08:55 -04:00
John Colvin
d12d798fa5
Add our own "SmallString" class for fixed cap strings
...
This will replace the std::array<char, N> we use in ExecutionReport, etc.
among other things
2025-06-16 11:08:55 -04:00
John Colvin
e75be2da2f
refactor: Move ExecutionReportStore to lib_spark
...
From lib_binance_execution
2025-06-16 11:08:55 -04:00
Adeel Khan
3ae3a7094e
CLEANUP: BYBIT Symbols.conf.
2025-06-16 11:08:55 -04:00
Adeel Khan
bafa2f71ba
ADD: BYBIT MD Working, NaN's also Added.
2025-06-16 11:08:55 -04:00
Adeel Khan
63bd29a49e
ADD: BYBIT Securities showing up on frontend. Default Contract Spec
2025-06-16 11:08:55 -04:00
Adeel Khan
5fdc9ca48b
ADD: GATEIO md_provider with dynamic contract specs using Batch REST call
2025-06-16 11:08:55 -04:00
Adeel Khan
a19e7bc7b4
ADD: GATEIO md_provider with dynamic contract specs 1 REST call per Contract
2025-06-16 11:08:55 -04:00
Adeel Khan
04b0a89644
ADD: GATEIO md_provider with default contract specs
2025-06-16 11:08:55 -04:00
John Colvin
363a43cb8b
Add gateio_dev config profile for Adeel
2025-06-16 11:08:55 -04:00
John Colvin
c9f9c5f0f6
cleanup: Remove unused spark.conf file
...
Originally, this spark.conf was hardcoded into compose.yaml, and copied
into the sparkd container on each deploy. It hasn't been used since
config profiles were introduced.
2025-06-16 11:08:54 -04:00
John Colvin
ab84a05644
Add py_dummy_md config profile to test py md plugin
2025-06-16 11:08:54 -04:00
John Colvin
4681f22e71
deploy: Move configurations to new stutter dir "app_sparkd"
2025-06-16 11:08:54 -04:00
John Colvin
a25e055e35
Add protos for gateio
2025-06-16 11:08:54 -04:00
John Colvin
2386612b51
Add protos for bybit
2025-06-16 11:08:54 -04:00
John Colvin
3606dac145
build: Move onix fix adapter to its own lib
...
We should keep order execution dependencies out of lib_spark.
2025-06-16 11:08:54 -04:00
John Colvin
bf936c9d13
fix build: Move OnixsUtils to from lib_spark to app_sparkd
...
They're only needed MDP3Main.cpp and ICEMain.cpp. We're one step closer
to removing lib_spark's dependency on onixs.
2025-06-16 11:08:54 -04:00
John Colvin
1aeeb7170e
cleanup: Remove unnecessary onixs include from TrafixOrderStateHandler.h
2025-06-16 11:08:54 -04:00
John Colvin
78ad8a3aed
feat: Support python md plugins
2025-06-16 11:08:54 -04:00
John Colvin
5ca8dbafaf
Add "clear" fn to Book & BookSide
...
Prelude to python market data bindings
2025-06-16 11:08:54 -04:00
John Colvin
b464b9466e
Add "smart" parse fns for a couple proto enums
...
Prelude to python market data bindings. Unlike the normal parse fns, the
"smart" variants aren't case sensitive, and can match strings with or
without TYPE_NAME_ prefixes.
2025-06-16 11:08:54 -04:00
John Colvin
7a5c676e99
build: Move okx execution to its own lib
2025-06-16 11:08:54 -04:00
John Colvin
1e8d699dd1
cleanup: Remove unnecessary onixs include from IDGenerator.h
2025-06-16 11:08:53 -04:00
John Colvin
aadd2dd594
cleanup: Remove unnecessary includes from MDP3Listener.h
2025-06-16 11:08:53 -04:00
John Colvin
5163ca4cae
refactor: Organize libspark proj w/ stuttering include
...
This new structure provides several benefits:
* Allows us to install multiple CMake projects in the same destination
dir (without include path conflicts).
- This makes shipping the trading algo sdk a trivial excercise. Simply
copy the lib and include dirs from the install dest. That's it!
* Puts us in a position to break up the monolithic libspark into smaller
modules, which are easier to manage.
Windows specific notes:
* Get rid of unnecessary script vs_filters.cmake
* Define WebsocketConnectionGateway directly in cs (instead of proto)
- The proto would no longer be in the include path passed to protoc.
Moving it to c# because there wasn't a compelling reason for it to
be a proto in the first place.
* Set spark output prefix to "lib" on windows
- So we get "libspark.dll" instead of "spark.dll", which conflicts
with the csharp lib, "Spark.dll".
* Copy libspark.dll to net8.0 & net8.0-windows in a single cmake build
target
fix build: Copy libspark to both dirs net8.0, net8.0-windows
2025-06-16 11:08:53 -04:00
John Colvin
b5d520d568
cleanup: Move TradeSummarySignalDispatcher to libspark
...
It is needed by TradeSummarySignal, which is in libspark. This prevents a
circular dependency from libspark -> sparkd.
2025-06-16 11:08:51 -04:00
John Colvin
83e4465a3b
build: Clean up protobuf cmake scripts
...
Make them more flexible, reusable. This is a precursor to porting over
python md provider code from another branch.
2025-06-16 11:08:51 -04:00
John Colvin
54cfb343a8
build: Rename external path vars w/ "EXTERNAL_" prefix in cmake
...
The old "SPARK_" prefix felt like a misnomer.
2025-06-16 11:08:51 -04:00
John Colvin
4e9b360e23
Add a few arithmetic ops to TimeSpan
2025-06-16 11:08:51 -04:00
John Colvin
16bc93b1a0
diag: Log time it takes to start md sources
...
So we can detect bottlenecks
2025-06-16 11:08:50 -04:00
John Colvin
f56d4d9062
refactor: Split MarketDataGateway up into providers/sources
...
MarketDataGateway still exists, but is greatly simplified. It creates
several `IMarketDataProvider`s, each of which generates a list of
`IMarketDataSource`s. The market data sources provide market data to an
`IMarketDataListener`.
* EmumbaMarketDataProvider
* CCApiMarketDataProvider
* PyMarketDataProvider
* OnixsMarketDataProvider
* ReplayMarketDataProvider
As a side effect, individual py market data sources are now explicitly
limited to a single venue (it probably should have been this way all
along, since the shared memory writers do not expect updates to involve
multiple venues).
2025-06-16 11:08:50 -04:00
John Colvin
36cc186626
cleanup: Remove dependency on quickfix for string to_lower
...
We shouldn't depend on external libs for trivial things like converting
strings to lower case.
2025-06-16 11:08:50 -04:00
John Colvin
2e78af1756
fix build: Upgrade IronPython to 3.4.2
...
All csprojs need to be on same version (Spark.csproj was on 3.4.1, while
DataStream.csproj was on 3.4.2)
2025-06-16 11:08:50 -04:00
Taha
70fae06a30
BlueOrb streams and MLP
2025-06-08 21:52:38 -04:00
John Colvin
07b59559a2
dev qol: Reduce md log level in config profile
2025-06-03 02:23:59 +00:00
John Colvin
cd1709563b
fix: Generate reject exe reports from ack at binance
...
This only applies to "new" order acks. In all other cases, exe reports
are generated from the order update stream. New order rejects are the
exception because they're not visible in the stream. Presumably, they're
never actually entered into binance's order system (they're "rejected"
before that happens).
2025-06-03 02:21:18 +00:00
arham
94f22ddf40
Update: changed url for okx ws exe in config
2025-05-30 12:59:26 +00:00
arham
7f6c08f956
Fix: positions now go through dispatch incoming
2025-05-29 16:26:05 +00:00
arham
1d737cdf82
Fix: change time to 5s for pos update okx
2025-05-23 16:12:12 +00:00
arham
b645192355
Add: OKX position turned on
2025-05-23 15:33:11 +00:00
John Colvin
5ad7755b28
tool: Make open orders symbol param optional in binance.env
2025-05-20 20:08:34 +00:00
John Colvin
df4ca124c3
cleanup: Remove redundant script get_binance_md.sh
...
I should have removed this when I added binance_rest_client.sh
2025-05-20 20:08:34 +00:00
John Colvin
2c8b9ee288
tool: Add envsource.fish to load .env files in fish
2025-05-20 20:08:34 +00:00
John Colvin
f02f28e347
tool: Add a few vars to binance.env
2025-05-20 20:08:34 +00:00
John Colvin
c8a89c1e8d
cleanup: Rename get_binance_md.py -> binance_client.py
...
This is a more appropriate name (it's not just md anymore).
2025-05-20 20:08:34 +00:00
John Colvin
7ae9e07a30
tool: Use binance.env file in python client
...
Like the bash impl (binance_rest_client.sh), it gets urls, params
(templates), and api credentials from env vars.
2025-05-20 20:08:34 +00:00
Taha
a130e47218
Arb Scoring stream
2025-05-20 00:24:17 -04:00
John Colvin
a377f6de7f
improvement (ui): Use sec qty display settings in rec report
2025-05-19 12:35:25 -04:00
John Colvin
855373c9e6
cleanup: Remove unnecessary code
2025-05-19 15:57:35 +00:00
John Colvin
e1b535b633
feat: Sync exch positions in binance usds futs gw
2025-05-19 15:57:35 +00:00
John Colvin
3b9ac71460
refactor: Abstract out start/stop state machine for gen use
...
* From binance usds futures gateway
* Thread-safe
* Handles re-entrancy gracefully
2025-05-19 15:57:35 +00:00
John Colvin
9a5075daaf
improvement: Use recursive_mutex in ws client to prevent deadlocks
2025-05-19 15:57:35 +00:00
John Colvin
4e7fe811a8
Add ConcurrentUnorderedMap (c++)
2025-05-19 15:57:35 +00:00
Taha
0a6733180d
Added SpreadStatsStream
2025-05-18 01:10:33 -04:00
Taha
f2169c6bd4
Adding more data to SpreadPerformanceStream
2025-05-16 01:42:32 -04:00
Taha
23ef478391
Spread analytics streams
2025-05-15 15:01:27 -04:00
John Colvin
236a5a9362
fix: Format binance order price up to 5 decimal places
...
Bugs (fixed):
* Binance JELLYUSDT order prices get rounded to 4 decimal places, even
though the price tick is 0.00001
2025-05-13 15:02:02 +00:00
John Colvin
cb560bbe33
fix: Sum realized & unrealized pnl correctly in header rows
2025-05-08 14:13:02 -04:00
John Colvin
248f2644ca
deploy: Automatically restart "sparkd-symbols" service
...
This is what we deploy to the alphaworx-zen-hk-live server
2025-05-07 14:49:53 +00:00
arham
b87d86f4d6
FIx: okx exe user cancel should be cxl not rjct
2025-05-07 08:38:28 +00:00
arham
8515196dcb
Add: okx exe fok orders working
2025-05-07 08:08:40 +00:00
Taha Afzal
e5e042f016
Spreader in RH mode now fires FOK, and hedges only whne the FOK is completely filled (ie not as we we might get partial fills)
2025-05-07 06:12:48 +00:00
John Colvin
da3593326d
Enable FOK orders at binance
2025-05-06 22:20:46 +00:00
John Colvin
313d3a2377
Enable FOK in EMNativeOrder.cpp
...
This class was originally designed for Emumba execution at deribit
(hence the "EM" prefix), but now serves all crypto exchanges.
2025-05-06 22:19:11 +00:00
Taha Afzal
22f0e126d0
Adding FillOrKill (FOK) to TimeInForce
2025-05-06 19:39:40 +00:00
arham
c76b061e20
fmt: okx exe
2025-05-02 09:00:00 +00:00
arham
0107b65281
add code for okx exe position updates
2025-05-02 08:59:46 +00:00
Taha Afzal
6b5496b799
OrderStore does not eleted all orders at sparkd boot-time now
2025-05-02 00:10:48 +00:00
arham
42f9daf9cb
Add: okx cancel all after code, not turned on for now
2025-05-01 09:48:06 +00:00
arham
bc5ddade7f
Add: okx exe small changes to maybe make ping better
2025-05-01 09:42:54 +00:00
arham
d8d198f7f4
Add: OKX only handle orders from this session
2025-05-01 09:41:20 +00:00
John Colvin
af9eb670c1
build: Update build scripts to target dotnet 8
2025-04-30 12:06:28 -04:00
John Colvin
d91837c377
improvement (pnl ui): Get qty display params from sec proto
...
Instead of config
2025-04-30 03:06:04 -04:00
John Colvin
7725ae33ef
fix: Fix total realized/unrealized pnl calc
2025-04-30 01:35:22 -04:00
John Colvin
9f56abaa38
vscode: Replace net7 w/ net8 in launch.json
2025-04-30 01:15:34 -04:00
John Colvin
95444d4a4c
fix: Exclude nans from pnl sum
2025-04-30 00:47:49 -04:00
John Colvin
a7e1c18fe2
diag: Improve trace logs in binance usds futures gw
2025-04-29 19:49:33 +00:00
John Colvin
d11fd23cb0
refactor: Drop "_" suffix in private vars (cpp)
...
It just looks ugly, and unnecessary with modern tooling. Plus it's ugly.
2025-04-29 19:35:15 +00:00
John Colvin
8499ac0dfb
diag: Clean up structured logging in binance order store
2025-04-29 19:35:15 +00:00
John Colvin
f4afe348e6
fix: Update timestamps in binance order store when orders change
2025-04-29 19:35:15 +00:00
John Colvin
096424b60b
refactor: Use WebsocketClient fn run_in_background in binance gws
2025-04-29 19:35:15 +00:00
John Colvin
b4f5e66466
fix: Set okx perp exp to year 2100
...
Because the year 3000 is too big for signed int64
nanosecond timestamp
2025-04-29 19:35:15 +00:00
John Colvin
69a0bcc9cd
fix: Request binance fut md on ccapi dispatcher thread
2025-04-29 19:32:21 +00:00
John Colvin
d72ebdf421
perf: Send Binance Usds Futs md rest reqs on background thread
2025-04-29 19:32:21 +00:00
John Colvin
8e93856767
fix: Send exchangeInfo req via our HttpClient instead of ccapi
...
For binance usds futs (we already did this for spots)
Bugs (fixed):
* We get a boost beast exception about the read limit being exceeded
when requesting securities from Binance prod url (testnet works).
2025-04-29 19:32:21 +00:00
John Colvin
19dae6cd34
fix build (cpp release): Don't return reference to local var
2025-04-29 19:32:21 +00:00
John Colvin
24f42550dd
fix build (cpp release): Explicitly cast int64 -> uint64
2025-04-29 19:32:21 +00:00
John Colvin
c2150ae450
refactor: Move BinanceUsdsFuturesFilterTuple fns to cpp file
2025-04-29 19:32:21 +00:00
John Colvin
40dbfaa068
improvement: Support both key types in binance spots gw
...
Namely, ED25519 and HMAC-SHA256. Only the former can be used
to login over websocket, which eliminates the need to sign requests
after that.
2025-04-29 19:32:21 +00:00
John Colvin
3756a8a8b7
Add api key type enum to binance exe gateway config
...
ED25519 keys offer performance benefits over the HMAC-SHA256
keys when using the websocket api, as you can log in on the websocket
connection to avoid signing each request.
2025-04-29 19:32:21 +00:00
John Colvin
46c8dcb18f
fix: Add binance spot order newOrderResp type to json/query params
2025-04-29 19:32:21 +00:00
John Colvin
024480f32e
improvement: Send binance usds fut orders via websocket api
...
Unlike the previous implementation, which sent REST requests, this won't
block the dispatch thread waiting for a response.
2025-04-29 19:32:21 +00:00
John Colvin
ba525372bb
cleanup: Move a few types from orderman_utils to their own headers
...
So they can be included without depending on OrderService.pb.h
2025-04-29 19:24:41 +00:00
John Colvin
8aa0f1b674
refactor: Move binance state machine out of "spots" dir
...
So it can be used in the implementation for all binance apis (usds futs,
cross margin, etc.)
2025-04-29 05:24:04 +00:00
John Colvin
fc14215d9b
fix: Don't assert-crash if ord key isn't found for outgoing req
...
The i/o is asynchronous, so it's possible for the gateway to receive
a terminal ack from the exchange, remove the order from its index,
then receive a cancel/modify req from an algo that submitted the
req before it knew the order reached a terminal state.
2025-04-29 05:24:04 +00:00
John Colvin
091ed5a327
fix: Don't fire exe report for internal rejections
...
Internal rejections don't mean the order isn't "working". Only
rejections from the exchange mean that.
2025-04-29 05:24:04 +00:00
John Colvin
30bb9d0837
fix: Make SpotsOrderStore threadsafe
2025-04-29 05:24:04 +00:00
John Colvin
9cb2c580d6
diag: Use structured logging in binance spots gw
2025-04-29 05:24:04 +00:00
John Colvin
2e2150ac1a
refactor: Send binance spot orders through websocket api
...
This offers performance benefits over the REST api, since we don't have
to sign each request.
Note: I consolidated the UserDataSubscription code into SpotsGateway
because it was cleaner than trying to share a single websocket
connection between the two classes (UserDataSubscription and
SpotsGateway).
2025-04-29 05:24:04 +00:00
John Colvin
21d9435a67
Add run_in_background fn to WebsocketClient (cpp)
2025-04-29 05:24:04 +00:00
John Colvin
e7e334fb5a
Add continous ping function to WebsocketClient (cpp)
2025-04-28 17:12:59 +00:00
John Colvin
61f04c4605
fix: Implement state machine for binance cxl/replace
2025-04-28 17:12:59 +00:00
John Colvin
24fb2e1e39
refactor: Remove 'get_internal_key' delegate from exe report store
...
It made constructing the exe report store awkward, and smelled like poor
design.
2025-04-28 17:12:59 +00:00
John Colvin
ece141c448
Add dedicated types for structured logging
2025-04-28 17:12:59 +00:00
John Colvin
617bac8e74
cleanup: Move CancelOrderMsg impl to .cpp
2025-04-28 17:12:59 +00:00
John Colvin
ab217401eb
cleanup: Move CreateOrderMsg impl to .cpp, get rid of unused code
2025-04-28 17:12:58 +00:00
John Colvin
77e5c1481b
Add security_id/name fns to ExecutionReport for convenience
...
So callers don't need to check if security is a nullptr
2025-04-28 17:12:58 +00:00
John Colvin
a0f77f92a3
cleanup: Move ExecutionReport impl to .cpp
...
Also add using aliases to get rid of repetitive namespace prefixes
2025-04-28 17:12:58 +00:00
John Colvin
0acdcb150d
fix: Format price & qty w/ 6 decimals in binance modify order json
2025-04-28 17:12:58 +00:00
John Colvin
ac2f3b9bc6
Add ConcurrentQueue (c++)
2025-04-28 17:12:58 +00:00
John Colvin
0e522f90e5
refactor: Add line num suffix to var name in LOCK_GUARD macro
...
Rename to "BTS_LOCK_GUARD" while we're at it to avoid potential naming conflicts
2025-04-28 17:12:58 +00:00
John Colvin
40ad1f1b48
Update binance websocket endpoints
...
The api just changed, and old urls will be retired in May 2025
2025-04-28 17:12:58 +00:00
John Colvin
1cb75d399f
devcontainer: Update dotnet7 -> dotnet8
2025-04-28 17:12:58 +00:00
John Colvin
cddecc0543
cleanup: Fix benign typo in websocket_client.h
...
The open_handler fn return type should be "OpenHandler", not
"CloseHandler" (obviously).
2025-04-28 17:12:58 +00:00
John Colvin
484e36b330
refactor: Make timestamp internal value to signed (int64)
...
I am sick of casting uint64 -> int64. The latter is a much more
common argument type than the former.
2025-04-28 17:12:58 +00:00
John Colvin
8a9c22fa2a
vscode: Add Python Debugger entry to launch.json
2025-04-28 17:12:58 +00:00
John Colvin
a6e8853cb9
fix build (warns): Account for nullable ref types
2025-04-28 17:12:58 +00:00
John Colvin
dafa193283
fix build (windows): Update externals to fix catch include path
...
I couldn't get the build to work with the new catch2 on windows, though
it does work on linux. Rather than revert the catch -> catch2 include
path changes (again, which do build on linux), I decided to rename the
old windows catch include dir 'catch2'.
2025-04-28 17:12:58 +00:00
John Colvin
23651970c9
fix: Fix improper 'OfType' linq cast
...
Previously, animations were expressed through polymorphism (derived from the
base class CellAnimation), but the new impl uses composition. This 'OfType' call
was leftover from the polymorphism implementation, an attempt to linq cast
elements of base class (CellAnimation) to BackgroundAnimation.
2025-04-28 17:12:58 +00:00
John Colvin
043efab5a2
build: Upgrade c# to net8.0
2025-04-28 17:12:58 +00:00
John Colvin
5476466d55
Fetch secrets for config profiles with new aws sso config
2025-04-28 17:12:58 +00:00
John Colvin
54cb5a2ab2
tool: Add script to configure aws sso
2025-04-28 17:12:58 +00:00
John Colvin
29bf7d8d7e
deploy (fix): Update okx urls
...
Their aws domain has been retired.
2025-04-28 17:12:58 +00:00
John Colvin
855d2fd790
build: Update vcpkg
2025-04-28 17:12:58 +00:00
John Colvin
8a6a652882
fix build: Get rid of databento's internal nlohmann-json
...
** Migrate nlohmann-json over to vcpkg in the process **
Make it so databento is built with the same nlohmann-json lib that our
code is built with. Recently, I started getting undefined symbol errors
in databento code from nlohmann headers - presumably a conflict between
their nlohmann dep and ours.
2025-04-28 17:12:58 +00:00
John Colvin
3f288704bf
build (linux): Migrate openssl to vcpkg, update to 3.5.0
...
Databento, as of version 0.32.0, requires openssl >= 3.0
https://github.com/databento/databento-cpp/releases/tag/v0.32.0
2025-04-28 17:12:58 +00:00
John Colvin
98bc78463d
build (linux): Migrate catch2 dep to vcpkg (from externals)
2025-04-28 17:12:58 +00:00
John Colvin
6d271ad7cd
build (linux): Manage fmt dep with vcpkg
2025-04-28 17:12:58 +00:00
John Colvin
2a17d656a5
fix: Dump json to str in LOG statements
...
The nlohmann::json obj isn't formattable (i.e., doesn't have a
fmt::formatter<T> specialization)
2025-04-28 17:12:58 +00:00
John Colvin
10436f0f3b
tool: Rename binance_orders_client.sh -> binance_rest_client.sh
...
It supports md and order/trade requests
2025-04-28 17:12:58 +00:00
arham
86b19f0ef0
Add: okx exe treats all cancelations except ioc as rejections
2025-04-26 05:35:46 +00:00
Taha Afzal
502024dd10
Removing an overzealous assert in the spreader, which was being false if the spread_book was not received once
2025-04-23 04:40:16 +00:00
John Colvin
623012b0b0
deploy: Add "sparkd_symbols" image to Dockerfile
...
Normally, we bake spark config into the image, which is simpler and less
error prone than managing separate configurations per host machine
and/or container. The sparkd_symbols deployment breaks the mold. Config
is mounted to a dir on the host machine so symbols can be added/removed
via config (i.,e., spark.conf) when we bounce sparkd.
2025-04-17 05:53:05 +00:00
John Colvin
5859f0f2d1
deploy (fix): chown bts for whole sparkdata dir in sparkd-symbols
...
Unlike normal deploys, the config dir isn't copied directly into the image
2025-04-16 13:48:24 +00:00
John Colvin
eeffb735ae
deploy (fix): Override existing values when loading env files
2025-04-16 13:48:24 +00:00
John Colvin
b602997f3a
deploy: Build sparkd if any service starts with "sparkd"
...
Previously, we only built sparkd for the "sparkd" service (i.e., exact match)
2025-04-16 13:48:24 +00:00
Taha Afzal
3530a77b38
Fixing the situation where an IOC cancels, cancels the spreader
2025-04-16 04:26:50 +00:00
arham
afdf635b31
fix: changes spread_book_tick_sz from int to double to fix low tick being zero for reverse hedge orders
2025-04-11 07:31:03 +00:00
John Colvin
4d4aefc0a7
fix: Bind SpreaderType dropdown to all algo types in synth ladder
...
Bugs (fixed):
* The SpreaderType dropdown only works for plain spreader orders. It
doesn't work for icebergs, leans, etc.
2025-04-10 16:16:43 -04:00
arham
960a8af77a
Fix: OKX futures working now
2025-04-10 05:06:17 +00:00
Taha Afzal
89828659a4
Reducing loggging on reverse hedge evaluation
2025-04-09 20:05:48 +00:00
John Colvin
76f83fd7f5
Revert "Set irrelevant spreader params to null in ladder"
...
This reverts commit 924386084b .
2025-04-09 15:52:00 -04:00
Taha Afzal
d4e40948d2
Changing the logic to determine if a leg is a quoting leg or not
2025-04-09 19:29:50 +00:00
arham
912ab512d2
Add: ioc orders to okx exe
2025-04-09 19:00:24 +00:00
Taha Afzal
5d3eb92175
Spreader gets the spread ticks from the right source now (it was hard coded at 0.1 earlier)
2025-04-09 18:42:06 +00:00
John Colvin
924386084b
Set irrelevant spreader params to null in ladder
...
Depending on the SpreaderType enum value (treat it like a oneof case)
2025-04-09 10:29:15 -04:00
Taha Afzal
acab3ac392
Reversehedging fixed
2025-04-09 09:42:47 -04:00
arham
d06a62f6ee
Format: ran clang format on file
2025-04-09 09:42:47 -04:00
arham
14e3eee0bc
Add: config changes to make server controller work and set up hk zen server
2025-04-09 09:42:47 -04:00
Taha Afzal
7277bf52f5
Fixing build:
2025-04-09 09:42:47 -04:00
Taha Afzal
b81dc7f5cf
Fixing a bug, where the spreader was isidentifying the quoting leg
2025-04-09 09:42:47 -04:00
John Colvin
b1a4635345
ui: Enable IOC for Iceberg orders
...
Previously, it was only enabled for Limit orders.
2025-04-08 13:22:09 -04:00
John Colvin
7c7c689170
fix build: Don't cast to unsigned int when checking bytesRead < 0
2025-04-04 15:36:55 -04:00
John Colvin
116506c8da
fix build: Lock databento version to v0.31.0
...
Previously, we were simply fetching HEAD. This was fine until they
released v0.32.0 (on 04/02/2025), which introduced a new dependency
requirement that broke our build: OpenSSL >= 3.0.
2025-04-04 15:36:55 -04:00
Adeel Khan
3feda49ef1
Fix: Taifex Time Conversion
2025-04-04 15:36:54 -04:00
Adeel Khan
10318be7bc
FIX: Taifex Crash Fixed, Added Diag Logs
2025-04-04 15:36:54 -04:00
Adeel Khan
f7d525a527
ADD: TBBO & Symbol wise Naming in Recorder
2025-04-04 15:36:54 -04:00
Adeel Khan
d2fe9f167a
ADD: TWSE Code Complete
2025-04-04 15:36:54 -04:00
Adeel Khan
696d91a5af
ADD: TWSE to WebApp
2025-04-04 15:36:54 -04:00
Adeel Khan
d575ba0ea8
ADD: TWSE throughout Sparkd
2025-04-04 15:36:54 -04:00
Taha
058b131760
Cleaning up some streams,new prompt etc
2025-04-04 15:36:52 -04:00
John Colvin
57f8f3ff58
ui: Add Sniper and Reverse Hedge controls
2025-04-04 00:19:38 -04:00
John Colvin
775df2850d
Add Sniper and Reverse Hedge params to spreader proto
2025-04-04 00:19:38 -04:00
Taha Afzal
cb2b15f10b
Adding IOC to SimExecution
2025-04-03 23:24:30 +00:00
Taha Afzal
60362a4c95
Adding reverse-hedge to spreader
2025-04-03 23:24:30 +00:00
John Colvin
c6d7a8f38c
ui (pnl): Use dark theme fg in "Selected" and "Total" labels
...
Previously, it was black and hard to read since the background is dark.
2025-03-26 14:46:44 -04:00
John Colvin
7931c62b46
ui (pnl): Hide "Override Cpp" menu item
...
The overrides aren't respected in (new) pnl calcs.
2025-03-26 14:35:23 -04:00
John Colvin
58518451c8
ui (pnl): Show realized/unrealized pnl in row group totals
2025-03-26 14:31:04 -04:00
John Colvin
e3bad78aef
fix (pnl ui): Get cash per point from security
...
Previously, it was hardcoded to 1.
2025-03-24 16:54:37 -04:00
John Colvin
8e9ab8f2eb
fix: Don't apply cme cpp overrides to non-cme securities
...
Bugs (fixed):
* The cash per point for many securities is wrong. For example, it is
5 for all BTC securities (which is wrong).
Cause: cpp override config that should be specific to cme is applied
to ALL securities.
2025-03-24 20:20:53 +00:00
John Colvin
8512f414d6
fix: Don't duplicate animations when migrating pnl window settings
2025-03-24 14:53:09 -04:00
John Colvin
513fc353cf
ui (pnl): Increase default width for a few columns
...
I noticed some data was truncated when connected to Alphaworx prod
server.
2025-03-24 14:39:40 -04:00
John Colvin
4a04beafd2
cleanup: Consoildate order extension methods
2025-03-24 11:19:51 -04:00
John Colvin
79a3251f26
fix: Make security comparison deterministic again
2025-03-24 11:19:51 -04:00
John Colvin
8785e0fecd
fix: Bootstrap all observers w/ snapshot on first update
...
Previously, the Subscribe impl only bootstrapped the first observer
(i.e., only in first call to Subscribe).
Bugs (fixed):
* The ladder crashes if there are working orders when the
ticks-per-row setting is changed.
Cause: The ladder viewmodel resubscribes to order updates each time
the ticks per row setting changes (as this determines the size of
the price buckets into which orders are grouped). The IObservable
"Bootstrap" extension fn doesn't properly clean up state leftover
from the previous subcsription when this happens.
2025-03-24 11:17:04 -04:00
John Colvin
04251c96b6
ui (pnl): Compute pnl as sum of realized + unrealized
...
This addresses a subtle inconsistency in the pnl window.
The old "start_pnl" and "day_pnl" values (from IPNLUnit) are computed
with respect to the mid market for both long and short positions, while
the new "unrealized" pnl values are computed with respect to the side of
the market needed to close the position (i.e., bid market for long pos,
ask for short). Consequently, the combined pnl columns weren't equal to
the sum of realized and unrealized pnl.
2025-03-24 10:13:24 -04:00
John Colvin
c2d400af1a
fix (pnl ui): Don't overwrite menu display settings w/ defaults
2025-03-24 10:13:24 -04:00
John Colvin
133cc16628
ui: Add menu item to open mview from pnl
...
Populated with the same securities as the pnl window
2025-03-24 10:13:24 -04:00
John Colvin
16f7aede93
fix: Add case for Synthetic sec type in proto conversions
2025-03-24 10:13:24 -04:00
John Colvin
2b9488faff
ui: Make pnl default settings pretty
...
* Make profit (green) and loss (red) easier to read against dark bg
* Hide 'SOD' and 'Day' columns (just show totals)
2025-03-24 10:13:24 -04:00
John Colvin
51b6d592d3
fix (pnl ui): Negate short avg cost when translating to avg price
2025-03-23 15:17:41 -04:00
John Colvin
1eb4ceba0a
ui: Add columns for long/short avg pnl
...
When looking at avg cost, you often really want to know the difference
between it and the current market. These columns provide that information
so you don't need to do the arithmetic in your head.
2025-03-23 15:17:41 -04:00
John Colvin
dbf5cadb44
ui: Add columns for realized/unrealized pnl
2025-03-23 15:17:41 -04:00
John Colvin
90f1bb0851
refactor: Move PNL data types to spark_api csproj
2025-03-23 13:08:46 -04:00
John Colvin
547d83e1b8
cleanup: Remove unused pnl enum WaterfallPriceFlags
2025-03-23 13:08:41 -04:00
John Colvin
e888149c41
Add order msg throttle to sparkd config profiles
2025-03-20 17:29:43 +00:00
John Colvin
b944c3980f
Show 6 decimals in spreader order log
...
So we can better track those rascally iceberg orders w/ small fractional
quantities
2025-03-20 14:30:35 +00:00
Taha Afzal
c2a8266106
merging
2025-03-20 01:23:05 +00:00
Mohammad Arham
4277ba61b9
Fix: Okx execution only sends
2025-03-19 13:26:57 -04:00
Mohammad Arham
35cfe9533b
Add filter for okx er we that were caused by orders from outside the system
2025-03-19 13:26:39 -04:00
John Colvin
d49efef93d
Increase ram to 8gb in alphaworx_prod config
...
The previous value of 4gb wasn't enough, as the container would
occasionally crash with a SIGKILL interrupt.
2025-03-19 15:26:14 +00:00
John Colvin
502ed13456
fix: Round IcebergAlgo remaining quantity
...
Bugs (fixed, 2nd attempt):
* Iceberg orders w/ fractional quantities can get into a state where
they send an infinite stream of orders w/ 0 qty
2025-03-19 14:43:24 +00:00
John Colvin
716c4dd674
tool: Add script to dump sparkd logs to sqlite db
2025-03-19 14:41:23 +00:00
John Colvin
4a2b189a07
diag: Format Price and Quantity w/ up to 6 decimal places
2025-03-17 13:00:15 +00:00
John Colvin
a42edaf9fa
diag: Trivial fix for log msgs in SpreaderOrder
...
* Format qty with fractional part
* Remove redundant log
2025-03-17 13:00:15 +00:00
John Colvin
cbc142ab15
fix: Account for rounding error in IcebergAlgo fills
...
Bugs (fixed):
* Iceberg orders w/ fractional quantities can get into a state where
they send an infinite stream of orders w/ 0 qty
Note: I also changed the number of decimal places used when rounding
order quantities from 6 to 7 (the level of precision guarenteed by
IEEE 754).
2025-03-17 13:00:15 +00:00
John Colvin
3980f2d030
Add order msg throttle to alphaworx_prod config
2025-03-17 13:00:15 +00:00
John Colvin
00f6d269c9
fix: Set binance routing record "operator_id" to username (not account)
...
Must have been a typo
2025-03-17 00:51:54 +00:00
John Colvin
10b2d38de8
tool: Add cmd to tail sparkd log file using docker compose
2025-03-17 00:51:54 +00:00
John Colvin
6d4d8f6ac1
fix: Disable ccapi order book cross check for Binance md
...
When enabled, the checks fail regularly (usually more than once per
hour). Sometimes many of them happen in rapid succession before sparkd
crashes. I'm disabling them in the hopes that the crosses are benign,
and this will stop the crashes.
2025-03-17 00:51:54 +00:00
John Colvin
5db89e1e8d
diag: Tone down noisy order management logs
...
* EMNativeOrder
* OrderRouter - only log routing record result if there are errors
* OrderServiceServer
* OrderStore - also avoid calling pb::to_string twice per change (it's
expensive)
* ServiceBridge
2025-03-17 00:51:47 +00:00
John Colvin
379a46afa2
Add order msg throttle to sim_dev config
2025-03-15 20:07:14 +00:00
John Colvin
d9eff37184
fix: Don't let rounding error accumulate in mview order summarizer
...
Bugs (fixed):
* Sometimes mview shows that a security has working orders when it
doesn't. It appears to be rounding-related as the price and qty of
the phony working orders is NaN and 0, respectively.
Cause: Rounding error accumulates in the order summarizer. If it gets
bad enough, the working quantity (which is computed incrementally as
orders transition to/from a working state) may still be non-zero after
all orders become non-working.
2025-03-15 15:21:56 -04:00
John Colvin
3d055de139
cleanup: Rename _tick_factors -> _qty_tick in mview order summarizer
...
It is specifically for quantity (not price).
2025-03-15 15:21:56 -04:00
John Colvin
648be7dd97
improvement: Don't resort orderbook on every order update
...
This causes performace issues when the orderbook is sufficiently large
(noticeable when around 20k orders).
2025-03-15 15:21:56 -04:00
John Colvin
2cac9affe4
fix: Use consistent price comp in ladder order stream
...
Bugs (fixed):
* The ladder occasionally crashes when orders are changing.
Cause: Subtle differences between the ordinal and equality comparers
made it possible for 2 prices to be considered equal by the former,
and not equal by the latter. Since the ladder indexes orders by
price, a "Key not found" exception could be thrown if the equality
comparer detected a price change that the ordinal comparer did not.
2025-03-15 15:21:55 -04:00
John Colvin
30ff3ba707
Add some null checks in LadderGridHost
2025-03-15 15:21:55 -04:00
John Colvin
9d3afd68aa
ui: Format Perp and Spot exps as "Perp" and "None"
2025-03-15 15:21:55 -04:00
John Colvin
8c3de1789b
fix: Apply leg price mults when computing synth book
...
This is necessary to support synths with BTC-USDC-SWAP legs at Okx
(because 1 "contract" == 0.01 BTC).
Bugs (fixed):
* Price multiplier has no effect in synthetic book
2025-03-14 19:32:04 +00:00
John Colvin
b251e1d47e
build: Generate semantic version header from git
...
Dump it to console and log file when sparkd runs
2025-03-14 16:00:06 +00:00
arham
6ccd48a8ba
Fix, 1e-5 orders working okx
2025-03-13 17:49:14 +00:00
John Colvin
41d84f7f73
Change externals submodule url from ssh to https
2025-03-13 17:08:04 +00:00
John Colvin
795062864d
fix: Fix typo in permission filter comparison (change == to !=)
...
Bugs (fixed):
* The orderbook and ladder blink/flicker when orders change if logged
in as non-bts user.
Cause: The permission filter acted as if the predicate itself
changed everytime an order changed. As a result, it would fire one
update to cleared all values, then fire another to add items that
pass the "new" predicate.
2025-03-13 10:39:33 -04:00
John Colvin
1ff72e9444
fix (ui): Show fractional order qty in activity log
...
Also fix formatting - change "price @ qty" to "qty @ price"
2025-03-13 10:39:33 -04:00
John Colvin
e13b5066a3
fix: Preserve fractional qty when modifying existing order in ticket
2025-03-13 10:39:33 -04:00
Mohammad Arham
5ba7901258
Fix: undo special case for btc swap and add ccp for swaps
2025-03-12 20:47:44 +00:00
Mohammad Arham
bc6eb59110
Fix: Change cttype from optional to string
2025-03-12 20:45:27 +00:00
John Colvin
5ae5a5dcbf
fix: Add switch case for PositionSide::BOTH in binance futs gw
...
In this case, the position is signed (unlike LONG and SHORT).
2025-03-12 14:55:56 +00:00
John Colvin
7966bcce77
fix: Change trade id from uint32 to int64
2025-03-12 04:27:54 +00:00
Mohammad Arham
ff86b10895
Fix: temp fix to move okx btc swap to config for min tick
2025-03-11 20:35:24 +00:00
John Colvin
ea3c0717d8
fix: Remove erroneous assert from binance usds futs gateway
2025-03-11 19:48:11 +00:00
John Colvin
d0abca3fd9
Fix symbols and account alphaworx_prod config
2025-03-11 19:47:34 +00:00
John Colvin
7895fe7357
Remove taifex and glbx from NotSimExecution in conf profiles
2025-03-11 15:24:08 +00:00
Mohammad Arham
6f5f4f5cc2
Fix: okx excution working now
2025-03-11 14:25:42 +00:00
John Colvin
066a0ba693
ui: Use dark theme foreground in mini orderbook, fills popups
...
It was black in a couple places before, making it hard to read.
2025-03-10 14:47:58 -04:00
John Colvin
c2b67b4813
fix: Check for null in orderbook root algo type filter
2025-03-10 14:47:58 -04:00
John Colvin
ecad456a2d
fix: Apply exp filter to popup grids in pnl window
2025-03-10 14:47:58 -04:00
Taha Afzal
2cb3cfdcdd
Spreader's orders were hardcoded to be DAY, now it gets it from the parent
2025-03-10 17:22:08 +00:00
Taha Afzal
0ebc8e5b4b
Orders in DB are deleted on server bounce
2025-03-10 15:42:12 +00:00
John Colvin
2072105c0f
cleanup: Add using aliases to make code more concise
2025-03-10 14:53:07 +00:00
John Colvin
d663c86aa0
improvement: Only load synth metadata from disk once in md dumper
...
Instead of every time a synth is dumped (i.e., do it lazily, outside the
loop body)
2025-03-10 14:53:07 +00:00
John Colvin
4a2dfa46a1
improvement: Print delimiter line when dumping secs to console
2025-03-10 14:53:07 +00:00
John Colvin
6e35db6a24
improvement: Apply filter to shared mem secs in md dumper
...
Previously, filter only applied to db securities (i.e., when --db flag
was used)
2025-03-10 14:52:39 +00:00
John Colvin
b45dc3bc2d
fix: Preserve create time when order is updated
...
Bugs (fixed):
* Order create time mirrors last update time (it should never change).
2025-03-10 05:24:49 +00:00
John Colvin
a1912907b8
Add brief readmes for sparkd config profiles
2025-03-10 05:24:49 +00:00
John Colvin
b1b3dcdeb8
Add sim_dev config profile for sparkd
2025-03-10 05:24:49 +00:00
John Colvin
03dc8ef767
Add boto3 to py venv, so we can retrieve secrets from aws
2025-03-10 05:24:49 +00:00
John Colvin
975bf56c2a
fix: Don't spam binance w/ listen key requests
...
Sleep for a few seconds before trying again on failure.
2025-03-10 05:24:49 +00:00
John Colvin
3f58c2ce84
Add script to install aws cli
...
The aws cli can be used to configure credentials in the shell in order
to retrieve secrets (e.g., production api keys for alphaworx, etc.).
2025-03-10 05:24:49 +00:00
John Colvin
96f28acf91
Add dev config profile for sparkd
2025-03-10 05:24:49 +00:00
John Colvin
c2c2658e21
Add alphaworx_prod config profile for sparkd
2025-03-10 05:24:49 +00:00
John Colvin
d6b8e359cd
Expand environment vars in binance gw config
2025-03-10 03:05:59 +00:00
John Colvin
84a0850d95
Tweak binance_dev config profile
...
* Factor out repeated symbol list into SymbolLists entry
* Change "binance_dev" account to "BINANCE_DEV" in exe gw json
* Adjust log level
* Remove unused config:
- ExchangeSubAccounts
- JetFIX_Connection.max_order_msgs_per_window
2025-03-10 03:05:59 +00:00
John Colvin
4effdc4160
diag: Log when http buffers are resized in binance exe gws
2025-03-10 03:05:59 +00:00
John Colvin
b9fe39f6e3
refactor: Make MDRecorder join the run thread in destructor
...
This wasn't broken, but detached threads bother me.
2025-03-10 03:05:59 +00:00
John Colvin
b555850015
build: Package & deploy sparkd with docker compose
2025-03-10 03:05:59 +00:00
John Colvin
23794fa30d
devcontainer: Add docker in docker
...
Also remove default port bindings in devcontainer (so the "docker in
docker" sparkd can use them). It's easy enough to manually bind ports
in vs code.
2025-03-08 05:20:08 +00:00
John Colvin
90b5a309a3
cleanup: Make a couple fns const
...
Probably a typo, since these fns don't mutate anything.
2025-03-08 05:20:08 +00:00
John Colvin
07beda3fc6
fix: Support fractional show qty in iceberg orders
2025-03-07 20:42:24 +00:00
Taha Afzal
e3c1ea70f2
Spreader orders can now be entered in fractional qty
2025-03-07 05:24:39 +00:00
Mohammad Arham
93b93969f5
Add: config to control databento md better
2025-03-06 02:44:40 +00:00
Mohammad Arham
0a10111846
Small changes to make cboe md work
2025-03-06 02:44:07 +00:00
John Colvin
eee00cc407
diag: Integrate websocketpp with our logging system
...
* Unifies log message structure for improved readability
* Grants us runtime control over log levels via config
2025-03-05 20:48:14 +00:00
John Colvin
0eb58eaba8
fix (preemptive): Pass websocket io_context to PImpl ctr
...
The 'io_context' member var is accessed in the constructor (it must be
initialized before that happens).
2025-03-05 20:42:50 +00:00
John Colvin
12f6733d1e
cleanup: Remove unnecessary account mapping in binance_dev conf
2025-03-05 18:04:04 +00:00
John Colvin
93a0984ed1
Filter out non usdc/usdt spots in binance_dev conf
2025-03-05 18:04:04 +00:00
John Colvin
b7e9c393a0
fix: Don't sign binance listen key requests
...
The spots and usds futures apis ignore the unnecessary signature params,
but the cross margin api responds with an error.
2025-03-05 18:04:04 +00:00
John Colvin
aa602b3fd6
tool: Add cross margin reqs to binance.env
2025-03-05 18:04:04 +00:00
John Colvin
b4b8cabdae
tool: Add listen key reqs to binance.env
2025-03-05 18:04:04 +00:00
John Colvin
cfe06869b3
tool: Support empty query params in binance_orders_client.sh
2025-03-05 18:04:04 +00:00
John Colvin
b177ed1cc0
fix: Include listenKey param in listen key refresh req (binance)
...
The cross margin api requires this. Spots and futures apis don't, but
including it for them is harmless. I also changed the response json impl
to treat the 'listenKey' field as optional (it's omitted in the cross
margin api).
2025-03-05 18:04:04 +00:00
John Colvin
ff17a64933
cleanup: Remove unnecessary includes
2025-03-05 18:04:04 +00:00
John Colvin
81a1fe11f6
feat: Add binance gw for cross margin orders
...
Spots only
2025-03-05 18:04:04 +00:00
John Colvin
ad5bf88fcf
refactor: Rename user_data_streams dir, ns without plural 's'
...
There is one user data stream for Binance Spots, and another for Usds
Futures. We model both of these user data streams in separate folders,
so each "user_data_stream" dir represents a single stream.
2025-03-05 18:04:04 +00:00
John Colvin
e15f155ac9
refactor: Abstract out common binance gw code
...
For all sub-venues to use
2025-03-05 18:04:04 +00:00
John Colvin
6d6822cd65
refactor: Rename binance exe gw filenames so they are unique
...
The debugger gets confused when a particular lib contains multiple
source files with the same name (in different dirs).
2025-03-05 18:04:04 +00:00
John Colvin
ace0c70516
fix: Trim account name string when converting to Fill proto
2025-03-05 18:04:04 +00:00
John Colvin
e9e5059bc2
fix: Disable CURLOPT_FAILONERROR to always get response body
...
Even when the http error code indicates failure. The payload may contain useful
information about the error.
2025-03-05 18:04:04 +00:00
John Colvin
cde72f51a0
Improve logging/error reporting in binance execution gw
...
In the process, consolidate binance exe report stores into single class.
2025-03-05 18:04:04 +00:00
John Colvin
e1fd49ae32
cleanup: Specify bts account directly in binance gw config
...
This eliminates the need for an ExchangeSubAccountResolver.
2025-03-05 16:00:43 +00:00
John Colvin
274f3f98e4
cleanup: Replace includes w/ fwd declares in header
2025-03-05 16:00:43 +00:00
John Colvin
a4b7d84318
fix: Reject orders w/ invalid TimeInForce at binance
...
Do this internally (before sending them to the exchange)
2025-03-05 16:00:43 +00:00
John Colvin
be17205094
fix: Report internal reject msgs to ExecutionReport
...
For Binance execution gateways (spots and usds futures)
2025-03-05 16:00:43 +00:00
John Colvin
6ef9bee8e4
Add reconnect behavior to WebsocketClient
2025-03-05 16:00:43 +00:00
John Colvin
2c8b3563f1
cleanup: Perform basic init in WebsocketClient ctr
2025-03-05 16:00:43 +00:00
John Colvin
71f7e64ccb
Move config profiles dir to server/sparkd
2025-03-05 16:00:43 +00:00
John Colvin
c31b44ecfb
diag: Log when config server starts and show url
2025-03-03 19:32:16 +00:00
John Colvin
19efb77e07
tool: Add some usds futures requests to binance.env
2025-02-24 15:12:45 +00:00
John Colvin
76de640421
diag: Show Binance sub-venue in log message prefix
...
Otherwise it's hard to tell which Binance venue the log is about (e.g.,
spots, usds futures, etc.).
2025-02-24 06:16:22 +00:00
John Colvin
fb030742d3
Clean up binance_orders_client.sh a bit, add docs
2025-02-24 06:06:15 +00:00
John Colvin
73608f1b45
perf: Run each ccapi md session on its own thread
2025-02-24 06:04:52 +00:00
John Colvin
58aebe7002
diag: Reduce log noise in CCApiMarketDepthAdaptor
2025-02-24 06:04:52 +00:00
John Colvin
299cb0b29e
diag: Log when curl_global_init/cleanup is called
...
It should only happen once for the whole app (so it shouldn't be noisy).
2025-02-23 20:08:52 +00:00
John Colvin
06663f711e
improvement: Only call curl_global_init/cleanup once
...
Avoids unnecessary overhead (plus their docs say it only needs to be
done once for the whole app). These fns have "global" in their name for
a reason.
https://curl.se/libcurl/c/curl_global_init.html
2025-02-23 20:08:52 +00:00
John Colvin
51479f4885
fix: Disable signal handling in curl
...
This avoids a strange stack corruption bug related to set/longjmp.
https://stackoverflow.com/questions/9191668/error-longjmp-causes-uninitialized-stack-frame
2025-02-23 20:08:50 +00:00
John Colvin
5b520ba241
cleanup: Get rid of custom fmt::formatter for security proto
...
Security proto objects are large, so we shouldn't pass them by value to
the log system. The custom formatter's presence makes this easy to do by
accident.
2025-02-23 19:05:42 +00:00
John Colvin
f0651a5341
tool: Add flag to roll logs without running sparkd
2025-02-23 15:36:10 +00:00
John Colvin
897e0990a2
diag: Improve logging in SecurityCache
2025-02-23 15:36:10 +00:00
John Colvin
30e1ae5c3c
Add Binance execution to config profile
2025-02-23 15:36:10 +00:00
John Colvin
efe83f81ec
fix: Index live securities (as well as db ones) in SecurityCache
2025-02-23 15:36:10 +00:00
John Colvin
5fb21ada97
diag: Add sec id to PositionMap log
2025-02-23 15:36:10 +00:00
John Colvin
58edd07f6b
fix: Check for empty string in strn fns
2025-02-23 15:36:10 +00:00
John Colvin
9d81354c3a
Add binance_orders_client.sh for quick/easy troubleshooting
2025-02-23 15:36:10 +00:00
John Colvin
0f37be89a4
Fix typos in binance.env
2025-02-23 15:36:10 +00:00
John Colvin
f6586fe636
Activate ccapi exchanges according to md config at runtime
...
Previously, the list of exchanges ccapi would activate was determined at
build time.
2025-02-23 15:36:08 +00:00
John Colvin
7827ccc0e7
Update externals to get ccapi v6.12.0
2025-02-23 15:35:35 +00:00
John Colvin
af6a2acc0a
perf: Send Binance md rest reqs on background thread
2025-02-23 06:10:36 +00:00
John Colvin
84abf59a55
feat: Add binance spots execution
2025-02-23 06:10:36 +00:00
John Colvin
2fc1d30bf1
Add binance_usds_futures_testnet config profile
...
To activate this config profile, run:
tools/sparkd_profile.sh binance_usds_futures_testnet
2025-02-23 06:10:36 +00:00
John Colvin
4dcd5d63a1
tool: Add script to easily switch between spark.conf profiles
2025-02-23 06:10:36 +00:00
John Colvin
63e5697b20
Unify conf and conf_json dirs
...
Put all files (.conf and .json) in sparkdata/conf. This allows us to
implement config profiles via symlink.
2025-02-23 06:10:36 +00:00
John Colvin
8e5cb100a7
fix: Assign BTS account to ExecutionReport
...
Not external exch account
2025-02-23 06:10:36 +00:00
John Colvin
938bee1817
Validate ExecutionReport before invoking callback
2025-02-23 06:10:36 +00:00
John Colvin
d48af15480
diag: Log warning if ExecutionReport acct not same as order def
2025-02-23 06:10:36 +00:00
John Colvin
e3b2e98cd9
fix build: Add JsonReader.cpp to client & api server builds
2025-02-20 00:55:48 -05:00
John Colvin
ee53fb854c
ui: Make buy/sell button bg darker in order ticket
...
The buttons looked washed out; too bright, yet pale.
2025-02-20 00:49:59 -05:00
Taha
d36e2d61f6
Fixing PNL for both crypto and non crypto
2025-02-19 19:06:12 -05:00
John Colvin
bfff3ea5fe
fix (preemptive): Check for null sec in ExecutionReport::to_string
...
I don't think it's ever a nullptr in practice, but if that changes, this
fn shouldn't seg fault.
2025-02-19 23:14:18 +00:00
John Colvin
059c03d289
cleanup: Removing docker submodules again
...
...they're back. It's like Night of the Living Dead submodules!
2025-02-19 22:50:29 +00:00
John Colvin
8c561f7886
improvement: Pass account/order updates by move (not value)
2025-02-19 22:50:29 +00:00
John Colvin
288567d1eb
cleanup: Pass JetFIX cfg directly to binance usds futs exe gw
2025-02-19 22:50:29 +00:00
John Colvin
ecfc5405fe
refactor: Move macro defined json fns to cpp file
...
Some fns are big (shouldn't be inlined); plus its best to avoid exposing
macros in headers.
2025-02-19 22:50:29 +00:00
John Colvin
8db34a2a11
Add json & fmt formatters for order & sec protos
2025-02-19 22:50:29 +00:00
John Colvin
91094c4d84
refactor: Make parse_uint support generic char iterator
2025-02-19 22:50:29 +00:00
John Colvin
dae8c85d73
fix: Check strlen when making order/fill keys
...
Although the string data is stored in a fixed size array, it isn't
necessarily the full length.
2025-02-19 22:50:29 +00:00
John Colvin
abdd98082f
Move BrokerProxy do_work fn to cpp
...
Doesn't make sense to inline; it's too big.
2025-02-19 22:50:29 +00:00
John Colvin
2f6aa785ab
Use object_pool wrapper to count allocations and assert if too many
2025-02-19 22:50:29 +00:00
John Colvin
7acf1f7043
diag: Improve logging in PositionMap
2025-02-19 22:49:30 +00:00
John Colvin
1c3c7a078a
diag: Improve logging in binance usds futures exe gateway
...
Also propagate error msgs through the execution report (so we can see
them in the front end).
2025-02-19 22:49:30 +00:00
John Colvin
93ace52bfa
diag: Periodically log ActorThread work cycle speed
...
So we can better detect slowness, blockage, etc.
2025-02-19 22:49:30 +00:00
John Colvin
b5d97c0b1b
refactor: Consolidate ExecutionGatewayProvider/Factory code
...
It's easier to read and manage code when you're not constantly jumping
to and fro across a bunch of files.
2025-02-19 22:49:30 +00:00
John Colvin
284aa53ad4
Add script to run sparkd and roll over log w/ timestamp suffix
...
So each run gets its own log file
2025-02-19 22:49:30 +00:00
John Colvin
8a9952f52d
diag: Improve logging in ServiceBridge
2025-02-19 22:49:30 +00:00
John Colvin
748aafaab3
cleanup: Remove unnecessary includes
2025-02-19 22:49:30 +00:00
John Colvin
3ba30c70de
cleanup: Normalize indentation in orderman_cfg.conf
...
It was all over the place before; unreadable
2025-02-19 22:49:30 +00:00
John Colvin
e72f465999
diag: Improve logging in OrderManager
2025-02-19 22:49:30 +00:00
John Colvin
f3d93f7749
fix leak: Replace object pool raw ptrs w/ shared ptrs
2025-02-19 22:49:30 +00:00
Mohammad Arham
8d142b8a74
Fix: forgot to push a file
2025-02-19 03:36:14 +00:00
Mohammad Arham
8210b9f35f
Fix: change onix exe so it uses dispatch fix msgs
2025-02-19 03:33:00 +00:00
Mohammad Arham
12b20b05ce
Fix: OKX executions sends er only when dispatch called
2025-02-18 16:02:10 +00:00
Mohammad Arham
72efbccdbd
Fix: USDC swaps md working for okx
2025-02-14 23:02:49 +00:00
Mohammad Arham
33643e9eab
Fix: make log resize work again
2025-02-14 22:59:35 +00:00
Mohammad Arham
0f6dc6f40b
Fix: Typo in Okx execution
2025-02-13 23:38:27 +00:00
Mohammad Arham
8c778f5e41
Fix: Added tdmode to config for okx execution
2025-02-13 23:32:53 +00:00
Mohammad Arham
f308654200
Fix: okx execution fix large spreader orderd
2025-02-13 23:27:32 +00:00
Mohammad Arham
bdf5b980fc
Add: config for sim trading
2025-02-13 22:06:23 +00:00
Mohammad Arham
914ac597cd
Vesrsion: Update to v2.2.5 deploying to jame prod
2025-02-13 14:40:27 +00:00
Mohammad Arham
4aa6a77680
Fix: OKX execution works as long as spreader not to big
2025-02-13 06:09:43 +00:00
Mohammad Arham
6b59dd84ee
Fix: Ordermanager will not try to remake orders from er after restart
2025-02-13 06:09:01 +00:00
Mohammad Arham
4e2c5f2d17
Fix: Okx USDT perps fix re added
2025-02-13 06:08:27 +00:00
Mohammad Arham
9a1b94a1b0
Add: fill id to okx execution report
2025-02-13 01:13:50 +00:00
Mohammad Arham
13b3c55771
Change: lst msg only entered if error
2025-02-12 23:13:59 +00:00
Mohammad Arham
d0a6fcdb9c
Add: Stricter Assert if websocket goes down okx execution
2025-02-12 22:23:53 +00:00
John Colvin
1eb13028c1
ui: Get qty display settings from md instead of config
2025-02-12 15:59:51 -05:00
John Colvin
424e5ff16b
Get Spark version at runtime via reflection
...
So it doesn't need to be hardcoded per release
2025-02-12 15:59:51 -05:00
John Colvin
bed389885b
Support qty tick size 10e-9
...
Some spots at OKX need this (e.g., BTC-AED)
2025-02-12 20:42:28 +00:00
John Colvin
85cf25a57a
fix: Get min qty and min increment from correct fields in okx json
...
"lotSz" represents min qty, and "minSz" represents min qty increment. We
had them backwards before.
2025-02-12 20:36:40 +00:00
John Colvin
61471d4675
diag: Show qty info fields in FixedSecurity json
2025-02-12 20:34:48 +00:00
John Colvin
d83435d49d
Move FixedSecurity to_json fn to cpp
...
Doesn't make sense to inline; it's too big.
2025-02-12 20:34:47 +00:00
John Colvin
582fc10dce
Add get_binance_md.sh for quick/easy troubleshooting
2025-02-12 20:34:47 +00:00
John Colvin
5969985d24
cleanup: Remove unnecessary includes
2025-02-12 20:34:47 +00:00
John Colvin
fabe7d665c
devcontainer: Install clang-format extension
2025-02-12 20:34:47 +00:00
John Colvin
7ce75bb8e7
devcontainer: Increase ram and shared mem
...
In order to support large symbol sets for Alphaworx
2025-02-12 20:34:47 +00:00
Mohammad Arham
b8e1c3cd60
Add: Fix build Issue
2025-02-12 19:48:52 +00:00
Taha
4909944694
Fixing PNL for perps (its base don mid market now)
2025-02-12 11:20:55 -05:00
Mohammad Arham
7258af09b4
Add: Basic Okx execution working
2025-02-12 11:55:50 +00:00
Mohammad Arham
7f2ce6c1a9
Add: Okx execution cfg
2025-02-12 11:06:45 +00:00
Mohammad Arham
4835725e1c
Add: diag logging to gttp client
2025-02-12 11:06:09 +00:00
Mohammad Arham
6a9e8de475
Deployment: Updating version numbers
2025-02-11 05:06:21 +00:00
Taha Afzal
4639951acc
Fixing Sprader's reportin of its filled qty
2025-02-11 03:04:30 +00:00
Taha
d8de211aac
Front end does NOT filter nay CBOE or CME expirations
2025-02-10 17:49:14 -05:00
John Colvin
87394bc60b
ui: Increase Currency col width so filter icon isn't clipped
2025-02-10 15:31:31 -05:00
John Colvin
beba30f65c
fix (ui): Remove temp hack that assumed synth qty always notional
2025-02-10 15:31:31 -05:00
John Colvin
2d58424d8f
ui (mview): Hook up currency filter in vm
...
Make it a discrete values filter (listbox) while we're at it. This seems
more practical than regex.
2025-02-10 15:31:31 -05:00
John Colvin
d1bbfa3791
Revert batch subscribe change in Binance
2025-02-10 19:59:42 +00:00
John Colvin
aab17f460f
cleanup: Remove docker submodules again
...
They just keep coming back.
2025-02-10 19:59:32 +00:00
John Colvin
461ef53322
devcontainer: Clean up and simplify
...
* Remove mounted dirs (the repo is automatically persisted in a volume)
* Remove user specific shell env initialization scripts (they're more
trouble than they're worth, often break the devcontainer build)
* Remove redundant settings already defined in settings.json
2025-02-10 16:55:53 +00:00
Taha Afzal
2486bc90b2
Spreader is hooked to the priceCal dropdown while creating the spreader
2025-02-10 03:10:11 +00:00
John Colvin
1c5cbaf60b
Change: Update version to 2.2.4-1
2025-02-08 13:19:41 -05:00
John Colvin
b8de447cfb
fix (pnl ui): Assume rows w/ exp DateTime.MinValue are Spots
...
Not the most elegant fix, but it'll do for now.
Bugs (fixed):
* The Pnl window groups perpetual futures with spots
2025-02-08 13:18:27 -05:00
Mohammad Arham
8bfc2d40f9
Update: V2.2.4 pnl working now
2025-02-08 12:43:13 -05:00
Taha
c09fd71c94
Fixing PNL for spots
2025-02-08 06:55:07 -05:00
Mohammad Arham
f3b037553e
Add: Batch subs in binance sub
2025-02-08 06:44:10 +00:00
Taha Afzal
d386d93166
SpreaderChildOrder::on_order_udpate, changing an asserrt to an error lgo to monitor condition
2025-02-07 15:06:16 +00:00
Mohammad Arham
5217742c71
Add: SendingTime tag to cboe execution
2025-02-07 12:26:11 +00:00
Taha
c59ec95460
Sheets and spreadbox theos can now optionally be set to market mid
2025-02-06 17:42:12 -05:00
Mohammad Arham
70f01583ec
ADD: CBOE TIME and SALES working
2025-02-06 20:42:58 +00:00
Mohammad Arham
66d2f9eb63
Add: Quote algo updates, theos removed
2025-02-06 03:08:46 +00:00
John Colvin
4625bed8b8
Change: Update version to v2.2.3-1
2025-02-05 17:01:11 -05:00
John Colvin
f18fb332b5
fix (orderbook): Make ColumnKeys enum 64bit
...
We rely on bit flags to indicate "dirty" columns in the vm. Need to do
this now that we have > 31 columns.
2025-02-05 17:01:11 -05:00
John Colvin
8746a6f5ca
fix (orderbook): Refresh Notional Value view when it changes in vm
2025-02-05 16:30:13 -05:00
John Colvin
702d47abb5
fix (orderbook): Fix typos in default settings for Notional Value
2025-02-05 16:29:29 -05:00
John Colvin
bf2a54717c
ui (orderbook): Add Filled Notional Value column
2025-02-05 16:13:34 -05:00
John Colvin
9db779fb25
fix (hack): Assume synth quantities are always notional values
2025-02-05 16:07:07 -05:00
Mohammad Arham
f55ddec708
Add: Vesrion logs to bothserver and client
2025-02-05 09:50:33 -05:00
Mohammad Arham
6e942dd2d4
Change: Updated version to v2.2.3
2025-02-05 08:38:31 -05:00
Mohammad Arham
142310a17a
Add: SPXW to default for cboe
2025-02-05 12:16:47 +00:00
John Colvin
7f99a89a95
ui (fills): Add notional value column
2025-02-05 00:00:41 -05:00
John Colvin
bfce663b4b
fix (orderbook): Hook up notional val filter in vm, persist settings
2025-02-04 23:42:58 -05:00
John Colvin
33c743643d
fix (orderbook): Include fractional qty in notional val
2025-02-04 23:34:47 -05:00
Taha Afzal
858e0a389b
Fixed the issue where a notioanal quoter would send a 0 qty order
2025-02-05 04:16:53 +00:00
Taha Afzal
9c3bb85674
Notional qty handling in spreader
2025-02-05 04:16:53 +00:00
Taha Afzal
f4930e8cd4
Added notional to PriceCalc type for the sprader, added code to compute notional books in SpreaderUtils
2025-02-05 04:16:53 +00:00
John Colvin
f21e7f3602
ui (orderbook): Show synth notional value in terms of first leg
2025-02-04 16:54:37 -05:00
John Colvin
b56c35aca4
ui (ladder): Show synth notional value in terms of first leg
2025-02-04 16:54:37 -05:00
John Colvin
e6ac83196e
fix (benign): Actually dispose _vm_sub in LadderGridHost
2025-02-04 16:54:37 -05:00
John Colvin
7b6f74fc4a
cleanup: Get rid of 'mimic tt' ladder config
...
This config was leftover from custom DJ work. We no longer need it.
2025-02-04 16:54:37 -05:00
John Colvin
f754953fac
cleanup: Remove duplicate equality check
2025-02-04 16:54:37 -05:00
Taha
2dd1143d9f
Fixed spx spreadbox theos
2025-02-04 16:31:17 -05:00
Mohammad Arham
e1ee3275d8
Add: Filter secs upto n months databento
2025-01-31 21:46:11 +00:00
John Colvin
2549e406f3
fix: Compute notional value correctly when ticks per row > 1
...
It's the sum of price * qty for each level in the row. Previously, we
were multiplying the total qty by the same price (smallest in the row).
2025-01-29 16:28:28 -05:00
John Colvin
d5024f1858
fix build: Don't check if unsigned int < 0
2025-01-28 20:20:06 +00:00
John Colvin
3623ac96db
fix: Multiply array size by sizeof(T)
2025-01-28 20:19:51 +00:00
John Colvin
19834625f9
fix: Don't fall through switch case
2025-01-28 19:59:32 +00:00
John Colvin
c8fc649b6b
fix build: Pass config arg to cmake --build
...
For Ninja Multi-Config builds (i.e., api server and data streams)
2025-01-28 19:59:09 +00:00
John Colvin
6b1fc1c413
cleanup: Remove unused code
2025-01-28 19:59:09 +00:00
John Colvin
6f086532ac
cleanup: Convert co2::TradeMessage in DomainSecurityUtils
...
Instead of domain::TradeRecord constructor. The domain types should have
minimal dependency on protos.
2025-01-28 19:59:09 +00:00
John Colvin
dc66282225
cleanup: Rename UINT32 -> VALUE in qty macro names
...
Should have done this when we changed qty primitive from uint32 to float
2025-01-28 19:59:09 +00:00
John Colvin
e2c74c0bb3
cleanup: Don't redefine CONTAINS macro
2025-01-28 19:59:09 +00:00
John Colvin
e89d50eec1
cleanup: Move large fns from header to cpp
2025-01-28 19:59:08 +00:00
John Colvin
8bebbc278d
cleanup: Get rid of 'using namespaces'
...
In headers, this is an objectively bad practice, since the type aliases
leak into dependent code and can cause unexpected naming conflicts. In
source files, it is debatable, but I prefer explicit aliases of the form
'using Foo = Bar'. The intentions are clear, and the resulting code is
naturally resistent to naming conflicts.
2025-01-28 19:59:08 +00:00
John Colvin
2e6d12c9c1
cleanup: Remove unnecessary includes
2025-01-28 19:59:08 +00:00
John Colvin
102a5e23ea
vscode: Add Tests Sparkd entries to tasks.json
2025-01-28 19:59:08 +00:00
John Colvin
582dc791db
vscode: Add Run Sparkd (Skip Build) entry to tasks.json
2025-01-28 19:59:08 +00:00
John Colvin
afb35d8c25
Compute synth book qty in terms of first leg
2025-01-28 19:59:08 +00:00
John Colvin
3a32ed5fe6
Fix BookSide equality operators
2025-01-27 16:35:22 +00:00
John Colvin
4e01697327
Add equality and output operators to BookLevel
2025-01-27 16:25:32 +00:00
John Colvin
22032c1742
Add fmt::formatter for domain::Side
...
So we can pass it to LOG macros, etc.
2025-01-27 16:25:15 +00:00
John Colvin
0a9bce317c
Add contract qty info to security proto (and FixedSecurity)
...
We should get this kind of security metadata from the exchange rather
than config.
2025-01-25 20:41:47 +00:00
Mohammad Arham
40e44db597
Fix: Added GCHandle in attempt to fix ui crash
2025-01-22 02:15:55 -05:00
Mohammad Arham
4a447e44e1
Add: Version to first log
2025-01-22 02:14:19 -05:00
John Colvin
b1f4654b3c
Fix benign typo in publish_spark.py
2025-01-21 17:28:50 -05:00
John Colvin
9d350334c0
fix build (installer): Fix install path typo in publish_spark.py
2025-01-21 17:28:05 -05:00
John Colvin
ac597f505f
fix build (installer): Set CMAKE_INSTALL_PATH for spark preset
2025-01-21 17:19:15 -05:00
John Colvin
147d9ceb1f
ui: Enable net change price display for synths in ladder
2025-01-21 13:11:16 -05:00
John Colvin
377804dfb9
fix (ui): Don't crash on bad regex in grid filters
...
Wrap Regex construction in try/catch. This is from user input, so of
course we can't assume it will be valid.
2025-01-21 09:26:37 -05:00
John Colvin
0d26401961
fix: Preserve tick_factors in TickDisplaySettings "with" ext
...
Not sure this was a bug in practice, but it was definitely wrong.
2025-01-21 01:24:23 -05:00
John Colvin
889e226151
feat: Add setting to show notional values in ladder
2025-01-21 01:24:23 -05:00
John Colvin
986d53f4e0
ui: Make ladder size format text editable
...
This is the best of both worlds: the user can choose from one of the
canned presets in the dropdown, or specify a custom string format.
2025-01-21 01:24:23 -05:00
Taha
d13ba2968a
DateTimeExpanstionStream col name changes
2025-01-20 15:34:11 -05:00
John Colvin
1a192c8117
feat: Add setting to show ladder prices in basis pts from mid
2025-01-20 14:15:49 -05:00
John Colvin
5e36d4801c
cleanup: Remove redundant if/else code
2025-01-20 13:31:41 -05:00
John Colvin
2cf6411ca7
fix: Send exchangeInfo req via our HttpClient instead of ccapi
...
Bugs (fixed):
* We get a boost beast exception about the read limit being exceeded
when requesting securities from Binance prod url (testnet works).
2025-01-19 14:21:11 +00:00
John Colvin
e99df129d5
cleanup: Remove erroneous include
2025-01-18 22:54:06 +00:00
John Colvin
d07c01bd17
ui: Support ladder ticks-per-row setting for synths
...
The ticks-per-row control was likely hidden by accident in the first
place. I'm guessing the original intent was to only hide the Price Terms
control for synthetics. At the time, there were no other controls in the
group box, so it made since to hide the whole thing. Some time later, we
added the ticks-per-row setting to the same group box, and forgot to
make the groupbox visible again.
2025-01-17 13:11:09 -05:00
Taha Afzal
d69673c524
Previous change was not setting the create time for orders, fixing that + changign one log level to Dag
2025-01-17 07:00:57 +00:00
Taha Afzal
4ee595cdaf
Fixing the issue were fractional fills on a working order could result in overfills being reported (it was only a reporting issue)
2025-01-17 06:38:38 +00:00
Mohammad Arham
6fcaf07df0
Fix: binance execution working on sim
2025-01-17 00:59:22 +00:00
Taha Afzal
492d48d886
Added qty_multiplier to spreader.proto
2025-01-16 20:18:26 +00:00
Taha Afzal
b6a56e768b
Adding config boolan config publish_boostrap_books_only to MarketDataReplayConfig which allows MDReplayer to only publish the bootstrap of all the books
2025-01-16 20:18:26 +00:00
Taha Afzal
3e2b5976a8
Minor changes and ocmments, to allow for fractional qtys
2025-01-16 20:18:26 +00:00
Mohammad Arham
edd3f6cbd6
FIX: Orderbook quantity display fixed
2025-01-15 14:47:09 -05:00
John Colvin
2e9347341b
fix: Don't throw ex just because leg qty is 0
...
ContractLegEnumerable shouldn't be responsible for security validation.
2025-01-14 12:58:58 -05:00
John Colvin
2061e38349
fix: Hook up mview venue filter in vm
2025-01-13 00:11:43 -05:00
Mohammad Arham
650dd57b6d
FIX: increase max symbo count from 32 to 1024
2025-01-13 04:42:13 +00:00
Mohammad Arham
82747f7dc0
Fix: OKX perp swaps are same as futures now
2025-01-13 04:41:29 +00:00
John Colvin
04b26904a3
ui: Sort exchanges in grid filter dropdown
2025-01-12 15:10:20 -05:00
John Colvin
f123e4c72c
ui: Add Exchange col to mview
2025-01-12 15:10:20 -05:00
John Colvin
b887cbebab
fix: Actually capture emumba deribit cfg obj by value
...
The previous "fix" didn't actually work because, although the lambda
captured the obj by value, the thread captured the lambda by reference!
2025-01-09 16:53:22 +00:00
John Colvin
14e975bc7e
vscode: Add Spark (Skip Build) entry to tasks.json
2025-01-09 11:23:30 -05:00
John Colvin
064f557057
ui: Sort accounts in ladder/ticket dropdown
2025-01-09 11:23:30 -05:00
John Colvin
4d106b5e6c
ui: Fix alignment issues in order ticket
2025-01-09 11:23:30 -05:00
John Colvin
9fbeaf0d68
ui: Make TickUpDown style match default (dark) Button
...
* Set BorderThickness to 0
* Use same mouse-hover background as Button
2025-01-09 11:23:30 -05:00
John Colvin
4018ac8b78
ui: Add tooltips to order ticket
2025-01-09 11:23:30 -05:00
John Colvin
81c9503300
ui: Apply dark theme to order ticket buttons
2025-01-09 11:23:30 -05:00
John Colvin
477210926d
ui: Fix text color in algo popup
2025-01-09 11:23:29 -05:00
John Colvin
0aa48a79a9
ui: Remove single quotes from tooltip
...
It looked busy
2025-01-09 01:38:26 -05:00
John Colvin
9e71c7af02
ui: Add tooltips to ladder
2025-01-09 01:38:26 -05:00
John Colvin
4647751409
vscode: Add Spark Test entry to launch.json, tasks.json
2025-01-09 01:03:33 -05:00
John Colvin
a3087b874b
cleanup: Remove unused code
2025-01-09 01:03:32 -05:00
John Colvin
5b2f5c5c8f
Don't throw/catch type exception needlessly
2025-01-09 01:03:32 -05:00
John Colvin
479227090e
ui: Remove overly rigid validation for "ticks per row" setting
2025-01-09 01:03:32 -05:00
John Colvin
2dfcf320f5
fix: Refresh ladder levels when "ticks per row" changes
2025-01-09 01:03:32 -05:00
John Colvin
b4c7213cb2
fix: Refresh ladder prices in view when they change
...
Bugs (fixed):
* Sometimes the prices in the left panel of the ladder get stale
2025-01-09 01:03:32 -05:00
John Colvin
c7dfd69b90
Consolidate tick rounding functions
...
Too many functions were performing essentially the same service:
rounding ticks. It was confusing.
2025-01-09 01:03:32 -05:00
John Colvin
4fdc866485
build: Propagate py script exit codes back to the shell
...
I noticed the build cmds were broken on Windows because the debugger
would still start even when the preLaunchTask (build) failed!
2025-01-09 01:03:32 -05:00
John Colvin
f5d5ca9a1c
ui: Use appropriate colors in font picker control
...
We must have missed this when we switched to dark theme.
2025-01-09 01:03:32 -05:00
John Colvin
cfafcf77b9
ui: Use appropriate text color in ladder settings dialog
...
We must have missed this black text when we switched to dark theme.
While we're at it, freeze the brushes it uses, since they are stored in
a StaticResource which could potentially be shared by multiple controls.
2025-01-09 01:03:32 -05:00
John Colvin
8ebe783139
fix: Clear low/high lines in ladder when scrolled out of view
2025-01-09 01:03:32 -05:00
John Colvin
1851a96a58
vscode: Add Spark (Skip Build) entry to launch.json
2025-01-09 01:03:32 -05:00
John Colvin
0b334ef870
fix: Remove erroneous early exit condition in Ladder render
...
Bugs (fixed):
* The ladder doesn't always render ask prices
2025-01-09 01:03:32 -05:00
John Colvin
ee811a729a
ui: Make ladder price line colors match button text
2025-01-09 01:03:32 -05:00
John Colvin
cb15be276d
ui: Show bid, ask, mid lines in ladder by default
2025-01-09 01:03:32 -05:00
John Colvin
7d5af6729b
ui: Show bid, ask, mid prices in ladder left panel
2025-01-09 01:03:31 -05:00
John Colvin
e5679cf082
ui: Make ladder price links prettier
...
* Change them from hyperlinks to buttons
* Add mouse hover animation
* Align text to the right (so decimal places line up)
2025-01-09 01:03:31 -05:00
John Colvin
98f89b9ad4
fix: Capture emumba deribit cfg by value in thread body
...
Bugs (fixed):
* Sparkd occasionally assert crashes on startup because it fails to
read emumba deribit config.
Cause: This is a simple race bug. The config obj isn't captured by
value, so it can be destructed before the thread references it.
2025-01-08 19:03:01 +00:00
John Colvin
ba4f8f67e3
fix: Disable binance usds futs symbol filter config
...
This was a premature optimization, and has become a burden now that we
need to subscribe to large symbol sets for Alphaworx.
2025-01-06 05:00:04 +00:00
John Colvin
9c78380490
fix build (gcc rel): Don't fallthrough switch cases
2025-01-06 03:52:25 +00:00
John Colvin
f6166f0674
fix: Show iceberg show qty in algo params grid
...
Bugs (fixed):
* In the ladder, you can't view/edit the iceberg show quantity.
Cause: No view template was defined for nullable double params. Most
likely, we changed the show qty in the vm from double to nullable
double at some point, and forgot to reflect this change in the view.
An easy mistake to make as it would not have broken the build.
2025-01-05 15:58:29 -05:00
John Colvin
6334a902e7
fix: Reclaim keyboard focus when mview popup is clicked
2025-01-05 15:29:11 -05:00
John Colvin
082ca62fff
fix: Focus price textbox when mview order ticket popup opens
2025-01-05 15:18:46 -05:00
John Colvin
9dbd8dd7c6
fix: Set focus through FocusScope element if one exists
2025-01-05 15:18:46 -05:00
John Colvin
8f7a4d0df6
Fix typo in AutoFocusTargetProvider dependency prop
2025-01-05 15:18:46 -05:00
John Colvin
f372dd56b3
fix: Return synthetic name in get_security_desc
...
If the security is a synthetic (of course).
Bugs (fixed):
* Changing limits in the risk ui causes sparkd to assert crash
2025-01-05 20:15:10 +00:00
John Colvin
86eba20d61
build: Make compiler/build type case insensitive
2025-01-05 06:25:11 +00:00
John Colvin
35a985280d
fix: Run heartbeat timer in MarketDataGateway
...
This is why we were getting those heartbeat asserts. Previously, only
ExchangeSimApplication would start this timer. Since MarketDataGateway
doesn't share its MDProducer with ExchangeSimApplication, it must start
one itself.
2025-01-03 16:59:07 +00:00
John Colvin
f5e1b3319c
cleanup: Don't write to cout in SparkDataDirectory
...
Don't need that noise.
2025-01-03 16:59:07 +00:00
John Colvin
65c3948e4f
cleanup: Remove unused code
2025-01-03 16:59:07 +00:00
John Colvin
002bdff9a3
fix: Don't parse non-existent json files in SparkDataDirectory
...
Bugs (fixed):
* Sparkd assert crashes if binance_usds_futures_execution.json file
doesn't exist
2025-01-03 16:59:07 +00:00
John Colvin
bb6e5dd548
diag: Fix sparkd log path in show_last_run.sh
2025-01-03 16:59:07 +00:00
John Colvin
7c1aed3339
fix build (api server): Remove old cpps from source file list
2025-01-03 16:59:07 +00:00
John Colvin
22ae51fb99
build: Clean up build py scripts
2025-01-03 16:59:07 +00:00
John Colvin
a53d347f81
fix build (warns): Update cmake min version to 3.10
...
Versions < 3.10 will soon be deprecated (we just started getting
warnings about it).
2025-01-03 16:59:07 +00:00
John Colvin
838dbcd886
build: In py build scripts, exit with code 1 on failure
...
Previously, they would exit with 0 no matter what happened.
2025-01-03 16:59:07 +00:00
John Colvin
5ed2309d22
fix: Add switch case for VENUE_BTS in VenueIndex
2025-01-03 16:59:07 +00:00
John Colvin
4d41704dd2
build: Configure admin webapp build entirely in CMakeLists
...
Get rid of build_admin_web_app.py
2025-01-03 16:59:07 +00:00
John Colvin
bd1000e518
build: Get rid of explicit "targets" in CMakePresets for sparkd
...
So we don't have to worry about keeping it synchronized with the actual
targets in the build (just one more thing that can break). You can build
specific targets via the cmake cli anyways.
2025-01-03 16:59:07 +00:00
John Colvin
dd4aa3c2ed
build: Update fable in dotnet-tools.json
2025-01-03 16:59:07 +00:00
John Colvin
fee9d3f597
build: Update webpack-cli in admin webapp
2025-01-03 16:59:07 +00:00
John Colvin
416f9aef31
vscode: Add Sparkd Test to launch.json
2025-01-03 16:59:07 +00:00
John Colvin
48f5e4c201
vscode: Fix sparkd launch.json
2025-01-03 16:59:07 +00:00
John Colvin
4438b4ca8d
feat: Support fractional quantities in synthetic book
2025-01-03 16:59:07 +00:00
John Colvin
73356e0d61
cli: Add cmd to print security info from the db to console
...
Usage: sparkd md dump --db
2025-01-03 16:59:07 +00:00
John Colvin
4a285fa581
cleanup: Make fn const
2025-01-03 16:59:07 +00:00
John Colvin
cfa82c2177
cleanup: Remove erroneous fwd declared class
2025-01-03 16:59:07 +00:00
John Colvin
0410a0431a
fix: Encode exp in valid date int when storing in sqlite
...
The strange bit shifting we were doing before doesn't support large
dates.
Bugs (fixed):
* Synthetics with Binance perpetual future legs don't appear on the
front end after sparkd restarts.
* Cause: Exp dates for Binance perpetual futures aren't properly
encoded in the database. As a result, when loaded from disk, they
are mistakenly identified as expired, along with any synthetics that
have them as legs.
2025-01-03 16:59:07 +00:00
John Colvin
02843f14ff
cleanup: Get rid of trivial fn that's only used in one place
...
Move the logic into the function that used to call it.
2025-01-03 16:59:07 +00:00
John Colvin
f1f1d4259c
diag: Add logs around security discovery in MDConsumer
...
This is multi-threaded event driven code. Without descriptive logs in
the right places, diagnosing problems is difficult and tedious.
2025-01-03 16:59:07 +00:00
John Colvin
71fc43af68
cleanup: Flatten nested if blocks with continue statements
...
Easier to read
2025-01-03 16:59:07 +00:00
John Colvin
350144ffd5
fix: Use security desc to get unique id (along with venue, type)
...
I'm guessing this typo went unnoticed because it didn't break the build
(another arg of the same was being used instead).
2025-01-03 16:59:07 +00:00
John Colvin
da227a5550
Support synths at Binance, Binance USDS Futures, Deribit, and Okx
2025-01-03 16:59:07 +00:00
John Colvin
0c92cf3a87
Add double ops to IntPrice
2025-01-03 16:59:07 +00:00
John Colvin
1188af391e
Add trivial fns to TimeStamp for convenience
...
* += and -= operators
* 'date' fn
* 'to_date_int' fn
2025-01-03 16:59:07 +00:00
John Colvin
0bb85cf915
refactor: Change month and day params from uint32 to uint8
...
They only need to hold values 1-12 and 1-31, respectively. A single byte
is more than sufficient. Also add ymdhms TimeStamp constructor.
2025-01-03 16:59:07 +00:00
John Colvin
d77f5996fe
cleanup: Make a couple TimeStamp fns const
2025-01-03 16:59:07 +00:00
John Colvin
940c41a274
Add int32 arithmetic operators to domain::Price
2025-01-03 16:59:07 +00:00
John Colvin
e671960dc3
Add double arithmetic operators to domain::Price
2025-01-03 16:59:07 +00:00
John Colvin
8fa34b6b39
fix: Multiply additive factors by exponent in domain::Price
...
We must not have been using these operators because the implementation
was very broken (yet it doesn't seemed to have caused any bugs).
2025-01-03 16:59:07 +00:00
John Colvin
f9d5429b5b
cleanup: Remove unused code
2025-01-03 16:53:05 +00:00
Mohammad Arham
2c342482eb
Merge branch 'development.crypto' of bitbucket.org:bluetradesys/spark into development.crypto
2025-01-02 15:30:13 -05:00
Mohammad Arham
3aefac26c2
fix: Timer condition was incorrect
2025-01-02 20:15:38 +00:00
John Colvin
1554594cdd
diag: Throttle noisy log in Ladder
2025-01-02 12:15:22 -05:00
John Colvin
727a0f38b5
fix: Prevent arithmetic overflow when converting long to int in Ladder
2025-01-02 12:13:58 -05:00
John Colvin
7a0e8cdb26
fix: Properly format spread/synth leg qty in generated desc
2025-01-01 13:12:17 -05:00
John Colvin
a4e97b04d7
fix: Don't propagate security criteria changes to other windows
...
Bugs (fixed):
* Security criteria changes in mview and time/sales propagate to other
windows of the same type.
2024-12-31 11:06:24 -05:00
John Colvin
463980a6ef
Fix typo in get_binance_md.py
2024-12-29 18:37:54 +00:00
John Colvin
9b25857042
fix: Ensure "earliest_expiration" fn actually returns min exp
...
Previously, it assumed the first leg would have the earliest exp. While
this may be true for most securities, it isn't guarenteed by our data
model.
2024-12-29 13:26:44 -05:00
John Colvin
ef19e7d76b
fix: Don't assume all securities have expirations in pnl vm
...
Bugs (fixed):
* PNL window crashes if data has Spots (i.e., securities without exps)
2024-12-29 13:17:17 -05:00
Taha Afzal
471faec2cb
Added more etsts for fractional qtys
2024-12-28 08:31:16 +00:00
Taha Afzal
ce96d42db8
Merging
2024-12-26 07:01:30 +00:00
Mohammad Arham
187894ca84
FIX: makes gateway factory work
2024-12-24 17:56:49 +00:00
John Colvin
63492c4575
fix: Remove erroneous assert in LadderRender
2024-12-21 23:24:52 -05:00
John Colvin
e150c611f6
fix: Round fractional order qty to 6 decimal places
...
It was previously rounding to 2 decimal places, which isn't enough for
many crypto currencies (e.g., BTCUSDT at Binance)
2024-12-21 23:17:52 -05:00
John Colvin
bb837d6c26
fix build: Trivial changes in DatabentoDataHandler
2024-12-22 04:00:23 +00:00
John Colvin
965e541ac8
Fix json structures and endpoints for Binance USDS Futures md
2024-12-22 04:00:23 +00:00
John Colvin
1b232aa610
feat: Add Binance USDS Futures execution
2024-12-22 04:00:23 +00:00
John Colvin
25439b094a
improvement: Perform sec lookup in const time
2024-12-21 20:08:19 +00:00
John Colvin
f927ef2c5e
Add conf_json dir to spark data file tree
...
So we don't have to add proto files to the build for trivial config.
2024-12-21 20:08:19 +00:00
John Colvin
d42389b75a
Don't needlessly download cme config xml
...
When we aren't subscribing to cme md (i.e., no glbx symbols in spark
conf).
2024-12-21 20:08:19 +00:00
John Colvin
671cbcf03a
Fix bts::Quantity string fmt so it includes fractional qty
2024-12-21 20:01:55 +00:00
John Colvin
e589e23f41
Add py script to get md at binance
...
For quick/easy troubleshooting (much easier than doing stuff in c++)
2024-12-21 20:01:55 +00:00
John Colvin
11fadc0a3e
Expose "run" fn in WebsocketClient
...
So it doesn't have to be hidden in a background thread
2024-12-21 20:01:55 +00:00
John Colvin
b8c2d93866
Add ping/pong facilities to WebsocketClient
2024-12-21 20:01:55 +00:00
John Colvin
67a23dd196
Add open/close callbacks to WebsocketClient
2024-12-21 20:01:55 +00:00
John Colvin
ff22c4eb5f
diag: Change WebsocketClient log category to "Http"
...
A better fit than "Gen"
2024-12-21 20:01:55 +00:00
John Colvin
5addff4c50
cleanup: Remove redundant fn HttpRequest::perform
...
This was doing the same thing as HttpClient::send. I likely refactored
this function during development, then forgot to remove it.
2024-12-21 20:01:55 +00:00
John Colvin
142e1bd2da
Add HttpMethod "DELETE"
...
In Binance's REST api, orders are canceled via DELETE requests.
2024-12-21 20:01:55 +00:00
John Colvin
d05d91bbe7
diag: Log Http requests
2024-12-21 20:01:55 +00:00
Mohammad Arham
3a90ca9cb5
Add: Johns changes to sec oracle
2024-12-17 14:47:37 +00:00
John Colvin
85a46727be
Improve c++ http client facilities
2024-12-11 19:28:36 +00:00
Mohammad Arham
624fd2c976
FIX: BUILD Error
2024-12-09 01:34:29 +00:00
Mohammad Arham
16d599d2d6
Add: OKX execution gateway connected
2024-12-08 20:14:16 -05:00
John Colvin
c00e4fcb6b
feat: Add Binance Coin Futures market data
2024-12-06 00:39:41 +00:00
John Colvin
03e19afe1d
Fix qty display conf for BTC/ETH futs at Binance
...
Need to show 3 decimal places (qty tick size is 0.001)
2024-12-05 21:05:53 +00:00
John Colvin
058e11eadc
fix build: Fix install path for sparkd
2024-12-05 18:31:36 +00:00
John Colvin
6ff52e7107
fix build: Add onixs libs again
...
Taha accidentally reverted externals
2024-12-04 16:56:46 +00:00
John Colvin
ee4c28ed41
Port ws client over from cme-websocket-connector
...
Need this to implement execution at crypto exchanges (beginning with
Binance and OKX)
2024-12-04 16:56:46 +00:00
Mohammad Arham
50d26f0060
Merge branch 'development' of bitbucket.org:bluetradesys/spark into development
2024-12-03 09:52:09 -05:00
Mohammad Arham
36378e6074
Add:: Recording md to sql
2024-12-03 14:51:32 +00:00
John Colvin
8cde8ee6e1
build: Add script to publish the api server
...
This will build the app as a self-contained executable here:
build/Release/bin/net7.0/linux-x64/publish
2024-11-28 16:35:21 +00:00
John Colvin
4f8fad444e
fix build: Add missing csproj deps to Spark.Api.Server.sln
2024-11-28 16:35:20 +00:00
John Colvin
832c645a30
build: Map cmake install dir to associated preset
...
Cpp bins are now installed to "install/<cmake_preset>/bin". For example,
"install/sparkd-release-gcc/bin", "install/spark-api-server-linux/bin", etc.
2024-11-28 16:35:20 +00:00
John Colvin
691343d9b6
feat: Add datastreams to spark api server
2024-11-28 06:14:00 +00:00
Taha
461cbad338
Adding FakeBBO stream
2024-11-27 13:36:14 -05:00
Taha
fecbcbb6b6
Merge branch 'development' of bitbucket.org:bluetradesys/spark into development
2024-11-26 22:33:31 -05:00
Taha
aab8e83543
Some rewrite of streams that fetch bars, and making the other business streams thin wrapper over stream primitives
2024-11-26 22:33:26 -05:00
Mohammad Arham
420b46741e
ADD: Config for sql writer in mdrecorder
2024-11-24 19:14:45 +00:00
Mohammad Arham
e8f65f95bd
Wrapped proto write in function in mdrecorder
2024-11-24 19:13:36 +00:00
Mohammad Arham
0e893b9b5d
Fix: Multiple Exchange data sources work without timee
2024-11-20 23:48:50 +00:00
Mohammad Arham
0593ae0320
Fix: Logging path was incorrect
2024-11-20 23:48:24 +00:00
Mohammad Arham
960149b9c5
Fix: Stupid bug on sleep time
2024-11-20 22:49:22 +00:00
Mohammad Arham
b5f5f2f3d5
Merge branch 'development' of bitbucket.org:bluetradesys/spark into development
2024-11-20 17:21:45 -05:00
John Colvin
bd8c97244a
refactor: Return row id in IStore::retrieve_one
...
This is a prerequisite to synchronizing security ids stored via IDGenerator
(the "new" way to store ids, added when we implemented Deribit), and
SecurityDB (the original way from spark days). The problem was that these
2 implementations were using separate databases. We need to unify the
interfaces between these storage services to avoid breaking existing code.
2024-11-20 22:17:08 +00:00
John Colvin
cb187e6997
Interpret db strng param as name or path in SqliteStore ctr
2024-11-20 22:17:08 +00:00
John Colvin
4004ed2267
refactor: Normalize sec key indexing for old md sources
...
Now both old and new (domain) md sources index securities by
the same string format.
2024-11-20 22:17:08 +00:00
John Colvin
a00347ac3f
Add ostream operator for Venue
2024-11-20 22:17:08 +00:00
John Colvin
c4d08840bd
Add ostream operator for SecurityType
2024-11-20 22:17:08 +00:00
John Colvin
84c0684b63
refactor: Normalize sec key indexing for new (domain) md sources
...
This is part of a larger effort to unify the way we index securities in order
to generate unique ids. Older md sources (e.g., MDP3Listener) use
SecurityIDRecorder to generate ids, which uses a different database and
different key-string formatting than the IDGenerator used by the newer md
sources (e.g., GeneralMarketDataListener and various ccapi-based sources).
2024-11-20 22:17:08 +00:00
Mohammad Arham
bca8df6d99
Fix: Release Crash fixed still has timer
2024-11-20 22:14:48 +00:00
Mohammad Arham
98f8e58f3f
Add optional tags to config
2024-11-20 22:12:06 +00:00
Mohammad Arham
76f18f3443
Fix: typo in fixed securities
2024-11-20 20:17:29 +00:00
Mohammad Arham
b67d748ba1
removed: moved onix glbx to main controler thread
2024-11-20 06:58:14 +00:00
Mohammad Arham
0113721355
Add: Config for marex reconnect attemps
2024-11-20 06:29:41 +00:00
Mohammad Arham
715f7f7c70
Fix: Race condition causes crash not sure if fixed
2024-11-20 06:17:28 +00:00
Mohammad Arham
89955597c1
Add: Onix/Marex fix adapter, listener, and state handler
2024-11-20 06:16:58 +00:00
Mohammad Arham
96d9aae048
Add: IFIXGenericAdapter
2024-11-20 04:45:36 +00:00
Mohammad Arham
f28d32946f
Add: General Execution Gateway (between system and IFIXAdapter )
2024-11-20 04:39:43 +00:00
Mohammad Arham
924efb2c3f
Add: Added proto for marex fix config
2024-11-20 04:26:39 +00:00
Mohammad Arham
50abb1d784
Fix: Refactored to Switch Statement instead of if and aadded cboe
2024-11-20 04:24:51 +00:00
Mohammad Arham
b5ef39ee4b
Fix: Change default tick_config for taifed
2024-11-20 04:18:58 +00:00
Mohammad Arham
5b61b4fe69
Add: Accounts for CBOE Execution, subaccounts dont work
2024-11-20 03:22:18 +00:00
Mohammad Arham
2d8523ae67
FIX: Order cancel now working in emnative order
2024-11-20 03:18:31 +00:00
Mohammad Arham
f0eef447d3
Add: Exchange parameter added to EMNAtive Order
2024-11-20 03:18:04 +00:00
Mohammad Arham
8604d5672c
ADD: Comments for CME execution for later
2024-11-20 03:11:54 +00:00
Mohammad Arham
30822fb1f1
Fix: move start glbx onix to seperate thread
2024-11-20 03:09:07 +00:00
Mohammad Arham
826bbf76a9
Add: load_reference_data config for databento
2024-11-20 02:53:19 +00:00
John Colvin
eb8c97a1b7
fix: Detect integer ticks of all sizes (not just 1)
...
Also don't divide by zero!
Bugs (fixed):
* Spark crashes when opening a ladder with integer tick size of 5.
This was first observed with TAIFEX futures.
Cause: The ladder throws a DivideByZero exception when calculating
the number of ticks per dollar. It isn't caught until it bubbles up
to the scope in which 'Dispatcher.Run' was called. The catch handler
then deliberately assert crashes.
2024-11-12 16:15:47 -05:00
John Colvin
5517c58724
fix (ui): Display trade time w/ microsecond precision
...
Was only showing milliseconds before
2024-11-12 11:42:23 -05:00
John Colvin
e9bcdd0c65
fix build: Update Newtonsoft.Json to 13.0.3 in Spark.csproj
2024-11-12 10:35:51 -05:00
Taha
83b37c6661
Merge branch 'development' of bitbucket.org:bluetradesys/spark into development
2024-10-30 13:36:33 -04:00
Taha
6163887cd3
Bunch of CME queries are now working
2024-10-30 13:36:23 -04:00
Mohammad Arham
214641aa1e
Add: cpp tp sec_common
2024-10-25 07:02:46 +00:00
Mohammad Arham
06bc7af232
FIx: Taifex md working
2024-10-25 06:59:52 +00:00
Mohammad Arham
36b6168dec
Taifex State working
2024-10-18 21:13:56 +00:00
Taha
f9aa9dd982
Merge branch 'development' of bitbucket.org:bluetradesys/spark into development
2024-10-15 23:48:34 -04:00
Taha
22f1ab44f9
Basic set of streams that can talks ot the LLM and get DSL back from few-shot prompt
2024-10-15 23:38:49 -04:00
John Colvin
e4c87c267d
Fix DateTimeConstant token regex for DataStreams
...
The previous regex was overly ambitious (and needlessly complex as a
result). The new regex is simpler, and doesn't impose bounds on month
and day ints, aside from requiring 1-2 digits.
Bugs (fixed):
* The DataStreamExpressionInterpretter fails to parse certain date
constants, such as "8/30/2024"
2024-10-15 11:46:58 -04:00
Mohammad Arham
86899a6ff0
FIX: TAifex state working
2024-10-14 17:19:03 +00:00
Adeel Khan
2324fd3ad2
CHANGE: Taifex State Mgmt
2024-10-14 20:59:34 +05:00
Taha
288b7712db
Added and modified more streams to address teh CME NL queries
2024-10-09 13:59:01 -04:00
Mohammad Arham
8850bf4903
Remove: Assert in taifex code
2024-10-09 14:53:20 +00:00
Mohammad Arham
11112b41d1
Add: Missing config entry databento
2024-10-09 02:36:35 +00:00
Adeel Khan
2a98889b67
UPDATE: TaifexDataGateway
2024-10-09 02:18:28 +00:00
Adeel Khan
8060518a28
FIX: Removed Taifex Bugs
2024-10-08 21:29:44 +00:00
Mohammad Arham
68440d3df3
ADD: Config for one year security filter
2024-10-08 13:50:38 +00:00
Mohammad Arham
7d775afc4c
Add: Taifex after hours ip change config
2024-10-08 13:49:36 +00:00
Mohammad Arham
50c7764ef7
Add: Databento security filter
2024-10-07 20:33:38 +00:00
Mohammad Arham
fb8b2d28f3
Fix: DATABENTO live working
2024-10-07 20:05:38 +00:00
Mohammad Arham
bcc2c4a514
CHANGE: Taifex Refactored
2024-10-07 16:26:12 +00:00
Mohammad Arham
9352298be0
Add: Taifex statehandler
2024-10-07 08:51:55 +00:00
Mohammad Arham
180bd11740
FIX: Taifex replay types of files handled
2024-10-06 21:31:34 +00:00
Mohammad Arham
57ea841381
ADD: i084 taifex
2024-10-06 20:31:07 +00:00
Mohammad Arham
7d9d176fa2
Change: Rename taifex files
2024-10-06 18:38:36 +00:00
Mohammad Arham
a8f4ec07c8
Change: moved file paths to config for taifex file replay
2024-10-06 08:29:27 +00:00
Mohammad Arham
58c5be268f
Add: Refactored taifex code to break it down
2024-10-06 08:01:22 +00:00
Mohammad Arham
bf4859a2a9
Change: Refactored read binary file taifex
2024-10-06 07:28:12 +00:00
Adeel Khan
db1cf93994
Add: Incremental Books
2024-10-04 23:23:54 +05:00
Adeel Khan
ecb1c4fea0
Add: Taifex Trades
2024-10-04 23:23:41 +05:00
Mohammad Arham
a513933044
Add: Exceptions handling for databento
2024-10-04 13:06:56 +00:00
Mohammad Arham
764d65b595
Remove: code to remove secuties
2024-10-03 12:06:55 +00:00
Mohammad Arham
bc0d606252
Add: code to connect to Live databento
2024-10-03 12:03:48 +00:00
Mohammad Arham
ceb8a68902
Add: databento repllay filename to cofig
2024-09-27 03:20:20 +00:00
Mohammad Arham
0b3183fd30
Add: added onix fix to lib to build dir
2024-09-26 16:23:19 +00:00
Mohammad Arham
b6dbe37818
Add: entry for onix fix
2024-09-24 13:41:45 +00:00
Mohammad Arham
c78b8bc8e1
Add databento to external
2024-09-24 13:37:57 +00:00
Taha Afzal
314f3e3715
Adding comments
2024-09-23 18:40:06 +00:00
Taha Afzal
26a1d5fc03
Creating scaffolding for incremental book updates in GeneralMarketDataListener
2024-09-23 18:39:12 +00:00
John Colvin
37e5003cf9
Merge remote-tracking branch 'origin/development.ps_linux' into development
2024-09-19 05:40:59 +00:00
John Colvin
07f3868ffc
Add CBOE to spark admin webapp
2024-09-18 19:03:43 +00:00
John Colvin
46f0b17138
Add TAIFEX to spark admin webapp
2024-09-18 19:03:43 +00:00
Taha
33a8e06a3a
Fixing implied vols for SPX (market data was getting divide dby the fwd)
2024-09-17 22:17:31 -04:00
John Colvin
74afdde8de
fix build: Add ".list" to SymbolLists config
...
For new exchanges, TAIFEX and CBOE
2024-09-17 13:01:05 +00:00
John Colvin
0a92ad2eb1
fix (build): Add break statement to new venue switch cases
...
Namely, TAIFEX and CBOE
2024-09-17 13:00:14 +00:00
John Colvin
56d207c3f1
fix: Preserve vs exp sec type (Option) when computing implied vol
...
Because the exp keys in the data map have sec type Option.
2024-09-16 13:46:53 -04:00
John Colvin
9172e752e3
ui: Highlight launcher buttons when selected
...
Namely, the expandable menu buttons in Analytics shell module
2024-09-16 13:46:53 -04:00
John Colvin
adc9f2acfc
Replace trivial refs to ThreadedBlackboard w/ IBus
2024-09-15 23:45:15 -04:00
John Colvin
41d17e67d2
fix: Move accounts subscription to dispatcher thread
...
We update a collection that is bound to the view in the callback (mutation
ops throw if performed on a different thread).
2024-09-15 23:45:15 -04:00
John Colvin
5cacff073b
fix: Check 'HasTimestamp' before accessing prop in man fill row
...
Pending adds (i.e., unpersisted new rows) don't have timestamps.
In such cases, the 'Timestamp' getter throws an exception.
2024-09-15 23:45:15 -04:00
John Colvin
c8da92c250
Cache/index options for faster lookups in SystemContractData
2024-09-15 23:45:15 -04:00
John Colvin
c4e76076e3
fix: Include venue w/ symbol when publishing interp greeks
2024-09-15 23:45:14 -04:00
John Colvin
6a715b9a68
Fix typo in comment
2024-09-15 23:45:11 -04:00
John Colvin
1f28940c95
fix: SelectorList.RemoveAt doesn't need 'convert_back' delegate
...
So don't throw an exception if it is null.
2024-09-15 23:45:11 -04:00
John Colvin
ebff40b88f
fix: Don't assume venue is in spread def string
...
Pass it as an argument to the parse fn. This should be fine for now, but
obviously a singular venue param won't work for cross exchange spreads.
2024-09-15 23:45:11 -04:00
John Colvin
a6d14a3ac3
improvement: Move system contracts sub off of dispatcher thread
...
Also add 200ms throttle. Contracts shouldn't change that often, so we don't need
lightening speed here. Furthermore, such changes may trigger expensive cpu ops.
Note: this change also replaces refs to ThreadedBlackboard w/ IBus (the
former is somewhat tethered to the dispatcher thread, particularly by
its timer_callback).
2024-09-15 23:45:11 -04:00
John Colvin
af96bcb9e3
fix: Pass symbol/venue from vs usage cfg to AnalyticsService proto
2024-09-15 23:28:59 -04:00
John Colvin
619702cb04
cleanup: Remove unused code
2024-09-15 23:28:59 -04:00
John Colvin
5baec19aff
refactor: Index vol and theos data by symbol/venue tuple
...
Rather than just symbol, to avoid conflicts when the same symbol exists
at multiple exchanges
2024-09-13 10:50:23 -04:00
John Colvin
39ef30c1e3
cleanup: Remove redundant type VolSurfaceSymbolStructure
...
There is an identical type (in terms of prop types and corresponding proto indices)
called VolSurfaceSymbolRelationship that is used in more places.
2024-09-13 10:50:23 -04:00
John Colvin
c2f2d28914
Move nested vm classes into own files
...
VolSurfaceSettingsEditor has too much code (still). It is hard to
navigate.
2024-09-13 10:50:23 -04:00
John Colvin
e06bc3bdf5
Simplify driver symbol structure in vs settings vm
...
The code was confusing and difficult to follow.
2024-09-13 10:50:23 -04:00
John Colvin
e19c19c013
fix: Raise prop changed event when is_selected changes
2024-09-13 10:50:23 -04:00
John Colvin
8f324bc507
Add venue to vs calc fns
2024-09-13 10:50:23 -04:00
John Colvin
12107c42ce
fix: Provide venue param w/ symbol when computing fut slide
2024-09-13 10:50:23 -04:00
John Colvin
91c6fb4e52
diag: Log added/removed exps when syncing vs input w/ system exps
...
This shouldn't happen often, but if it does, we should know about it.
PS: We recently fixed a bug caused by inequalities between
VolSurfaceExpirations in vs input, and in SystemContractData. User
inputs to the vs grid would be immediately reverted, as the exps
associated with the inputs were not found in the SystemContractData.
This bug would have been easier to track down with this logging in
place.
2024-09-13 10:50:23 -04:00
John Colvin
7044355547
Add venue to VolSurface(DisplayGrid)Expiration
...
So we can handle symbol/exp date conflicts for securities that differ
only by venue (exchange)
2024-09-13 10:50:23 -04:00
John Colvin
583ae65fb2
Clean up VolSurfaceExpiration
...
* Enable nullable lang feat
* Use file-scoped namespace
* Sort members
* Include sec type in Compare impl
2024-09-13 10:50:22 -04:00
John Colvin
552fec0edc
Add null check in SelectorList ctr
2024-09-13 10:50:22 -04:00
John Colvin
fcf6927fbf
cleanup: Remove unused code
2024-09-13 10:50:22 -04:00
John Colvin
23f6d827d5
refactor: Replace ThreadedBlackboard w/ IBus
...
In SingleVolSurfaceCommittedOutputRetriever
2024-09-13 10:50:22 -04:00
John Colvin
f083d0abab
Clean up VolSurfaceTimeParameters
...
* Enable nullable lang feat
* Use file-scoped namespace
* Implement ICloneable
2024-09-13 10:50:22 -04:00
John Colvin
a895e36922
cleanup: Remove unnecessary ternary op
2024-09-13 10:50:22 -04:00
John Colvin
59d7a72288
fix: Actually use (sec) type arg in to_security_expiration
2024-09-13 10:50:22 -04:00
John Colvin
7731f2f4d1
fix: Check security_type in vs exp Equals impl
2024-09-13 10:50:22 -04:00
John Colvin
41ae5897c0
fix: Maintain sec type when converting SecurityExpiration to vs exp
...
Avoids data loss
2024-09-13 10:50:22 -04:00
John Colvin
c7949ccfba
refactor: Get rid of sec type 'Option' default (magic const)
...
Better to be explict
2024-09-13 10:50:22 -04:00
John Colvin
567af79c7f
Fix selection binding issues in vs settings editor
...
This specifically addresses issues in the driver symbol grids.
Unfortunately, we can't bind directly to ComboCheckBox.SelectedItems
because the view is contained in TabControl. When the selected tab
changes, the previously selected tab clears its ItemsSource (and
consequently the SelectedItems) as a side effect of being unloaded.
2024-09-13 10:50:22 -04:00
John Colvin
6bb4a7b25c
refactor: Enable nullable (ref types) in ImmutableListExtensions
2024-09-13 10:50:22 -04:00
John Colvin
bc7cef4c34
Fix typo in namespace bts.collection -> bts.collections
2024-09-13 10:50:22 -04:00
John Colvin
ab9dc361d6
fix: Use ComboCheckBoxItem.IsSelectedProperty
...
Instead of Selector.IsSelectedProperty. Unlike ComboBoxItem, ComboCheckItem's
IsSelectedProperty isn't linked to Selector.IsSelectedProperty.
2024-09-13 10:50:21 -04:00
John Colvin
39da0a4dde
Add currency column to mview
2024-09-13 10:50:21 -04:00
John Colvin
44e864d697
fix: Use SetCurrentValue instead of SetValue
...
So we propagate values to/from the binding data source,
rather than overwrite the data source itself.
2024-09-13 10:50:21 -04:00
John Colvin
0a02da3aba
Add venue, sec type dropdown filters to exp params ui
2024-09-13 10:50:21 -04:00
John Colvin
836771a7f0
fix: Add null check in ComboCheckBox.SelectedItemsCollectionChanged
2024-09-13 10:50:21 -04:00
John Colvin
f386badcfb
fix: Disconnect old INotifyCollectionChanged source when dp changes
2024-09-13 10:50:21 -04:00
John Colvin
1d43a484db
fix: Index opt/fut exps by venue in vs analytics
2024-09-13 10:50:21 -04:00
John Colvin
91042cb41e
fix: Support multiple exchanges in exp params view/vm
2024-09-13 10:50:21 -04:00
John Colvin
10c0e4247d
cleanup: Remove unused 'option_symbol' param from compute_atm_data
2024-09-13 10:50:21 -04:00
John Colvin
8e34264e6d
fix: Filter by exch when getting atm data for sheets
...
This is necessary now that we support exchanges with overlapping securities
2024-09-13 10:50:21 -04:00
John Colvin
0a840d256b
fix: Remove sec id prop from SecurityExpiration
...
This class isn't designed to represent a particular contract; it's
basically a key to get option theos.
2024-09-13 10:50:21 -04:00
John Colvin
8a69c51cac
refactor: Add 'onDispatcher' param to 'ThreadedBlackboard.subscribe'
...
This will allow us to gradually move subscriptions off of the dispatcher
(render) thread. When the render thread is pegged, the ui is laggy and
unresponsive. The new param defaults to true, so it won't change
anything until we explicitly set it to false in 'subscribe' calls.
2024-09-13 10:50:21 -04:00
John Colvin
0262fd40bc
fix: Remove redundant SparkTimer threads
...
This addresses some slowness on the front end. Prior to this commit,
each ThreadedBlackboard would start its own timer thread. Since multiple
ThreadedBlackboard run concurrently, this multiplied the frequency of
the SparkTimer callbacks by the number of active ThreadedBlackboards.
The fix is to start this thread in an IShellDaemon rather than the
ThreadedBlackboard. This avoids duplicate threads because the former is
guarenteed to be instantiated only once, via dependency injection.
2024-09-13 10:50:21 -04:00
John Colvin
859dfe3544
refactor: Simplify FakeMDDaemon code that delays post
...
Use Task.Delay/ContinueWith instead of subscribing to the SparkTimer tag.
2024-09-13 10:50:21 -04:00
John Colvin
9c624a2f79
refactor: Use IBus instead of ThreadedBlackboard
...
This should address some slowness/mouse lag in the front end (recently
reported by Brandon at Immutable Cap)
2024-09-13 10:50:20 -04:00
John Colvin
dec1061b40
fix: Detect and log ccapi depth msg parse errors
...
This fixes a crash that would happen if a depth item's price or quantity
were an empty string (now we log an error and keep going).
2024-09-13 10:50:20 -04:00
John Colvin
be4895faa6
fix: Actually store currency in shared mem
...
This field has always been in the Security proto, though it until now it
had been omitted from the shared memory representation (FixedSecurity).
2024-09-13 10:50:20 -04:00
Mohammad Arham
3c9755f0dd
ADD: Added delay to databento file reader
2024-09-12 15:10:34 +00:00
Mohammad Arham
61ed847d7d
Add: Replay speed added to mdreplayer
2024-09-11 06:50:37 +00:00
Mohammad Arham
fc9094fea8
Added: CBOE working on databento
2024-09-11 06:17:26 +00:00
Mohammad Arham
e36c0efc4b
Added: GLBX working on databento
2024-09-11 06:16:42 +00:00
Mohammad Arham
4158a14cff
Added: option to send timestamp in onbook IMDListener
2024-09-11 06:15:40 +00:00
Mohammad Arham
201a45cba0
taifex deo
2024-09-05 00:59:00 +00:00
Mohammad Arham
ab31b9b253
Add: Multiple files working taifex
2024-09-03 15:15:55 +00:00
Adeel Khan
780ddc79a7
Add: Taifex I083 Message
2024-09-03 20:06:48 +05:00
Mohammad Arham
d5a2adaed6
fix: removed errernous code
2024-09-03 13:09:54 +00:00
Mohammad Arham
008ab3f068
Merge branch 'development.new_exchanges' of bitbucket.org:bluetradesys/spark into development.new_exchanges
2024-09-02 16:31:14 -04:00
Mohammad Arham
7d926f588d
FIX: typo, changed date to day
2024-09-02 16:31:10 -04:00
Mohammad Arham
707cd07876
Add: CBOE L1 books and contracts working
2024-08-28 16:22:51 +00:00
Mohammad Arham
f7ecb9090c
add: code to start working on cboe
2024-08-27 12:33:23 +00:00
Mohammad Arham
a19e8a7315
change: changes to make taifex work
2024-08-27 07:45:59 -04:00
John Colvin
cf189ecf81
refactor: Start cboe and taifex md 'start_databento'
...
Similar to how we start all ccapi implemented exchanges in "_start_cc"
(in MarketDataGateway)
2024-08-26 19:13:59 +00:00
John Colvin
4405b7adcb
refactor: Rename cc_md_listeners -> venue_listeners
...
They are general purpose, not specific to ccapi.
2024-08-26 19:09:57 +00:00
John Colvin
e51ee7b7b9
refactor: Wrap IDGenerator and IMarketDataListeners in shared_ptrs
...
Our various market data providers (ccapi, emumba, replay, onixs)
dereference pointers to common instances of these types concurrently on
different threads. Although we've correctly managed memory ownership
thus far, we really should be using smart pointers (this is why they
were made).
2024-08-26 19:03:29 +00:00
John Colvin
dfaa74f426
build: Add databento dep via FetchContent
...
Now we actually build libdatabento in the devcontainer along with our
code (as we do with the vcpkg deps, though this is not part of vcpkg)
2024-08-26 17:13:51 +00:00
Mohammad Arham
1d2ce59009
add: added taifex entries
2024-08-22 23:13:05 +00:00
Adeel Khan
3602cc05bb
added: TaifexDataHandler, temp
2024-08-23 02:06:47 +05:00
Adeel Khan
ac88a81ac6
Security Extensions
2024-08-23 01:42:24 +05:00
Adeel Khan
01fa8e0c63
added: files for Taifex, Cboe
2024-08-23 00:57:40 +05:00
Taha
322c06bfbe
Fixing build relted to BCG vs bts namespace
2024-08-21 16:24:22 -04:00
Taha
b68fceb362
Cleaning up FileStreams
2024-08-21 16:11:57 -04:00
John Colvin
59547e0f37
fix build: Lock fmt version to 10.2.1 in vcpkg
...
This fixes the front-end (Windows) libspark build
2024-08-21 13:45:43 -04:00
John Colvin
3ce100a9cf
build: Add databento to sparkd
...
* Install databento deps in docker
- libzstd-dev
* Add databento cmake module to externals
2024-08-21 13:49:53 +00:00
John Colvin
6513fb0cd5
cleanup: Remove unused include 'date.h' from MDSource
...
This was an external header, and may interfere with a dependency in the
databento lib (which we are about to add).
2024-08-21 13:49:24 +00:00
Taha Afzal
fd0fb1f760
Adding incremental book updates to IMarketDataListener
2024-08-12 22:49:27 -04:00
Taha Afzal
5db586c4d3
spaces, newlines?
2024-08-12 21:27:43 -04:00
Taha
31df1a3827
Added benzinga streamd to MDPub's test config
2024-07-17 14:32:57 -04:00
Taha
7e219aec7c
Merging
2024-07-17 02:06:17 -04:00
Taha
37f8b98df9
Icomplete SharedStreamsConfig was breaking build, fixing
2024-07-17 02:03:43 -04:00
Taha
f06beff22f
Added new benzinga streams
2024-07-17 02:02:01 -04:00
Taha Afzal
6aa39a716a
PubSubSingleTagState is not needed any more
2024-07-16 01:15:51 +00:00
Taha Afzal
5088b5ebe0
Fixing problems where PSS's client socket's IP was different than what the client was sending in PubSubClientSubscription
2024-07-08 23:09:53 +00:00
Taha Afzal
010dd3a26f
Fixing PubsubEchoTestStream
2024-07-07 06:13:22 +00:00
Taha Afzal
1a18755176
Added pss_echo()
2024-07-06 06:53:35 +00:00
Taha Afzal
233e2576af
Merging
2024-07-05 21:21:27 +00:00
Taha Afzal
543c655bca
src/DataStreamSuite/PubSub/ManagedTCP/AsyncTCPConnection_AsyncWrapper.cs
2024-07-05 21:13:36 +00:00
Taha
050e21e22b
Adding TableMetaDataStream
2024-07-04 23:39:04 -04:00
Taha Afzal
672ad6d237
Cleaning Config 1.0 (not completely done)
2024-06-30 23:17:20 -04:00
Taha
0a8f6e866f
Added pubsub routes to the Title of DSQA
2024-06-27 14:04:07 -04:00
Taha
b177ffef04
Fixing the bug where messages were being sent by the pubsubserver out of order
2024-06-27 00:17:22 -04:00
Taha
393f304be4
some buf gixes + cleanup
2024-06-26 00:23:19 -04:00
Taha
151816bf5e
PubSuMessageReactor empties the message queue (instead oif one elem ata a time) in the main loop
2024-06-24 02:29:28 -04:00
Taha
5cf7eacd82
Fixing the extra .0 in the default pubsub server routeplusfixing acorner cae issue of gettign a fast streaming data cakback after an unsubscribe
2024-06-24 00:28:07 -04:00
Taha
87b6a2044c
PubSubServer's linux issues fixed
2024-06-23 21:13:37 -04:00
John Colvin
087d8ab8c9
Set text=auto in gitattributes
...
Otherwise git can cause files to show as "permanently modified" when you
apply changes from the index (e.g., by running 'git rebase', or 'git
stash pop')
2024-06-11 17:39:12 +00:00
John Colvin
b69b7b3350
vscode: Use '\' dir separator in show_last_run.cmd
...
This script only runs on Windows
2024-06-08 15:08:25 -05:00
John Colvin
5620f33524
vscode: Add task to view logs for last PubSubServer run
2024-06-08 14:42:13 +00:00
John Colvin
f3e33fa390
vscode: Add task to run PubSubServer without building
2024-06-07 16:41:42 +00:00
John Colvin
3b631e0d5f
Fix a couple more typos in logs
2024-06-07 16:35:06 +00:00
John Colvin
f93d28ee76
vscode: Add task to view logs for last DSQA run
2024-06-07 11:34:36 -05:00
John Colvin
23d7d3775d
cleanup: Delete unused files
2024-06-07 10:40:34 -05:00
John Colvin
4940771166
fix: Don't login to sparkd from DSQA
2024-06-07 10:02:42 -05:00
John Colvin
c1967a8f6a
vscode: Add tasks to edit Spark, DataStream config
2024-06-06 11:12:56 -05:00
John Colvin
8c797ee76b
Fix typo in log msg
2024-06-06 09:21:01 -05:00
John Colvin
ce80957d07
DSQA shouldn't connect to sparkd by default
2024-06-06 09:21:01 -05:00
John Colvin
c60018b59e
DataStreamPublisher shouldn't connect to sparkd by default
...
It only needs to connect to sparkd if it's publishing market data
2024-06-06 09:21:01 -05:00
John Colvin
3a27af87d1
PubSubServer shouldn't download config from sparkd by default
...
As far as I know, it only needs spark/protobuf config to initialize the
c++ logging system. Eventually, we should get rid of that dependency and
use builtin Microsoft Logging facilities for c# instead. In the
meantime, PubSubServer will load config from local files by default
(i.e., spark.conf).
2024-06-06 09:21:01 -05:00
John Colvin
068c1c6973
cleanup: Remove erroneous comment
2024-06-06 09:21:01 -05:00
John Colvin
645a58c97d
vscode: Add launch.json
2024-06-06 09:21:01 -05:00
John Colvin
b7f455b29a
vscode: Remove launch/tasks.json from gitignore
2024-06-06 09:21:00 -05:00
John Colvin
4a2735839e
Change line endings to crlf in a few cs files
2024-06-06 09:21:00 -05:00
John Colvin
d2e9fe8237
Change line endings to crlf in a few cs files
2024-06-06 09:21:00 -05:00
John Colvin
cfc60766d8
Change line endings to crlf in a few cs files
2024-06-06 09:21:00 -05:00
John Colvin
b03fec7f79
Change line endings to crlf in a few cs files
2024-06-06 09:20:59 -05:00
John Colvin
fb8e7c5b81
Ensure crlf line endings for cs files
2024-06-06 09:20:05 -05:00
John Colvin
e44601d1b1
fix merge: Bring back few fns that SchemaGuesserStream needs
...
* BlueDataUtility.get_typed_schema
* BlueDataUtility.get_typed_table
2024-06-06 09:20:05 -05:00
Taha
07bade53c0
Fixing AsyncTCPCOnnection_APM's double partial messages for messages > 100 MB
2024-06-06 09:20:05 -05:00
Taha
fac0a9c25b
RemoteQueryStream
2024-06-06 09:20:05 -05:00
John Colvin
1d1de3dd18
Ensure proper line endings for sh, cmd scripts
...
Via new gitattributes file
2024-06-06 09:20:05 -05:00
John Colvin
cfcbf004ba
vscode: Use consistent naming conventions in tasks.json
2024-06-06 09:20:05 -05:00
John Colvin
78c7dadbbe
vscode: Add parameterless task variants
...
This allows us to build as a 'preLaunchTask' when debugging (without
requiring users to redundantly specify 'Debug' as the build type).
2024-06-06 09:20:01 -05:00
John Colvin
7b5a6031af
build: Use consistent naming conventions in CMake presets
...
We'll use similar naming conventions for our (vscode) tasks
2024-06-06 09:15:17 -05:00
John Colvin
c13a7782c8
build: Name sparkd binary dirs after cmake preset
...
All other builds follow this pattern. Given the fact that we have so
many different builds now (spark, sparkd, data-streams, api-server,
etc.), it behooves us to adopt consistent naming conventions.
2024-06-06 09:15:17 -05:00
John Colvin
45f02391d1
cleanup: Remove sparkd login creds from PubSubServerConfig
...
It only needs a Sparkd url to download logging config. TBH, we can probably
get rid of that too, and simply read the local spark.conf.
2024-06-06 09:15:17 -05:00
John Colvin
a3ca06addc
Reduce 'internal_publish' log level to Diag
...
It was a bit noisy for the Info level
2024-06-06 09:15:17 -05:00
John Colvin
32869e54ec
fix: Fix typos in c# Logs
...
Prefix a few msgs with '$' so vars in curly braces are expanded via
interpolation
2024-06-06 09:15:17 -05:00
John Colvin
ca73220a2e
Add new PubSub cfg to DSQA
...
At least the server address (port) needs to be exposed
2024-06-06 09:15:17 -05:00
John Colvin
9fc3c2618e
cleanup: Remove unnecessary reference to SharedStreams
...
From DataStreamPublisher
2024-06-06 09:15:17 -05:00
John Colvin
e72e0c0a30
fix: Make PubSubServer actually use sparkd url in config
2024-06-06 09:15:17 -05:00
John Colvin
a5bfc75278
cleanup: Fix a few nullability warnings in DSQA
2024-06-06 09:15:17 -05:00
John Colvin
261a184d25
refactor: Replace legacy Crono type w/ dotnet Stopwatch
2024-06-06 09:15:16 -05:00
John Colvin
54e96a06cb
build: Build DSQA on Windows in build_data_stream.py
...
It was only building DataStreamSuite.sln before, which is crossplatform
now so it no longer includes DSQA.
2024-06-06 09:15:16 -05:00
John Colvin
17e60df32b
cleanup: Remove unused file PubSub3Config.cs
2024-06-06 09:15:16 -05:00
John Colvin
c99d5fe24c
Add support for core PubSub config via json
2024-06-06 09:15:16 -05:00
John Colvin
6049826e90
build: Make build_data_stream.py cross platform
...
Previously it only worked on Windows
2024-06-06 09:15:16 -05:00
John Colvin
672149dd89
Add more descriptive logs (via stdout) during initialization
...
The C++ configure/initialize routines were particularly opaque
2024-06-06 09:15:16 -05:00
John Colvin
8c99147f04
build: Build cmake presets in distinct binary dirs
...
It may use more disk space, but we really don't have a choice. CMake
projects with different parameters, preprocessors, source files, etc.
are not designed to share binary dirs. For example, the libspark.so file
that the Spark front-end is supposed to use is not necessarily the same
as the libspark.so file the Api Server is supposed use.
2024-06-06 09:15:16 -05:00
John Colvin
4d4b0b7ae3
build: Give api server distinct shm prefix
2024-06-06 09:15:16 -05:00
John Colvin
1a7955c1b1
build: Remove unnecessary cache var/macro for api server
...
* BUILD_API_SERVER (cmake cache var)
* SPARK_ENABLE_API_SERVER (preprocessor defined conditionally w/ BUILD_API_SERVER)
They were originally added to control whether or not MD Consumer extern
fns were built. They have been made redundant by the more appropriately
named EXPORT_MD_CONSUMER cache var/macro (both have same name now).
2024-06-06 09:15:16 -05:00
John Colvin
f8fc2a1d96
cleanup: Remove static service Locator class
...
It relied on a static singleton 'ServiceContext'. Replace references
with a properly scoped (i.e., non-static) ServiceContext
2024-06-06 09:15:16 -05:00
John Colvin
9a7b3cb77c
refactor: Reduce code (simplify) in BTS.init
...
Remove the 'args' parameter, since the cmd line can be obtained from
Environment.GetCommandLine anyways
2024-06-06 09:15:16 -05:00
John Colvin
8a3e4edb4d
Dispose connections in SimpleTCPServer.Dispose
2024-06-06 09:15:15 -05:00
John Colvin
e59183bb1e
improvement: Allow app data dir to be overriden w/ env var
...
Historically, this has been hardcoded to ~/sparkdata on Linux, and
%APPDATA%/bts on Windows.
2024-06-06 09:15:15 -05:00
John Colvin
dd4d4e8dc6
fix: Expand path vars in cmake presets at runtime
...
The binaries should be portable to other machines (paths shouldn't
be fixed to the user's home dir at compile time).
2024-06-06 09:15:15 -05:00
John Colvin
f1744450ca
build: Add script to build api server
2024-06-06 09:15:15 -05:00
John Colvin
3a7827a75b
build: Automatically load vs dev prompt for win builds
2024-06-06 09:15:15 -05:00
John Colvin
8d22e91c0d
fix build: Remove references to HighPerformanceTimer
...
An old custom timer class from Edge that we no longer need. We're
replacing it w/ a dotnet builtin timer class called 'Stopwatch'.
2024-06-06 09:15:15 -05:00
John Colvin
c425145fc9
Remove mongo stream from MarketDataPublisher.json
...
It doesn't work unless you have mongodb running (a bit of a burden when you
just want to publish market data)
2024-06-06 09:15:15 -05:00
John Colvin
4855935ae0
improvement: Get PubSubRoutes from config (json)
...
Previously, we were using hardcoded default routes.
2024-06-06 09:15:15 -05:00
John Colvin
65f412ec9f
fix: Fix myLoadAllTypes impl
...
* Fix lowercase typo in hardcoded assemblies (case matters now that we
run on Linux)
* Fix resolution to full path (previous impl used deprecated functions,
and incorrectly appended relative paths)
2024-06-06 09:15:15 -05:00
John Colvin
40681c6093
Add MaybeNullWhen(false) attribute to TryGetFirst result
...
So callers can assume non-null when it returns true (for IEnumerables of
non-null ref types)
2024-06-06 09:15:15 -05:00
John Colvin
ab2d36e523
vscode: Add tasks.json
...
This provides a good base of operations to build/launch our now numerous
projects
2024-06-06 09:15:09 -05:00
John Colvin
b1ab3a66d2
build: Fix old build_sparkd.py script
...
Things have become a lot simpler since it was originally written.
2024-06-05 15:44:36 +00:00
John Colvin
5e7e7e8ef4
test: Add FilterStreamTests
2024-06-05 15:44:35 +00:00
John Colvin
9291d30ae9
build: Add cmake opt to export MDConsumerInterop fns
...
Now that we have all these different builds (sparkd, spark, api server,
data streams) - many of which are cross platform - we need a proper
toggle for this (relying on 'IF WIN32' preprocessors doesn't cut it)
2024-06-05 15:44:35 +00:00
John Colvin
549708fcc2
fix: Use ".dll" file ext when loading assemblies by path
2024-06-05 15:44:35 +00:00
John Colvin
1d5551f301
build: Add cmake preset to build data streams on linux
2024-06-05 15:44:35 +00:00
John Colvin
283db0f9ea
build: Define shm prefix and app data dir in CMakePresets
...
It's more organized this way, with the input variables for each
configuration defined in one place.
2024-06-05 15:44:35 +00:00
John Colvin
9fb04b2db9
cleanup: Remove unused cmake preset 'spark-api-server-base'
...
This was a hidden preset designed for common Linux/Windows config.
Neither Linux nor Windows configs were inheriting it.
2024-06-05 15:44:35 +00:00
John Colvin
ceca96e10a
Add build_data_stream.sh script for Linux
2024-06-05 15:44:35 +00:00
John Colvin
fc48cc64af
fix build (webapp): Add --include=dev arg to npm install
...
The apt repo finally updated npm from 8.X.X to 10.8.0. This broke the
Spark.Admin.WebApp build because the new version doesn't install dev
dependencies by default.
2024-06-05 15:44:35 +00:00
John Colvin
015d846941
feat: Add Pandas(Script)Stream
2024-06-05 15:44:35 +00:00
John Colvin
e39108bbda
fix build (sparkd): Fix typo
...
'git' was appended to a type name. I suspect a keyboard focus conflict
between vs code editor/terminal panes.
2024-06-03 13:46:51 +00:00
John Colvin
6240d89a08
Map PubSubServerConfig json to historic PubSub3Config
2024-06-03 13:46:51 +00:00
John Colvin
c76700e5e5
cleanup: Remove unused PubSubRoute array from DSQA
2024-06-03 13:46:51 +00:00
John Colvin
a8a0e8dea0
cleanup: Remove unused PubSubConfigureForm
2024-06-03 13:46:51 +00:00
John Colvin
d65a2d6ed6
test: Add DataPacketBuilderTests
...
This tests somes facilities used in the new MongoCollectionStream
2024-06-03 13:46:51 +00:00
John Colvin
1824d5f9a4
feat: Add [incremental] MongoCollectionStream
2024-06-03 13:46:51 +00:00
John Colvin
d3eb29087b
refactor: Impl IDictionary<int, IBlueRow> in BlueTable
2024-06-03 13:46:51 +00:00
John Colvin
d6fae238f9
Add IObservable ViewStream wrapper
2024-06-03 13:46:51 +00:00
John Colvin
44b46033e3
cleanup: Remove HighPerformanceTimer
2024-06-03 13:46:51 +00:00
John Colvin
5741ec9ab2
cleanup: Remove unnecessary usings (namespaces)
2024-06-03 13:46:51 +00:00
John Colvin
596975b309
style: Apply modern formatting to Crono
2024-06-03 13:46:51 +00:00
John Colvin
9050318ceb
refactor: Replace HighPerformanceTimer w/ Stopwatch
...
This is needed to build DataStreamSuite on Linux.
'Stopwatch' is a platform independent timer class included with the
System.Runtime library in modern dotnet. 'HighPerformanceTimer' is a BTS
class that predates Stopwatch, and is NOT platform independent as it
relies on Windows kernel32.dll QueryPerformanceCounter/Frequency fns.
2024-06-03 13:46:51 +00:00
John Colvin
7ee6933789
refactor: Add formal config classes for prepublish streams
...
They can be passed to command line via --config <json_file>
2024-06-03 13:46:51 +00:00
John Colvin
306c572b53
improvement: Polish clis in DataStreamSuite
...
Specifically PubSubServer, DataStreamPublisher and DSQA
* Use CommandLineParser (nuget pkg)
* Clean up csproj files a bit while we're at it
2024-06-03 13:46:51 +00:00
John Colvin
63306173b1
refactor: Reorganize NumericComparer, support all types
...
Get rid of redundant UniversalNumericComparer while we're at it
2024-05-25 15:37:33 +00:00
John Colvin
df1b45ca3e
cleanup: Remove leftover legacy files
2024-05-25 15:37:33 +00:00
John Colvin
c31a1f1bf3
cleanup: Switch assert statements to 'constraint model'
...
This gets rid of a few more build warnings in DataStreamSuite
2024-05-25 15:37:33 +00:00
John Colvin
e95997ea17
cleanup: Chip away a few nullability warnings in DataStreamSuite
...
This is a start, but there're still > 700 of them!
2024-05-25 15:37:33 +00:00
John Colvin
12a070fd8d
build: Remove DSQA from DataStreamSuite.sln to its own sln
...
So we can build DataStreamSuite.sln on Linux (DSQA depends on WinForms)
2024-05-25 15:37:33 +00:00
John Colvin
e0459c54fd
build: Fix 'pubsub' -> 'PubSub' typo in csproj ProjectReferences
...
Now all DataStream projects build on Linux (except DSQA, which is a winforms app)
2024-05-25 15:37:33 +00:00
John Colvin
15262a5a95
build: Move winform deps from BlueRegistry.csproj to DSQA
...
So we can build all DataStream projects (except DSQA) on Linux
2024-05-25 15:37:33 +00:00
Taha Afzal
90ec64d25c
deleting old serializers
2024-05-25 15:37:33 +00:00
Taha Afzal
061481419e
Streams can be published and subscribed now
2024-05-25 15:37:33 +00:00
Taha Afzal
8035255868
Cleaned up the cashe held by PublisherCore
2024-05-25 15:37:33 +00:00
Taha Afzal
7feaf869dc
Stream serialization revamp, all tests pass (TestsStream.s)
2024-05-25 15:37:33 +00:00
Taha Afzal
7104e90fc9
Adding tests for stream serialization
2024-05-25 15:37:33 +00:00
Taha Afzal
52c1e500e6
Re-fixing PubSubserver exception caused by null server in internal tags
2024-05-25 15:37:33 +00:00
Taha Afzal
07274f897a
Pubsub stats
2024-05-25 15:37:33 +00:00
Taha Afzal
f626e53415
CPublisherCore, SubscriberCore, PubsubMessageReactor claned
2024-05-25 15:37:33 +00:00
Taha Afzal
5a7b2965be
PubUBState cleaned up and modernized
2024-05-25 15:37:33 +00:00
Taha Afzal
0aab1628bc
removing old handlers from PubsubMessageReactor
2024-05-25 15:37:33 +00:00
Taha Afzal
b9649e1d8e
Clearing up PubSubMessageReactor
2024-05-25 15:37:33 +00:00
Taha Afzal
3bc9bf2207
Starting the server cleanup work
2024-05-25 15:37:33 +00:00
Taha Afzal
888c06262b
PublisherCore cleaned up
2024-05-25 15:37:33 +00:00
Taha Afzal
03e80dabcb
CHanging member names from old style
2024-05-25 15:37:33 +00:00
Taha Afzal
54184d0875
Cleaning up ISubscriberCore to use ne PubSubClientMssage class
2024-05-25 15:37:33 +00:00
Taha Afzal
90bcf5a455
Adding testss for PubSubMessages
2024-05-25 15:37:33 +00:00
Taha Afzal
d32d5ef40c
Cleaning up PubSubMessages, as types messages instead of PubSubMetaData untypes dictionary
2024-05-25 15:37:33 +00:00
Taha Afzal
2e0e0658c6
Fixing merging
2024-05-23 10:34:40 -04:00
Taha Afzal
bbdb50a41e
FIX bridge for Trafix
2024-05-23 10:34:40 -04:00
John Colvin
90705cac83
Integrate theos into Algo Testing Framework
...
(For Laguna)
2024-05-13 14:28:16 +00:00
John Colvin
b91cdd6d91
Expose get/set test secs fn for Algo Testing Framework
...
Laguna needs this. Historically, the test securities were extracted from
coconut md replay files. Eventually, we'll extract them from the new
replay file format, but until then, this gives Laguna a way to manually
set securities for algo tests (I hardcoded 1 ES sec for convenience by
default).
2024-05-13 14:23:31 +00:00
Mohammad Arham
64f2bd7a88
removed quoting algo so can be loaded dynamically
2024-04-04 20:38:15 -04:00
Mohammad Arham
3e32f684e3
removed redundant code
2024-04-03 16:56:31 -04:00
Mohammad Arham
ff73756a55
removed assert if delete order that is completed
2024-04-03 16:55:29 -04:00
Mohammad Arham
1e64811d93
changed error code
2024-04-03 16:54:51 -04:00
John Colvin
2488597cf2
fix build (linux api server): Remove srcs that moved to sparkd
2024-04-02 16:25:13 +00:00
John Colvin
2d8047ba80
build: Tidy up build/publish_spark scripts
...
In preparation for the first deployment of the "development" branch.
2024-04-02 01:47:47 -05:00
John Colvin
3cb602b77e
dev: Remove irrelevant comment from devcontainer.json
...
It was added by accident
2024-04-02 06:10:25 +00:00
John Colvin
51e1005e79
dev: Fix compile-commands-dir in devcontainer.json
...
It should be '${workspaceFolder}/build/clang/debug'
2024-04-02 06:01:14 +00:00
John Colvin
ef7ea64973
dev: Add -y to 'apt install libpcap' in Dockerfile
...
Fixes the docker build
2024-04-02 05:57:03 +00:00
Mohammad Arham
0267c16e5a
Merge branch 'development' of bitbucket.org:bluetradesys/spark into development
2024-04-01 19:02:32 -04:00
Mohammad Arham
ca40e278ff
quoting crash fix and comments
2024-04-01 19:01:27 -04:00
John Colvin
1c5da7ad48
cleanup: Tidy up hdrs included in cpp algo sdk
2024-04-01 10:09:35 -04:00
John Colvin
9bb3d5c569
refactor: Move ChildOrder defs to cpp file
...
So deps can be omitted in header
2024-04-01 10:09:35 -04:00
John Colvin
92ba43bfad
refactor: Move ccapi logger into its own file
2024-04-01 10:09:35 -04:00
John Colvin
ddeb5e59f0
fix: Add missing exchs to risk webapp
2024-04-01 10:09:35 -04:00
John Colvin
6b0b3e31cc
cleanup: Move nested Trade class into its own file
...
Cut down on dep clutter
2024-04-01 10:09:35 -04:00
John Colvin
3f6c904ccc
cleanup: Get rid of redundant SecurityKey def
...
Make bts::Security an alias for bts::md::Security instead
2024-04-01 10:09:35 -04:00
John Colvin
d433919f1b
refactor: Move OrderBuilder defs to cpp file
...
So impl deps don't leak into the sdk through headers
2024-04-01 10:09:35 -04:00
John Colvin
75da9af39d
improvement: Add fmt impl for SecurityKey
2024-04-01 10:09:34 -04:00
John Colvin
22eff44926
cleanup: Tidy up Book.h
2024-04-01 10:09:34 -04:00
John Colvin
cd6a052d0c
cleanup: Tidy up IAlgoOrder, add fmt impl for ChildState
...
Part of a larger effort to polish the sdk for laguna
2024-04-01 10:09:34 -04:00
John Colvin
3b6cf0b187
cleanup: Remove unnecessary includes
2024-04-01 10:09:34 -04:00
John Colvin
c170dd7a9e
refactor: Include ostream in EnumMacros.h
...
So it doesn't need to be explicitly included to support ENUM_CLASS_DEF
2024-04-01 10:09:34 -04:00
John Colvin
154775bc70
refactor: Move misplaced code from libspark to sparkd
...
This will reduce the dependency load on libspark and the c++ algo sdk.
This has been on the backburner for a while; we're doing it now so we
can deliver a clean sdk to Laguna.
Migrating the following code:
* general server-side md utils (MarketDataGateway, etc.)
* emumba dependent code, including md impl for:
- deribit
* ccapi dependent code, including md impl for
- binance
- binance_us
- binance_usds_futures
- ftx
- coinbase
- okx
- deribit
* md recording
Note: I first began this migration to eliminate the c++ algo sdk's
wependency on emumba. There seemed to be a conflict related to ssl
versions (the build in the sdk docker would fail with unresolved
external symbols for OPENSSL macros)
2024-04-01 10:09:34 -04:00
John Colvin
7f936f3ba9
fix: Inline defs in hdr file
2024-04-01 10:09:24 -04:00
John Colvin
9b30a947a8
fix build: Add a few includes, suppress some werrors
...
Not sure how this slipped through the cracks, but I'm guessing the build
broke when we upgraded the cpp compilers (to clang 17 and gcc 13).
* Include <cstdint> in a few places
* Suppress -Wstringop-overflow in sparkd-gcc-release build
* Change const char * param to explicit std::string for proper type
deduction in call to 'std::make_tuple'. For some reason, the gcc build
was failing with an array-bounds werror (changing the arg fixed it).
2024-04-01 10:09:24 -04:00
John Colvin
2ee58d8f99
dev: Add devcontainer.json (for linux)
2024-04-01 10:09:24 -04:00
John Colvin
0537b3e9c7
cleanup: Remove mongodb.js file that was added by accident
2024-04-01 10:09:24 -04:00
John Colvin
8ae940bcbc
fix: Change "init" call to "initAsync" in spark_api_server main
...
WebsocketDataSync.init was refactored to initAsync
2024-04-01 10:09:23 -04:00
Mohammad Arham
966ccb5f2c
fixed order entry
2024-04-01 07:48:36 -04:00
Mohammad Arham
fae9c3e59e
quantity connected, fills reported, limit set as default
2024-04-01 07:46:36 -04:00
Mohammad Arham
cb1dfb257d
updated defaults to have max_order_msgs_per_window
2024-03-29 16:00:36 -04:00
Mohammad Arham
76c3fe86c2
quote algo quantity hooked
2024-03-29 14:42:15 -04:00
Mohammad Arham
7e3b1de70f
bug fix
2024-03-29 00:38:33 -04:00
Mohammad Arham
2a07e3cf3c
custom orders working on api
2024-03-29 00:37:04 -04:00
Mohammad Arham
6d0249825d
quotingalgo working
2024-03-29 00:35:49 -04:00
Mohammad Arham
63810e3966
build fix
2024-03-26 18:07:48 -04:00
Mohammad Arham
4c013a8886
reorganise code
2024-03-26 18:07:01 -04:00
Mohammad Arham
9509818978
set theos
2024-03-26 18:06:07 -04:00
Mohammad Arham
4fdf3b8b60
just cleaner code
2024-03-26 02:10:06 -04:00
Mohammad Arham
ee9a58486d
cleaner code same logic
2024-03-25 02:52:51 -04:00
Mohammad Arham
75d6939c8b
added new rules for onix decode ticks
2024-03-25 01:48:22 -04:00
Mohammad Arham
fb1697b571
spelling fix
2024-03-24 23:17:02 -04:00
Mohammad Arham
4080d606d8
fixed get trades
2024-03-24 23:16:38 -04:00
Mohammad Arham
7655fd47dd
fixed spelling
2024-03-24 23:14:11 -04:00
Taha Afzal
d029b0fce7
SimExecution works with GLBX now
2024-03-18 23:54:19 -04:00
John Colvin
60df5b2a68
refactor: Move OrderDataHelper to spark_api.csproj
...
So it can be called from the websocket api server to parse
CustomOrderDefinitions
2024-03-13 12:45:17 -05:00
Taha Afzal
bada5dc961
Fixing SimExecution for CME
2024-03-06 14:11:02 -05:00
Taha Afzal
4510e120ab
enabling some streams like max, that were dependent on AddCOlumns (thta John fixed in the last commit)
2024-02-12 13:21:28 -05:00
Taha Afzal
da2af06ee6
Merging
2024-02-08 15:48:51 -05:00
Taha Afzal
8c1342230b
RowDiff added
2024-02-08 14:11:10 -05:00
Taha Afzal
8dd2775092
Added volatility to trade bars
2024-02-04 23:22:52 -05:00
Taha Afzal
8966b73ac9
Merge branch 'development' of bitbucket.org:bluetradesys/spark into development
2024-02-04 18:04:06 -05:00
Taha Afzal
ddb2a91763
Adding TradBar stream
2024-02-04 18:03:51 -05:00