Commit Graph
14413 Commits
Author SHA1 Message Date
John Colvin 4221cb879d feat: Add server side csharp sdk
For Ophil
2025-12-06 17:35:08 +00:00
John Colvin 410a07da28 refactor: Move position model to spark_api
So we can use it in the new sdk (c# server-side api) for Ophil
2025-12-06 17:32:11 +00:00
John Colvin e7e37a64eb diag: Fix typo in TableAccumulator2 log 2025-12-05 15:56:24 +00:00
John Colvin 2b80c79eae check pt: Make subscriber core shared (so all observers see every update) 2025-12-05 15:56:24 +00:00
John Colvin af452e8fff refactor: Move RefCountMap into spark_core 2025-12-05 15:56:24 +00:00
John Colvin 45994a2bb9 fix use dependency injection in dsqa 2025-12-05 15:56:23 +00:00
John Colvin 5a0b90278b cleanup: Remove unused code 2025-12-05 15:56:23 +00:00
John Colvin 861f60a342 fix: Publish data for all subscribed tags (not just the first one)
Remove erroneous early return
2025-12-05 15:56:23 +00:00
John Colvin 23503c2d50 diag: Reduce log noise in pubsub 2025-12-05 15:56:23 +00:00
John Colvin 7798dfd825 improve: Reduce lock contention in RemoteQueryManager 2025-12-05 15:56:23 +00:00
John Colvin b7f600b968 diag: Use structured logging in pubsub/streams 2025-12-05 15:56:23 +00:00
John Colvin 4dbb88da4d diag: Preserve file and line number in "log_at" ext fn 2025-12-05 15:56:23 +00:00
John Colvin 25ae34b970 cleanup: Remove unused code 2025-12-05 15:56:23 +00:00
John Colvin 0f5fa229c3 cleanup: Remove unnecessary code from AsyncTCPConnectionAcceptor 2025-12-05 15:56:23 +00:00
John Colvin a17596d4a6 refactor: Decouple PublisherStream from PubTableBroker (impl) 2025-12-05 15:56:23 +00:00
John Colvin 62ebe1f1a2 dev: Add tools/py to PYTHONPATH 2025-12-05 15:56:23 +00:00
John Colvin 76dc178e14 cleanup (build): Remove unnecessary CommandLineParser dep from csprojs 2025-12-05 15:56:23 +00:00
John Colvin a1e905cf0a fix add pubsub service modules 2025-12-05 15:56:23 +00:00
John Colvin ada00edf81 build: Remove HtmlAgilityPack dependency from DataStream
I was experiencing strange runtime errors, not sure if this nuget pkg is to blame,
but I'm removing it from the csproj to rule it out. It was only needed for the
WebScrapingStream (of course, we can bring it back if we want).
2025-12-05 15:56:23 +00:00
John Colvin 8da6fd8613 fix: Check book level index before accessing 2025-12-05 15:56:23 +00:00
John Colvin 02fb78b091 improve: Check security.HasContractSpec before accessing prop
All securities "should" have a (non-null) contract spec, but might as
well make the system resilient now that we are integrating md client w/
streams, replay files, etc.
2025-12-05 15:56:23 +00:00
John Colvin 8080f61653 refactor: Decouple IMarketDataClient trades from shared mem
So we can use it in BlueOrb.
2025-12-05 15:56:23 +00:00
John Colvin 70bd9af80b deploy: Lower default verbosity
The dotnet builds were too noisy
2025-12-05 15:56:23 +00:00
John Colvin 2ecf2000e2 deploy: Update sparkdata path to ~/.local/share in Dockerfile 2025-12-05 15:56:23 +00:00
John Colvin e2e39f9d2e fix build: Copy libspark to "net8.0" dir (was net7.0) 2025-12-05 15:56:23 +00:00
John Colvin b96027ae4f diag: Change log dir to ~/.local/share/bts/<exe>/data/logging 2025-12-05 15:56:23 +00:00
John Colvin 3e12a6570c config: Disable PubSub by default in Spark appsettings.json
Rename the key "Enabled" -> "IsEnabled" to be consistent with other appsettings
2025-12-05 15:56:23 +00:00
John Colvin 832c9b51ce cleanup: Change default DSQA pubsub config server to 127.0.0.1 2025-12-05 15:56:23 +00:00
John Colvin 313a6fb808 refactor: Support null values in Update, TableUpdate ctr 2025-12-05 15:56:23 +00:00
John Colvin f39a41bfcc build: Add local appsettings config to a couple csprojs
* DataStreamPublisher
* DataStreamQueryAnalyzer
* PubSubServer

Also modify gitignore to exclude DataStreamSuite local appsettings
2025-12-05 15:56:23 +00:00
John Colvin 9a3d996e71 fix: Use List instead of Array in WorkScheduler
I should have used a list in the first place...the array wasn't getting trimmed
after removing workers, which was causing null refs at the back of the array.
2025-12-05 15:56:23 +00:00
John Colvin 7fae875ee5 refactor: Hide AdminClient behind interface 2025-12-05 15:56:23 +00:00
John Colvin 02356cf97c refactor: Make AtomicGate actually atomic (lockless) 2025-12-05 15:56:23 +00:00
John Colvin 2cdeab394b vscode: Add "Local" pub sub config option to tasks.json 2025-12-05 15:56:23 +00:00
John Colvin 3684c2e781 vscode: Add PubSub config env var to launch.json 2025-12-05 15:56:23 +00:00
John Colvin 0674475566 refactor: Hide OrdersClient behind interface 2025-12-05 15:56:23 +00:00
John Colvin 546a52cb02 feat: Add historic md streams and replay client 2025-12-05 15:56:23 +00:00
John Colvin 2b1a1f262c fix build (api server): Fix typo DOTNET_BINARY_DIR cmake var (path)
This is the dir into which libspark (c++) is copied for dotnet (c#) exes.
2025-12-05 15:56:23 +00:00
John Colvin ea8bd4082f cleanup: Remove redundant IBlueTableSchema 'ResolveIndices' ext fns 2025-12-05 15:56:23 +00:00
John Colvin 4de85a978e refactor: Move a couple ViewStream extensions into DataStreamCore 2025-12-05 15:56:23 +00:00
John Colvin 4a42bc1600 fix: Change DataPacket to IDataPacket 2025-12-05 15:56:23 +00:00
John Colvin 3dcde6d089 refactor: Remove spark api server dependency on SecurityMD
This is part of a larger effort to decouple the api server from spark grpc
(protobuf) services
2025-12-05 15:56:23 +00:00
John Colvin e0e9989eda refactor: Move ObservableViewStream into DataStreamCore 2025-12-05 15:56:23 +00:00
John Colvin 5f4a811598 feat: Add ParquetReader/Writer streams 2025-12-05 15:56:23 +00:00
John Colvin dd136afad7 chore: Copy/paste some log utils from py interop dir to tools/py 2025-12-05 15:56:23 +00:00
John Colvin 4591755423 improve: Support arbitrary columns in py table_formatter
Not just key/values
2025-12-05 15:56:23 +00:00
John Colvin 7490fd107f refactor: Support dict and list of tuples in py fmt_table 2025-12-05 15:56:23 +00:00
John Colvin 5fb64e114c refactor: Hide md client behind interface, create via dep injection 2025-12-05 15:56:21 +00:00
John Colvin 0571429375 refactor: Decouple ViewStreamDatabase from TableStream, ISubscriberCore 2025-12-05 15:54:03 +00:00
John Colvin 8d10d512dd cleanup: Drop ".exe" from log dir name on windows 2025-12-05 15:54:03 +00:00