Commit Graph
13470 Commits
Author SHA1 Message Date
John Colvin e93f970706 SparkApiService check pt 2024-09-18 14:02:36 +00:00
John Colvin 844677dae0 pubsub server checkpt 2024-09-18 14:02:36 +00:00
John Colvin 3541feef87 pubsub checkpt 2024-09-18 14:02:36 +00:00
John Colvin 776a589945 datastreamwebapp server checkpt 2024-09-18 14:02:36 +00:00
John Colvin 396f72d7c1 datastream webapp client check pt 2024-09-18 14:02:35 +00:00
John Colvin 62a9917a48 fix add pubsub services 2024-09-18 14:02:35 +00:00
John Colvin a3dd7c3815 Remove unused PubSubConfig members from MarketDataPublisher cfg json 2024-09-18 14:02:35 +00:00
John Colvin e669cf6b77 Remove unused PubSubConfig members from PubSubServer cfg json 2024-09-18 14:02:35 +00:00
John Colvin c5e3c4aa18 fix Remove unused PubSubConfig members 2024-09-18 14:02:35 +00:00
John Colvin be35ed120b Remove unused PubSubConfig members from DSQA config json 2024-09-18 14:02:35 +00:00
John Colvin 9846bf3936 Remove unused PubSubConfig members from WebApp config json 2024-09-18 14:02:35 +00:00
John Colvin cb5fcee2ce Remove unused PubSubConfig props 2024-09-18 14:02:35 +00:00
John Colvin 08319cfd38 Fix dsqa config json 2024-09-18 14:02:35 +00:00
John Colvin 777c276437 fix: Always call 'handle_internal_pubs' in PubSubMessageReactor main loop 2024-09-18 14:02:35 +00:00
John Colvin 5ab0e1f886 improvement: Accept CancellationToken, catch exceptions when running ds websocket server 2024-09-18 14:02:35 +00:00
John Colvin a90896f606 Fix mismatched types in PubSubServerTagStatsStream schema 2024-09-18 14:02:35 +00:00
John Colvin 3f6f5ab0e0 Add DataStreamPublisher service modules designed for Dependency Injection
* Change DataStreamPublisher json fields to match cs class property names verbatim
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
2024-09-18 14:02:35 +00:00
John Colvin 24e5683ad2 refactor: Decouple ViewPublisher from IPubTableBroker impl 2024-09-18 14:02:35 +00:00
John Colvin 34999d0a5a Add PubSub service modules designed for Dependency Injection
* Get rid of JsonPropertyName attributes in PubSubConfig
  - The Dependency Injection framework (based around dotnet ApplicationBuilder)
    does not support this attribute, and maps property names verbatim to json fields.

* Change PubSubServer json fields to match cs class property names verbatim
2024-09-18 14:02:35 +00:00
John Colvin ace7ae2695 refactor: Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl 2024-09-18 14:02:35 +00:00
John Colvin f9016907e6 refactor: Decouple ManagedSubscriber from ISubscriberCore impl 2024-09-18 14:02:35 +00:00
John Colvin 0913fe28d4 refactor: Only support 1 tag type in IPubSubServerTagHandler 2024-09-18 14:02:35 +00:00
John Colvin 400b4ff848 refactor: Decouple PubTableBroker from IPublisherCore impl
Reference interface rather than the PublisherCore impl.
2024-09-18 14:02:35 +00:00
John Colvin 5a36a5f4d4 improvement: Accept CancellationToken in JsonRpcRequestObservable 2024-09-18 14:02:35 +00:00
John Colvin 0f6539ed00 fix: Dispose market data sub in various md streams 2024-09-18 14:02:35 +00:00
John Colvin 475bf1effc cleanup: Remove unused pubsub tag handler code
* Removed 'IPubSubServerTagHandler.register'
  - All implementations of this function were empty
* Removed 'IPubSubServerTagHandler.supportedTagTypes'
  - This property wasn't referenced anywhere
* Remove IWorker inheritance from IPubSubServerTagHandler
  - No reason these interfaces have to be tied together
2024-09-18 14:02:35 +00:00
John Colvin ec1e657687 refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
This matches the file name
2024-09-18 14:02:35 +00:00
John Colvin 284c84dfe6 Change pub sub schema prefix 'bluto://' -> 'bts-cloud://'
In config json files (not actual code change)
2024-09-18 14:02:35 +00:00
John Colvin b78c61bf44 fix build: Fix prop names after refactoring PubSubRoute
* Change 'server_address' -> 'ServerAddress'
* Change 'tag_reg_exp' -> 'TagRegExp'
2024-09-18 14:02:35 +00:00
John Colvin 09a41775bb build: Add scripts to publish data stream apps 2024-09-18 14:02:35 +00:00
John Colvin b9ef3ddf91 Fix typo in log msg 2024-09-18 14:02:35 +00:00
John Colvin e516496fe8 Fix build scripts so they propagate the exit code back to the shell
I noticed this was broken because launch.json configurations would still start the debugger
even when the preLaunchTask (build) failed!
2024-09-18 14:02:35 +00:00
John Colvin a55705db75 vscode: Add DataStreamPublisher debug cfg to skip build 2024-09-18 14:02:35 +00:00
John Colvin e39ad91e1c build: Use separate cmd to build DataStreamSuite/QueryAnalyzer.sln
The DataStreamSuite.sln builds the new webapp now, which will
(eventually) replace DSQA. We'll keep it around for now, but with a
clean separation from the main build.
2024-09-18 14:02:35 +00:00
John Colvin 658472b2a8 feat: Add DataStream WebApp
Included in DataStreamSuite.sln, but not DataStreamQueryAnalyzer.sln
2024-09-18 14:02:35 +00:00
John Colvin 1ad94cfe49 Add spark service modules designed for Dependency Injection 2024-09-18 14:02:35 +00:00
John Colvin 22a5a9a4c1 build: Copy libspark to all dotnet dirs in single cmake target
This is cleaner than the previous impl, which used separate build
targets to copy libspark to "net7.0", and "net7.0-windows" (dirs).
2024-09-18 14:02:35 +00:00
John Colvin e5bbc26da5 fix: Parse ServerAddress as normal string
The "JsonValueOrArrayConverter<T>" doesn't work the way I thought it
would. When it is associated with a property via the JsonConverter
attribute, dotnet json parsing code throws an exception if "T" isn't
assignable to the property type.
2024-09-18 14:02:35 +00:00
John Colvin 6a628a3070 fix build (api server): Change namespace to bts.websocket_core 2024-09-18 14:02:35 +00:00
John Colvin ac120658eb Remove docker submodules again 2024-09-18 14:02:35 +00:00
John Colvin 85a36384ce Change externals url from ssh to https style
So we don't need ssh keys to clone it (the builtin credential manager
makes https authentication safe/easy)
2024-09-18 14:02:35 +00:00
John Colvin 563833bc67 cleanup: Fix benign typos in build scripts
Change BUILD_SPARK_PY var name to BUILD_PY (because these scripts to actually build 'spark')
2024-09-18 14:02:35 +00:00
John Colvin d77ac1600b fix build (warn): Only use Registry on windows 2024-09-18 14:02:35 +00:00
John Colvin 3b41a2065a cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2024-09-18 14:02:35 +00:00
John Colvin f993f01634 cleanup: Normalize indentation in TradeBarsStream 2024-09-18 14:02:35 +00:00
John Colvin e7067a708b cleanup: Disable nullability warnings
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2024-09-18 14:02:35 +00:00
John Colvin 04ac2b9d3d cleanup: Modernize PubSubRoute code, json parsing 2024-09-18 14:02:35 +00:00
John Colvin 66c6b0bf8c cleanup: Remove unused DateTime grammar from exp parser 2024-09-18 14:02:34 +00:00
John Colvin 03dd4000fe Add JsonRpcRequestObservable, AsyncEnumerable 2024-09-18 14:02:34 +00:00
John Colvin e928295e71 Add json facilities for DataStream websocket api 2024-09-18 14:02:34 +00:00