Commit Graph
13390 Commits
Author SHA1 Message Date
John Colvin ea2d9441cd pubsub checkpt 2024-07-14 13:17:29 -04:00
John Colvin 18ee4724fe datastreamwebapp server checkpt 2024-07-14 13:17:29 -04:00
John Colvin 96e0823811 datastream webapp client check pt 2024-07-14 13:17:29 -04:00
John Colvin 3bda3d1b08 fix add pubsub services 2024-07-13 00:03:21 -04:00
John Colvin 87f656a489 Remove unused PubSubConfig members from MarketDataPublisher cfg json 2024-07-13 00:02:26 -04:00
John Colvin 91ccbc138e Remove unused PubSubConfig members from PubSubServer cfg json 2024-07-13 00:02:06 -04:00
John Colvin 5cab7f11b8 fix Remove unused PubSubConfig members 2024-07-12 23:41:00 -04:00
John Colvin 6f34a014ed Remove unused PubSubConfig members from DSQA config json 2024-07-12 23:28:16 -04:00
John Colvin 2a6fb6340c Remove unused PubSubConfig members from WebApp config json 2024-07-12 23:22:46 -04:00
John Colvin 913578a2d7 Remove unused PubSubConfig props 2024-07-12 23:22:22 -04:00
John Colvin 572d058817 Fix dsqa config json 2024-07-12 18:28:48 -04:00
John Colvin 79b984f575 fix: Always call 'handle_internal_pubs' in PubSubMessageReactor main loop 2024-07-12 18:28:48 -04:00
John Colvin a8808a043e improvement: Accept CancellationToken, catch exceptions when running ds websocket server 2024-07-12 18:28:48 -04:00
John Colvin 1cc8e901e7 Fix mismatched types in PubSubServerTagStatsStream schema 2024-07-12 18:28:48 -04:00
John Colvin c29eaa1a53 Add DataStreamPublisher service modules designed for Dependency Injection
* Change DataStreamPublisher json fields to match cs class property names verbatim
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
2024-07-12 18:28:48 -04:00
John Colvin 252fea8466 refactor: Decouple ViewPublisher from IPubTableBroker impl 2024-07-12 18:28:48 -04:00
John Colvin 4ac1382751 Add PubSub service modules designed for Dependency Injection
* Get rid of JsonPropertyName attributes in PubSubConfig
  - The Dependency Injection framework (based around dotnet ApplicationBuilder)
    does not support this attribute, and maps property names verbatim to json fields.

* Change PubSubServer json fields to match cs class property names verbatim
2024-07-12 18:28:47 -04:00
John Colvin 8ceaffe488 refactor: Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl 2024-07-12 18:28:47 -04:00
John Colvin cef5a636c6 refactor: Decouple ManagedSubscriber from ISubscriberCore impl 2024-07-12 18:28:47 -04:00
John Colvin 334b9fd6b8 refactor: Only support 1 tag type in IPubSubServerTagHandler 2024-07-12 18:28:47 -04:00
John Colvin 58b4266a27 refactor: Decouple PubTableBroker from IPublisherCore impl
Reference interface rather than the PublisherCore impl.
2024-07-12 18:28:47 -04:00
John Colvin 0ba85cdf3d improvement: Accept CancellationToken in JsonRpcRequestObservable 2024-07-12 18:28:47 -04:00
John Colvin dcd534e3e5 fix: Dispose market data sub in various md streams 2024-07-12 18:28:47 -04:00
John Colvin a214edd44f cleanup: Remove unused pubsub tag handler code
* Removed 'IPubSubServerTagHandler.register'
  - All implementations of this function were empty
* Removed 'IPubSubServerTagHandler.supportedTagTypes'
  - This property wasn't referenced anywhere
* Remove IWorker inheritance from IPubSubServerTagHandler
  - No reason these interfaces have to be tied together
2024-07-12 18:28:47 -04:00
John Colvin 23c687d3d9 refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
This matches the file name
2024-07-12 18:28:47 -04:00
John Colvin 7445a1777f fix build: Update vcpkg
On some vs versions, vcpkg cmake scripts would fail w/ an error saying
"Could not locate a complete Visual Studio instance"
2024-07-12 18:28:47 -04:00
John Colvin 8a81d44818 Change pub sub schema prefix 'bluto://' -> 'bts-cloud://'
In config json files (not actual code change)
2024-07-12 18:28:47 -04:00
John Colvin 22fbc0001f fix build: Fix prop names after refactoring PubSubRoute
* Change 'server_address' -> 'ServerAddress'
* Change 'tag_reg_exp' -> 'TagRegExp'
2024-07-12 18:28:47 -04:00
John Colvin 3ec863f02c build: Add scripts to publish data stream apps 2024-07-12 18:28:47 -04:00
John Colvin c43151a28c Fix typo in log msg 2024-07-12 18:28:46 -04:00
John Colvin 7d2e7e4665 Fix build scripts so they propagate the exit code back to the shell
I noticed this was broken because launch.json configurations would still start the debugger
even when the preLaunchTask (build) failed!
2024-07-12 18:28:46 -04:00
John Colvin 0ffc1a1a47 vscode: Add DataStreamPublisher debug cfg to skip build 2024-07-12 18:28:46 -04:00
John Colvin 6c4364989c build: Use separate cmd to build DataStreamSuite/QueryAnalyzer.sln
The DataStreamSuite.sln builds the new webapp now, which will
(eventually) replace DSQA. We'll keep it around for now, but with a
clean separation from the main build.
2024-07-12 18:28:46 -04:00
John Colvin 3095253aa9 feat: Add DataStream WebApp
Included in DataStreamSuite.sln, but not DataStreamQueryAnalyzer.sln
2024-07-12 18:28:46 -04:00
John Colvin 8227304aab Add spark service modules designed for Dependency Injection 2024-07-12 18:28:46 -04:00
John Colvin 4e022a8a12 build: Copy libspark to all dotnet dirs in single cmake target
This is cleaner than the previous impl, which used separate build
targets to copy libspark to "net7.0", and "net7.0-windows" (dirs).
2024-07-12 10:49:04 -04:00
John Colvin 450773f588 fix: Parse ServerAddress as normal string
The "JsonValueOrArrayConverter<T>" doesn't work the way I thought it
would. When it is associated with a property via the JsonConverter
attribute, dotnet json parsing code throws an exception if "T" isn't
assignable to the property type.
2024-07-09 15:14:29 -04:00
John Colvin f09d6badaf fix build (api server): Change namespace to bts.websocket_core 2024-07-09 15:14:29 -04:00
John Colvin 6b054e0718 Remove docker submodules again 2024-07-09 15:14:29 -04:00
John Colvin 5e2789d16e Change externals url from ssh to https style
So we don't need ssh keys to clone it (the builtin credential manager
makes https authentication safe/easy)
2024-07-09 15:14:29 -04:00
John Colvin 9ebdb2e460 cleanup: Fix benign typos in build scripts
Change BUILD_SPARK_PY var name to BUILD_PY (because these scripts to actually build 'spark')
2024-07-09 15:14:29 -04:00
John Colvin d7bf82722b fix build (warn): Only use Registry on windows 2024-07-09 15:14:29 -04:00
John Colvin e1526e25a7 cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2024-07-09 15:14:26 -04:00
John Colvin 089ba9b369 cleanup: Normalize indentation in TradeBarsStream 2024-07-09 15:09:02 -04:00
John Colvin 567976043f cleanup: Disable nullability warnings
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2024-07-09 15:07:24 -04:00
John Colvin 2fd3382939 cleanup: Modernize PubSubRoute code, json parsing 2024-07-09 15:06:34 -04:00
John Colvin a96f1c8f17 cleanup: Remove unused DateTime grammar from exp parser 2024-07-09 15:03:30 -04:00
John Colvin 6f623f9a37 Add JsonRpcRequestObservable, AsyncEnumerable 2024-07-09 15:03:30 -04:00
John Colvin 827189cdb4 Add json facilities for DataStream websocket api 2024-07-09 15:03:30 -04:00
John Colvin 123f12ab88 refactor: Add handy WebsocketExtensions
* IsSocketStateOpen
* IsClosedProperly
2024-07-09 14:59:06 -04:00