Commit Graph
14272 Commits
Author SHA1 Message Date
John Colvin debaaa0d04 Improve error reporting in DataStreamWebSocketServer 2025-07-23 15:59:52 -04:00
John Colvin 43735ed42e data stream web app client check pt 2025-07-23 15:59:43 -04:00
John Colvin 44b214ec55 fix build: Fix Spark build after making config init async 2025-07-23 15:58:33 -04:00
John Colvin 4188b8c00d refactor: Tweak SparkCoreServices to synchronize Configuration.init globally 2025-07-23 15:55:46 -04:00
John Colvin 44214aa195 refactor: Change Configuration.initialize to be async 2025-07-23 15:54:41 -04:00
John Colvin 16430e2b42 vscode: Add 'isSparkApiEnabled' input to launch.json 2025-07-23 15:54:21 -04:00
John Colvin 856fb6410b SparkApiService check pt 2025-07-23 15:53:22 -04:00
John Colvin ed8420f346 datastreamwebapp server checkpt 2025-07-23 15:53:06 -04:00
John Colvin ca90b5dc43 datastream webapp client check pt 2025-07-23 15:52:40 -04:00
John Colvin 04829b1226 Remove unused PubSubConfig props 2025-07-23 15:52:09 -04:00
John Colvin d725fe4ba1 Fix dsqa config json 2025-07-23 15:51:30 -04:00
John Colvin fc8a39745d fix: Always call 'handle_internal_pubs' in PubSubMessageReactor main loop 2025-07-23 15:49:41 -04:00
John Colvin 476dd44838 improvement: Accept CancellationToken, catch exceptions when running ds websocket server 2025-07-23 15:48:25 -04:00
John Colvin 92d301831c Fix mismatched types in PubSubServerTagStatsStream schema 2025-07-23 15:48:09 -04:00
John Colvin c47ddf041f Add DataStreamPublisher service modules designed for Dependency Injection
* Change DataStreamPublisher json fields to match cs class property names verbatim
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
2025-07-23 15:47:51 -04:00
John Colvin 2b7fa477fb refactor: Decouple ViewPublisher from IPubTableBroker impl 2025-07-23 15:47:39 -04:00
John Colvin 850d6a1ce4 Add PubSub service modules designed for Dependency Injection
* Get rid of JsonPropertyName attributes in PubSubConfig
  - The Dependency Injection framework (based around dotnet ApplicationBuilder)
    does not support this attribute, and maps property names verbatim to json fields.

* Change PubSubServer json fields to match cs class property names verbatim
2025-07-23 15:45:01 -04:00
John Colvin 0c3f063727 refactor: Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl 2025-07-23 15:42:41 -04:00
John Colvin 742001fbb8 refactor: Decouple ManagedSubscriber from ISubscriberCore impl 2025-07-23 15:42:20 -04:00
John Colvin 3abb05ca7a refactor: Only support 1 tag type in IPubSubServerTagHandler 2025-07-23 15:41:56 -04:00
John Colvin 6f56471b91 refactor: Decouple PubTableBroker from IPublisherCore impl
Reference interface rather than the PublisherCore impl.
2025-07-23 15:41:41 -04:00
John Colvin 19a3541546 improvement: Accept CancellationToken in JsonRpcRequestObservable 2025-07-23 15:41:27 -04:00
John Colvin ef64417d27 fix: Dispose market data sub in various md streams 2025-07-23 15:41:05 -04:00
John Colvin 988139d549 cleanup: Remove unused pubsub tag handler code
* Removed 'IPubSubServerTagHandler.register'
  - All implementations of this function were empty
* Removed 'IPubSubServerTagHandler.supportedTagTypes'
  - This property wasn't referenced anywhere
* Remove IWorker inheritance from IPubSubServerTagHandler
  - No reason these interfaces have to be tied together
2025-07-23 15:36:48 -04:00
John Colvin 14edcc00f3 refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
This matches the file name
2025-07-23 15:36:29 -04:00
John Colvin 25ecaca6f1 Change pub sub schema prefix 'bluto://' -> 'bts-cloud://'
In config json files (not actual code change)
2025-07-23 15:36:11 -04:00
John Colvin a0b9a9b2cf fix build: Fix prop names after refactoring PubSubRoute
* Change 'server_address' -> 'ServerAddress'
* Change 'tag_reg_exp' -> 'TagRegExp'
2025-07-23 15:33:49 -04:00
John Colvin f43e7d4724 build: Add scripts to publish data stream apps 2025-07-23 15:33:31 -04:00
John Colvin 5e856dd974 Fix typo in log msg 2025-07-23 15:30:52 -04:00
John Colvin c6e7cdcf35 Fix build scripts so they propagate the exit code back to the shell
I noticed this was broken because launch.json configurations would still
start the debugger even when the preLaunchTask (build) failed!
2025-07-23 15:30:32 -04:00
John Colvin a7b3ba07dc vscode: Add DataStreamPublisher debug cfg to skip build 2025-07-23 15:29:00 -04:00
John Colvin 11bb3234da build: Use separate cmd to build DataStreamSuite/QueryAnalyzer.sln
The DataStreamSuite.sln builds the new webapp now, which will
(eventually) replace DSQA. We'll keep it around for now, but with a
clean separation from the main build.
2025-07-23 15:26:30 -04:00
John Colvin 7d6d63365e feat: Add DataStream WebApp
Included in DataStreamSuite.sln, but not DataStreamQueryAnalyzer.sln
2025-07-23 15:21:58 -04:00
John Colvin ddf9bc7c0b Add spark service modules designed for Dependency Injection 2025-07-23 15:14:34 -04:00
John Colvin b214198dd6 cleanup: Fix benign typos in build scripts
Change BUILD_SPARK_PY var name to BUILD_PY (these scripts build other
projects than 'spark').
2025-07-23 15:11:19 -04:00
John Colvin 15738f65e7 fix build (warn): Only use Registry on windows 2025-07-23 13:23:10 -04:00
John Colvin f2aaf524cb cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2025-07-23 13:22:48 -04:00
John Colvin f43f0d3950 cleanup: Disable nullability warnings
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2025-07-23 13:16:22 -04:00
John Colvin f48e403a1f cleanup: Modernize PubSubRoute code, json parsing 2025-07-23 13:15:32 -04:00
John Colvin 22bfbd8ddb cleanup: Remove unused DateTime grammar from exp parser 2025-07-23 13:15:13 -04:00
John Colvin 44349f3f53 Add JsonRpcRequestObservable, AsyncEnumerable 2025-07-23 13:14:43 -04:00
John Colvin f79d897f62 Add json facilities for DataStream websocket api 2025-07-23 13:14:18 -04:00
John Colvin df7672832a refactor: Add handy WebsocketExtensions
* IsSocketStateOpen
* IsClosedProperly
2025-07-23 13:13:57 -04:00
John Colvin b3f23c1210 feat: Add JsonRpc types for websocket paradigm 2025-07-23 13:13:38 -04:00
John Colvin 353449f3d8 cleanup: Remove empty file (clearly added by accident) 2025-07-23 13:13:19 -04:00
John Colvin 096432e432 editorconfig: Use standard c# capitalization rules
* Use 'PascalCase' for public members (instead of 'snake_case')
* Use 'camelCase' for private members (instead of '_snake_case')
2025-07-23 13:10:59 -04:00
John Colvin eecc70f151 Add generic ViewStream, BlueRow types/interfaces 2025-07-23 12:52:23 -04:00
John Colvin b121849189 Add BlueRowColumnMask to hide columns w/ minimal overhead
Less overhead than ProjectStream
2025-07-23 10:55:31 -04:00
John Colvin bbc07ce5fe fix: Remove query from literal error stream
The parser can't handle colons in the expression (it throws/crashes)
2025-07-23 10:55:31 -04:00
John Colvin 186ebb5309 Add async SQLiteEnumerator to read sqlite tables as IBlueRowCollection 2025-07-23 10:55:31 -04:00