Commit Graph
13518 Commits
Author SHA1 Message Date
John Colvin 1f4f3b032e Assert that System publishers 'doWork' is always called on same thread 2024-10-30 19:31:08 +00:00
John Colvin f6e54254a9 fix: Make InternalPublisherQueue thread safe 2024-10-30 19:31:08 +00:00
John Colvin b5adcb1a84 Improve error reporting in DataStreamWebSocketServer 2024-10-30 19:31:08 +00:00
John Colvin 179b034bc7 data stream web app client check pt 2024-10-30 19:31:08 +00:00
John Colvin 2cf8a3963d fix build: Fix Spark build after making config init async 2024-10-30 19:31:08 +00:00
John Colvin 49e4e05a3c refactor: Tweak SparkCoreServices to synchronize Configuration.init globally 2024-10-30 19:31:08 +00:00
John Colvin ae9311a2a7 refactor: Change Configuration.initialize to be async 2024-10-30 19:31:08 +00:00
John Colvin 21c3d57324 vscode: Add 'isSparkApiEnabled' input to launch.json 2024-10-30 19:31:08 +00:00
John Colvin 9352f529de SparkApiService check pt 2024-10-30 19:31:08 +00:00
John Colvin 0b76ac4763 datastreamwebapp server checkpt 2024-10-30 19:31:08 +00:00
John Colvin b887801cf5 datastream webapp client check pt 2024-10-30 19:31:08 +00:00
John Colvin c8ffa1489e Remove unused PubSubConfig props 2024-10-30 19:31:08 +00:00
John Colvin f72ea96d20 Fix dsqa config json 2024-10-30 19:31:08 +00:00
John Colvin 7025ffbc4d fix: Always call 'handle_internal_pubs' in PubSubMessageReactor main loop 2024-10-30 19:31:08 +00:00
John Colvin bf989e1f70 improvement: Accept CancellationToken, catch exceptions when running ds websocket server 2024-10-30 19:31:08 +00:00
John Colvin 933db6ea96 Fix mismatched types in PubSubServerTagStatsStream schema 2024-10-30 19:31:08 +00:00
John Colvin 0e92e7f30c Add DataStreamPublisher service modules designed for Dependency Injection
* Change DataStreamPublisher json fields to match cs class property names verbatim
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
2024-10-30 19:31:08 +00:00
John Colvin 25c42e388f refactor: Decouple ViewPublisher from IPubTableBroker impl 2024-10-30 19:31:08 +00:00
John Colvin 3187722159 Add PubSub service modules designed for Dependency Injection
* Get rid of JsonPropertyName attributes in PubSubConfig
  - The Dependency Injection framework (based around dotnet ApplicationBuilder)
    does not support this attribute, and maps property names verbatim to json fields.

* Change PubSubServer json fields to match cs class property names verbatim
2024-10-30 19:31:08 +00:00
John Colvin 352cb97818 refactor: Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl 2024-10-30 19:30:30 +00:00
John Colvin 0215b7990e refactor: Decouple ManagedSubscriber from ISubscriberCore impl 2024-10-30 19:30:30 +00:00
John Colvin 1644b5b58b refactor: Only support 1 tag type in IPubSubServerTagHandler 2024-10-30 19:30:30 +00:00
John Colvin b2ea6820d7 refactor: Decouple PubTableBroker from IPublisherCore impl
Reference interface rather than the PublisherCore impl.
2024-10-30 19:30:30 +00:00
John Colvin 5c9fc5ccc2 improvement: Accept CancellationToken in JsonRpcRequestObservable 2024-10-30 19:30:30 +00:00
John Colvin 4bd6f149b9 fix: Dispose market data sub in various md streams 2024-10-30 19:30:30 +00:00
John Colvin 293f7f3833 cleanup: Remove unused pubsub tag handler code
* Removed 'IPubSubServerTagHandler.register'
  - All implementations of this function were empty
* Removed 'IPubSubServerTagHandler.supportedTagTypes'
  - This property wasn't referenced anywhere
* Remove IWorker inheritance from IPubSubServerTagHandler
  - No reason these interfaces have to be tied together
2024-10-30 19:30:30 +00:00
John Colvin ad2f9a8624 refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
This matches the file name
2024-10-30 19:30:30 +00:00
John Colvin 99922725e1 Change pub sub schema prefix 'bluto://' -> 'bts-cloud://'
In config json files (not actual code change)
2024-10-30 19:30:30 +00:00
John Colvin 60c3cc0015 fix build: Fix prop names after refactoring PubSubRoute
* Change 'server_address' -> 'ServerAddress'
* Change 'tag_reg_exp' -> 'TagRegExp'
2024-10-30 19:30:30 +00:00
John Colvin 775567968b build: Add scripts to publish data stream apps 2024-10-30 19:30:30 +00:00
John Colvin e1555387ec Fix typo in log msg 2024-10-30 19:30:30 +00:00
John Colvin d044f711a3 Fix build scripts so they propagate the exit code back to the shell
I noticed this was broken because launch.json configurations would still start the debugger
even when the preLaunchTask (build) failed!
2024-10-30 19:30:30 +00:00
John Colvin eba801845c vscode: Add DataStreamPublisher debug cfg to skip build 2024-10-30 19:30:30 +00:00
John Colvin 04f85818c6 build: Use separate cmd to build DataStreamSuite/QueryAnalyzer.sln
The DataStreamSuite.sln builds the new webapp now, which will
(eventually) replace DSQA. We'll keep it around for now, but with a
clean separation from the main build.
2024-10-30 19:30:29 +00:00
John Colvin 545dcdf9ff feat: Add DataStream WebApp
Included in DataStreamSuite.sln, but not DataStreamQueryAnalyzer.sln
2024-10-30 19:30:29 +00:00
John Colvin 8c2d643e0c Add spark service modules designed for Dependency Injection 2024-10-30 19:30:29 +00:00
John Colvin ae70f731b7 build: Copy libspark to all dotnet dirs in single cmake target
This is cleaner than the previous impl, which used separate build
targets to copy libspark to "net7.0", and "net7.0-windows" (dirs).
2024-10-30 19:30:29 +00:00
John Colvin 828b06d4e2 fix: Parse ServerAddress as normal string
The "JsonValueOrArrayConverter<T>" doesn't work the way I thought it
would. When it is associated with a property via the JsonConverter
attribute, dotnet json parsing code throws an exception if "T" isn't
assignable to the property type.
2024-10-30 19:30:29 +00:00
John Colvin 6fa8de176c fix build (api server): Change namespace to bts.websocket_core 2024-10-30 19:30:29 +00:00
John Colvin fec918aefd Remove docker submodules again 2024-10-30 19:30:29 +00:00
John Colvin aa57b3bb8c Change externals url from ssh to https style
So we don't need ssh keys to clone it (the builtin credential manager
makes https authentication safe/easy)
2024-10-30 19:30:29 +00:00
John Colvin ab206ccf87 cleanup: Fix benign typos in build scripts
Change BUILD_SPARK_PY var name to BUILD_PY (because these scripts to actually build 'spark')
2024-10-30 19:30:29 +00:00
John Colvin 06ad7c51be fix build (warn): Only use Registry on windows 2024-10-30 19:30:29 +00:00
John Colvin 60a52b3922 cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2024-10-30 19:30:29 +00:00
John Colvin 5426ba0aa7 cleanup: Disable nullability warnings
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2024-10-30 19:24:18 +00:00
John Colvin b1fdee5a22 cleanup: Modernize PubSubRoute code, json parsing 2024-10-30 19:24:18 +00:00
John Colvin 59461aadff cleanup: Remove unused DateTime grammar from exp parser 2024-10-30 19:24:18 +00:00
John Colvin 2d3a9a77ae Add JsonRpcRequestObservable, AsyncEnumerable 2024-10-30 19:24:18 +00:00
John Colvin 868b43ef8b Add json facilities for DataStream websocket api 2024-10-30 19:24:18 +00:00
John Colvin fc7043f6e1 refactor: Add handy WebsocketExtensions
* IsSocketStateOpen
* IsClosedProperly
2024-10-30 19:24:18 +00:00