Commit Graph
13472 Commits
Author SHA1 Message Date
John Colvin c14bade720 fix add pubsub services 2024-09-22 18:33:41 +00:00
John Colvin 104de5df7b Remove unused PubSubConfig props 2024-09-22 18:33:41 +00:00
John Colvin bd52ef0a8e Fix dsqa config json 2024-09-19 20:03:31 +00:00
John Colvin a015ff98e4 fix: Always call 'handle_internal_pubs' in PubSubMessageReactor main loop 2024-09-19 20:03:31 +00:00
John Colvin 12fc295825 improvement: Accept CancellationToken, catch exceptions when running ds websocket server 2024-09-19 20:03:31 +00:00
John Colvin 587e853f21 Fix mismatched types in PubSubServerTagStatsStream schema 2024-09-19 20:03:31 +00:00
John Colvin 909c8f5bcd Add DataStreamPublisher service modules designed for Dependency Injection
* Change DataStreamPublisher json fields to match cs class property names verbatim
* Rename Publishers -> PublisherNodes in DataStreamPublisherConfig
2024-09-19 20:03:31 +00:00
John Colvin 448d49f38d refactor: Decouple ViewPublisher from IPubTableBroker impl 2024-09-19 20:03:31 +00:00
John Colvin f67c24f7f5 Add PubSub service modules designed for Dependency Injection
* Get rid of JsonPropertyName attributes in PubSubConfig
  - The Dependency Injection framework (based around dotnet ApplicationBuilder)
    does not support this attribute, and maps property names verbatim to json fields.

* Change PubSubServer json fields to match cs class property names verbatim
2024-09-19 20:03:31 +00:00
John Colvin 69a72a1035 refactor: Decouple RemoteQueryManager from IManagedSubscriber, IPubTableBroker impl 2024-09-19 20:03:31 +00:00
John Colvin 819a8f6c22 refactor: Decouple ManagedSubscriber from ISubscriberCore impl 2024-09-19 20:03:31 +00:00
John Colvin 42db20316b refactor: Only support 1 tag type in IPubSubServerTagHandler 2024-09-19 20:03:31 +00:00
John Colvin 633a8590a1 refactor: Decouple PubTableBroker from IPublisherCore impl
Reference interface rather than the PublisherCore impl.
2024-09-19 20:03:31 +00:00
John Colvin eff979fb2b improvement: Accept CancellationToken in JsonRpcRequestObservable 2024-09-19 20:03:31 +00:00
John Colvin 23771909fd fix: Dispose market data sub in various md streams 2024-09-19 20:03:31 +00:00
John Colvin 9de21a3116 cleanup: Remove unused pubsub tag handler code
* Removed 'IPubSubServerTagHandler.register'
  - All implementations of this function were empty
* Removed 'IPubSubServerTagHandler.supportedTagTypes'
  - This property wasn't referenced anywhere
* Remove IWorker inheritance from IPubSubServerTagHandler
  - No reason these interfaces have to be tied together
2024-09-19 20:03:31 +00:00
John Colvin 8d231ce46c refactor: Rename CloudStreamManager.cs -> RemoteQueryManager.cs
This matches the file name
2024-09-19 20:03:31 +00:00
John Colvin fc9d5ab04f Change pub sub schema prefix 'bluto://' -> 'bts-cloud://'
In config json files (not actual code change)
2024-09-19 20:03:29 +00:00
John Colvin 3562f6848e fix build: Fix prop names after refactoring PubSubRoute
* Change 'server_address' -> 'ServerAddress'
* Change 'tag_reg_exp' -> 'TagRegExp'
2024-09-19 20:02:09 +00:00
John Colvin 1511bcf252 build: Add scripts to publish data stream apps 2024-09-19 20:02:09 +00:00
John Colvin b4c902c023 Fix typo in log msg 2024-09-19 20:02:09 +00:00
John Colvin 438ea44c88 Fix build scripts so they propagate the exit code back to the shell
I noticed this was broken because launch.json configurations would still start the debugger
even when the preLaunchTask (build) failed!
2024-09-19 20:02:09 +00:00
John Colvin 75acf12ef3 vscode: Add DataStreamPublisher debug cfg to skip build 2024-09-19 20:02:09 +00:00
John Colvin be407e3e1e build: Use separate cmd to build DataStreamSuite/QueryAnalyzer.sln
The DataStreamSuite.sln builds the new webapp now, which will
(eventually) replace DSQA. We'll keep it around for now, but with a
clean separation from the main build.
2024-09-19 20:02:09 +00:00
John Colvin 2dfa44c1e4 feat: Add DataStream WebApp
Included in DataStreamSuite.sln, but not DataStreamQueryAnalyzer.sln
2024-09-19 20:02:09 +00:00
John Colvin 022cad3122 Add spark service modules designed for Dependency Injection 2024-09-19 20:02:09 +00:00
John Colvin 5c42b4fe8b build: Copy libspark to all dotnet dirs in single cmake target
This is cleaner than the previous impl, which used separate build
targets to copy libspark to "net7.0", and "net7.0-windows" (dirs).
2024-09-19 20:02:09 +00:00
John Colvin 9f81c0ab39 fix: Parse ServerAddress as normal string
The "JsonValueOrArrayConverter<T>" doesn't work the way I thought it
would. When it is associated with a property via the JsonConverter
attribute, dotnet json parsing code throws an exception if "T" isn't
assignable to the property type.
2024-09-19 20:02:09 +00:00
John Colvin c941fa78d2 fix build (api server): Change namespace to bts.websocket_core 2024-09-19 20:02:09 +00:00
John Colvin e6f2c8d1a5 Remove docker submodules again 2024-09-19 20:02:09 +00:00
John Colvin faf296254e Change externals url from ssh to https style
So we don't need ssh keys to clone it (the builtin credential manager
makes https authentication safe/easy)
2024-09-19 20:02:09 +00:00
John Colvin 41712c774f cleanup: Fix benign typos in build scripts
Change BUILD_SPARK_PY var name to BUILD_PY (because these scripts to actually build 'spark')
2024-09-19 20:02:09 +00:00
John Colvin 565c70bb92 fix build (warn): Only use Registry on windows 2024-09-19 20:02:09 +00:00
John Colvin 97bc186110 cleanup: Fix numerous build warnings in DataStreamSuite
Nearly all warnings were due to nullability issues, since the concept of
nullable reference types did not exist when the DataStream code was
originally written.

* PubSubClientMessage
  - Normalize indentation/whitespace
  - Fix some typos
  - Address nullability issues to fix build warnings
  - Modernize code

* Compression related files
  - ZipEntry
  - ZipFile
  - ZipInputStream
  - TarHeader

* RemoteMessage
  - make 'message' nullable to resolve warnings elsewhere (in calling code)

* PubSubSingleTagStateEvent.cs
* PubSubMessageReactor
* PubSubState
* SystemHeartbeatPublisher
* SystemStatsPublisher
* SingleTagData
* ManagedTCPMessageHub
* PubsubDeserializationController
* PublisherCore
* InflaterHuffmanTree
* AsyncTCPConnection_APM
* ThreadSafeQueue
* ManagedSubscriber
* SubscriberCore
* PubSubServerManagedTCPConnection
* AsyncTCPConnection_APM
* SuperMapStream
* MapStream
* TradeBarsStream
2024-09-19 20:02:09 +00:00
John Colvin 158f4fb6c8 cleanup: Normalize indentation in TradeBarsStream 2024-09-19 20:02:09 +00:00
John Colvin c7961f53c0 cleanup: Disable nullability warnings
* BZip2InputStream
* TarInputStream
* Inflater
* InflaterDynHeader
* BZip2OutputStream
* DeflaterHuffman
* ParallelJoinTest
2024-09-19 20:02:09 +00:00
John Colvin 6e89f42bc8 cleanup: Modernize PubSubRoute code, json parsing 2024-09-19 20:02:09 +00:00
John Colvin 0ee450b12a cleanup: Remove unused DateTime grammar from exp parser 2024-09-19 20:02:09 +00:00
John Colvin b2e3a6bf82 Add JsonRpcRequestObservable, AsyncEnumerable 2024-09-19 20:02:09 +00:00
John Colvin f87aa6c4b6 Add json facilities for DataStream websocket api 2024-09-19 20:02:09 +00:00
John Colvin 40a46dd6d7 refactor: Add handy WebsocketExtensions
* IsSocketStateOpen
* IsClosedProperly
2024-09-19 20:02:09 +00:00
John Colvin 75d04b2cd2 feat: Add JsonRpc types for websocket paradigm 2024-09-19 20:02:09 +00:00
John Colvin 423e20ae41 cleanup: Remove empty file (clearly added by accident) 2024-09-19 20:02:09 +00:00
John Colvin de439579cf fix: Fix improper 'OfType' linq cast
Previously, animations were expressed through polymorphism (derived from the
base class CellAnimation), but the new impl uses composition. This 'OfType' call
was leftover from the polymorphism implementation, an attempt to linq cast
elements of base class (CellAnimation) to BackgroundAnimation.
2024-09-19 20:02:09 +00:00
John Colvin adbe72258b editorconfig: Use standard c# capitalization rules
* Use 'PascalCase' for public members (instead of 'snake_case')
* Use 'camelCase' for private members (instead of '_snake_case')
2024-09-19 20:02:09 +00:00
John Colvin 13646a5a16 refactor: Add WebsocketClient class
Basically an ISubject (I/O) wrapper for IWebsocketConnectionGateway
2024-09-19 20:02:09 +00:00
John Colvin 9a1e51eacf refactor: Change WebsocketServerConfig.port to uint32
It should always be a number between 1-65535
2024-09-19 20:02:09 +00:00
John Colvin 9982bd2203 refactor: Rename WebsocketServerConfig proto pkg 'bts.websocket_core'
This matches the namespace used in spark_websocket csproj. Eventually,
I'd like to use conventional capitalization (TitleCase namespaces/types,
etc.), but let's shelf that until we have enough time/energy to do a
more comprehensive refactoring.
2024-09-19 20:02:09 +00:00
John Colvin e7d7352941 refactor: Remove nullability constraint in GetValueOrDefault 2024-09-19 20:02:09 +00:00
John Colvin d2efe037f9 refactor: Use events to avoid circular deps for ws sync/gateway 2024-09-19 20:02:09 +00:00